The Data Engineer is responsible for building and optimizing big data pipelines, analyzing data to address business questions, and supporting data modeling and implementation. They handle data transformation processes, work with unstructured datasets, and manage data architecture, ensuring effective data workflows and deployments.
- Build and optimizing ‘big data’ data pipelines, architectures and data sets.
- Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- To support and develop Data modeling, Implementation activities.
- Strong analytic skills related to working with unstructured datasets.
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Manipulating, processing and extracting value from large disconnected datasets.
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Perform CI/CD Activities, Deployment activities in both cloud and on premises or as required.
- Experience with Data engineering tools (Python, Scala, Hadoop).
- Experience with relational (oracle, MySQL, MS Sql) or NoSQL databases, including Postgres and Cassandra.
- Experience with data pipeline and workflow management tools.
- Experience with stream-processing systems
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala,etc
- Experience on Cloudera/ Data bricks/ Snowflakes platforms will be added advantage
- Experience with Cloud (Aws, Azure or Google).
Top Skills
C++
Java
Python
Scala
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
Similar Jobs
Be an Early Applicant
As a Junior Data Engineer, you will design, develop, and maintain data pipelines using Hadoop and Spark. You will collaborate with teams to ensure data quality, and implement CI/CD processes with DevOps. The role requires optimizing workflows for containerized deployment and mentoring junior members.
Be an Early Applicant
The Data Engineer (Python) will be responsible for ingesting data from various sources, curating data assets, collaborating with teams, deploying ML models, and architecting data pipelines to enable informed decision-making. The role focuses on enhancing the user experience through data-driven solutions.
Be an Early Applicant
As a Junior Data Engineer, you will support the Regional Data team by building data pipelines, ingesting data sources into the data warehouse, and creating tools for data consumers. You will ensure high-quality data assets for data scientists and collaborate with cross-functional teams to enhance data-driven decision-making.
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
