The Data Engineer is responsible for building and optimizing big data pipelines, analyzing data to address business questions, and supporting data modeling and implementation. They handle data transformation processes, work with unstructured datasets, and manage data architecture, ensuring effective data workflows and deployments.
- Build and optimizing ‘big data’ data pipelines, architectures and data sets.
- Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- To support and develop Data modeling, Implementation activities.
- Strong analytic skills related to working with unstructured datasets.
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Manipulating, processing and extracting value from large disconnected datasets.
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Perform CI/CD Activities, Deployment activities in both cloud and on premises or as required.
- Experience with Data engineering tools (Python, Scala, Hadoop).
- Experience with relational (oracle, MySQL, MS Sql) or NoSQL databases, including Postgres and Cassandra.
- Experience with data pipeline and workflow management tools.
- Experience with stream-processing systems
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala,etc
- Experience on Cloudera/ Data bricks/ Snowflakes platforms will be added advantage
- Experience with Cloud (Aws, Azure or Google).
Top Skills
C++
Java
Python
Scala
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
Similar Jobs
Financial Services
As a Data Engineer III at JPMorgan Chase, you will design and maintain scalable and secure data solutions, develop critical data pipelines, and analyze data risks. Collaborating with various stakeholders, you will support analytics teams while utilizing Agile DevOps practices and cloud technologies. Your role emphasizes building robust systems capable of handling large volumes of data and ensuring data accuracy and timeliness. You will also document processes and enhance team culture.
Top Skills:
Python
Fintech • Information Technology • Payments
The Data Engineer will design and develop systems for Visa's Loyalty & Marketing Services, focusing on high-availability and performance. Responsibilities include developing new products, enhancing existing systems, coordinating CI activities, and instilling best practices in software development. The role requires collaboration with architects and stakeholders to ensure high quality and timely delivery of products.
Top Skills:
JavaScala
Fintech • Financial Services
The Quantexa Data Engineer will design and implement data solutions using Quantexa technology, develop and optimize ETL pipelines, and work with collaborative teams to enhance data quality and performance. The role requires strong programming skills and the ability to communicate technical concepts effectively.
Top Skills:
JavaPythonScala
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
