Design, develop and maintain ETL pipelines using Spark, Hadoop and related frameworks while collaborating with stakeholders to ensure data quality.
- Design, develop, and maintain ETL pipelines and data workflows using Spark, Hadoop, Hive, and related frameworks.
- Work with large-scale structured and unstructured data to enable business insights.
- Collaborate with Data Scientists, Analysts, and other stakeholders to ensure data availability and quality.
- Implement data ingestion and transformation processes using distributed computing frameworks.
- Optimize performance of big data applications and troubleshoot issues in the data pipelines.
- Ensure data security, compliance, and governance across all platforms.
- 3–6 years of experience as a Data Engineer or in a similar role.
- Strong hands-on knowledge of Apache Spark, Hadoop, Hive, HDFS, and SQL.
- Proficiency in Python / Scala / Java for data engineering tasks.
- Solid understanding of data warehousing concepts, ETL design patterns, and data modeling.
- Experience with workflow orchestration tools (Airflow, Oozie, etc.) is a plus.
- Familiarity with cloud platforms (Azure, AWS, or GCP) is advantageous.
- Strong problem-solving skills and ability to work in a collaborative team environment.
Top Skills
Airflow
AWS
Azure
GCP
Hadoop
Hive
Java
Oozie
Python
Scala
Spark
SQL
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
Similar Jobs
Gaming • Hardware
The Data Engineer will design and develop data pipelines for AI applications, ensuring scalable and optimized data infrastructure while collaborating with AI/ML engineers and data scientists.
Top Skills:
AirflowAWSAzureDaskData Build Tool (Dbt)DockerGoogle Cloud PlatformKubernetesNeo4JPrefectPythonSparkSQLTerraform
Information Technology • Consulting
The Data Engineer is responsible for designing, building, and maintaining ETL/ELT data pipelines, optimizing SQL queries, and collaborating with teams to ensure data quality and compliance.
Top Skills:
EltETLGitHiveLinuxSparkSQLUnix
Information Technology • Consulting
The Junior Data Engineer will write and optimize SQL queries, work with Apache Spark, and develop Hive tables for data pipelines, ensuring data quality and performance.
Top Skills:
SparkHiveSQL
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
