Design, develop and maintain ETL pipelines using Spark, Hadoop and related frameworks while collaborating with stakeholders to ensure data quality.
- Design, develop, and maintain ETL pipelines and data workflows using Spark, Hadoop, Hive, and related frameworks.
- Work with large-scale structured and unstructured data to enable business insights.
- Collaborate with Data Scientists, Analysts, and other stakeholders to ensure data availability and quality.
- Implement data ingestion and transformation processes using distributed computing frameworks.
- Optimize performance of big data applications and troubleshoot issues in the data pipelines.
- Ensure data security, compliance, and governance across all platforms.
- 3–6 years of experience as a Data Engineer or in a similar role.
- Strong hands-on knowledge of Apache Spark, Hadoop, Hive, HDFS, and SQL.
- Proficiency in Python / Scala / Java for data engineering tasks.
- Solid understanding of data warehousing concepts, ETL design patterns, and data modeling.
- Experience with workflow orchestration tools (Airflow, Oozie, etc.) is a plus.
- Familiarity with cloud platforms (Azure, AWS, or GCP) is advantageous.
- Strong problem-solving skills and ability to work in a collaborative team environment.
Top Skills
Airflow
AWS
Azure
GCP
Hadoop
Hive
Java
Oozie
Python
Scala
Spark
SQL
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
Similar Jobs
Cloud • Information Technology • Software
As a Data Engineer, you will design and develop data pipelines, collaborate with teams to deliver data-driven insights, and ensure code quality through practices like pair programming. Strong SQL and ETL experience are essential.
Top Skills:
AthenaAWSAws DmsAws LambdaAws S3AzureBeautifulsoupCassandraEcsEventbridgeGlueGCPHadoopKafkaMongoDBMySQLNode.jsPandasPhantomjsPostgresPythonRRabbitMQSeleniumSparkSpringSQLSqliteSsisVoltdb
Retail
The Data Engineer will design, develop, and maintain ETL pipelines, collaborate with stakeholders for data solutions, and ensure compliance with data governance practices.
Top Skills:
AlicloudAWSGoogle Cloud PlatformAzure
Information Technology • Consulting
The Data Engineer will develop and maintain data pipelines using Spark and SQL, collaborating with teams to enhance data quality for analysis and processing.
Top Skills:
MySQLOraclePostgresSparkSQL
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
