The Data Engineer will design, develop, and optimize big data pipelines using SQL, Python, Spark, Hive, and Hadoop, ensuring data quality and performance.
Role Overview
We are looking for a skilled Data Engineer with 5 years of hands-on experience in designing, developing, and optimizing big data pipelines and solutions. The ideal candidate will have strong expertise in SQL, Python, Apache Spark, Hive, and Hadoop ecosystems and will be responsible for building scalable data platforms to support business intelligence, analytics, and machine learning use cases.
Key Responsibilities- Design, develop, and maintain scalable ETL pipelines using Spark, Hive, and Hadoop.
- Write efficient SQL queries for data extraction, transformation, and analysis.
- Develop automation scripts and data processing workflows using Python.
- Optimize data pipelines for performance, reliability, and scalability.
- Work with structured and unstructured data from multiple sources.
- Ensure data quality, governance, and security throughout the data lifecycle.
- Collaborate with cross-functional teams (Data Scientists, Analysts, and Business stakeholders) to deliver data-driven solutions.
- Monitor and troubleshoot production data pipelines.
Required Skills & Qualifications
- 5+ years of experience in Data Engineering / Big Data development.
- Strong expertise in SQL (query optimization, performance tuning, stored procedures).
- Proficiency in Python for data manipulation, scripting, and automation.
- Hands-on experience with Apache Spark (PySpark/Scala) for large-scale data processing.
- Solid knowledge of Hive for querying and managing data in Hadoop environments.
- Strong working knowledge of Hadoop ecosystem (HDFS, YARN, MapReduce, etc.).
- Experience with data pipeline orchestration tools (Airflow, Oozie, or similar) is a plus.
- Familiarity with cloud platforms (AWS, Azure, or GCP) is preferred.
- Excellent problem-solving, debugging, and communication skills.
Top Skills
Spark
Hadoop
Hive
Python
SQL
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
Similar Jobs
Information Technology • Software
The Deal Desk Analyst collaborates with sales teams to create, review, and approve commercial quotes, ensuring compliance and managing deal governance. Responsibilities include negotiating commercial terms and improving deal desk productivity.
Top Skills:
ExcelPowerPointSalesforceSalesforce Cpq
.jpg)
Fintech • Analytics
The Accounts Receivable Executive will manage invoicing, resolve discrepancies, maintain accounting records, and support finance teams while ensuring effective operations.
Top Skills:
ExcelMS OfficeOracle
Information Technology • Software
As a SaaS Operations Engineer, you will manage incidents, execute services, deploy applications, and collaborate with teams to ensure quality service delivery for Sitecore's SaaS platform.
Top Skills:
AksAWSAzureElasticElastic SearchHelmJira Service DeskKubernetesLinuxAzureMongo DbNoSQLOpsgeniePagerdutyRedisRedisService Now
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
