Data Engineer

Role Purpose

The Data Engineer designs, builds and deploys scalable data lakehouse platforms for our clients - architecting end-to-end pipelines, establishing data quality frameworks, and delivering production-ready solutions that form the foundation of an organisation's data infrastructure. You will work across modern lakehouse stacks (Databricks, Spark, Microsoft Fabric or AWS) alongside architects, AI engineers and business analysts on the same delivery, not in a separate data silo.

What You'll Own

Data Pipeline & Architecture Design

• Design and implement scalable ETL/ELT pipelines on modern cloud data platforms - AWS (Glue, Step Functions, Lambda, S3), Databricks, Spark or Microsoft Fabric.
• Architect and build data lakehouse solutions using open table formats such as Apache Iceberg, Delta Lake or S3 Tables, including schema evolution, partition evolution and ACID transactions.
• Optimise pipelines for performance, cost and reliability at enterprise scale.

Data Quality & Governance

• Define, implement and maintain automated data quality validation frameworks, with metrics and monitoring for accuracy, completeness and consistency.
• Enforce data governance standards, access controls, lineage and PII handling across the platform, appropriate to public sector and regulated client environments.

Application Development & Deployment

• Write production-quality code and deploy solutions on cloud infrastructure, including Government Commercial Cloud (GCC) environments.
• Apply DevOps practices - CI/CD pipelines and infrastructure-as-code (Terraform, CloudFormation or equivalent) - for repeatable and auditable deployments.

AI & Analytics Enablement

• Prepare and serve data for AI and agentic workloads - feature pipelines, retrieval sources and knowledge bases.
• Support analytics and BI teams with reliable, well-modelled datasets and clear semantics.

Collaboration, Handover & Day 2 Operations

• Work across cross-functional teams and communicate technical designs clearly to non-technical stakeholders.
• Produce thorough technical documentation to support handover to Day 2 operations teams.
• Own the pipelines you build in production - monitoring, incident response, root-cause fixes and cost tuning as data volumes grow.

AI-Assisted Delivery

We deliver with an in-house AI workbench that pairs practitioners and AI across the lifecycle. As a Data Engineer you'll use it to:

• Accelerate pipeline scaffolding, transformation logic and test data generation.
• Support schema mapping, profiling and technical documentation of source systems.
• Assist with code review, data quality rule suggestion and runbook drafting.

AI accelerates the work; you own the design decisions, the data correctness and the outcome.

What You'll Bring

Must-Have

• Degree in Computer Science, Data Engineering, Information Systems or a related field.
• 4+ years of hands-on experience in data engineering, ETL/ELT development or data platform roles, building and operating production pipelines.
• Strong hands-on experience with at least one modern data platform - Databricks, Apache Spark, Microsoft Fabric or AWS (Glue, Step Functions, Lambda, S3, Redshift).
• Proficiency with open table formats and lakehouse architecture - Apache Iceberg, Delta Lake or S3 Tables - including schema evolution, partitioning and ACID transactions.
• Strong SQL and proficiency in Python (or Scala/Java) for data processing and automation.
• Experience designing data models and warehouse/lakehouse layers for analytics, reporting and AI workloads.
• Experience defining and automating data quality validation, monitoring and reconciliation.
• Working knowledge of DevOps for data - CI/CD, version control and infrastructure-as-code (Terraform, CloudFormation or equivalent).
• Ability to work directly with business and technical stakeholders and communicate technical designs clearly to non-technical audiences.

Good-to-Have

• Experience delivering data projects in the Singapore Public Sector, particularly in a Government Commercial Cloud (GCC / GCC+) environment.
• Cloud or platform certification - AWS Certified Data Analytics, AWS Certified Solutions Architect, Databricks Data Engineer, or Azure/Fabric Data Engineer.
• Exposure to AI/ML workloads - feature pipelines, vector stores, RAG data preparation or MLOps.
• Experience with data migration from legacy systems, including reconciliation and cutover.
• Familiarity with governance frameworks and PII handling (e.g. masking, tokenisation, Presidio).
• Experience with BI and semantic layers (Power BI, Tableau, Looker) and enabling self-service analytics.

Not for You If

This role is not a fit if:

• You want to work only on ad-hoc analysis and dashboards rather than building and running pipelines.
• You see data quality, testing and documentation as someone else's job.
• You're uncomfortable supporting what you build once it's in production.
• You prefer a single fixed tech stack over learning new platforms as client needs change.
• You treat technical documentation and operational handover as an afterthought.