Design, build, and optimize end-to-end data ingestion and ETL pipelines on AWS into governed data lake zones. Implement data quality frameworks, error handling, and metadata/catalog management. Use IaC (Terraform/AWS CDK), write tests, support UAT, produce runbooks/documentation, and participate in Agile delivery.
About OneByZero
OneByZero is a Frontier Systems Integrator building agentic AI systems for leading banks, telcos, insurers, and retailers across Asia Pacific. We work on mission-critical problems for organizations operating the region's financial and digital infrastructure. Operating exclusively on AWS, we combine deep technical expertise with strong partnership programs that enable innovation, scale, and delivery excellence. We build with people who are technically serious, outcome-focused, and passionate about solving complex enterprise challenges.
The Role
We are seeking a highly skilled Data Pipeline Engineer to design, build, and optimize scalable data ingestion and transformation pipelines on AWS. The ideal candidate will be responsible for developing end-to-end data pipelines, implementing data quality frameworks, and enabling reliable data delivery into a governed data lake environment.
This role requires strong expertise in AWS data services, ETL development, Infrastructure as Code (IaC), and modern data engineering best practices.
- Develop and implement end-to-end data ingestion pipelines from source systems through AWS S3, AWS Glue ETL, and AWS Lake Formation curated zones.
- Design and implement ETL transformations based on defined data mapping specifications.
- Build and maintain data quality validation frameworks to ensure completeness, schema conformance, and referential integrity.
- Configure error handling, exception management, and dead-letter queue patterns for failed ingestion records.
- Register and manage datasets within AWS Glue Data Catalog, ensuring standardized metadata tagging and governance compliance.
- Develop and maintain Infrastructure as Code (IaC) using Terraform or AWS CDK for data platform resources.
- Execute unit testing and integration testing to validate pipeline functionality and data accuracy.
- Support User Acceptance Testing (UAT) by validating output datasets, schemas, and record counts with business stakeholders.
- Create and maintain technical documentation including pipeline configurations, runbooks, and data flow diagrams.
- Participate in Agile ceremonies including sprint planning, stand-ups, demos, and code reviews.
RequirementsRequired Skills & Experience
- 5+ years of experience in Data Engineering or Data Platform Development.
- Strong hands-on experience with AWS Glue, AWS Lake Formation, Amazon S3, and Amazon Athena.
- Proficiency in Python, PySpark, and SQL.
- Experience building ETL/ELT pipelines in cloud-native environments.
- Hands-on experience with Infrastructure as Code (IaC) tools such as Terraform or AWS CDK.
- Strong understanding of data modeling, data governance, and metadata management.
- Experience with automated testing and deployment practices.
- Experience with Great Expectations or similar data quality frameworks.
- Exposure to SHIP-HATS CI/CD pipelines.
- Knowledge of enterprise data cataloging and governance solutions.
- Prior experience working with Government Agencies or Public Sector projects.
Education
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
Benefits
What We Offer
Competitive compensation benchmarked to market. The opportunity to work on AI transformation programmes that matter to the institutions millions of people in Asia Pacific depend on. A team that spans 10 markets, brings together some of the most experienced AI, data, and enterprise technology practitioners in the region, and moves fast without losing quality. Access to the AWS partnership programmes and frontier AI tooling that most firms in this region do not have. A firm that is building something genuinely new, with a leadership team that has scaled and delivered before.
OneByZero is an equal opportunity employer. We are committed to building a diverse and inclusive team and welcome applications from candidates of all backgrounds.
Why OneByZero
Agentic AI in regulated enterprise is the hardest version of this problem. The clients are the institutions that run the financial infrastructure of Asia Pacific. The constraints are real. The stakes are high.
OBZ is one of the few firms in the region doing this work at production scale in regulated industries, with an AWS partnership that puts you at the frontier of what is deployable. You will build systems that matter, work in squads small enough that your contribution is visible, and use AI coworkers as a genuine force multiplier.
If you are at the start of your career and want to be shaped by the hardest agentic engineering problems in the region, this is where that happens. If you are experienced and want to work on problems that are still genuinely unsolved, this is also where that happens.
Note - Only candidates with existing unrestricted work authorization in Singapore will be considered, as visa sponsorship is not available for this position
OneByZero Singapore Office
Similar Jobs
Information Technology • Software • Financial Services • Quantitative Trading
Quantitative Researchers at Citadel Securities develop automated trading strategies using statistical techniques and implement them in live environments.
Top Skills:
C++PythonR
Information Technology • Software • Financial Services • Quantitative Trading
12-week internship designing, implementing, and testing FPGA solutions for ultra-low latency trading systems. Work in small teams to improve trading system efficiency, perform RTL coding, simulation, synthesis, validation and integration, and collaborate with senior engineers.
Top Skills:
BashCFpga SynthesisHardware ValidationPythonRtl CodingSimulationStatic AnalysisSystemverilogTclVhdl
Information Technology • Software • Financial Services • Quantitative Trading
The intern will develop trading strategies, back test models, analyze data, and collaborate with the research team to innovate and refine monetization systems.
Top Skills:
C++PythonR
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
