Design, build, and optimize end-to-end data ingestion and ETL pipelines on AWS into governed data lake zones. Implement data quality frameworks, error handling, and metadata/catalog management. Use IaC (Terraform/AWS CDK), write tests, support UAT, produce runbooks/documentation, and participate in Agile delivery.
About OneByZero
OneByZero is a Frontier Systems Integrator building agentic AI systems for leading banks, telcos, insurers, and retailers across Asia Pacific. We work on mission-critical problems for organizations operating the region's financial and digital infrastructure. Operating exclusively on AWS, we combine deep technical expertise with strong partnership programs that enable innovation, scale, and delivery excellence. We build with people who are technically serious, outcome-focused, and passionate about solving complex enterprise challenges.
The Role
We are seeking a highly skilled Data Pipeline Engineer to design, build, and optimize scalable data ingestion and transformation pipelines on AWS. The ideal candidate will be responsible for developing end-to-end data pipelines, implementing data quality frameworks, and enabling reliable data delivery into a governed data lake environment.
This role requires strong expertise in AWS data services, ETL development, Infrastructure as Code (IaC), and modern data engineering best practices.
- Develop and implement end-to-end data ingestion pipelines from source systems through AWS S3, AWS Glue ETL, and AWS Lake Formation curated zones.
- Design and implement ETL transformations based on defined data mapping specifications.
- Build and maintain data quality validation frameworks to ensure completeness, schema conformance, and referential integrity.
- Configure error handling, exception management, and dead-letter queue patterns for failed ingestion records.
- Register and manage datasets within AWS Glue Data Catalog, ensuring standardized metadata tagging and governance compliance.
- Develop and maintain Infrastructure as Code (IaC) using Terraform or AWS CDK for data platform resources.
- Execute unit testing and integration testing to validate pipeline functionality and data accuracy.
- Support User Acceptance Testing (UAT) by validating output datasets, schemas, and record counts with business stakeholders.
- Create and maintain technical documentation including pipeline configurations, runbooks, and data flow diagrams.
- Participate in Agile ceremonies including sprint planning, stand-ups, demos, and code reviews.
RequirementsRequired Skills & Experience
- 5+ years of experience in Data Engineering or Data Platform Development.
- Strong hands-on experience with AWS Glue, AWS Lake Formation, Amazon S3, and Amazon Athena.
- Proficiency in Python, PySpark, and SQL.
- Experience building ETL/ELT pipelines in cloud-native environments.
- Hands-on experience with Infrastructure as Code (IaC) tools such as Terraform or AWS CDK.
- Strong understanding of data modeling, data governance, and metadata management.
- Experience with automated testing and deployment practices.
- Experience with Great Expectations or similar data quality frameworks.
- Exposure to SHIP-HATS CI/CD pipelines.
- Knowledge of enterprise data cataloging and governance solutions.
- Prior experience working with Government Agencies or Public Sector projects.
Education
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
Benefits
What We Offer
Competitive compensation benchmarked to market. The opportunity to work on AI transformation programmes that matter to the institutions millions of people in Asia Pacific depend on. A team that spans 10 markets, brings together some of the most experienced AI, data, and enterprise technology practitioners in the region, and moves fast without losing quality. Access to the AWS partnership programmes and frontier AI tooling that most firms in this region do not have. A firm that is building something genuinely new, with a leadership team that has scaled and delivered before.
OneByZero is an equal opportunity employer. We are committed to building a diverse and inclusive team and welcome applications from candidates of all backgrounds.
Why OneByZero
Agentic AI in regulated enterprise is the hardest version of this problem. The clients are the institutions that run the financial infrastructure of Asia Pacific. The constraints are real. The stakes are high.
OBZ is one of the few firms in the region doing this work at production scale in regulated industries, with an AWS partnership that puts you at the frontier of what is deployable. You will build systems that matter, work in squads small enough that your contribution is visible, and use AI coworkers as a genuine force multiplier.
If you are at the start of your career and want to be shaped by the hardest agentic engineering problems in the region, this is where that happens. If you are experienced and want to work on problems that are still genuinely unsolved, this is also where that happens.
Note - Only candidates with existing unrestricted work authorization in Singapore will be considered, as visa sponsorship is not available for this position
OneByZero Singapore Office
Similar Jobs
Cloud • Information Technology • Security • Software • Cybersecurity
Coordinate interview logistics across video, phone, and in-person formats while delivering an excellent candidate experience. Maintain accurate ATS records, scheduling tools, dashboards, and candidate trackers. Communicate with candidates, recruiters, hiring managers, and interviewers; handle confidential information; support recruiting operations projects; identify process improvements; and collaborate with team members in a fast-paced environment.
Top Skills:
Applicant Tracking System (Ats)Candidate TrackersDashboardsScheduling Tools
Cloud • Information Technology • Security • Software • Cybersecurity
Investigate sensitive and escalated abuse, compliance, legal-request, financial-fraud, and Trust & Safety cases. Ensure timely SLA delivery, maintain stakeholder satisfaction, coordinate with Policy, Legal, and Product teams, and handle emotionally charged content tactfully. The role requires knowledge of DNS and Internet operations, collaboration with lawyers and law enforcement, and participation in rotating weekend or off-hours coverage.
Top Skills:
Dns
Cloud • Information Technology • Security • Software • Cybersecurity
Generate qualified sales opportunities and pipeline from Cloudflare’s existing PAYGO customer base. Manage a customer book of business, identify expansion opportunities, conduct outreach, explain product solutions, meet sales quotas, and track activities in Salesforce. The role requires fluent Mandarin and Cantonese, strong customer communication, technical learning ability, and collaboration with account executives, marketing, and sales operations. It offers a path toward customer success, account management, or mid-market sales.
Top Skills:
AirtableGoogle SheetsOutreachSales NavigatorSalesforceZoominfo
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
