The Senior Data Engineer develops and maintains scalable data infrastructure and pipelines, ensuring data quality and accessibility for data-driven decision-making.
The Senior Data Engineer plays a critica role in designing, developing, and maintaining scalable data infrastructure and pipelines. This position supports data-driven decision-making across the organization by ensuring data quality, accessibility, and performance.
Roles and Responsibilities
- Data Pipeline Development: Design, build, and maintain robust, scalable data pipelines using tools like Apache Airflow, Kafka, and Spark.
- Data Architecture: Develop and optimizedata models, data warehouses, and ETL processes to support analytics and machine learning initiatives.
- Collaboration: Work closelywith data scientists, analysts, and software engineers to understand data requirements and deliver tailored solutions.
- Data ǪualityG Governance: Ensure data integrity, implement validation checks, and enforce data governance policies.
- Performance Optimization: Monitor and improvedata processing speed,system uptime, and query performance.
- Leadership G Mentorship: Guide junior engineers, promote best practices, and foster a culture of continuous learning.
- Innovation G Research: Stay updated with emerging technologies in AI, automation, and data streaming; lead initiatives to integrate them into existing systems.
Profile
- Bachelor's or Master's in computer science, data science, information technology, or a related field, with 6+ years of experience.
- Proficiency in SǪL, Python, and ETL/ELT tools
- Experience in Microsoft solutions are as follows:
- Azure Data Factory
- Microsoft Fabric
- Azure Synapse Analytics
- Azure Databricks
- Power BI
- Azure SǪL Database/ SǪL Server
- Azure Purview
- Experience with big data technologies (Hadoop, Spark)
- Familiarity with cloud platforms(Azure, AWS, and GCP)
- Strong understanding of data modelling, data warehousing, and stream processing
- Knowledge of data security, compliance, and governance
- Knowledge of CI/CD pipelines using Azure DevOps
- Understanding of data security and compliance(e.g., GDPR, HIPAA)
Top Skills
Apache Airflow
AWS
Azure Data Factory
Azure Databricks
Azure Devops
Azure Purview
Azure Sql Database
Azure Synapse Analytics
Ci/Cd
ETL
GCP
Hadoop
Kafka
Microsoft Fabric
Power BI
Python
Spark
SQL
SQL Server
Similar Jobs
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
Oversee strategy and architecture of data and AI platforms, resolving challenges, promoting best practices, and mentoring the team.
Top Skills:
AirflowBigQueryDatabricksDataflowDataprocGCPGoogle Cloud StorageKafkaKubernetesSpark
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
Lead technical decision-making and development for core infrastructure, optimize distributed systems, and mentor engineers while working with AI-driven products.
Top Skills:
FlinkGoJavaKafkaKotlinKubernetesOlap EnginesSpark
Information Technology
As a Data Engineer, you will develop and manage infrastructure for data science and machine learning applications, engage in digital transformation for MOE, and ensure data systems meet organizational needs.
Top Skills:
Cloud TechnologyData ScienceDatabase AdministrationDevOpsETLMachine Learning
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
