Similar Jobs
Artificial Intelligence • HR Tech • Information Technology • Professional Services
Lead and scale the Data Engineering and DataOps functions. Oversee data ingestion, transformation, and operations to enable self-service analytics. Manage a hybrid team, ensuring end-to-end delivery excellence and data quality.
Top Skills:
AirbyteAirflowAWSEltETLFivetranPythonSagemakerSnowflakeSQL
Cloud • Information Technology • Software • Analytics
Lead Data Engineers design and implement data solutions, manage projects, collaborate with stakeholders, and ensure data governance and security.
Top Skills:
AWSAzureGCPPythonSparkSQL
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
Lead the Data DevOps team by developing and maintaining data infrastructure, enhancing tooling for self-service capabilities, and ensuring software quality and functionality.
Top Skills:
FlinkGitopsKafkaKubernetesSparkTerraform
As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data solution.
Your main responsibilities will include:
- Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data.
- Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data.
- Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch.
- Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions.
- Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques.
- Optimize data engineering workflows for containerized deployment and efficient resource utilization.
- Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability.
- Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements.
- Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure.
- Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference.
- Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps.
- Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank.
- Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.
- At least 10 years of experience as a Data Engineer, working with Hadoop, Spark, and data processing technologies in large-scale environments.
- Strong expertise in designing and developing data infrastructure using Hadoop, Spark, and related tools (HDFS, Hive, Ranger, etc)
- Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration using Kubernetes.
- Proficiency in programming languages commonly used in data engineering, such as Spark, Python, Scala, or Java.
- Knowledge of DevOps practices, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)
- Experience with Grafana, Prometheus, Splunk will be an added benefit
- Strong problem-solving and troubleshooting skills with a proactive approach to resolving technical challenges.
- Ability to manage multiple priorities, meet deadlines, and deliver high-quality results in a fast-paced environment.
- Experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services is a plus.
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
