The Senior Data Engineer will design and optimize data pipelines and models using Databricks, PySpark, and cloud platforms, leveraging deep data engineering expertise to enhance data processing and ensure quality delivery.
Essential Technical Skills
- Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns
- PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization
- Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration
- Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration
- Data Modelling: Experience implementing data models including dimensional modeling, data vault, or lakehouse architectures
- Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques
- Python: Strong Python programming skills for data processing and automation
Additional Technical Skills
SQL proficiency for data querying and transformation
- Experience with cloud platforms (Azure, AWS, or GCP)
- Understanding of data governance and security best practices
- Knowledge of streaming data processing (Structured Streaming)
- Familiarity with DevOps practices and CI/CD pipelines
- Experience with version control systems (Git)
- Understanding of data quality frameworks and testing methodologies
Professional Experience
- Minimum 8 years in data engineering or related roles
- At least 2-3 years of hands-on experience with Databricks platform
- Proven track record of refactoring legacy code to modern frameworks
- Experience building and maintaining production data pipelines at scale
- Background working across multiple data sources and formats
- Experience in agile development environments
Required Certifications
- Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional
Additional Certifications (Preferred)
- Databricks Certified Associate Developer for Apache Spark
- Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer)
- Relevant data engineering or big data certifications
Soft Skills
- Strong problem-solving and analytical thinking abilities
- Excellent communication skills to explain technical concepts clearly
- Ability to work collaboratively in cross-functional teams
- Self-motivated with strong attention to detail
- Adaptable to changing priorities and technologies
- Client-focused mindset with commitment to quality delivery
Top Skills
AWS
Azure
Data Engineering
Data Pipeline
Databricks
Delta Lake
Elt
ETL
GCP
Git
Pyspark
Python
SQL
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
Similar Jobs
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Manage strategic projects, support business planning and strategy, conduct analyses for growth opportunities, evaluate M&A partnerships, and enhance team engagement.
Top Skills:
BusinessEconomicsFinanceManagement
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Design and maintain automation solutions for factory operations, optimizing productivity and system efficiency by collaborating with global teams and utilizing data analysis.
Top Skills:
AngularCSSGoogle Cloud PlatformHTMLLinuxPerlPower BIPythonSQLTableau
Artificial Intelligence • Hardware • Information Technology • Machine Learning
The Senior Data Scientist will optimize production planning through AI/ML solutions, develop agentic AI systems, and mentor junior data scientists while ensuring code quality and performance.
Top Skills:
BitbucketDockerGitGoogle Cloud PlatformKubernetesPower BIPythonStreamlitTableau
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
