The Senior Data Engineer will design and optimize data pipelines and models using Databricks, PySpark, and cloud platforms, leveraging deep data engineering expertise to enhance data processing and ensure quality delivery.
Essential Technical Skills
- Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns
- PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization
- Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration
- Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration
- Data Modelling: Experience implementing data models including dimensional modeling, data vault, or lakehouse architectures
- Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques
- Python: Strong Python programming skills for data processing and automation
Additional Technical Skills
SQL proficiency for data querying and transformation
- Experience with cloud platforms (Azure, AWS, or GCP)
- Understanding of data governance and security best practices
- Knowledge of streaming data processing (Structured Streaming)
- Familiarity with DevOps practices and CI/CD pipelines
- Experience with version control systems (Git)
- Understanding of data quality frameworks and testing methodologies
Professional Experience
- Minimum 8 years in data engineering or related roles
- At least 2-3 years of hands-on experience with Databricks platform
- Proven track record of refactoring legacy code to modern frameworks
- Experience building and maintaining production data pipelines at scale
- Background working across multiple data sources and formats
- Experience in agile development environments
Required Certifications
- Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional
Additional Certifications (Preferred)
- Databricks Certified Associate Developer for Apache Spark
- Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer)
- Relevant data engineering or big data certifications
Soft Skills
- Strong problem-solving and analytical thinking abilities
- Excellent communication skills to explain technical concepts clearly
- Ability to work collaboratively in cross-functional teams
- Self-motivated with strong attention to detail
- Adaptable to changing priorities and technologies
- Client-focused mindset with commitment to quality delivery
Top Skills
AWS
Azure
Data Engineering
Data Pipeline
Databricks
Delta Lake
Elt
ETL
GCP
Git
Pyspark
Python
SQL
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
Similar Jobs
Cloud • Information Technology • Security • Software • Cybersecurity
The Manager, Solutions Engineering leads a team, providing technical guidance, coaching, and collaboration across departments while ensuring customer satisfaction and achieving sales targets.
Top Skills:
Customer SupportSoftware DevelopmentSolutions Consulting
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Develop backend services for AI features, integrate generative AI technologies, ensure performance and reliability, and mentor junior engineers.
Top Skills:
AWSAzureCi/CdGCPGoJavaLangchainLlamaindexLlm ApisNoSQLPythonRestful ApisSpring BootSQL
Fintech • Mobile • Payments • Software • Financial Services
The Senior Account Manager will drive growth with existing partners, manage relationships and churn risk, and influence decision-making through data, collaborating across various teams at Wise.
Top Skills:
Account Management SoftwareFinancial ServicesPayment SolutionsSwift
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
