The Data Engineer will develop and maintain data pipelines using Spark and SQL, collaborating with teams to enhance data quality for analysis and processing.
We are looking for a proficient Data Engineer with strong skills in Spark and SQL to join Unison Group. The successful candidate will be responsible for designing, building, and managing data pipelines that facilitate real-time data processing. This role involves optimizing data flow and enhancing data quality to support the analytical needs of various teams within the organization.
Key Responsibilities:
- Develop and maintain robust data pipelines using Apache Spark to process large datasets efficiently.
- Utilize SQL for data querying, manipulation, and management across various relational database systems.
- Collaborate with data scientists and analytics teams to understand data requirements and deliver suitable solutions.
- Implement data transformation and aggregation processes to optimize data for analysis.
- Monitor and troubleshoot data pipelines, ensuring high availability and reliability.
- Perform data quality checks and maintain data integrity throughout the data lifecycle.
- Stay updated on industry trends and best practices in data engineering, particularly in Spark and SQL technologies.
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- Experience as a Data Engineer, with a focus on Spark and SQL.
- Good to have experience in Banking/Finance domain.
- Proficiency in Spark for distributed data processing and ability to write efficient Spark jobs.
- Strong SQL skills, with experience working with relational databases (e.g., MySQL, PostgreSQL, Oracle).
- Experience with data integration, ETL processes, and data warehousing concepts.
- Knowledge of big data technologies and frameworks is a plus.
- Strong analytical and problem-solving skills.
- Excellent communication and teamwork capabilities.
- Salary: SGD 6000 - 8000
- Great Work Culture
- Includes Standard Annual and Sick Leave
Top Skills
MySQL
Oracle
Postgres
Spark
SQL
Unison Consulting Singapore Office
1 Changi Business Park Crescent, , Plaza 8 #03-06 Tower A, Singapore, , Singapore, 486025
Unison Consulting Singapore Office
#12-00, 63 Market Street, Bank of Singapore Center, Singapore, , Singapore, 048942
Similar Jobs
Information Technology • Consulting
The Data Engineer will design, develop, and optimize big data pipelines using SQL, Python, Spark, Hive, and Hadoop, ensuring data quality and performance.
Top Skills:
SparkHadoopHivePythonSQL
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
The Customer Performance Director oversees risk management and compliance, optimizes fraud performance, engages stakeholders, and enhances Mastercard services.
Top Skills:
AutomationBusiness DevelopmentCompliance ProgramsRisk ManagementTechnology
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Lead B2B2C marketing initiatives, collaborating with partners to create effective campaigns and strategies while managing stakeholder relationships and marketing assets.
Top Skills:
Marketing
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
