Data Engineer (Big Query, Composer, Dataproc, ETL, GCP Tools)

We are seeking a skilled and motivated Data Engineer with expertise in Google Cloud Platform tools such as BigQuery, Composer, Dataproc, and ETL processes. The ideal candidate will have a strong background in designing, implementing, and optimizing data pipelines and workflows. You will play a key role in ensuring our data infrastructure is scalable, efficient, and reliable to support our data-driven decision-making processes.

Key Responsibilities:

• Design and Development:
• Develop, construct, test, and maintain scalable data pipelines using Big Query, Composer, Dataproc, and other GCP tools.
• Design and implement ETL processes to extract, transform, and load data from various sources into the data warehouse.
• Ensure the architecture supports business requirements and allows for easy reporting and analysis.
• Data Management:
• Monitor and manage data workflows and pipelines to ensure data quality, integrity, and reliability.
• Perform data profiling and analysis to identify and address data quality issues.
• Optimization and Performance:
• Optimize ETL processes and data pipeline performance to handle large-scale data efficiently.
• Implement best practices for data governance, security, and compliance.
• Collaboration
• Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Troubleshooting and Support:
• Identify, diagnose, and resolve data-related issues and discrepancies.
• Provide support and troubleshooting for data pipelines and ETL processes.

Minimum of 5 years of experience as a Data Engineer or in a similar role.

Proven experience with Google Cloud Platform, specifically Big Query, Composer, Dataproc, and GCP Tools.

Strong experience with ETL processes and tools.