Centre for Strategic Infocomm Technologies
Jobs
Staff Platform Engineer - Big Data Platform Management
Centre for Strategic Infocomm Technologies
Staff Platform Engineer - Big Data Platform Management
Be an Early Applicant
Design, implement, and manage large-scale Big Data infrastructure, including Hadoop, Spark, data lakes, storage systems, clusters, and Kubernetes environments. Optimize performance, scheduling, scalability, security, governance, and availability. Support data engineers and scientists, manage capacity planning and infrastructure deployments, and lead engineering teams. The role also involves building streaming and batch pipelines, supporting search workloads, and deploying AI/ML infrastructure.
CSIT develops products to advance the national security interests of Singapore. We use our products in a wide range of operations, including but not limited to Counter-terrorism and Computer Network Defence. We are looking for talented Senior Big Data Platform Staff Engineer. Join the awesome CSIT family and use cutting-edge technologies to protect the nation.
We are seeking an experience Big Data Platform Staff Engineer to join our team, responsible for managing and optimizing our Big Data Infrastructure platform. The successful candidate will have a deep understanding of Big Data technologies, architectures, and systems, as well as experience with managing large-scale data processing and analytics environments. The role will involve designing, implementing, and maintaining the Big Data Infrastructure platform, ensuring high availability, scalability, and performance.
Responsibilities
- Design, implement and manage the Big Data Infrastructure platform, including Hadoop, Spark, NoSQL databases, and data lakes.
- Manage Big Data clusters, including monitoring, troubleshooting, and optimizing cluster performance, as well as managing job scheduling and resource allocation.
- Design and implement storage solutions for Big Data workloads, including HDFS, S3, and other object storage systems.
- Ensure the security and governance of the Big Data Infrastructure platform, including managing access controls, implementing data encryption, and conducting regular security audits.
- Collaborate with data engineers and data scientists to optimize Big Data application performance and provide technical support and guidance on using the Big Data Infrastructure platform.
- Plan and manage Big Data resource capacity, including forecasting, procurement, and deployment of new hardware and software.
Requirements (Minimum Qualifications)
- Background in Computer Science, Engineering, or a related field
- At least 8 years of experience in managing Big Data Platforms. Proven experience with technologies like Spark, Iceberg, Trino, Elastic Stack, NiFi, Kafka, Airflow.
- Expertise in Big Data Engineering, especially in building streaming/batch pipelines, architecting data warehouses, and deploying AI/ML infrastructure.
- Capable of managing and utilizing diverse storage protocols, including S3, NFS, SMB, and HDFS.
- Strong knowledge in securing applications through SSL/TLS encryption, data at rest encryption, and robust authentication/authorization (AuthN/AuthZ) using OIDC/OAuth2.
- Comprehensive understanding of core networking principles, including firewall configuration, DNS management, routing and subnetting.
- Experience with scripting languages, such as Bash, Python, Scala and Java.
- Proficient in Kubernetes orchestration, leveraging its ecosystem for automated scaling, self-healing, and seamless CI/CD integration.
- Experience in supporting search workloads in multi-terabytes and billions of records
- Experience with agile development methodologies and version control systems, such as Git.
- Experience in leading engineering teams.
Nice-to-haves
- Certifications in Big Data, such as Hadoop Certified Developer or Spark Certified Developer.
- Experience in managing physical and virtual infrastructures.
- knowledgeable in deploying and scaling LLM-based and agentic AI/ML applications.
Why join us?
- The work is purposeful and meaningful
- You will work with the best engineers
- We work with modern technologies and tech stacks
- We have excellent engineering culture and work-life balance
- We aspire to engineering and operational excellence
- We empower to innovate
- We grow together as a family
As CSIT is an agency under the Ministry of Defence (Singapore), only Singapore Citizens will be considered.
Centre for Strategic Infocomm Technologies Singapore Office
Similar Jobs
Fintech • Mobile • Payments • Software • Financial Services
Leads a team of up to 10 Enterprise Support Specialists serving Wise Platform business partners. Responsibilities include coaching, hiring, performance management, SLA and workflow oversight, account ownership, feedback analysis, escalation handling, cross-functional collaboration, and using partner insights to improve products, processes, and partner experience.
.png){kind=logo}
AdTech • eCommerce • Information Technology • Software • Travel • Generative AI
Leads Expedia Group’s hotel supply and commercial performance across APAC, owning revenue, margin, inventory competitiveness, and market share. Develops regional commercial strategy, negotiates complex hotel contracts, and leads a geographically dispersed leadership organization. Partners with global teams across marketing, finance, analytics, legal, operations, and air to execute growth initiatives, improve operating models, and expand hotel partnerships.
Cloud • Information Technology • Security • Software • Cybersecurity
Provides advanced technical support for enterprise customers using Cloudflare Zero Trust products. Troubleshoots identity, device security, authentication, secure application access, networking, cloud, and integration issues. Creates technical documentation, collaborates with Product, Engineering, and Security teams on escalations, supports security incident response, and communicates customer feedback to improve products.
Top Skills:
AWSAzureAzure Active DirectoryCloudflare AccessCloudflare GatewayCloudflare TeamsCloudflare Zero TrustDnsFidoFirewallsGCPIamLdapMicrosoft Sql ServerMySQLOidcOktaPing IdentityPostgresPush MfaSaaSSAMLSms MfaSsoThreat IntelligenceTotpU2FVpnWebauthn
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
