Hyphen Connect Limited
LLM Pre-training & Distributed Engineer (AI Infrastructure)
Be an Early Applicant
Design, run, and optimize large-scale LLM pre-training on 1,000+ GPUs. Manage distributed training with PyTorch/DeepSpeed/Megatron-LM, optimize networking and memory, and automate checkpointing and failure recovery for long runs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Plans and performs chemical and analytical experiments under supervision, establishes methods using existing instruments, validates data, and assesses analytical accuracy. Prepares technical reports and presentations, maintains laboratory safety and organization, supports laboratory administration, and collaborates across teams. Provides basic equipment maintenance and mentors junior team members in analytical and laboratory procedures.
Top Skills:
Analytical Laboratory InstrumentsChemical Analysis MethodsLaboratory EquipmentStatistical Analysis Software
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Leads food product development and process deployment projects from concept through commercialization. Plans and coordinates technical, pilot plant, and factory trials while meeting GMP and HACCP requirements. Develops consumer-led specifications, interprets analytical results, influences cross-functional decisions, manages stakeholders, and communicates project status and risks. The role supports innovation, portfolio maintenance, productivity, and cost optimization initiatives across multiple geographies.
Top Skills:
GmpHaccpMS Office
Information Technology • Software • Financial Services • Big Data Analytics
Develop modern C++ systems for systematic investing, including low-latency data processing, high-frequency trading execution, quantitative analytics libraries, and machine-learning inference tools. Collaborate with quantitative researchers to build scalable, reliable, and performant production platforms, design distributed data pipelines, evaluate system architectures, and maintain fail-safe live-market trading infrastructure.
Top Skills:
C++Distributed SystemsHigh-Frequency Trading SystemsLow-Latency SystemsMachine-Learning LibrariesReal-Time Data ProcessingStatistical Libraries
What you need to know about the Singapore Tech Scene
The digital revolution has driven a constant demand for tech professionals across industries like software development, data analytics and cybersecurity. In Singapore, one of the largest cities in Southeast Asia, the demand for tech talent is so high that the government continues to invest millions into programs designed to develop a talent pipeline directly from universities while also scaling efforts in pre-employment training and mid-career upskilling to expand and elevate its workforce.
