Design, orchestrate, and optimize large-scale LLM pre-training across 1,000+ GPUs. Implement 3D parallelism, manage GPU clusters (SLURM/Kubernetes), optimize InfiniBand/RDMA networking and memory, and automate checkpointing and failure recovery for long training runs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
.png)
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Develop and manage Square’s always-on B2B content engine across social, thought leadership, customer stories, partner content, and industry narratives. Build content calendars, formats, briefs, campaigns, and AI-assisted workflows; coordinate cross-functional teams and external partners; oversee production through launch; and analyze performance to optimize messaging, formats, and distribution.
Top Skills:
Ai ToolsLinkedInSocial AnalyticsSocial Media Platforms
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Develop and manage Square’s always-on B2B content engine across social, thought leadership, customer stories, partner content, and industry narratives. Build content calendars, formats, briefs, campaigns, and AI-assisted workflows; collaborate with marketing, sales, creative, communications, and external partners; manage production through launch; and use performance insights to optimize content and distribution.
Top Skills:
Ai ToolsLinkedInSocial Media Platforms
Cloud • Security • Software • Cybersecurity • Automation
Leads secure enterprise architecture design and GitLab Professional Services implementations across application security, cloud security, DevSecOps, identity, vulnerability management, and compliance. Guides SaaS, Self-Managed, and Dedicated deployment decisions; leads CI/CD, source control, and security-tool migrations; conducts risk assessments and threat modeling; supports pre-sales scoping and statements of work; coordinates cross-functional delivery; mentors consultants; and develops security enablement materials.
Top Skills:
AnsibleCheckmarxContainer ScanningDastDependency ScanningFedrampGdprGitguardianGitlabGitlab Ci/CdGitlab DedicatedGitlab DuoGitlab SaasGitlab Self-ManagedIso 27001JenkinsNist 800-53Owasp Top 10 For Llm ApplicationsSastSecret DetectionSnykSoc 2SonarqubeTerraformVeracode
What you need to know about the Seattle Tech Scene
Home to tech titans like Microsoft and Amazon, Seattle punches far above its weight in innovation. But its surrounding mountains, sprinkled with world-famous hiking trails and climbing routes, make the city a destination for outdoorsy types as well. Established as a logging town before shifting to shipbuilding and logistics, the Emerald City is now known for its contributions to aerospace, software, biotech and cloud computing. And its status as a thriving tech ecosystem is attracting out-of-town companies looking to establish new tech and engineering hubs.
Key Facts About Seattle Tech
- Number of Tech Workers: 287,000; 13% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Amazon, Microsoft, Meta, Google
- Key Industries: Artificial intelligence, cloud computing, software, biotechnology, game development
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Madrona, Fuse, Tola, Maveron
- Research Centers and Universities: University of Washington, Seattle University, Seattle Pacific University, Allen Institute for Brain Science, Bill & Melinda Gates Foundation, Seattle Children’s Research Institute
