Operate and optimize large-scale LLM pre-training on 1,000+ GPU clusters using PyTorch, DeepSpeed, or Megatron-LM. Improve networking (InfiniBand/RDMA), memory management, checkpointing, and failure recovery. Manage SLURM/Kubernetes GPU clusters and apply systems engineering (C++, CUDA, Python) and 3D parallelism techniques.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
Beauty • Robotics • Design • Appliances • Manufacturing
Provides Tier 1 IT support for global associates, troubleshooting Windows and macOS endpoints, networks, identity and access, Microsoft 365, and software issues. Manages onboarding, offboarding, endpoint provisioning, and device administration using Intune, Autopilot, and Jamf. Documents resolutions, supports major incidents, escalates complex issues, and improves support processes. Requires up to 10% domestic and international travel and occasional after-hours support.
Top Skills:
Active DirectoryConfluenceiOSJAMFJira Service ManagementmacOSMicrosoft 365Microsoft Entra IdMicrosoft IntuneWifiWindowsWindows Autopilot
Information Technology
Owns the strategy, roadmap, backlog, and delivery of professional services products. Partners with technical and business stakeholders to define product vision, prioritize features, manage Agile release planning, track KPIs, conduct research, and communicate product direction to senior leaders. The role requires strong analytical, presentation, negotiation, and cross-functional collaboration skills, plus experience with professional or managed services and Agile methodologies.
Top Skills:
AgileCertinia PsaKanbanLucidchartMS OfficePowerPointSafeSalesforce CpqSalesforce CRMScrum
Information Technology
Leads strategic technology consulting engagements and advises clients on transformation, risk, and business objectives. Defines and governs enterprise integration architectures across ServiceNow and business systems, using APIs, workflows, data architecture, and security controls. Develops recommendations, roadmaps, executive presentations, and architectural standards; manages stakeholders, project governance, business development, and client relationships. Mentors consultants and designs secure, scalable, maintainable integration solutions across ServiceNow modules and enterprise technologies.
Top Skills:
AIAnsibleAPIsArtifactoryCommvaultCyberarkData ArchitectureDigicertGithub Ci/CdHashicorp VaultKubernetesPanoramaSecurity ControlsServicenowWorkflow OrchestrationXsoar
What you need to know about the Seattle Tech Scene
Home to tech titans like Microsoft and Amazon, Seattle punches far above its weight in innovation. But its surrounding mountains, sprinkled with world-famous hiking trails and climbing routes, make the city a destination for outdoorsy types as well. Established as a logging town before shifting to shipbuilding and logistics, the Emerald City is now known for its contributions to aerospace, software, biotech and cloud computing. And its status as a thriving tech ecosystem is attracting out-of-town companies looking to establish new tech and engineering hubs.
Key Facts About Seattle Tech
- Number of Tech Workers: 287,000; 13% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Amazon, Microsoft, Meta, Google
- Key Industries: Artificial intelligence, cloud computing, software, biotechnology, game development
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Madrona, Fuse, Tola, Maveron
- Research Centers and Universities: University of Washington, Seattle University, Seattle Pacific University, Allen Institute for Brain Science, Bill & Melinda Gates Foundation, Seattle Children’s Research Institute
