Designworks Talent LLC Logo

Designworks Talent LLC

Senior AI Training Infrastructure Engineer

Reposted 14 Days Ago
Be an Early Applicant
Hybrid
Bellevue, WA, USA
Senior level
Hybrid
Bellevue, WA, USA
Senior level
Build and scale distributed infrastructure for large-scale AI model training across GPU clusters. Responsibilities include improving reliability, fault tolerance, checkpointing, recovery, resource utilization, training pipelines, developer tooling, and operational processes. The role partners with platform, orchestration, performance, and machine learning teams to diagnose training issues and support production-ready AI workloads.
The summary above was generated by AI
AI Training Infrastructure Engineer

Location: Hybrid | Bellevue, WA (downtown)
Titles:
Senior and Staff (multiple roles available)


Build the Training Infrastructure Powering Next-Generation AI Models
About the Opportunity

A well-funded, rapidly growing AI infrastructure company is building a next-generation cloud platform designed to power the full lifecycle of artificial intelligence. The organization is developing a comprehensive AI infrastructure, platform, and services portfolio that supports the full spectrum of AI workloads—including large-scale compute, model training, fine-tuning, inference, and emerging agentic AI applications.

Backed by significant long-term investment, the company combines the speed, ownership, and innovation of a startup with the stability and resources of an established parent organization. Engineering teams are intentionally lean, highly collaborative, and AI-native, leveraging modern tooling and automation to build infrastructure capable of supporting the industry's most demanding AI workloads.

We're seeking AI Training Infrastructure Engineers to build and scale the distributed systems that power large-scale AI model training. This team focuses on reliability, efficiency, and operational excellence across GPU clusters, enabling researchers and engineers to train and deploy advanced AI models at scale.

 
 
The Opportunity

This is a foundational engineering role focused on building the infrastructure layer behind large-scale AI training workloads. You'll work on distributed training systems, GPU clusters, model pipelines, and the tooling required to make AI development more reliable, efficient, and scalable.

You'll collaborate closely with infrastructure, orchestration, performance, and machine learning teams to solve complex challenges around distributed computing, fault tolerance, training efficiency, and production readiness.

This opportunity is ideal for engineers who enjoy building highly scalable systems and working at the intersection of AI research, infrastructure engineering, and distributed computing.

 
 
What You'll Do
  • Build and scale distributed training infrastructure supporting large AI models across large GPU clusters.

  • Design and improve systems that increase training reliability, efficiency, and resource utilization.

  • Develop solutions for fault tolerance, checkpointing, recovery, and large-scale training operations.

  • Integrate AI models into production training pipelines in partnership with platform, orchestration, and performance engineering teams.

  • Diagnose and resolve issues impacting training throughput, stability, reliability, and cost efficiency.

  • Build tools and automation that improve the developer experience for AI researchers and engineers.

  • Establish best practices for training infrastructure, operational processes, and platform reliability.

  • Contribute to the evolution of the AI infrastructure platform as an early member of the engineering team.

 
What We're Looking For
  • Hands-on experience building and operating distributed training systems or large-scale machine learning infrastructure.

  • Experience supporting large AI models, foundation models, post-training workflows, or similar ML systems.

  • Strong understanding of the reliability, scalability, and efficiency challenges associated with multi-node GPU training.

  • Experience integrating training systems with production machine learning pipelines.

  • Strong programming skills and experience working with complex distributed systems.

  • Ability to independently own technically challenging projects in a fast-moving engineering environment.

  • Comfortable operating with high ownership and limited process overhead.

 
 
Preferred Qualifications
  • Experience with distributed training frameworks such as PyTorch Distributed, DeepSpeed, Megatron-LM, Ray, or similar technologies.

  • Experience with supervised fine-tuning (SFT), reinforcement learning from human feedback (RLHF), or other post-training workflows.

  • Background operating AI training infrastructure at scale within a hyperscaler, AI research organization, cloud provider, or GPU cloud environment.

  • Experience optimizing GPU utilization, training performance, or distributed system reliability.

  • Familiarity with Kubernetes, containerized AI workloads, and large-scale infrastructure platforms.

 
Compensation
  • Competitive base pay for Bellevue market

  • Certain roles are eligible for additional rewards, including merit increases, annual bonus, and long term incentives. These awards are allocated based on individual performance

  • U.S. based employees have access to medical, dental, and vision insurance, a 401(k) plan and company match, employees also receive per calendar year, paid holidays

 
Location
  • Hybrid role based in downtown Bellevue, WA.

  • Approximately three days per week in the office.

  • Candidates elsewhere in the U.S. who are open to relocation are encouraged to apply.

  • U.S. work authorization is required. Visa sponsorship is not currently available.

 
Why Join?
  • Build the infrastructure powering the next generation of AI models and applications.

  • Work directly on distributed training systems, GPU clusters, and large-scale AI platforms.

  • Solve some of the industry's most challenging problems around AI scalability, reliability, and efficiency.

  • Join early enough to influence architecture, tooling, and engineering practices.

  • Collaborate with a highly experienced team building critical AI infrastructure from the ground up.

  • Enjoy the ownership and technical impact of a startup environment backed by significant long-term investment.

Similar Jobs

7 Minutes Ago
Remote or Hybrid
USA
25-26 Hourly
Junior
25-26 Hourly
Junior
Healthtech • Social Impact • Software
Processes and responds to medical record and release-of-information requests from patients, payors, attorneys, and other authorized parties. Reviews authorizations, subpoenas, and records for completeness, accuracy, HIPAA compliance, and state and federal privacy requirements. Maintains confidentiality, communicates with requesters and providers, tracks work in EMRs and ticketing systems, supports chart audits, and assists teams with records-related questions and process improvements.
Top Skills: Electronic Medical Records (Emr)Ticketing Software
19 Minutes Ago
Hybrid
Seattle, WA, USA
Senior level
Senior level
Financial Services
Designs, develops, and troubleshoots secure, scalable ML software systems and services. Builds production code, architecture artifacts, APIs, and distributed microservices; analyzes data to improve applications and system architecture. Uses AI-assisted development tools responsibly, validates generated outputs, and guides peers on secure engineering practices. Requires Python and ML framework expertise, cloud-native technologies, public cloud experience, distributed systems knowledge, and Cassandra or equivalent NoSQL experience. Preferred experience includes GPU workloads, model serving, distributed inference, model compression, and edge deployment.
Top Skills: AWSC++CassandraDockerGCPGpuJavaKubernetesNoSQLPythonPyTorchTensorFlowTensorflow ServingTorchserveTriton Inference Server
24 Minutes Ago
Hybrid
Bellevue, WA, USA
195K-343K Annually
Expert/Leader
195K-343K Annually
Expert/Leader
Artificial Intelligence • Cloud • Machine Learning • Mobile • Software • Virtual Reality • App development
Design, architect, and implement high-performance iOS applications for Snapchat’s camera experiences. Develop and launch stable, scalable features using Objective-C, optimize mobile performance and multithreading, conduct code reviews, evaluate technical tradeoffs, collaborate with Design, and apply AI-assisted engineering workflows while maintaining code quality, security, and architectural integrity.
Top Skills: CC++iOSMetalObjective-COpenglSwift

What you need to know about the Seattle Tech Scene

Home to tech titans like Microsoft and Amazon, Seattle punches far above its weight in innovation. But its surrounding mountains, sprinkled with world-famous hiking trails and climbing routes, make the city a destination for outdoorsy types as well. Established as a logging town before shifting to shipbuilding and logistics, the Emerald City is now known for its contributions to aerospace, software, biotech and cloud computing. And its status as a thriving tech ecosystem is attracting out-of-town companies looking to establish new tech and engineering hubs.

Key Facts About Seattle Tech

  • Number of Tech Workers: 287,000; 13% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Amazon, Microsoft, Meta, Google
  • Key Industries: Artificial intelligence, cloud computing, software, biotechnology, game development
  • Funding Landscape: $3.1 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Madrona, Fuse, Tola, Maveron
  • Research Centers and Universities: University of Washington, Seattle University, Seattle Pacific University, Allen Institute for Brain Science, Bill & Melinda Gates Foundation, Seattle Children’s Research Institute

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account