San Francisco, United States
Machine Learning Infrastructure Engineer
Raydar · San Francisco · United States
- Salary
- $200,000–$350,000 per year
- Contract
- Full time
- Occupation
- DevOps / Platform Engineer
Posted: 11/14/2026 · Last checked: 11/16/2026
Job description
Quoted in full from Raydar's job posting. Gavroche does not edit it.
About the company
Our client is a software company.
The role
Raydar is recruiting for this role on behalf of our client. Build and tune the serving and training infrastructure behind specialized language models, from early experiments through to live production use. The work centers on making model execution quicker and less costly on accelerators while keeping releases dependable and observable.
What you'll do
- Lead end-to-end care of the systems that run and train custom language models, from early testing through live traffic.
- Benchmark and raise response speed, token throughput, hardware utilization and cost under production-like loads.
- Refine request grouping, caching, memory footprint and model execution, and test approaches such as reduced-precision formats and accelerated decoding.
- Keep model inference fast for evaluation runs, synthetic data generation and reinforcement learning sampling, and build dependable release and monitoring pipelines.
- Work alongside leadership and researchers to split accelerator capacity between training and serving, and to roll post-trained models out into client environments.
Requirements
What we're looking for
- Proven record building or tuning language model serving software in a live company setting, with clear credit for performance or stability gains.
- Solid low-level engineering and troubleshooting ability, including familiarity with GPU memory and compute ceilings.
- Deep familiarity with hardware chokepoints, serving tuning and observability for model workloads, including how output quality trades against speed.
- Hands-on work with a mainstream LLM serving framework or a comparable in-house engine, written in PyTorch or JAX.
- Experience owning an infrastructure system end to end, ideally built from scratch on a small team or at an AI-focused startup.
- Skill in balancing output accuracy against response time, capacity and expense when choosing pragmatic approaches.
- Willingness to move an idea from proof of concept to a live service within a compact group.
Bonus points
- Low-level kernel programming, reduced-precision numerics, accelerated decoding, or running many models across several machines.
- Background at an AI-focused company that trains its own models.
- A technical degree from a highly regarded program.
- Published research or public writing about systems work.
Benefits
Compensation and benefits
- Base salary: USD 200,000 to 350,000 per year
- Equity
Location and work model
- Bay Area, California, United States
- On-site, 5 days a week in person
- Full-time
From Gavroche data
Figures computed from live adverts on Gavroche; the job text above is the employer's.
Similar roles in San Francisco
- Member of Technical Staff - ML Platform Engineer · Preference Model, San Francisco
- Member of Technical Staff - ML Infrastructure Engineer - RL Post-training · Preference Model, San Francisco
- Senior DevOps Engineer · Treasury Prime, San Francisco
- 2027 Internship Onboard Infrastructure Engineer - ML Inference · Bedrock Robotics Inc, San Francisco
- Cloud Infrastructure Engineer · 42dot, San Francisco
- Staff Software Engineer - Site Reliability Engineer · Harvey, San Francisco
200 open roles like this one; at most three per employer shown.
What devops / platform engineer adverts in San Francisco state
Middle half: 196,250 – 260,112 USD a year; median 221,250 USD.
From 30 adverts by 21 employers that state a salary, annualised.
How devops / platform engineer roles in the United States are worked
1% fully remote · 2% hybrid · 97% do not say
Of 176 adverts. An advert that does not state a work mode is counted as not saying, not as on-site.
New openings in San Francisco, by week
- 8/24/2026 323
- 8/31/2026 415
- 9/7/2026 365
- 9/14/2026 446
- 9/21/2026 515
- 9/28/2026 623
- 10/5/2026 574
- 10/12/2026 683
- 10/19/2026 562
- 10/26/2026 562
- 11/2/2026 602
- 11/9/2026 554
- 11/16/2026 44
6,268 openings over 12 weeks, by the week they were published.
Other roles at Raydar
- Senior Software Engineer · San Francisco
- Labor and Employment Attorney · Concord
- Deployment Strategist · New York City
- Commercial Real Estate Lawyer · Concord
- Research Engineer · Mountain View
- Sales Development Representative · San Francisco
765 open, including roles outside the United States.