Senior Ai Infrastructure Engineer, Model Serving Platform

Scale

San Francisco, CA, US
Base: $216,000 - $270,000 usd; equity: included su...
On-site
5+ years backend system experience
Python go rust or c++ programming
Llm serving routing fundamentals
The role involves designing and building platforms for the scalable and reliable serving of LLMs across various environments

Job Summary

  • The role involves designing and building platforms for the scalable and reliable serving of LLMs across various environments.
  • Candidates will collaborate with researchers to integrate and optimize models for both production and research use cases.
  • Compensation includes a base salary range of $216,000 to $270,000 USD along with equity and comprehensive benefits.

Matching Summary

The role involves designing and building platforms for the scalable and reliable serving of LLMs across various environments.

Salary

Base: $216,000 - $270,000 USD; Equity: Included subject to Board approval; Benefits: Health, dental, vision, retirement, PTO, and stipends

Skills & Requirements

Must-have

  • 5+ years backend system experience
  • Python Go Rust or C++ programming
  • LLM serving routing fundamentals
  • Docker and Kubernetes orchestration
  • AWS or GCP cloud infrastructure

Nice-to-have

  • Experience with vLLM or SGLang frameworks
  • Familiarity with TensorRT-LLM
  • Knowledge of text-generation-inference
  • Infrastructure as code with Terraform

Key Requirements

  • 5+ years building large-scale backend systems
  • Strong proficiency in Python, Go, Rust, or C++
  • Experience with containerization and orchestration tools

Work Rights

Not specified

Tailored Resume

Cover Letter