Senior Ai Infrastructure Engineer, Model Serving Platform

Scale AI

San Francisco, CA, USA
$216,000—$270,000 usd py
On-site
Scalable, reliable, and efficient llm serving
Fault-tolerant, high-performance systems
Llm serving and routing fundamentals
Design and build platforms for scalable, reliable, and efficient serving of LLMs

Job Summary

  • Design and build platforms for scalable, reliable, and efficient serving of LLMs.
  • Build and maintain fault-tolerant, high-performance systems for serving LLMs workloads at scale.
  • Collaborate with researchers and engineers to integrate and optimize models for production and research use cases.

Matching Summary

Design and build platforms for scalable, reliable, and efficient serving of LLMs.

Salary

$216,000—$270,000 USD

Skills & Requirements

Must-have

  • Scalable, reliable, and efficient LLM serving
  • Fault-tolerant, high-performance systems
  • LLM serving and routing fundamentals
  • Containers and orchestration tools
  • Cloud infrastructure and IaC

Nice-to-have

  • Modern LLM serving frameworks
  • LLM capabilities and concepts

Key Requirements

  • 5+ years of experience
  • Strong programming skills
  • Experience with LLM serving
  • Experience with containers and orchestration
  • Familiarity with cloud infrastructure

Work Rights

Not specified

Tailored Resume

Cover Letter