Senior Backend Engineer, Inference Platform

Together AI

San Francisco, California, United States
Base: $160,000 - $250,000; equity: startup equity ...
On-site
5+ years building large-scale distributed systems
Expert-level programming in rust, go, python, or typescript
Deep understanding of low-level os concepts and networking
The role involves shaping the core inference backbone that powers frontier models while solving performance-critical challenges in global request routing

Job Summary

  • The role involves shaping the core inference backbone that powers frontier models while solving performance-critical challenges in global request routing.
  • Candidates will work hands-on with tens of thousands of GPUs to fully utilize every FLOP and optimize latency down to the last millisecond.
  • The position offers competitive compensation, startup equity, health insurance, and a culture of deep technical ownership where work makes models faster and cheaper.

Matching Summary

The role involves shaping the core inference backbone that powers frontier models while solving performance-critical challenges in global request routing.

Salary

Base: $160,000 - $250,000; Equity: Startup equity included; Benefits: Health insurance and other competitive benefits

Skills & Requirements

Must-have

  • 5+ years building large-scale distributed systems
  • Expert-level programming in Rust, Go, Python, or TypeScript
  • Deep understanding of low-level OS concepts and networking

Nice-to-have

  • Experience with open source inference projects like vLLM
  • Familiarity with GPU software stacks including CUDA and Triton
  • Knowledge of modern LLMs and generative model serving

Key Requirements

  • Bachelor's or Master's degree in Computer Science or equivalent experience
  • Strong background in designing fault-tolerant API microservices
  • Experience with Kubernetes or container orchestration is a strong plus

Work Rights

Not specified

Tailored Resume

Cover Letter