Principal Software Engineer - Ai Inference

NVIDIA

Base: $272,000 - $431,250 usd; bonus/equity: eligi...
**
15+ years production software experience
Llm inference systems expertise
Rust c++ python cuda programming
** NVIDIA is seeking a Principal Software Engineer specializing in AI Inference to contribute to open-source LLM serving, particularly with upstream inference engines like vLLM and SGLang. The ideal candidate will have extensive experience in systems engineering, GPU performance, and distributed systems, with a strong focus on optimizing inference runtime features. **

Job Summary

  • This role focuses on advancing open-source LLM serving by contributing to upstream engines like vLLM and SGLang.
  • The engineer will optimize core hot paths across the stack from Python orchestration down to CUDA kernels for high-throughput inference.
  • Candidates must have a proven track record of owning ambiguous, high-impact technical problems end-to-end with significant depth in systems engineering.

Matching Summary

Match Score: 75

** NVIDIA is seeking a Principal Software Engineer specializing in AI Inference to contribute to open-source LLM serving, particularly with upstream inference engines like vLLM and SGLang. The ideal candidate will have extensive experience in systems engineering, GPU performance, and distributed systems, with a strong focus on optimizing inference runtime features. **

Salary

Base: $272,000 - $431,250 USD; Bonus/Equity: Eligible for equity; Benefits: Comprehensive benefits package included

Skills & Requirements

Must-have

  • 15+ years production software experience
  • LLM inference systems expertise
  • Rust C++ Python CUDA programming
  • GPU performance analysis tools
  • Distributed systems concurrency knowledge

Nice-to-have

  • Open-source maintainer experience
  • Strong communication skills
  • Mentoring senior engineers
  • Building robust benchmarking infrastructure

Key Requirements

  • 15+ years building production software
  • BS/MS in Computer Science or equivalent
  • Demonstrated expertise in LLM inference/serving systems

Work Rights

Not specified

Tailored Resume

Cover Letter