Senior Product Manager, Ai Inference - Dynamo

NVIDIA

Multiple Locations
Base: 208,000 usd - 327,750 usd; bonus/equity: equ...
Ai inference
Distributed systems
Gpu-accelerated computing
Drive the product strategy for Dynamo’s modular components, including the KV-aware Router, KV Block Manager (KVBM), and communication planes

Job Summary

  • Drive the product strategy for Dynamo’s modular components, including the KV-aware Router, KV Block Manager (KVBM), and communication planes.
  • Define requirements for sophisticated routing logic that minimizes redundant prefill and optimizes Time to First Token (TTFT) across substantial GPU clusters.
  • NVIDIA offers highly competitive salaries and a comprehensive benefits package.

Matching Summary

Drive the product strategy for Dynamo’s modular components, including the KV-aware Router, KV Block Manager (KVBM), and communication planes.

Salary

Base: 208,000 USD - 327,750 USD; Bonus/Equity: equity; Benefits: comprehensive benefits package

Skills & Requirements

Must-have

  • AI inference
  • distributed systems
  • GPU-accelerated computing
  • LLM inference lifecycle
  • KV cache mechanics
  • distributed serving techniques

Nice-to-have

  • Agentic frameworks
  • multi-turn AI applications
  • Responsible AI
  • MLOps
  • intuition for ML models

Key Requirements

  • 12+ years product management experience
  • Bachelors Degree in Computer Science
  • Proven experience in AI inference
  • Proven experience in distributed systems
  • Proven experience in GPU-accelerated computing

Work Rights

Not specified

Tailored Resume

Cover Letter