Human Evaluation - Program Manager

Netflix

New York, United States of America
Base: $230,000.00 - $340,000.00; bonus/equity: no ...
Remote
4+ years experience in human evaluations
Genai and ml output evaluation frameworks
End-to-end program management execution
This role offers a rare opportunity to shape how Netflix measures and guides the performance of AI systems from the ground floor

Job Summary

  • This role offers a rare opportunity to shape how Netflix measures and guides the performance of AI systems from the ground floor.
  • You will be accountable for driving delivery from start to finish, including scoping, planning, rater onboarding, and calibration.
  • Netflix provides comprehensive benefits including health plans, a 401(k) retirement plan with employer match, and flexible time off.

Matching Summary

This role offers a rare opportunity to shape how Netflix measures and guides the performance of AI systems from the ground floor.

Salary

Base: $230,000.00 - $340,000.00; Bonus/Equity: No bonuses; salary vs stock options chosen annually; Benefits: Comprehensive health, 401(k), stock options, paid leave

Skills & Requirements

Must-have

  • 4+ years experience in human evaluations
  • GenAI and ML output evaluation frameworks
  • End-to-end program management execution
  • Rater onboarding and calibration workflows
  • Cross-functional collaboration with engineering

Nice-to-have

  • Experience managing human annotation vendors
  • Familiarity with responsible AI principles
  • Ability to synthesize feedback into recommendations
  • Strong organizational skills and executional focus
  • Proven ability to build foundational documentation

Key Requirements

  • 4+ years of experience in human evaluations or data operations
  • Track record of implementing process improvements for data collection
  • Prior experience managing human annotation vendors or raters
  • Strong understanding of evaluation design including rubrics and scoring protocols

Work Rights

Not specified

Tailored Resume

Cover Letter