Evals Engineer, Applied Ai

Scale AI

San Francisco, CA, US
Base: $179,400—$224,250 usd; equity: subject to bo...
On-site
Large language models (llms)
Generative ai evaluation
Ai evaluation methodologies
This high-impact role is critical to our mission of delivering the industry's leading GenAI Evaluation Suite

Job Summary

  • This high-impact role is critical to our mission of delivering the industry's leading GenAI Evaluation Suite.
  • Design, research, and develop LLM-as-a-Judge autorater frameworks and AI-assisted evaluation systems.
  • Compensation packages at Scale for eligible roles include base salary, equity, and benefits.

Matching Summary

This high-impact role is critical to our mission of delivering the industry's leading GenAI Evaluation Suite.

Salary

Base: $179,400—$224,250 USD; Equity: subject to Board of Director approval; Benefits: Comprehensive health, dental and vision coverage, retirement benefits, learning and development stipend, generous PTO, commuter stipend

Skills & Requirements

Must-have

  • Large Language Models (LLMs)
  • Generative AI evaluation
  • AI evaluation methodologies
  • Python and ML frameworks
  • data-driven assessment methods

Nice-to-have

  • integrating novel research ideas
  • dynamic, fast-paced research environment
  • thinking outside the box
  • collaboration with enterprise customers

Key Requirements

  • Bachelor’s degree in CS or related field
  • 2+ years ML or Applied Research experience
  • Hands-on LLM and Generative AI experience
  • Strong understanding of frontier model evaluation
  • Proficiency in Python and major ML frameworks

Work Rights

Not specified

Tailored Resume

Cover Letter