Applied Science Manager, Sponsored Products and Brands

Amazon

New York, NY, US
Not specified; not specified; not specified
On-site
Applied science leadership experience
Machine learning engineering background
Llm-as-judge rubric design
The role involves leading a team of applied scientists and engineers to define quality standards for an agentic brand-intelligence system

Job Summary

  • The role involves leading a team of applied scientists and engineers to define quality standards for an agentic brand-intelligence system.
  • You will design evaluation frameworks including LLM-as-Judge rubrics and establish rigorous offline-to-online consistency protocols.
  • This is a business-critical greenfield initiative focused on developing responsible AI technologies to transform the advertising lifecycle.

Matching Summary

The role involves leading a team of applied scientists and engineers to define quality standards for an agentic brand-intelligence system.

Salary

Not specified; Not specified; Not specified

Skills & Requirements

Must-have

  • Applied Science leadership experience
  • Machine Learning engineering background
  • LLM-as-Judge rubric design
  • A/B testing and drift monitoring
  • Prompt optimization strategies
  • Synthetic training pair generation

Nice-to-have

  • Culture of scientific excellence
  • Customer obsession mindset
  • Agentic brand-intelligence system knowledge
  • Orchestrator-worker pattern experience
  • Multi-model ensemble judging skills

Key Requirements

  • Experience leading applied scientists and ML engineers
  • Proven track record in model evaluation frameworks
  • Expertise in continuous model learning and optimization

Work Rights

Not specified

Tailored Resume

Cover Letter