Agent Evaluation Engineer

Comcast

Base: $142,651.46 - $213,977.19; bonus: eligible f...
Fully remote
Ai agent evaluation pipelines
Conversational ai quality metrics
Ci/cd integration for ai
Comcast is seeking an Agent Evaluation Engineer to join their innovative technology team, responsible for assessing and validating the performance of AI agents. The role involves developing evaluation frameworks, metrics, and automated systems to ensure agent accuracy and reliability

Job Summary

  • The role involves designing and developing agent evaluation pipelines across development, staging, and production environments to ensure AI agents perform consistently.
  • Candidates will define standardized evaluation metrics for conversational AI quality, including accuracy, relevance, customer experience, and safety.
  • Comcast offers a competitive base pay range of $142,651.46 - $213,977.19 along with best-in-class benefits and bonus eligibility.

Matching Summary

Match Score: 85

Comcast is seeking an Agent Evaluation Engineer to join their innovative technology team, responsible for assessing and validating the performance of AI agents. The role involves developing evaluation frameworks, metrics, and automated systems to ensure agent accuracy and reliability.

Salary

Base: $142,651.46 - $213,977.19; Bonus: Eligible for non-sales positions; Benefits: Best-in-class array of options provided

Skills & Requirements

Must-have

  • AI Agent Evaluation Pipelines
  • Conversational AI Quality Metrics
  • CI/CD Integration for AI
  • Automated Evaluation Systems
  • Production Monitoring of Agents

Nice-to-have

  • Customer Support AI Experience
  • Responsible AI Knowledge
  • Curious Mindset
  • Mentoring Junior Engineers
  • A/B Testing in Production

Key Requirements

  • Bachelor's Degree preferred
  • 5-7 Years relevant work experience
  • Experience with Large Language Models (LLMs)

Work Rights

Not specified

Tailored Resume

Cover Letter