Sky (Comcast) is seeking an AI Agentic Test Engineer for their core automation team, focusing on building and maintaining automated evaluation frameworks for AI agents. The ideal candidate should have extensive experience in Python, web and API automation, and CI/CD environments, with a strong emphasis on quality engineering in AI applications
Job Summary
The role focuses on building the next generation of chat experiences by transitioning from human-assisted workflows to fully agentic, AI-driven systems.
Candidates will design automated evaluation loops using Large Language Models to assess agent quality, accuracy, and safety in real-time.
Comcast offers a collaborative environment where employees can bring their authentic selves while working on cutting-edge products that reach hundreds of millions of customers.
Matching Summary
Match Score: 85
Sky (Comcast) is seeking an AI Agentic Test Engineer for their core automation team, focusing on building and maintaining automated evaluation frameworks for AI agents. The ideal candidate should have extensive experience in Python, web and API automation, and CI/CD environments, with a strong emphasis on quality engineering in AI applications.
Salary
Base: $114,985.89 - $172,478.83; Bonus: Eligible for non-sales bonus plan; Benefits: Best-in-class physical, financial, and emotional support options
Skills & Requirements
Must-have
Expert-level Python coding skills
LLM-as-a-Judge evaluation frameworks
Web and API automation expertise
CI/CD pipeline integration experience
Agentic AI testing validation
Nice-to-have
TypeScript and Java cross-functional knowledge
React framework familiarity
Streamlit dashboard development
Jenkins or Concourse pipeline management
Passion for customer experience innovation
Key Requirements
5+ years as SDET or Automation Engineer
Legal authorization to work in the United States
No visa sponsorship eligibility
Work Rights
Must be legally authorized to work in the US without sponsorship