Ai Tester

Omegahires

Los Angeles, CA, United States
On-site
Ai/ml and genai output evaluation
Model behavior and failure mode analysis
Golden dataset maintenance for regression
This role partners with Product, Engineering, and SMEs to validate AI-driven products across real-world scenarios

Job Summary

  • This role partners with Product, Engineering, and SMEs to validate AI-driven products across real-world scenarios.
  • The candidate will design and execute model evaluation strategies while maintaining golden datasets for regression testing.
  • Responsibilities include monitoring post-release performance to identify drift, regressions, and user feedback trends.

Matching Summary

This role partners with Product, Engineering, and SMEs to validate AI-driven products across real-world scenarios.

Skills & Requirements

Must-have

  • AI/ML and GenAI output evaluation
  • Model behavior and failure mode analysis
  • Golden dataset maintenance for regression
  • End-to-end workflow testing
  • Responsible AI fairness validation

Nice-to-have

  • Experience testing GenAI prompts
  • Strong domain understanding
  • User-centric mindset
  • Excellent documentation skills

Key Requirements

  • 3-7+ years of QA or testing experience
  • Solid understanding of AI/ML fundamentals
  • ISTQB or AI/Data certifications preferred

Work Rights

Not specified

Tailored Resume

Cover Letter