Senior Qa (gen Ai - Must Have) Exp: 5+ Years

Fractal

Multiple Locations
On-site
Gen ai experience
Api test automation
Llm evaluation strategies
Fractal is seeking a Senior QA professional with over 5 years of experience, particularly in API test automation and validating generative AI systems. The ideal candidate should possess strong technical skills, particularly in Python and TypeScript, and have a solid understanding of backend testing, performance testing, and CI/CD processes

Job Summary

  • Design, automate, and execute API test suites including functional, contract, performance, and resilience testing.
  • Define LLM evaluation strategies, including performance testing, golden sets, guardrails, hallucination/consistency checks, and relevance & latency KPIs.
  • Integrate test infrastructure with Azure DevOps for CI/CD, test orchestration, quality gates, and dashboards, enforcing build-time gates for API and LLM quality.

Matching Summary

Match Score: 85

Fractal is seeking a Senior QA professional with over 5 years of experience, particularly in API test automation and validating generative AI systems. The ideal candidate should possess strong technical skills, particularly in Python and TypeScript, and have a solid understanding of backend testing, performance testing, and CI/CD processes.

Skills & Requirements

Must-have

  • Gen AI experience
  • API test automation
  • LLM evaluation strategies
  • CI/CD integration
  • Python/TypeScript development

Nice-to-have

  • Azure DevOps pipelines
  • FHIR/HL7 familiarity
  • Healthcare data privacy/security
  • Frontend testing capabilities

Key Requirements

  • 5+ years of experience
  • Backend/API test automation
  • Performance testing
  • CI/CD
  • ML/LLM systems validation
  • Regulated product lifecycles

Work Rights

Not specified

Tailored Resume

Cover Letter