Senior Ai Quality Engineer

Roofr

Remote
Remote
Llm eval framework development
Ci/cd pipeline quality integration
Human-in-the-loop ai review
Roofr is building the application foundation that will define how AI is integrated across their entire product and needs a Senior AI Quality Engineer to ensure it works effectively

Job Summary

  • Roofr is building the application foundation that will define how AI is integrated across their entire product and needs a Senior AI Quality Engineer to ensure it works effectively.
  • The role involves owning evaluation frameworks, testing standards, and quality gates for AI, working horizontally across teams to embed best practices and raise quality standards.
  • Roofr offers generous vacation policies, employer-paid benefits, parental leave, remote work culture, and learning and development opportunities.

Matching Summary

Roofr is building the application foundation that will define how AI is integrated across their entire product and needs a Senior AI Quality Engineer to ensure it works effectively.

Skills & Requirements

Must-have

  • LLM eval framework development
  • CI/CD pipeline quality integration
  • Human-in-the-loop AI review
  • AI Platform team collaboration
  • Automated AI output evaluation
  • Software engineering fundamentals

Nice-to-have

  • Early-stage startup environment
  • Cross-team coaching and training
  • Ownership mentality
  • AI quality landscape awareness
  • Remote first culture

Key Requirements

  • 5–8 years software engineering or QA experience
  • Experience building eval frameworks for LLM features
  • Experience integrating automated quality checks into CI/CD
  • Familiarity with LLM APIs and agent frameworks
  • Experience designing human review workflows

Work Rights

Not specified

Tailored Resume

Cover Letter