Peach Pilot — Principal Qa Engineer (ai Systems & Platform) Remote — Latin America

Peach Pilot

Argentina, Argentina
Competitive contractor rate; pyid monthly via deel...
On-site
Build and own qa function from scratch
Design llm evaluation pipelines
Test ai agent orchestration
Peach Pilot is building an enterprise AI operating system where trust is the product, aiming to close the AI trust gap

Job Summary

  • Peach Pilot is building an enterprise AI operating system where trust is the product, aiming to close the AI trust gap.
  • This role involves building and owning the QA function from scratch, focusing on AI-specific challenges like non-deterministic LLM outputs and agent orchestration.
  • You will be responsible for establishing the testing framework, defining quality standards, and ensuring the platform's reliability for enterprise users.

Matching Summary

Peach Pilot is building an enterprise AI operating system where trust is the product, aiming to close the AI trust gap.

Salary

Competitive contractor rate; Paid monthly via Deel in USD; Not specified

Skills & Requirements

Must-have

  • Build and own QA function from scratch
  • Design LLM evaluation pipelines
  • Test AI agent orchestration
  • Validate Enterprise Knowledge Graph
  • Test file ingestion pipeline
  • Ensure UX quality for non-technical users
  • Hands-on Python test coding

Nice-to-have

  • Experience with LLM evaluation frameworks
  • Experience testing agent frameworks
  • Background in enterprise software
  • Insurance industry background

Key Requirements

  • 7+ years of QA engineering experience
  • 3+ years in a lead or senior role
  • Hands-on experience testing LLM-powered applications
  • Experience building CI/CD-integrated test suites
  • Comfortable testing complex API chains
  • Built or significantly improved a QA function
  • Strong English communication skills
  • Minimum 5 hours daily overlap with US Eastern Timezone

Work Rights

Not specified

Tailored Resume

Cover Letter