Ai Observability Engineer

AIG

Machine learning model monitoring
Llm evaluation metrics knowledge
Data quality and bias detection
This role is critical for ensuring AI solutions align with enterprise Responsible AI principles and standards within AIG's innovative Generative AI team

Job Summary

  • This role is critical for ensuring AI solutions align with enterprise Responsible AI principles and standards within AIG's innovative Generative AI team.
  • The engineer will track key performance indicators like context relevance and accuracy while monitoring data inputs for anomalies, bias, and drift.
  • AIG offers a comprehensive Total Rewards Program focused on health, wellbeing, financial security, and professional development for its global team members.

Matching Summary

This role is critical for ensuring AI solutions align with enterprise Responsible AI principles and standards within AIG's innovative Generative AI team.

Skills & Requirements

Must-have

  • Machine Learning model monitoring
  • LLM evaluation metrics knowledge
  • Data quality and bias detection
  • Responsible AI principles implementation
  • Cross-functional collaboration skills

Nice-to-have

  • Experience with Fiddler or Arize tools
  • OpenTelemetry distributed tracing experience
  • Financial services regulatory background
  • Dashboard and alert building capabilities
  • Strong ownership in ambiguous environments

Key Requirements

  • Minimum 3 years of relevant experience
  • Bachelor's degree required
  • Knowledge of ML, LLMs, and evaluation metrics
  • Experience in regulated industries preferred

Work Rights

Not specified

Tailored Resume

Cover Letter