Staff Applied Ai Engineer (observability)

Komodo Health

New York, NY, United States
On-site
Production ai system experience
Llm tool and function calling expertise
Agent orchestration and prompt systems
The role focuses on building the foundation that makes AI systems trustworthy at scale within a newly formed team

Job Summary

  • The role focuses on building the foundation that makes AI systems trustworthy at scale within a newly formed team.
  • You will establish the observability foundation including logging, tracing, and evaluation pipelines to measure LLM performance and reliability.
  • This is a greenfield, backend-heavy systems role where you will define core infrastructure powering AI across the company.

Matching Summary

The role focuses on building the foundation that makes AI systems trustworthy at scale within a newly formed team.

Skills & Requirements

Must-have

  • Production AI system experience
  • LLM tool and function calling expertise
  • Agent orchestration and prompt systems
  • AI observability and evaluation systems
  • Python backend engineering skills
  • Distributed systems architecture

Nice-to-have

  • Internal observability platform building
  • Request tracing and replay systems
  • Large-scale distributed data systems
  • Healthcare data domain knowledge
  • Model monitoring and drift detection

Key Requirements

  • Strong experience building production AI systems
  • Deep expertise in LLMs and applied AI systems
  • Hands-on experience with AI observability and debugging
  • Proven ability to operate in ambiguous environments
  • Track record of owning systems end-to-end

Work Rights

Not specified

Tailored Resume

Cover Letter