This role involves designing and building end-to-end evaluation frameworks for AI platforms including LLM-based chat bots and voice bots
Job Summary
This role involves designing and building end-to-end evaluation frameworks for AI platforms including LLM-based chat bots and voice bots.
The candidate will work hands-on with Python, Java, and JavaScript/TypeScript to build production-grade test infrastructure integrated with CI/CD pipelines.
Lilly is a global healthcare leader seeking a Principal Engineer to shape engineering direction as a leader in Pharma-tech.
Matching Summary
This role involves designing and building end-to-end evaluation frameworks for AI platforms including LLM-based chat bots and voice bots.
Skills & Requirements
Must-have
10+ years Quality Engineering experience
Python Java JavaScript TypeScript proficiency
AI platform evaluation framework design
LLM chat bot and voice bot testing
Test automation using pytest Selenium Appium
CI/CD pipeline integration expertise
Performance testing with JMeter
Nice-to-have
Experience with RAGAS DeepEval LangChain
Multi-modal AI application testing
Speech recognition quality metrics knowledge
GxP HIPAA compliance testing knowledge
Open-source contribution to test tooling
Self-healing automation research
Global multi-region testing experience
Key Requirements
Bachelor's or Master's degree in Computer Science or related field
10+ years of Quality Engineering experience
Hands-on experience with AI model evaluation pipelines