This role will define how LSEG measures the accuracy, reliability and consistency of AI-powered interactions across our data, analytics and market solutions
Job Summary
This role will define how LSEG measures the accuracy, reliability and consistency of AI-powered interactions across our data, analytics and market solutions.
You will establish the frameworks, processes and standards that evaluate LLM behaviour, MCP tool usage, and natural-language output quality across multiple AI stacks—ensuring our customers receive precise, trustworthy, and explainable results.
LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives.
Matching Summary
This role will define how LSEG measures the accuracy, reliability and consistency of AI-powered interactions across our data, analytics and market solutions.
Skills & Requirements
Must-have
AI evaluation framework ownership
Test automation and scenario design
LLM orchestration and prompt engineering
Evaluation metrics and rubric scoring
Cross-functional collaboration with product teams
Responsible AI governance alignment
Nice-to-have
Experience with Python/JS tooling ecosystems
Familiarity with financial data pipelines
Strong analytical skills for AI quality issues
Partnering with senior stakeholders
Sustainability and diversity commitment
Key Requirements
Background in product management or QA/test automation
Experience designing structured test systems at scale
Strong knowledge of LLMs and prompt engineering
Familiarity with financial data and analytics workflows
Ability to translate ambiguous tasks into evaluation rubrics