This role involves analyzing business specifications and executing test plans for Large Language Models and AI agents within the financial services sector
Job Summary
This role involves analyzing business specifications and executing test plans for Large Language Models and AI agents within the financial services sector.
Candidates will maintain automated tests using Selenium with Python and manage defects through Jira in a hybrid work environment.
The position supports emerging AI initiatives, requiring strong analytical skills to ensure reliability and safety of AI-driven workflows.
Matching Summary
This role involves analyzing business specifications and executing test plans for Large Language Models and AI agents within the financial services sector.
Skills & Requirements
Must-have
3-8 years software quality assurance experience
Selenium with Python test automation
Pytest framework for automated testing
Postman or Python web service automation
Jira bug tracking and management
Nice-to-have
Experience with LLM prompt validation
Exposure to red teaming for vulnerabilities
Familiarity with OpenAI Evals or Ragas tools
Knowledge of AI governance and bias detection
Experience with BDD & TDD (Gherkin)
Key Requirements
3–8 years of software quality assurance testing experience
Hands-on experience with LLM testing and Agents testing
Basic understanding of artificial intelligence and machine learning concepts