Associate Director, Generative Ai Evaluation & Standards
J&J FAMILY OF COMPANIES
Barcelona, Spain
Hybrid
Generative ai evaluation frameworks
Llm quality and rag performance assessment
Scientific accuracy benchmarking in regulated environments
Johnson & Johnson is seeking an Associate Director for Generative AI Evaluation & Standards to lead a newly established evaluation function in their Data Science and Digital Health team. This role focuses on developing evaluation frameworks and governance for generative AI in a regulated pharmaceutical R&D environment
Job Summary
This newly created leadership role within the Generative AI organization reports directly to the Head of Generative AI and owns evaluation and governance for all GenAI work across R&D.
The successful candidate will design evaluation frameworks spanning LLM quality, RAG performance, agent reliability, safety, and scientific accuracy specifically for a regulated pharmaceutical environment.
The role requires building and leading a team that establishes a culture of scientific rigor while defining what 'production-ready' means for GenAI solutions.
Matching Summary
Match Score: 85
Johnson & Johnson is seeking an Associate Director for Generative AI Evaluation & Standards to lead a newly established evaluation function in their Data Science and Digital Health team. This role focuses on developing evaluation frameworks and governance for generative AI in a regulated pharmaceutical R&D environment.
Skills & Requirements
Must-have
Generative AI evaluation frameworks
LLM quality and RAG performance assessment
Scientific accuracy benchmarking in regulated environments
Cross-functional governance leadership
Team building and technical management
Nice-to-have
Experience with FDA or EMA regulatory compliance
Therapeutic area expertise in oncology or neuroscience
Multi-modal AI system evaluation experience
Knowledge of CDISC and FHIR data standards
Strategic influence on build-versus-buy decisions
Key Requirements
Advanced degree (PhD strongly preferred) in computational biology, bioinformatics, data science, computer science, AI/ML, biomedical engineering, or applied mathematics
Minimum 8 years of post-academic industry experience in pharmaceutical or biotechnology R&D
Demonstrated track record designing evaluation frameworks, scientific benchmarks, or quality standards for AI/ML systems
Strong people leadership experience including managing technical teams in a matrixed organization