Associate Director, Generative Ai Evaluation & Standards
Johnson & Johnson
Barcelona, Spain
Hybrid
Generative ai evaluation framework design
Llm quality and rag performance assessment
Scientific accuracy and regulatory compliance standards
This newly created leadership role within the Generative AI organization reports directly to the Head of Generative AI and owns evaluation and governance for all generative AI work across R&D
Job Summary
This newly created leadership role within the Generative AI organization reports directly to the Head of Generative AI and owns evaluation and governance for all generative AI work across R&D.
The incumbent will define what quality means for generative AI in a regulated pharmaceutical environment, establish benchmarks for scientific validity, and enforce standards across every platform and vendor.
The position requires building and leading a dedicated team focused on scientific rigor, independent judgment, and developing internal evaluation capabilities through training and documentation.
Matching Summary
This newly created leadership role within the Generative AI organization reports directly to the Head of Generative AI and owns evaluation and governance for all generative AI work across R&D.
Skills & Requirements
Must-have
Generative AI evaluation framework design
LLM quality and RAG performance assessment
Scientific accuracy and regulatory compliance standards
Cross-functional governance board leadership
Team building in matrixed pharmaceutical environment
Nice-to-have
Experience with FDA or EMA regulated environments
Therapeutic area expertise in oncology or neuroscience
Multi-modal AI system evaluation experience
Enterprise data standards like CDISC and FHIR
Influencing build-versus-buy decisions
Key Requirements
Advanced degree (PhD strongly preferred) in computational biology, bioinformatics, data science, computer science, AI/ML, biomedical engineering, or applied mathematics
Minimum 8 years of post-academic industry experience in pharmaceutical or biotechnology R&D
Hands-on expertise with large language models, retrieval-augmented generation, agentic frameworks, and prompt engineering
Demonstrated track record designing evaluation frameworks, scientific benchmarks, or quality standards for AI/ML systems
Strong people leadership experience including building and managing technical teams