Not specified; benefits: comprehensive health bene...
On-site
10+ years ml experience
5+ years technical leadership
Design large-scale llm evaluations
LawZero is seeking a Director of Evaluations to build a world-class, independent team responsible for rigorously assessing the safety and capabilities of their Scientist AI system
Job Summary
LawZero is seeking a Director of Evaluations to build a world-class, independent team responsible for rigorously assessing the safety and capabilities of their Scientist AI system.
The role requires defining evaluation strategies, constructing novel benchmarks, and overseeing red-teaming programs to ensure trust in AI claims both internally and externally.
Candidates will benefit from comprehensive health benefits, a minimum of 20 days vacation, and a 4% retirement savings employer contribution while working on a high-impact non-profit mission.
Matching Summary
LawZero is seeking a Director of Evaluations to build a world-class, independent team responsible for rigorously assessing the safety and capabilities of their Scientist AI system.
Salary
Not specified; Benefits: Comprehensive health benefits; Vacation: Minimum 20 days per year; Retirement: 4% employer contribution
Skills & Requirements
Must-have
10+ years ML experience
5+ years technical leadership
Design large-scale LLM evaluations
Build evaluation datasets from scratch
Lead automated and manual red-teaming
Nice-to-have
Experience with third-party partners
Open-source dataset release history
Familiarity with AI safety policy standards
Collaboration with government bodies
External safety institute engagement
Key Requirements
MSc or higher in machine learning or computer science
10+ years experience in machine learning
5+ years in a leadership role building technical teams