Ai Auditing Engineer (temporary) - Penn Center For Media, Technology, And Democracy - Penn Engineering At University Of Pennsylvania

University of Pennsylvania

Philadelphia, Pennsylvania, US
Base: $7.25 - $22.00 hourly rate; bonus/equity: no...
On-site
Develop and productionize auditing tools
Unify research prototypes into interoperable tools
Longitudinal monitoring of llms
The Penn Center for Media, Technology, and Democracy (Penn MEDIATED) seeks a part-time AI Auditing Engineer to develop and productionize auditing and monitoring of large language models (LLMs)

Job Summary

  • The Penn Center for Media, Technology, and Democracy (Penn MEDIATED) seeks a part-time AI Auditing Engineer to develop and productionize auditing and monitoring of large language models (LLMs).
  • The AI Auditing Engineer will unify existing research prototypes from several codebases in different academic projects to create an interoperable set of tools that is generally useful for longitudinal monitoring of LLMs.
  • This role can be fully remote or hybrid, and is a temporary role with the expected term during summer 2026 at between 20-40 hours per week.

Matching Summary

The Penn Center for Media, Technology, and Democracy (Penn MEDIATED) seeks a part-time AI Auditing Engineer to develop and productionize auditing and monitoring of large language models (LLMs).

Salary

Base: $7.25 - $22.00 Hourly Rate; Bonus/Equity: Not specified; Benefits: Health and Life, Retirement

Skills & Requirements

Must-have

  • Develop and productionize auditing tools
  • Unify research prototypes into interoperable tools
  • Longitudinal monitoring of LLMs
  • Monitor rates of refusal or soft censorship
  • Monitor political lean of LLM responses
  • Monitor sourcing information of LLM responses
  • Develop efficient and scalable data pipelines in Python

Nice-to-have

  • Interest in political outputs and impacts of LLMs
  • Experience with cloud cost engineering
  • Experience with data visualization in interactive dashboards

Key Requirements

  • Bachelor's degree in a relevant field
  • Proficiency in designing and managing NoSQL databases
  • Proficiency in working with cloud computing platforms
  • Experience working with LLM APIs
  • Experience with natural language processing methods for LLM evaluations

Work Rights

Not specified

Tailored Resume

Cover Letter