The Data Engineer III will collect, manage, and convert raw data into usable information for analytics and decision-making, interpreting trends and patterns, and building algorithms and prototypes
Job Summary
The Data Engineer III will collect, manage, and convert raw data into usable information for analytics and decision-making, interpreting trends and patterns, and building algorithms and prototypes.
Responsibilities include leading the development of social media data collection pipelines, inference methods, classification methods, and visualization dashboards, collaborating with researchers on cutting-edge social computing research.
This role involves designing and managing data ETL pipelines processing over 10 million data points daily, deploying scalable microservices, and managing operational infrastructure for system reliability.
Matching Summary
The Data Engineer III will collect, manage, and convert raw data into usable information for analytics and decision-making, interpreting trends and patterns, and building algorithms and prototypes.
Salary
$85,000; Not specified; Benefits Eligible
Skills & Requirements
Must-have
Data pipelines and ETL processes
Python and Golang microservices
Kafka for real-time streaming
MongoDB NoSQL storage
Kubernetes cluster management
CI/CD pipelines with Jenkins
Prometheus and Grafana monitoring
Nice-to-have
Social computing research collaboration
Cutting-edge research opportunities
Problem-solving in data challenges
Interacting with partner organizations
Key Requirements
Master's degree in Computer Science or related field
2+ years of data engineer/software developer experience
1+ years of experience leading a data engineering team
Work Rights
Proof of legal authority to work in the United States