The Data Engineer IV will collect, manage, and convert raw data into usable information for analytics and decision-making
Job Summary
The Data Engineer IV will collect, manage, and convert raw data into usable information for analytics and decision-making.
Responsibilities include leading the development of social media data collection pipelines, inference methods, classification methods, and visualization dashboards.
This position requires collaboration with researchers, participation in research projects, and interaction with other developers to achieve optimal performance metrics.
Matching Summary
The Data Engineer IV will collect, manage, and convert raw data into usable information for analytics and decision-making.
Salary
$95,000; Not specified; Not specified
Skills & Requirements
Must-have
Develop and maintain datasets
Improve data quality and efficiency
Interpret trends and patterns of data
Build algorithms and prototypes
Lead social media data collection pipelines
Design and manage data ETL pipelines
Deploy scalable microservices
Nice-to-have
Contribute to cutting-edge research
Collaborate with researchers and developers
Enhance system fault tolerance
Prepare data for analysis and reporting
Key Requirements
Master's degree in Computer Science, Information Science, or related discipline with 4+ years of experience OR PhD degree with 2+ years of experience
2+ years of experience leading a team of data engineers
Expert proficiency in data models, pipelines, ETL, data stores, data mining, segmentation
Expert proficiency in programming/scripting languages (Java, Python)
Expert proficiency in SQL database design and data integration platforms
Expert proficiency in numerical, analytical, and data security skills
Expert proficiency in collecting raw data from social media platforms
Expert proficiency in CI/CD pipelines
Expert proficiency in front-end development (HTML/CSS, JavaScript, Node.js)
Expert proficiency in training and deploying machine learning models
Ability to lead a large team of data engineers (5+ members)
Expert proficiency with Kafka, MongoDB, Kubernetes
Expert proficiency with microservices, Python, Golang, FlaskAPI, GraphQL, Docker
Expert proficiency with Elasticsearch, Grafana, Prometheus, Kibana
Expert proficiency in data modeling concepts and data APIs
Expert proficiency in data processing software and algorithms
Expert proficiency in cloud platforms and data warehousing solutions
Expert proficiency in technical communications
Work Rights
Proof of legal authority to work in the United States