Manage batch, scheduling & informational capability team
Define technical roadmap and backlog
Hands-on creation and maintenance of dags in airflow
Manage the technical and administrative aspects of the Batch, Scheduling & Informational Capability team, defining the technical roadmap and backlog for processing and analytics capabilities
Job Summary
Manage the technical and administrative aspects of the Batch, Scheduling & Informational Capability team, defining the technical roadmap and backlog for processing and analytics capabilities.
Design, implement, and optimize data pipelines using AWS services like EMR, Glue, and Athena, ensuring reliability, performance, and scalability, while also supporting business areas with analytical datasets and dashboards.
Explore and implement innovative use cases with AI Agents integrated into data pipelines and analytics, and promote continuous improvement, standardization, and documentation of solutions.
Matching Summary
Manage the technical and administrative aspects of the Batch, Scheduling & Informational Capability team, defining the technical roadmap and backlog for processing and analytics capabilities.
Skills & Requirements
Must-have
Manage Batch, Scheduling & Informational Capability team
Define technical roadmap and backlog
Hands-on creation and maintenance of DAGs in Airflow
Design, implement, and optimize data pipelines using EMR, Glue, and Athena
Define and monitor processing SLAs and batch windows
Support business areas in building analytical datasets and dashboards
Explore and implement use cases with AI Agents
Troubleshoot pipeline failures and data inconsistencies
Establish metrics for quality, observability, and governance
Promote continuous improvement and standardization
Nice-to-have
AWS certifications
AI tools for productivity
Modern Data Lake and Lakehouse architectures
Advanced Spark optimization
CI/CD integration for data pipelines
DataOps practices
Data governance and cataloging
Advanced English and Spanish for global interaction
Key Requirements
Solid experience in technical leadership and management of Data, Analytics, and Batch Processing teams
Advanced knowledge of workflow orchestration and scheduling with Apache Airflow
Practical experience with AWS EMR for distributed data processing
Knowledge of AWS Glue for ETL and data governance
Experience with AWS Athena for analytical queries
Solid knowledge of batch data architecture and ETL/ELT pipelines
Experience with Analytics and data visualization tools like PowerBI
Experience with AI Agents integration
Experience with S3 storage optimization
Ability to define data pipeline quality, governance, and observability standards
Experience defining and monitoring quality indicators, SLAs, SLOs, and operational metrics
Ability to act technically in pipeline implementation, architectural reviews, and code reviews