Design and maintain optimal data pipeline architecture to ensure efficient, scalable, and reliable data flow across the organization
Job Summary
Design and maintain optimal data pipeline architecture to ensure efficient, scalable, and reliable data flow across the organization.
Build and optimize infrastructure for extraction, transformation, and loading (ETL/ELT) of data from diverse sources using SQL, Python, distributed data processing frameworks, cloud data platforms, and cloud services.
Our benefits include Health & Wellness, Flexible Downtime, Continuous Learning, Invest in Your Future, and Family Friendly Perks.
Matching Summary
Design and maintain optimal data pipeline architecture to ensure efficient, scalable, and reliable data flow across the organization.
Skills & Requirements
Must-have
Python
SQL
Cloud data platforms
AWS, Azure, or GCP
Distributed data processing frameworks
Databricks
RESTful services
Nice-to-have
System design concepts
Object-oriented programming principles
Agile environments
Continuous learning
Data quality and consistency
Key Requirements
Minimum 5 years of experience in a data engineering role
Experience with workflow orchestration / scheduling platforms
Experience working with version control systems
Experience maintaining and developing software in production environments utilizing CI/CD platforms