Publix Serving (Civica) is looking for a Lead Engineer specializing in Big Data and PySpark to join their analytics team. The ideal candidate should have extensive experience in Python programming, PySpark, and large-scale data processing, along with strong analytical and problem-solving skills
Job Summary
The role involves designing, developing, and maintaining efficient, scalable, and reliable data pipelines using PySpark.
Candidates will collaborate with stakeholders to translate data requirements into technical specifications and optimize jobs for performance.
The ideal candidate will mentor junior developers and contribute to the continuous improvement of the team's technical capabilities.
Matching Summary
Match Score: 85
Publix Serving (Civica) is looking for a Lead Engineer specializing in Big Data and PySpark to join their analytics team. The ideal candidate should have extensive experience in Python programming, PySpark, and large-scale data processing, along with strong analytical and problem-solving skills.
Skills & Requirements
Must-have
PySpark for large-scale data processing
Python programming with object-oriented design
Apache Spark architecture expertise
Distributed file systems like HDFS or S3
Relational databases and SQL proficiency
Nice-to-have
Cloud platform experience AWS Azure GCP
Workflow orchestration tools Apache Airflow
Streaming data processing Kafka Spark Streaming
Containerization technologies Docker Kubernetes
Data warehousing concepts and modeling
Key Requirements
8-12 years of relevant experience
Bachelor's or Master's degree in Computer Science
5+ years professional Big Data software development