Lead Engineer Bigdata - Pyspark

Work From Home With CiCi

**
Pyspark for large-scale data processing
Python programming
Apache spark architecture
** The job posting is for a Senior Bigdata/PySpark Engineer at a remote company seeking an experienced professional with a strong background in Python and extensive knowledge of Apache Spark and PySpark. The role focuses on designing and maintaining data pipelines, optimizing performance, and collaborating with stakeholders. **

Job Summary

  • We are seeking a highly skilled and experienced Senior Bigdata/PySpark Engineer to join our dynamic Big Data Analytics team.
  • This role involves designing, developing, and optimizing robust and scalable data pipelines, working with vast datasets, and contributing to the architecture of our Big Data solutions.
  • Mentor junior developers and contribute to the continuous improvement of the team's technical capabilities and processes.

Matching Summary

Match Score: 75

** The job posting is for a Senior Bigdata/PySpark Engineer at a remote company seeking an experienced professional with a strong background in Python and extensive knowledge of Apache Spark and PySpark. The role focuses on designing and maintaining data pipelines, optimizing performance, and collaborating with stakeholders. **

Skills & Requirements

Must-have

  • PySpark for large-scale data processing
  • Python programming
  • Apache Spark architecture
  • design and develop data pipelines
  • optimize PySpark jobs
  • distributed file systems

Nice-to-have

  • cloud platforms and services
  • workflow orchestration tools
  • streaming data processing
  • data warehousing concepts
  • containerization technologies

Key Requirements

  • 8-12 years of relevant experience
  • 5+ years of professional experience in software development
  • 5+ years of hands-on experience with PySpark
  • Bachelor's or Master's degree

Work Rights

Not specified

Tailored Resume

Cover Letter