Senior Pyspark Data Engineer - Assistant Vice President

Work From Home With CiCi

Pyspark data pipelines
Apache airflow workflows
Big data ecosystems
Design, develop, and maintain robust, scalable, and high-performance data pipelines using PySpark

Job Summary

  • Design, develop, and maintain robust, scalable, and high-performance data pipelines using PySpark.
  • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality data solutions.
  • Mentor junior data engineers and promote best practices in data engineering.

Matching Summary

Design, develop, and maintain robust, scalable, and high-performance data pipelines using PySpark.

Skills & Requirements

Must-have

  • PySpark data pipelines
  • Apache Airflow workflows
  • Big Data ecosystems
  • distributed query engines
  • SQL and databases
  • Linux/Unix environment
  • GIT HUB, CI/CD Pipeline

Nice-to-have

  • mentor junior engineers
  • data governance and security

Key Requirements

  • 6+ years of professional experience
  • Bachelor’s or Master’s degree
  • PySpark and advanced Python
  • Cloudera and/or DataBricks experience
  • Starburst (Trino/Presto) experience
  • Apache Airflow proficiency
  • Data warehousing concepts
  • ETL/ELT processes
  • Data modeling techniques

Work Rights

Not specified

Tailored Resume

Cover Letter