Own the end-to-end architecture for data ingestion and processing pipelines, from external data sources through Airflow-orchestrated Python workflows into storage on S3 and downstream services
Job Summary
Own the end-to-end architecture for data ingestion and processing pipelines, from external data sources through Airflow-orchestrated Python workflows into storage on S3 and downstream services.
Design and evolve microservices built with NestJS/Node.js and TypeScript, exposing APIs for lead generation, vendor management, and intent resolution at scale.
Provide technical thought leadership to create and grow an agile, high-performing engineering team culture around CI/CD, trunk-based development, testing, observability, and rapid iteration.
Matching Summary
Own the end-to-end architecture for data ingestion and processing pipelines, from external data sources through Airflow-orchestrated Python workflows into storage on S3 and downstream services.
Skills & Requirements
Must-have
NestJS/Node.js and TypeScript backend systems
Python data ingestion and ETL/ELT pipelines
PostgreSQL and MongoDB database design
Auth0, OAuth2, and JWT authentication
Event-driven architectures and messaging systems
Google Cloud Pub/Sub
CI/CD, trunk-based development, observability
Nice-to-have
MCP architectures and tools development
Machine learning in data pipelines
B2B SaaS or marketplaces experience
B2B buyer intent data platforms
Key Requirements
Expert-level experience designing and building large-scale backend systems
Strong hands-on experience with Python data ingestion and ETL/ELT pipelines
Deep understanding of relational and NoSQL database design
Proven experience with secure authentication/authorization in SaaS environments
Hands-on experience with event-driven architectures
Solid understanding of cloud infrastructure (preferably GCP)