Production-grade scraping and ingestion infrastructure
Scalable, observable, and resilient systems
Web data acquisition or scraping experience
Own and evolve systems responsible for large-scale web data collection, designing and maintaining production-grade scraping and ingestion infrastructure
Job Summary
Own and evolve systems responsible for large-scale web data collection, designing and maintaining production-grade scraping and ingestion infrastructure.
Build a scalable web data acquisition platform used across teams, enabling safer and more efficient data ingestion.
Collaborate cross-functionally to support new data sources and evolving product requirements, shaping the future of AI-driven technologies.
Matching Summary
Own and evolve systems responsible for large-scale web data collection, designing and maintaining production-grade scraping and ingestion infrastructure.
Skills & Requirements
Must-have
Production-grade scraping and ingestion infrastructure
Scalable, observable, and resilient systems
Web data acquisition or scraping experience
Networking fundamentals (TLS/SSL, timeouts)
Browser automation tools
CSS selectors and XPath
SQL and NoSQL databases
AWS production systems experience
Nice-to-have
Browser extensions experience
Browser internals exposure
Large-scale crawling experience
Key Requirements
5+ years of experience
Experience with frequent change, partial failure
Product-oriented engineering mindset
Hands-on experience with browser automation tools
Proficiency with CSS selectors and XPath
Experience with retry logic, error handling, testing, observability