Senior Data Engineer (Web Scraping)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Engineer (Web Scraping) based in India.

This is a senior, hands-on opportunity to own the next generation of reliable web-data acquisition within a modern data platform. You will design, build, and operate production-grade Python scrapers and scalable ingestion pipelines that support trusted research and data products. The role focuses on improving the maturity, reliability, and scalability of web-scraping capabilities across a growing data environment. You will establish reusable patterns for extraction, scheduling, storage, monitoring, validation, and failure handling. From investigating new data sources to deploying and supporting production workloads, you will have substantial ownership over the full engineering lifecycle. The role combines deep Python engineering with cloud infrastructure, data pipelines, observability, and practical problem-solving in a remote-first environment. You will work independently while collaborating with a distributed engineering team through code reviews, documentation, and structured development workflows.