Senior Developer - Web Scraping

We're looking for a seasoned Python developer with deep expertise in large-scale web scraping and data extraction. You'll design and maintain robust pipelines that collect structured data from across the web — reliably, ethically, and at scale — to power our core data products. What you'll do: Architect and build distributed web scraping systems capable of handling millions of requests per day Develop robust anti-detection strategies including proxy rotation, user-agent management, and CAPTCHA handling Write parsers and extractors for HTML, JSON, and XML data using libraries like BeautifulSoup, lxml, and Parsel Monitor data pipeline health, handle failures gracefully, and maintain high data quality standards Collaborate with data science and product teams to define collection requirements and deliver clean datasets Mentor junior engineers, conduct code reviews, and champion engineering best practices across the team #J-18808-Ljbffr