Senior Python Data Scraping Engineer (Freelance)

Team: IT

Location: India

Commitment: Contract

Workplace Type: remote

Accountabilities:

• Design, develop, and maintain end-to-end web scraping workflows that efficiently collect structured data from complex and dynamic websites.
• Build reliable data extraction solutions using Python and modern scraping frameworks while adapting techniques to handle JavaScript-rendered content, APIs, and evolving website structures.
• Implement scalable data collection processes through automation, batching, parallel processing, and monitoring to ensure efficient execution across large datasets.
• Validate, clean, normalize, and structure extracted data while maintaining strict quality standards and ensuring consistency across multiple data sources.
• Troubleshoot anti-bot mechanisms, optimize scraping performance, and continuously improve the reliability and resilience of extraction pipelines.
• Utilize cloud infrastructure, containerization technologies, and automation frameworks to support scalable and maintainable data processing workflows.
• Collaborate effectively within a distributed project environment, documenting processes and delivering high-quality datasets that meet project specifications.

Requirements

• Minimum of 5 years of professional experience in data engineering, web scraping, automation, software development, or a related technical field.
• Strong expertise in Python for web scraping using frameworks such as BeautifulSoup, Selenium, or similar technologies.
• Proven experience extracting data from dynamic websites, JavaScript-rendered applications, REST APIs, and complex HTML structures.
• Solid understanding of data cleaning, normalization, validation, and delivery of structured datasets in formats such as CSV, JSON, or Google Sheets.
• Experience overcoming anti-bot protections, managing proxies, and maintaining stable scraping solutions for changing website architectures.
• Hands-on experience with cloud platforms such as AWS (or equivalent), Docker, and scalable automation workflows.
• Familiarity with modern AI and LLM frameworks such as LangChain, OpenRouter, or similar technologies is highly desirable.
• Strong analytical thinking, troubleshooting skills, attention to detail, and the ability to work independently with minimal supervision.
• Bachelor's or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related technical discipline is preferred.
• Upper-intermediate (B2) English proficiency or higher is required. A GitHub portfolio showcasing relevant projects is considered an advantage.

Benefits

• Freelance, fully remote opportunity based in India.
• Flexible part-time workload of approximately 10–20 hours per week during active project phases.
• Competitive compensation of up to $30 per hour equivalent, depending on experience, contribution level, and project scope.
• Opportunity to work on innovative AI-focused projects involving advanced data extraction and automation technologies.
• Exposure to modern Python ecosystems, cloud infrastructure, containerization, and LLM-powered development tools.
• Flexible work environment with the autonomy to manage your schedule while collaborating with a global technical team.
• Opportunity to expand your expertise through challenging, real-world data engineering and web scraping projects.