Staff Backend Engineer Data Acquisition

What you will do:

We are in search of a Staff Backend Engineer with a deep fascination for the inner workings of the internet and a desire to impart this passion to our team. You will play a critical role in our data extraction process, ensuring that we can extract and process billions of data points every day.

Where you will have impact:

• Drive the optimization and expansion of our crawling infrastructure, ensuring both high scalability and superior performance, while maintaining effective web scraping capabilities.
• Introduce innovative methods for enhancing our monitoring and troubleshooting processes and tools.
• Engineer scraping solutions by applying findings from our R&D.
• Actively collaborate with diverse engineering teams to address challenges and advance initiatives related to backend infrastructure.
• Serve as a subject matter expert, providing mentorship and support to team members while fostering an environment of continuous learning and development.

About our team:

Our team is at the helm of our sophisticated data extraction infrastructure. Tasked with the responsibility of acquiring pricing data from over 200 unique data sources, we process nearly 200 million requests each day. The data we collect is the lifeblood of our products. We are committed to ensuring its reliable and timely delivery, thereby powering our diverse product portfolio.

Who you are:

• Master’s or Bachelor’s degree in a STEM field
• 5+ years experience in low-level or systems programming, with proficiency in languages such as Rust, C, C++, Golang, or similar.
• You are adept at navigating through complex codebases and have the confidence to delve into and modify libraries/tools as needed, demonstrating a deep understanding and not just surface-level usage.
• Strong problem-solving and analytical skills, with the ability to identify and troubleshoot complex technical issues
• Excellent communication and collaboration skills, with the ability to work effectively in a team environment
• Fluent in English

Technologies you’ll work with:

Google Cloud, Kubernetes, Rust, Golang, NodeJS, TypeScript, Scrapy, Python, Prometheus, Grafana, Foundationdb 

 (feel free to ask us why these are the best fit for us)

We welcome:

• You have experience in the web scraping field, demonstrating a deep understanding of networking libraries and proficiency with tools such as Wireshark and Burp Suite.
• You have experience writing code for web browsers.
• You have experience reverse engineering web applications.
• You're comfortable with data analysis and manipulation.
• You're no stranger to the open-source community, with a track record of meaningful contributions

What’s in it for you:

• Flexible working environment: Work from home or at one of our global offices.
• Flexible time off: Autonomy to manage your work-life balance.
• Collaborative team: High-bar, friendly, creative, and passionate colleagues.
• Career development: Workshops, frameworks, tools, training, and processes to realize your full potential.
• Impactful work: Shape products relied on by 85,000+ users worldwide.
• Competitive compensation: Proactively maintained to value your work.
• Referral bonuses: Earn rewards for bringing in new talent.