What You'll Do
- Use modern tooling to build robust, extensible, and performant web scraping platform
- Build thoughtful and reliable data acquisition and integration solutions to meet business requirements and data sourcing needs.
- Deliver best in class infrastructure solutions for flexible and repeatable applications across disparate sources.
- Troubleshoot, improve and scale existing data pipelines, models and solutions
- Build upon data engineering's CI/CD deployments, and infrastructure-as-code for provisioning AWS and 3rd party (Apify) services.
Qualifications, Skills, & Abilities
- 3+ years of work experience as a data engineer/full stack engineering, coding in Python.
- 3+ years of experience building web scraping tools in python, using Beautiful Soup, Scrapy, Selenium, or similar tooling
- 1-3 years of deployment experience with CI/CD
- Strong experience of HTML, CSS, JavaScript, and browser behavior.
- Experience with RESTful APIs and JSON/XML data formats.
- Knowledge of cloud platforms and containerization technologies (e.g., Docker, Kubernetes).
- Advanced understanding of how at least one big data processing technology works under the hood (e.g. Spark / Hadoop / HDFS / Redshift / BigQuery / Snowflake)
- Excellent analytical, problem solving, and troubleshooting skills to manage complex process and technology issues without guidance
Preferred Experience
- 2+ years of experience developing in Airflow
- 2+ deploying Infrastructure as Code within AWS or similar
- 2+ deploying microservices and/or APIs within cloud environment
- 1+ years using ML / AI workflows for data enrichment and/or sentiment analysis by integrating scraped data into ML pipelines.
Other Jobs from Brightwheel
Senior Software Engineer, Backend
Lead Salesforce Systems Administrator
Engineering Manager, Growth/Activation
Senior Software Engineer
Staff Data Engineer
Similar Jobs
Data Engineer - Sr. Consultant
Senior Software Engineer - Fullstack
Full Stack Developer
Specialist Software Engineer
Senior Machine Learning Engineer (VOIS)
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got about 70,000 jobs from 5,000 vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 5,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say