As a Senior Data Engineer, you will play an instrumental role in advancing our core machine learning solution. Your technical expertise and leadership skills will be leveraged to drive the development and deployment of high-quality, scalable data pipelines and products. You will work alongside scientists and engineers in a collaborative environment, contributing features and data pipelines to power our core recommender systems and our DaVinci Personalization product. This is an opportunity to work end-to-end on a large-scale machine-learning system that touches millions of customers, and a chance to continuously learn and help improve our solution as the field evolves.
This role will be reporting to the Director of Engineering (AI).
Responsibilities
- Implement production data products and pipelines that are scalable, reliable, and of high quality.
- Build, maintain, and optimize our machine learning Data Lake.
- Continuously improve data infrastructure for greater scalability.
- Support the data needs of ML Engineers and Scientists for machine learning model development.
- Release features that deliver measurable and tangible business value.
Requirements
- 4+ years of data engineering experience
- Experience with large-scale data processing frameworks (we use PySpark, SQL)
- Expertise in Spark DataFrame API
- Experience with event stream data (we use Kafka)
- Strong software development skills in Python (unit testing, git, code review, CI/CD)
- Experience with cloud computing platforms and cluster configuration, optimization, and scaling (GCP)
- Experience in data storage formats (we use Parquet, Delta Lake)
- Ability to collaborate with technical partners – you’ll be working closely with other teams to determine requirements for your work and to make design decisions that affect our stack
- Enjoys working in a fast-paced, goal-driven environment
Studies have shown that women, communities of color, and historically underrepresented people are less likely to apply to jobs unless they meet every single qualification. We are committed to building a diverse and inclusive culture where all Inkers can thrive. If you’re excited about the role but don’t meet all of the abovementioned qualifications, we encourage you to apply. Our differences bring a breadth of knowledge and perspectives that makes us collectively stronger.
We welcome and employ people regardless of race, color, gender identity or expression, religion, genetic information, parental or pregnancy status, national origin, sexual orientation, age, citizenship, marital status, ethnicity, family or marital status, physical and mental ability, political affiliation, disability, Veteran status, or other protected characteristics. We are proud to be an equal opportunity employer.
Jobs from our Partners
Sr Software QA Engineer
Senior Software Engineer, UX Engineering
Other Jobs from Movable Ink
Distributed Systems Engineer
Front End Developer L1
Front End Developer, Client Solutions
Principal Data Engineer - Data Platform & Ingress
Software Developer, Integrations
Senior Full Stack Engineer (Mobile)
Similar Jobs
Senior Application Security Engineer
Lead Software Engineer (Mobile)
Software Engineer II, Compliance
Senior Staff Software Engineer, Data and ML Platform
Principal Engineer (CloudInfra/SRE)
Sr Staff Software Engineer (Malware Research - Antivirus Systems)
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 307 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
Cancel anytime / Money-back guarantee