Data Engineer - Collision Avoidance System

In this role, you will:

• Design the data pipelines to support Zoox’s machine learning systems and data mining at scale
• Develop and maintain ETL (Extract, Transform, Load) processes for data ingestion and transformation to make data readily available for ML models training and validation.
• Maintain high quality data pipelines implementing data quality checks, ensuring data consistency and accuracy and adherence to data engineering best practices.
• Design and implement data models and data storage solutions for fast, reliable and user-friendly data querying.
• Build self-serve data dashboards for quick fact checking and ongoing reporting purposes.
• Collaborate with data scientists, software engineers, and other stakeholders to ensure that the data pipelines meet the requirements for machine learning models and make recommendations for changes or upgrades.
• Collaborate with legal, infrastructure, platform teams to develop effective solutions that aligns with data access, retention, privacy protection policies and regulations.

Qualifications

• BS/MS degree in a technical field
• Experience designing and building complex data infrastructure at scale
• Advanced Structure Query Language (SQL) and data warehousing experience
• Experience operating a workflow manager such as Airflow
• Experience with large scale streaming platforms (e.g. Kafka, Kinesis), processing frameworks (e.g. Spark, Hadoop) and storage engines (e.g. HDFS, HBase)

Bonus Qualifications

• Exceptional Python or Scala skills
• Basic fluency in C++
• Familiarity with or exposure to experimentation platforms
• A strong DataOps mindset and opinions on next-generation warehousing tools