Staff Software Engineer, Evaluation Quality

The ability to rapidly develop workflows to evaluate safety, compliance, and service quality in new cities, novel situations, and issues identified in the field is critical to scaling Waymo’s business and is one of the top priorities in the company. The Behavior Sets team is focused on accelerating the development of driving sets and evaluation workflows used to evaluate the Waymo Driver. We develop ML-based tooling for data mining, identification of interesting behaviors, sampling, and optimization for evaluation workflows based on simulations of logged and synthetically generated driving scenarios. 

In this hybrid role, you will report to our Senior Manager of Evaluation Engineering.

In this Software Engineering role, you will:

• Understand and refine product requirements for tools and measurements to understand and control the quality of Waymo’s driving evaluation tools (aka “eval of eval”)
• Translate product requirements into engineering and data science tasks, and execute on these as the primary individual contributor
• Critically examine and improve the data and statistical methods used to evaluate our eval tools
• Extend and improve our underlying infrastructure to meet product needs and ensure consistency across our various tools
• Work with stakeholders on integrating eval of eval tools into workflows and processes
• Lead coworkers on various teams to execute on high priority projects 

You have:

• MSc level statistics or equivalent work experience
• 5+ years industry experience with 
• Data processing pipelines
• SQL and databases
• C++ and/or Python
• Data visualization methods

We prefer:

• Experience developing evaluation tools, metrics, statistical algorithms, and related data
• Experience iterating on product vision together with Product Managers and leadership