Staff Software Engineer (TLM), Multiverse

Location: Mountain View, CA, USA

Department: Simulation (7XW)

The RO Performance Team is accountable for the largest metrics system deployment in the AV industry. The team manages data pipelines that monitor Waymo’s AV fleet in the real world at various latencies: from minutes to hours; supporting critical business decisions for deployments in new markets and identifying emergent business critical challenges. In addition, the same pipeline technologies are deployed for the virtual world to measure key outcomes from Waymo’s massive scale simulations, generating insights to support new Driver releases. Important objectives for the next couple of years include:

• Reliably scaling as the business scales, Waymo is in its fast expansion phase and systems have to be built to support 100x growth.
• Timeliness: As Waymo’s deployments accelerate across the globe, time to first detection needs to consistently improve.
• Comprehensive coverage: Expanding into new weather conditions and new markets brings new challenges. Increasing the metric and detection offerings to meet these challenges is key to achieving our business goals.

The team works closely with many other teams such as: metrics development teams including large AI deployments, System engineers and Data scientists that produce definitions and quality controls, infrastructure and UI teams that support underlying capabilities, product teams to stay sensitive to changing business needs, and site reliability engineering (SRE) teams to maintain high system uptime. 

You Will:

• As TLM of the Event Insights sub-team, you will start with a small team (3-5) to work on key initiatives revolving around the enrichment, understanding, and management of metric events at low latency. Once metric events, e.g., a hard brake occurred, are minted from heavy logs processing pipelines, an event lifecycle emerges where such events may get sent to other systems for further refinement, including but not limited to: human triage, VLM inference, clustering. Key workstreams that the TLM will be accountable for include:
• Clustering: Lead the creation of a dynamic event clustering product to produce event clusters at low latency for rapid understanding of emerging issues during new deployments of the Waymo Driver in simulation or in the real world.
• Storage & APIs: Events need to be stored and queried by different downstream systems. Consumption APIs include pull or push based paradigms. Labels received from human triage or VLMs update existing events. The TLM is accountable for storage that scales to meet all these requirements.
• Low Latency Orchestration: Gathering further event insights requires interaction with other systems at Waymo such as for human triage or large VLM inference. Orchestrating these interactions reliably and at low latency is key for meeting Waymo’s challenges.
• Visualization: Work closely with partner UI teams to ensure that event insights can be delivered to end consumers and provide value for understanding the Waymo Driver’s performance.

You have:

• 7+ years of full-time software engineering experience, or a quantitative PhD with 2+ years of professional software engineering experience
  • C++ proficiency
  • Python familiarity
  • SQL familiarity
• Strong verbal and written communication skills
• Excited about autonomous driving, Sim+Eval, eager to learn

We prefer:

• 10+ years of industry experience, including 2+ in a quantitative- or quality- focused engineering role in which you had experience as a technical lead for a team in which you were performing tasks like developing hypotheses, designing and running experiments, processing data from experiments, synthesizing conclusions, and ensuring the long term stability and health of the production system.
• B.S. in Computer Science
• Experience working with large FAANG scale distributed systems.
• Experience building software reliability evaluation systems