Software Engineer, Perception, VLM Foundation

Semantics Foundation is a model platform team in Waymo Perception, and we build model platforms for other problem domain teams to use for addressing production issues. In this role, you will work on VLM for driving focusing on model pre-training, fine-tuning, and post-training, for Waymo applications.

You will report to the Technical Lead Manager of Semantics Foundation.

You will:

• Develop and deploy deep learning models, including vision language models (VLMs) and large language models (LLMs)
• Explore leveraging VLM's world knowledge and reasoning capabilities to improve various driving tasks
• Work on different phases of model development from data mining, training and eval, and deploy the models on the autonomous vehicles
• Collaborate and work in partnership with research teams across Alphabet

You have:

• BS degree in Computer Science or a similar discipline, or an equivalent amount of deep learning experience
• 4+ years of industry or research experience with deep Learning, VLM/LLMs, and/or computer vision
• 4+ years of experience with Python
• Hands-on experience with deep learning frameworks such as Jax, PyTorch, Tensorflow

We prefer:

• Experience with training large VLM/LLMs with distributed training
• Experience with model optimization
• Experience of modeling with real-world sensor data (cameras, LIDAR, radars)
• Publications on top-tier conferences like CVPR, ICCV, ECCV, ICLR, ICML, NeurIPS, ICLR, AAAI, IJCV, PAMI
• Experience with C++

#LI-hybrid