Research Engineer - Large Behavior Models, Computer Vision

Responsibilities

• Collaborate with internal research scientists and our partner labs at top academic research universities, including MIT, Stanford, Berkeley, CMU, Columbia, and Princeton to drive pioneering research at scale.
• Build, improve, and robustify end-to-end integrated ML pipelines for training multimodal (language, images, video, actions) models at scale.
• Train, finetune, and serve robot foundation models with a strong MLOps mindset.
• Build processes for integrating collaboration-produced and open-source advancements and code into our internal stack.
• Build and improve large data pipelines for foundation model training.
• Be a key member of the team and play a critical role in rapid progress measured by both the development of internal capabilities and high-impact external publications.

Qualifications

• 2+ years of professional ML engineering experience at an AI/ML-focused organization.
• Familiarity with the state-of-the-art in behavior learning, language, and/or computer vision.
• Experience training large-scale foundation models (VLMs, text-to-video models, etc) utilizing distributed training and high-performance optimization techniques such as quantization, mixed precision, model parallelism, data parallelism or FSDP.
• Extensive practical experience with PyTorch.
• Strong proficiency in Python and software development best practices such as unit testing, documentation, code review, continuous integration, and dependency management.
• Familiarity with data pipelines, model serving and optimization, cloud training, and dataset management.
• An ability to move fast and switch between modes of rapid prototyping and robust implementation as required.

Nice to Haves

• Experience deploying models on embodied systems/robots.
• Experience working in mixed teams of research scientists and engineers.
• Experience Amazon EC2, S3, and/or Sagemaker.
• Experience with Bazel.