Machine Learning Software Engineer Intern

Machine Learning Software Engineer Intern
Hybrid (Palo Alto, CA or Toronto, Canada)

Groq's Summer Internship Program runs from May - September and is full-time during the duration of the internship.

Mission: As a member of our Multimodality team, you will play a key role in driving the launch of the latest models that process and generate audio, image, and video content. Our team operates at the intersection of Machine Learning Models, Groq's Inference Engine, and Groq's Compiler, with a mission to expand Groq's model portfolio and demonstrate the universality of our architecture and the maturity of our software stack. 

To succeed in this role, you will need to bring a deep understanding of ML model architecture, as well as the ability to recognize effective partitioning strategies and custom model-level optimizations that can unlock the potential of Groq's LPU system. Additionally, you will be responsible for ensuring state-of-the-art quality of our deployments, including verifying that quantization or other precision-affecting changes have minimal impact on evaluation scores. 

You will collaborate closely with multiple teams, including the Compiler team, the Hardware team, and the System Software team, as well as our product and strategy leadership, to ensure the successful launch of new models.

Responsibilities & opportunities in this role:

• Implement optimizations and decompositions at the PyTorch level while working with the latest machine learning models across research and industry
• Coordinate with the compiler team to compile model partitions to executables to be run on Groq's LPU systems and enable further model-specific optimizations
• Implement orchestration software to run compute pipelines on large systems of hardware, enabling blazing-fast inference
• Develop profiling tools to understand bottlenecks across the runtime stack and drive improvement based on rigorous metrics that customers and developers care about

Ideal candidates have/are:

• Strong understanding of transformer-based multimodal LLM architectures (e.g Llama, Mixtral, Pixtral) and  common machine learning operators and techniques (e.g. Attention, Cross-attention, Embeddings, Tokenization, Quantization)
• High proficiency in Python, particularly in PyTorch in the context of machine learning models
• Familiarity with huggingface and experience with quality evaluation of multimodal LLMs 
• Experience with Linux-based development environments and ability to work comfortably in a terminal-based environment
• Familiarity with Test-Driven Development principles and practices, including writing unit tests and integrating testing frameworks
• Ability to write clear, concise, and well-documented code, and to collaborate with others to ensure high-quality software development
• Knowledge of Rust and C++ is a plus, as is familiarity with MLIR, LLVM, ONNX and distributed systems (MPI)
• Must be authorized to work in the United States or Canada

Attributes of a Groqster:

• Humility - Egos are checked at the door
• Collaborative & Team Savvy - We make up the smartest person in the room, together
• Growth & Giver Mindset - Learn it all versus know it all, we share knowledge generously
• Curious & Innovative - Take a creative approach to projects, problems, and design
• Passion, Grit, & Boldness - no limit thinking, fueling informed risk taking

If this sounds like you, we’d love to hear from you!

Compensation: The US pay range for our technical internships is $30-$50 / per hour. The US pay range for our non-technical internships is $30-$40 / per hour. Individual compensation will be commensurate with the candidate’s qualifications and experience, country of internship and aligned with Groq’s internal leveling guidelines and benchmarks. #LI-DNI