Senior Machine Learning Engineer, Scaling and Performance

Responsibilities

• Scaling Expertise: Design and implement strategies to efficiently scale machine learning models across diverse hardware platforms (GPU/TPU).
• Performance Optimisation: Analyse and profile ML systems under heavy load, pinpointing bottlenecks, and implementing targeted optimisations.
• Distributed Systems Architecture: Create robust distributed training and inference solutions for maximum computational efficiency.
• Algorithmic Optimisation: Research and understand the latest deep learning literature to implement and optimise state-of-the-art algorithms and architectures, ensuring compute efficiency and performance.
• Low-Level Mastery: Write high-quality Python, C/C++, XLA, Pallas, Triton, and/or CUDA code to achieve performance breakthroughs.

Required Skills

• Understanding of Linux systems, performance analysis tools, and hardware optimisation techniques
• Experience with distributed training frameworks (Ray, Dask, PyTorch Lightning, etc.)
• Expertise with Python and/or C/C++
• Development with machine learning frameworks (JAX, Tensorflow, PyTorch etc.)
• Passion for profiling, identifying bottlenecks, and delivering efficient solutions.

Highly Desirable

• Track record of successfully scaling ML models.
• Experience writing custom CUDA kernels or XLA operations.
• Understanding of GPU/TPU architectures and their implications for efficient ML systems.
• Fundamentals of modern Deep Learning
• Actively following ML trends and a desire to push boundaries.

Example Projects:

• Profile algorithm traces, identifying opportunities for custom XLA operations and CUDA kernel development.
• Implement and apply SOTA architectures (MAMBA, Griffin, Hyena) to research and applied projects.
• Adapt algorithms for large-scale distributed architectures across HPC clusters.
• Employ memory-efficient techniques within models for increased parameter counts and longer context lengths.

What We Offer:

• Real-World Impact: Directly contribute to the performance and reach of our AI solutions.
• Cutting-Edge Challenges: Tackle complex problems at the forefront of machine learning and large-scale system design.
• Growth-Oriented Environment: Expand your expertise in a team of talented engineers dedicated to advancing ML scalability.