Inference Stack Engineer

Inference Stack Engineer

(AI Systems / Compiler & Runtime)

We are building a next-generation AI inference stack designed for high-performance execution on modern and custom compute architectures. Our mission is to deliver industry-leading low-latency and high-throughput AI systems by designing and optimizing the full execution path — from model representation to hardware-level execution.

This is a deeply technical role at the intersection of compiler systems, AI runtimes, and high-performance computing.

You will work on core infrastructure that defines how modern AI models are executed efficiently at scale.

What you will do

• Design and build components of an AI inference stack, from high-level model representation to low-level execution

• Develop and extend a Python-based DSL for expressing AI workloads and kernels

• Work on compiler infrastructure including:

  • IR design and transformation pipelines

  • graph lowering and optimization passes

  • backend code generation for target execution environments

• Optimize model execution for:

  • latency

  • throughput

  • memory efficiency

  • numerical stability

• Contribute to runtime systems responsible for model execution and scheduling

• Profile and analyze inference workloads to identify system bottlenecks

• Collaborate closely with hardware and systems engineers on execution efficiency

• Influence architecture decisions for next-generation AI execution platforms

What we are looking for

• Strong software engineering background (C++ and Python)

• Experience with performance-critical systems or compiler-related work

• Understanding of AI model execution (especially transformers / LLMs)

• Familiarity with compute graphs, tensor operations, or execution frameworks

• Ability to analyze complex systems end-to-end (model → runtime → hardware)

• Experience working with large codebases and system-level debugging

• Strong communication skills and ability to work in cross-functional teams

Nice to have

• Experience with compiler frameworks such as:

  • LLVM

  • MLIR

  • Triton

  • TVM

  • XLA

• Experience contributing to deep learning frameworks (PyTorch, TensorFlow, JAX)

• Understanding of GPU or accelerator execution models

• Experience with kernel optimization or operator-level performance tuning

• Knowledge of distributed inference systems (e.g. NCCL, RPC-based serving)

• Familiarity with hardware-aware optimizations (memory hierarchy, vectorization, scheduling)

What we offer

• Work on the core execution layer of modern AI systems

• Direct impact on inference performance of large-scale AI workloads

• Collaboration with experts in compilers, systems, and AI infrastructure

• Highly technical environment with strong engineering autonomy

• Opportunity to shape the architecture of a next-generation inference stack

• Competitive compensation and flexible working model

Why this role is different

This is not a typical ML engineering or application role.

You will not be training models.

You will be working on how models actually run efficiently, at scale, across compute systems, shaping the performance layer that sits between AI models and hardware.