ML Infrastructure Engineer

Department: Engineering

Location: San Mateo

Employment Type: FullTime

About the Role

This is a hands-on ML Infrastructure Engineer role at an early-stage enterprise AI startup, where you'll own the end-to-end inference and model-serving infrastructure that keeps production AI agents running reliably and at scale. You'll sit at the intersection of ML and platform engineering, directly shaping the systems that power real-world, high-stakes deployments in regulated industries like insurance, banking, and healthcare.

What You'll Do

• Own inference and model-serving infrastructure end to end, from design through production deployment.

• Build and scale systems that enable AI agents to run reliably and efficiently under increasing concurrency.

• Collaborate closely with ML and infrastructure teams to ensure seamless integration and performance optimization.

• Identify infrastructure bottlenecks and drive cross-functional solutions across engineering teams.

What We're Looking For

• 5+ years of experience building and operating ML inference systems, model-serving platforms, or ML infrastructure in production.

• Hands-on experience designing and scaling inference-serving infrastructure using frameworks such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom systems.

• Strong track record optimizing production ML systems for latency, throughput, and reliability at scale.

• Experience with containerization and orchestration (Docker, Kubernetes) for deploying and scaling ML workloads.

• Experience building distributed systems that handle concurrent requests and manage resource allocation under load.

• Proficiency with observability and debugging tooling for production systems (e.g., Prometheus, Grafana, ELK, distributed tracing).

• Cloud platform experience on AWS, GCP, or Azure for deploying and managing ML systems.

• Proficiency in at least one systems or backend language — Python, Go, Rust, C++, or Java.

• Nice to have: experience with knowledge graphs, semantic search, or graph databases (e.g., Neo4j, Amazon Neptune); real-time or low-latency inference systems; agentic or multi-step reasoning pipelines; enterprise data infrastructure or integration platforms.

Location

On-site in San Mateo, CA. No visa sponsorship is available for this role.