Senior Software Engineer, Eval Efficiency & Optimization

Location: Mountain View, CA, USA; San Francisco, CA, USA; New York, NY, USA

Department: Simulation (7XW)

As a Software Engineer, you will solve ill-defined problems at the intersection of statistics, machine learning, and scalable systems engineering. You will develop creative quantitative models to address eval challenges, while providing rigorous mathematical and experimental proof of the correctness and statistical efficiency of your solutions.

You will:

• Tackle ill-defined, ambiguous problems and reduce complex systems into tractable quantitative models.
• Develop statistical techniques that enable our evaluation pipelines to deliver better signals with fewer resources.
• Design and build automation to enhance the efficiency and scalability of our evaluation infrastructure.
• Collaborate with other engineers, data scientists, product managers, and the leadership team to deliver evaluation products and help make data driven decisions.
• Champion code health and best practices in a large and complex code base.

You have:

• BS in Math, Physics, or another quantitative field
• Fluency in implementing statistical, ML, and general mathematical algorithms for large datasets
• Experience adapting techniques in research papers to applied problems
• Strong self-motivation and comfort with ambiguity to navigate complex systems and pursue open-ended problems to completion
• 3-5 years of experience with
• Navigating and modifying a large code base containing a variety of languages, such as C++, Python, and SQL
• Performing statistical analyses
• Building data processing pipelines
• Writing, reviewing, and merging code following industry standards for code health and maintainability

We prefer:

• PhD in a quantitative field
• Experience programming in C++ and Python
• Experience developing, evaluating, and deploying products at scale
• Experience with Agentic AI