Software Engineer, Developer Experience & Evals
Location: San Mateo, CA
Department: Engineering
About Us:
At Fireworks, we’re building the future of generative AI infrastructure. Our platform delivers the highest-quality models with the fastest and most scalable inference in the industry. We’ve been independently benchmarked as the leader in LLM inference speed and are driving cutting-edge innovation through projects like our own function calling and multimodal models. Fireworks is a Series C company valued at $4 billion and backed by top investors including Benchmark, Sequoia, Lightspeed, Index, and Evantic. We’re an ambitious, collaborative team of builders, founded by veterans of Meta PyTorch and Google Vertex AI.
The Role:
We are seeking a Software Engineer, DevEx & Evals to play a highly impactful role in shaping the Fireworks platform. You will be responsible for defining and building a cohesive developer journey that bridges the gap between experimentation and production.
In the AI application lifecycle, Evals and Training is a continuous loop. You will build the experiences that allow developers to seamlessly navigate this cycle: starting with serverless experimentation, moving to fine-tuning custom models, evaluating them rigorously, and finally deploying to on-demand GPUs.
You will also lead the engineering efforts on our open-source initiative and productize these capabilities to help users author evals and train custom models. As the public platform serves as the product-led growth (PLG) engine for Fireworks, your work will directly drive business impact by removing friction at every stage of the developer onboarding and adoption process.
Key Responsibilities:
- Unify the AI Lifecycle: Build a streamlined experience that connects our serverless inference, fine-tuning, and on-demand deployment products into a single, intuitive workflow.
- Streamline Developer Onboarding: Obsess over the "Time to First Token" and "Time to First Fine-tune." Identify and eliminate friction points for new developers entering the platform.
- Architect Scalable Eval Tooling: Design full-stack features that support the continuous cycle of training and evaluation, providing deep insights into model quality that directly inform the next round of fine-tuning.
- Productize Inference Optimization: Build experiences that help developers optimize their GPU deployments for their specific workloads, guiding them on the right balance of throughput, latency, and model quality.
Minimum Requirements:
- 1 - 7 years of software engineering experience (We are hiring at multiple levels for this role).
- Passion for Developer Experience: You care deeply about API design, documentation, and the ergonomics of CLI/SDK tools.
- Understanding of the GenAI Lifecycle: You understand the end-to-end workflow—from prompting a base model to curating a dataset, fine-tuning, and productionizing agents—and how these steps interconnect.
- User-Centric Mindset: Willing to talk to users, triage GitHub issues for open-source projects, and build products from scratch to serve emerging needs.
Preferred Qualifications:
- 3+ years of software engineering experience.
- Domain-Specific Evaluation Experience: Strong familiarity with designing and running evaluations for domain-specific use cases (e.g. medical, legal, coding, or custom internal datasets)
- Open Source Contributions: Prior contributions to developer tools or AI/ML repositories
- Inference & Hardware Knowledge: Interest in the hardware side of AI—understanding GPU constraints, inference optimization techniques, and how they relate to model performance.
- Startup DNA: Experience in fast-paced environments where you own features end-to-end.
Why Fireworks AI?
- Solve Hard Problems: Tackle challenges at the forefront of AI infrastructure, from low-latency inference to scalable model serving.
- Build What’s Next: Work with bleeding-edge technology that impacts how businesses and developers harness AI globally.
- Ownership & Impact: Join a fast-growing, passionate team where your work directly shapes the future of AI—no bureaucracy, just results.
- Learn from the Best: Collaborate with world-class engineers and AI researchers who thrive on curiosity and innovation.
Fireworks AI is an equal-opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all innovators.
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say