Salary not listedPosted 1h ago
Real job — pulled straight from Retool’s careers page · Verified September 20, 2026 · No reposts.
Job description
Retool is hiring a Software Engineer, Agent Platform — a full-time, based in San Francisco, CA role. Apply directly on Retool's careers page below.
Software Engineer, Agent Platform
Location: San Francisco
Department: Engineering
Location Type: HYBRID
Employment Type: FULL_TIME
WHY WE’RE LOOKING FOR YOU
We’re building AI-native products where model behavior is part of the product, not just an implementation detail. As LLMs become more capable, the bottleneck is no longer access to models—it’s making non-deterministic systems reliable, evaluable, and trustworthy in production.
We’re hiring AI Engineers to own that problem end to end. This is not a role for someone who simply integrates AI APIs into features. It’s for engineers who take responsibility for how probabilistic systems behave over time, how quality is measured in the presence of variance, and how capabilities improve without regressing.
If model regressions, subtle behavior drift, or edge-case failures keep you up at night—and you enjoy that kind of ownership—we want to talk.
WHAT YOU’LL DO
As an AI Engineer, you’ll own model-driven behavior in production systems, working across product, infrastructure, and evaluation layers. Your work will directly shape what users experience—and how confidently the team can ship. You might:
- Own the behavior of AI-powered features across multiple product surfaces, including quality, safety, variance, and failure modes
- Design and evolve prompting, retrieval, routing, and tool-use strategies that embrace non-determinism while bounding its downside
- Build and maintain evaluation systems that measure model performance using statistical signals, distributions, and trends—not just pass/fail tests
- Detect, diagnose, and resolve non-deterministic failures such as hallucinations, partial correctness, instruction drift, or sensitivity to context changes
- Define and implement guardrails, fallbacks, and degradation paths that keep systems useful even when models behave unexpectedly
- Partner with product and infra teams to decide when probabilistic behavior is “good enough” to ship—and when it isn’t
- Influence model selection, model behavior, and tool design to balance quality, cost, latency, and robustness for real user workflows
You’ll work across the stack (e.g., TypeScript, Node.js, React), but your leverage won’t come from code volume alone—it will come from shaping runtime behavior with precision, measurement, and intent.
WHAT THIS ROLE IS (AND IS NOT)
This role is:
- Accountable for AI behavior, not just system correctness
- Grounded in evaluation, iteration, and regression prevention under non-determinism
- Comfortable designing systems where outputs vary, confidence is probabilistic, and correctness is contextual
- Focused on shipping dependable products on top of imperfect components
This role is not:
- Adding LLM calls to existing features and moving on
- Treating models as black boxes with undefined behavior
- Shipping AI features without owning their long-term reliability, drift, or user trust
THE SKILLSET YOU’LL BRING
- 6+ years of professional engineering experience, with ownership over complex systems in production
- Demonstrated experience owning AI/LLM behavior beyond basic integration, including mitigation of variance and failure modes
- Comfort reasoning about probabilistic systems and tradeoffs (quality vs. cost, recall vs. precision, speed vs. robustness)
- Experience designing or maintaining evaluation frameworks, golden datasets, regression detection, or human-in-the-loop feedback loops
- Strong product intuition—you care deeply about what “good” looks like even when outputs are non-deterministic
- Ability to operate independently in ambiguous problem spaces and set quality standards others rely on
- Strong opinions, weakly held—you iterate quickly and adjust based on evidence and observed runtime behavior
BONUS POINTS
- Experience with RAG, agentic systems, or tool-using models in production
- Familiarity with vector databases, embeddings, or retrieval pipelines
- Exposure to fine-tuning, model routing, or post-training techniques
- Experience building shared AI infrastructure used by multiple teams
- History of mentoring engineers on designing for non-determinism and evaluation-driven development
WHO YOU’LL WORK WITH
You’ll join a small, senior team focused on advancing AI capabilities across the product. You’ll collaborate closely with product engineers, infra engineers, designers, and PMs—often acting as the final owner of AI behavior and quality before features reach users.
Your work will set standards that others build on. If you enjoy being the person teams rely on when AI behavior matters most—and certainty is never guaranteed—you’ll thrive here.
READY TO BUILD RELIABLE AI SYSTEMS?
If you’re excited to move beyond demos and take real ownership of non-deterministic behavior in production—defining quality, preventing regressions, and turning variability into a strength—we’d love to meet you.
Get Software Engineer jobs like this→
New roles from thousands of companies land hourly, straight from their careers pages. Get the freshest matches by email so you never miss one.
Email me new jobsSimilar jobs
Director, Product Manager, AI/Renter's Insurance (Remote)
$121k–$206kRemote · US-eligible
✓ From careers page· 7h ago
Frequently asked questions
What skills are required for Software Engineer, Agent Platform at Retool?
The required skills for Software Engineer, Agent Platform at Retool include: AI, LLM, TypeScript, Node.js, React, RAG, Machine Learning.
What is the seniority level for Software Engineer, Agent Platform at Retool?
Software Engineer, Agent Platform at Retool is a Senior level position.
How do I apply for Software Engineer, Agent Platform at Retool?
You can view the full description and apply for Software Engineer, Agent Platform at Retool on EchoJobs: https://echojobs.io/job/retool-software-engineer-agent-platform-yqbl0.