Software Quality & Reliability Engineer (Founding Role)

Location: United States

Workplace: remote

Employment Type: full

Description

The stack: Node.js/Express, TypeScript, React, Postgres, Clickhouse, MongoDB, Heroku                                                                   

The product: We're building agentic software for Finance and Operations teams to automate high-volume workflows across systems—from AP and AR to payroll, inventory, and reconciliations. See www.woodrow.ai                                                                 

The way we work:                                                                                                                               

• This is a remote job—work anywhere you want, in any timezone on earth (or outside it).                                                       
• We are an asynchronous team—no scheduled meetings. We rely on each team member to pick their own schedule and communicate clearly in writing and recorded videos.                                                                                                             
• We operate completely autonomously—no one will tell you what to do, we'll tell you the goals and you'll figure out how to work towards them with the rest of the team.                                                                                                                     

The role:

We're looking for our first Quality & Reliability Engineer who thinks about quality as a system, not a phase at the end of development. 

You'll own quality and reliability across the stack—from test infrastructure to production monitoring to LLM evals. This is a founding role: you'll assess what we need, prioritize ruthlessly, and build the foundations that let us ship fast without breaking things.

This is not a traditional QA role. You may review some pull requests and help write e2e tests, but you will ALSO own the monitoring side and build guardrails that scale—all with the goal of preventing and catching issues before customers do.

This role combines elements of software engineering, traditional QA, reliability engineering, and devops.

What you'll work on:             

  CI & Test Health                                                                                                                             

• Make sure tests consistently run fast and green means green
• Maintain staging and test environments that are useful
• Help review code with a focus on reliability, edge cases, and failure handling
• Build and maintain integration and end-to-end test suites
• Close gaps in test coverage by writing tests yourself              

  AI Quality & Evals 

• Monitor AI workflow correctness and audit individual workflows to catch regressions
• Design and maintain eval suites for LLM features
• Build a repeatable process for validating prompt changes before they ship

Observability & Production Health

• Design and implement monitoring, alerting, and dashboards so we understand system health at a glance
• Own observability for our data pipelines external integrations, and AI workflows
• Make sure we know about failures before customers tell us     

Requirements

• You're an excellent engineer—you can solve hard problems, you're AI Native, and ship quickly
• Experience with observability tooling (Datadog, Sentry, or similar)
• Strong testing background (unit, integration, e2e; automation frameworks) both writing, as well as architecting
• Familiarity with LLM-based features and eval approaches (or willingness to learn quickly)
• Great communicator via writing and recorded video
• Comfortable working in ambiguous, fast-changing environments
• Excited about working on a globally-distributed, diverse team where autonomy, accountability, and ownership matter as much as your skillset

Benefits

• Unlimited vacation
• Completely flexible work schedule—work literally anytime (and anywhere) you want
• Benefits appropriate to your location (health/dental/vision in the USA)
• Company-wide retreats multiple times per year
• Market-rate compensation including benefits—we're scrappy, but not scrappy on compensation