Security Engineer

We’re doing something ambitious at Fly.io: a new public cloud,running on our own hardware around the world, designed to make it easyto run distributed and real-time apps close to users anywhere in theworld.

Security is part of what we promise our users. We run containers byconverting them into micro-VMs running under Firecracker, amemory-safe, stripped-down Rust hypervisor. Apps on Fly.io areisolated from other customers with their own kernel, private networks,and WireGuard, Jason Donenfeld’s next-gen VPN, which provides thenetworking dial-tone for our entire fleet.

Security at Fly.io is part of the engineering team, and we’relooking for security engineers to join it.

Security engineering goes beyond the code we write and the productionservers we manage; it’s responsible for our corporate security and allour endpoints as well. We’re looking for engineers who are interestedin working on the full gamut of security challenges facing anintensely technical, relatively small team competing with the largestcompanies in the industry.

This is sort of a golden moment to come work with us at Fly.io. We’vehammered out our basic service and have a base of enthusiastic usersshippingsuper coolstuffonit. Our team is growing but still ata point where everyone knows everyone and what they’re workingon. Nobody’s off in a corner on a solitary death march. It’s stilleasy to have a good idea here, float it to the team, and have it takeoff. We’re having fun. For some of us, this kind of environment is whywe work in startups.

It’s not all sparkles and lollipops. Security Engineering is ano-fooling serious role. We don’t want to sell anyone a bill ofgoods. Here are some messy things we want you to know:

• We’re smaller than you’d guess given our competitors. There is somechaos. We’ve held the line on a lot of nuts-and-bolts securitystuff, but there’s lots left to do.

• We’re ruthless about working on stuff that our users will see andcare about, to the exclusion of a lot of engineeringformalism. “How will this immediately help users?” is astandard we hold ourselves to, even when it makes us uncomfortable.

• We’re on call, 24/7. Everyone shares a rotation. We’ve chosen acortisol-intensive domain to work in: when our stuff breaks, ourusers notice, and because we’re global, they notice in every timezone.

• We’re a helpful bunch, but all of us are learning stuff as we goalong and we expect you to do the same.

• We don’t care what the cool kids are using. We’re addicted to codethat works, right away, with minimal ceremony. We like SQLite, andwe get nervous when people talk about Raft. The engineering culturehere is pragmatic to what Hacker News would consider a fault.

Extrapolate all the bad implications you can from that list. Then askus about them, and we’ll be candid.

The Role

This is a mid-level job. The salary ranges from $90k to$134k USD. We also offer competitive equity grants.

We’re remote-first, with team members in Colorado, Quebec, Chicago,London, Mexico, Spain, Virginia, Brazil, and Utah. Most internalcommunication is written, and often asynchronous. You’ll want to becomfortable with not getting an immediate response for everything.

The role reports to our security practice lead, and up to engineering management.

Here’s some of what you’ll be working on in this role:

• CorpSec/endpoint security for everyone in the company. Which in ourcase means designing a security CorpSec strategy from the ground up:our processes and controls are OK for the size we’re at now, butwon’t be a year from now.

• InfraSec for our production fleet: visibility and monitoring, andtaking advantage of the platform capabilities we’ve spent the lastseveral years investing in. Our prod fleet is all Linux, we nerd outabout BPF, and we’re happy to sink resources into interestingsecurity projects.

• Software security (and software supply chain security). You’ll wantto be comfortable with the idea of picking up a PR in a language youdon’t write all the time, understanding it, and spotting flaws. Thiscould be the year we end up getting semgrep religion!

• Designing and building security features for our users. We get towrite the rulebook for this platform as we go, and there are azillion opportunities to improve the security status quo for appsthat deploy here.

Does that sound like “all of security”, like we’re lookingfor people to work in multiple different security subspecialties?Probably, we are. If that’s appealing to you, we’d like to talk.

You’ll Dig This Role If You:

• Are looking for a technical firehose for your next gig, and you’re happiest when there’s always more important stuff to learn.
• Want to work in a diverse and respectful team that values communication, glue work, and small, autonomous teams making decisions for themselves.
• Are deeply comfortable with software development and with building solutions to challenges when existing tools and libraries invariably prove to be insufficient. Our problems are idiosyncratic!
• Look forward to working on problems with immense scope and a million degrees of freedom, while relentlessly focusing on impact and building incrementally. Managing scope is here is even harder than it looks.
• Like the idea of a high-profile role working in public communicating directly with our users. We value prose writing more than most tech companies.
• Naturally ask lots of questions and can function effectively in situations where you don’t immediately have the right answer to every question.
• Believe in what we’re doing. Every company says this. We’re a startup where it’s unusually easy to explain how big the project can get to, and the risks are easy to see too.

How We Hire

We are weird about hiring. We’re skeptical of resumes and we don’ttrust interviews (we’re happy to talk, though). We respect careerexperience but we aren’t hypnotized by it, and we’re thrilled at theprospect of discovering new talent.

We hire with work-sample testing. That means we give candidateschallenges that model the work we actually do. We’ve taken the time tobuild scoring rubrics for those challenges in advance. The challengesare the whole process; they’re not a hoop you jump through before wehaze you with interviews.

For this role, we’ve got 3 at-home challenges: a relatively lightsoftware security assessment (we’ll give you working code, you give usthe dumb bugs), an osquery exercise, and a quick thought exerciseabout the security of our platform. We think these challenges shouldtake substantially less time than a series of phone screens, but we’renot timing you; knock them out in your spare time.

If the results of those challenges suggest you’ll be happy in therole, we’ll invite you onto our Slack for an hour or two to talkthrough a design exercise: we’ll be designing new security monitoringcapabilities for our production hosts.

There’s nothing up our sleeves with these challenges. If you’reinterested, we’ll tell you much more about them, including all theresources we can think of to bone up on technical subjects. We want tosee you in your best light, not surprise you with tricky questions.

If you’re interested, mail jobs+security@fly.io. Tell us a bit about yourself, if you like. We’re happy to chat, online or voice.