Software Engineer - Distributed Storage

We’re looking for software engineers to join our Event Platform Storage team. Datadog gathers telemetry data from millions of hosts, adding up to trillions of data points we must ingest and process each day in order to make them observable in real time for our 20,000+ customers. 

Our Event Platform ingests, transforms and stores events to provide more than 30 Datadog products data retrieved by a query API at a rate of ~15 million messages/second. The goal is to let our product engineers focus on building products while the Event Platform handles everything else, with a high-level of reliability. 

The Storage team within Event Platform built and owns Husky, a distributed, time-series oriented, columnar store optimized for streaming ingestion and hybrid analytical and search queries. Husky’s technology decouples storage and compute, enabling teams to store and query massive volumes of logs for long periods of time at very low costs. 

We have challenges to continue scaling these components as well as adding new features that are important for our customers. This is an opportunity to accelerate the growth of one of the most profitable parts of the business. With a focus on reliability, you will contribute to a platform using Java, Go and Rust. 

This is a unique opportunity to contribute to one of the most critical platforms at Datadog. This platform provides a scalable, exactly-once, cost-effective, and reliable storage engine for timestamped payloads with a full-featured and fast query engine with multiple integration options. Engineers with a background in the challenges of optimizing distributed systems for durability, high availability, and low latency at internet-scale are encouraged to apply. 

Learn more about Husky: 

1. Datadog Engineering Blog: Introducing Husky
2. Datadog Engineering Blog: Husky Deep Dive
3. YouTube: Datadog on Building an Event Storage System
4. YouTube: Changing a Tire on a Moving Car: Migrating to Datadog’s New Event Store without Downtime

What You’ll Do:

• Build fault-tolerant, horizontally scalable solutions running in multi-tenant environments
• Write in Go, Java Rust or C++, amongst other languages
• Use Kafka, Redis, Cassandra, Elasticsearch and other open-source components
• Own meaningful parts of our service, have an impact, grow with the company

Who You Are:

• You have a BS/MS/PhD in a scientific field or equivalent experience
• You have significant backend programming experience in one or more languages (Go, Java, Rust, C++) 
• You have worked on distributed storage systems 
• You can get down to the low-level when needed
• You care about simple designs and performance
• You want to work in a fast, high-growth startup environment that respects its engineers and customers