Junior Data Infrastructure Engineer

Location: Brighton, England, United Kingdom

Department: 710 - Software Product Engr

As part of the Data Infrastructure team you will be supporting mission critical big data platforms, to ensure they are fully performant, reliable, available and secure. We call it data infrastructure engineering, also known as DataOps or Database Administration or SRE.

The role is a mixture of development of tooling and operational support for our platforms, two aspects that go hand-in-hand. It requires attention to detail and curiousity about how the systems work under the hood, and gives you a wide base of skills from low level system tuning to general coding. 

We manage four main storage platforms, namely:

• Apache Solr (~2.2 PB)
• Apache HBase (~450 TB)
• PostgreSQL (~15 TB)
• Kafka (~60 TB)

These platforms are all open source, written in Java, Scala or C, and we maintain in-house builds and patching of them. We use a variety of open-source and in-house developed tooling to manage these services, mostly written in Rust and Python, which run primarily on hundreds of servers in multiple data centres and in the cloud.

We maintain a balance between project work and operational/ad-hoc work for all members in the team, whether they are senior or recent graduates, and your day-to-day work will be a mix of these.

The exact project work will depend on your experience, but we will scope it out collectively to ensure it is both achievable and delivers benefit. Some example projects given to recent graduates are:

• Coding mapreduce jobs to validate data between two multi-TB+ hbase clusters (to run in hours rather than months)
• Building out a python code stack to transparently migrate production clients between clusters (without losing any data or requiring any downtime)
• Building and testing new HBase builds and deploying them live (without putting everyone in Brandwatch out of a job)
• Creating a platform for rolling out our data management systems using Apache Airflow (to stop people re-writing custom job management systems)
• Rolling out a DR solution for hadoop on AWS (while keeping the system performant and not bankrupting the company)

If you're joining the team as a graduate engineer we'll be focusing on developing your knowledge of the storage platforms, how to program against them and how to troubleshoot issues. The aim is that you'd be able to help with the staging support by 3-6 months, and join the rest of the team supporting the production systems in 12 months. You'll also be expected to take a full part in the life of the team and, as your experience grows, start designing the infrastructure we run.

This role is based in Brighton, UK, but working remotely will be possible once the initial 6-12 months of training is complete.

Who we are looking for

In house training for the storage systems and our tooling is a necessary part of this role, but we are looking for someone with a keen interest in large scale storage infrastructure.A successful candidate will be expected to have the following:

• An interest in how computer infrastructure actually works, and a passion for learning
• Interest, and ideally production experience, running storage systems, eg. as part of a selfhosted service, a home lab or as part of academic studies
• Experience with Linux systems administration, including experience of trouble shooting
• Fluency with one or more scripting languages, ideally Bash or Python
• Experience helping your peers
• Pride in the quality of your work
• Experience with any of the following will be beneficial:
  • Development experience, in Python, Java, Rust, C/C++ or Golang.
  • Upstream open-source contributions
  • Academic research in scalability, distributed systems or storage infrastructure
  • Kubernetes experience, ideally running or writing Operators Experience with Docker and with CI/CD pipelines