Sr. IT Linux Site Reliability Engineer

Location: Hawthorne, CA

Department: Information Technology - Corporate

SR. LINUX SITE RELIABILITY ENGINEER

SpaceX is looking for an experienced engineer with deep working knowledge of Kubernetes and related containerized technologies. This employee will be a member of the Information Technology Linux Infrastructure team and will provide expertise in Kubernetes design, maintenance, scaling and optimization in support of critical business functions. The ideal candidate will be flexible and flourish in a fast paced and challenging environment. They should be a self-starter, self-motivator and possess ingenuity to excel at this position.

RESPONSIBILITIES:

• Install, manage, scale and optimize Kubernetes and RKE clusters using Ansible, Terraform and adjacent technologies in production environments.
• Work closely with other SpaceX engineers to gather requirements, research, evaluate, design, plan, deploy, and support software platforms and related technologies running in Kubernetes within a world-class environment that meets the needs of the demanding SpaceX engineering teams. Build highly resilient, high-performance, scalable, and robust systems.
• Exercise a high degree of personal responsibility for the processes, systems, and tools you create and manage; all supporting the goal of making humanity an interplanetary species.
• Make recommendations, justify, and implement improvements using an accepted change control methodology.
• Work within a diverse group to design and deliver creative solutions and resolve problems in a timely and proactive manner by interacting with internal business units.
• Define, document and follow standards and best practices for systems design, testing, and implementation.
• Foster an environment of collaboration and cross-training, upskilling the team in Kubernetes expertise and ensuring peers are developed into capable engineers.
• Drive scripting, self-service and automation to develop solutions to reduce administrative overhead and TOIL.
• Participate in on-call rotation to handle urgent after-hours work when necessary.

BASIC QUALIFICATIONS:

• Bachelor’s degree in Computer Science or a STEM discipline and 5+ years of systems engineering experience; OR 7+ years of systems engineering experience in lieu of a degree.
• Experience deploying and supporting Linux servers in physical and virtualized environments (e.g. VMware via automation).
• Experience with the Linux shell as well as configuring and extending Linux instances (e.g. kernel modules, cgroups, pki, iptables, interfaces).
• Experience supporting and scaling containerized applications in Linux environments.
• Experience using automation frameworks (e.g. Ansible, Terraform) to manage provisioning and post-provisioning lifecycles of infrastructure and Kubernetes installations.

PREFERRED SKILLS AND EXPERIENCE:

• Expertise in creating repeatable, reliable, scalable systems architectures, with high availability, fault tolerance, performance tuning, monitoring, and statistics/metrics collection.
• Expertise in source code version control tools such as Git and Subversion and collaborating on source code via Pull Requests and other Git-based workflows.
• Strong understanding of Linux Container Runtime.
• Experience implementing configuration management provisioning and workflow automation solutions via Infrastructure as Code, CI/CD and GitOps (e.g. Ansible, AWX/Tower, Vagrant, Puppet, Redfish, Jenkins, cloud-init, ArgoCD, etc).
• Experience writing test automation to ensure backwards compatibility of feature and change development for automation processes and Kubernetes deployments.
• Experience with programming and scripting languages such as Python and Golang to develop software solutions and integrate with external systems to implement automation against RESTful API services.
• Experience installing, configuring and troubleshooting Kubernetes internals, CNI, CRI and CSI plugins (e.g. Docker, Cri-O, Ceph, Cilium), load balancing (e.g. MetalLB), Service Mesh (e.g. Istio) and software-defined storage (e.g. rook-ceph) in cloud or on-premise environments.
• Experience developing solutions using Kubernetes patterns to extend system functionality and solve custom use cases (e.g. webhooks, controllers, operators, sidecars).
• Experience implementing proactive alert/monitoring workflows and dashboards for Linux systems and Kubernetes deployments using Prometheus, Grafana, InfluxDB or similar technologies.
• Experience with dynamic system configuration templating using Jinja, Jsonnet, YAML and Helm.

ADDITIONAL REQUIREMENTS:

• Must be willing to work extended hours and weekends as needed.

COMPENSATION AND BENEFITS:    
    
Pay Range:         
Site Reliability Engineer/Senior: $160,000.00 - $220,000.00/year

Your actual level and base salary will be determined on a case-by-case basis and may vary based on the following considerations: job-related knowledge and skills, education, and experience.

Base salary is just one part of your total rewards package at SpaceX. You may also be eligible for long-term incentives, in the form of company stock, stock options, or long-term cash awards, as well as potential discretionary bonuses and the ability to purchase additional stock at a discount through an Employee Stock Purchase Plan. You will also receive access to comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short and long-term disability insurance, life insurance, paid parental leave, and various other discounts and perks. You may also accrue 3 weeks of paid vacation and will be eligible for 10 or more paid holidays per year. Employees accrue paid sick leave pursuant to Company policy which satisfies or exceeds the accrual, carryover, and use requirements of the law.