Principal Engineer, Site Reliability Engineering - Observability

Due to Federal Government contract requirements, U.S. Citizenship is required for this position.

FedRamp Staff may be subject to customer or third party background checks up to and including Secret Clearance if required by their role at SentinelOne. 

What are we looking for?

We are seeking to hire an experienced Principal Engineer to join our Site Reliability Engineering (SRE) Team at SentinelOne. This role can be 100% remote for individuals based in the US, or hybrid if local to a corporate office location.

As a Principal Engineer, you will architect and lead the implementation of advanced observability, automated triage, and self-healing capabilities within our microservices-based SaaS environment. You will be instrumental in driving our organization’s evolution towards proactive, scalable incident management by enabling smart alert correlation, automated root cause analysis, and autonomous remediation systems. Additionally, you will define and implement Service Level Objectives (SLOs) that align with business goals, ensuring our systems meet reliability standards and exceed customer expectations.

What will you do? 

• Design and guide the implementation of end-to-end alert correlation, auto-triage, and uto-remediation frameworks that meet the needs of a microservices-based SaaS architecture.

• Ensure solutions align with business priorities and customer impact goals.

• Define, implement, and monitor SLOs in collaboration with product and engineering teams. 

• Establish reliability standards that meet business and customer expectations, driving accountability and transparency around service performance.

• Partner with software engineers, SREs, and data scientists to implement and refine monitoring, alerting, alert correlation, auto-remediation, and SLO solutions.

• Lead initiatives to promote best practices and knowledge sharing across all of SentinelOne engineering.

• Mentor engineers and contribute to a culture of reliability engineering excellence through thought leadership and guidance on advanced SRE principles and practices.

What skills and knowledge should you bring?

• Extensive SRE Experience: Proven experience in architecting and implementing SRE solutions at scale within a microservices or distributed systems environment.
• 15+ years of progressive professional experience, with 5+ years of recent experience supporting enterprise SaaS environments (or equivalent combination of education, experience, and certifications).

• Technical Expertise: Deep knowledge of incident management, alert correlation, automated triage, self-healing strategies, and SLO frameworks. Strong understanding of observability platforms, including monitoring, logging, and tracing solutions.

• Programming & Scripting: Proficient in one or more programming languages (e.g., Python, Go, Java) with experience in automation and scripting for incident management workflows.

• Machine Learning & Data Analysis: Experience with machine learning, anomaly detection, or data analytics techniques for real-time alert correlation and triage systems.

• Cloud Infrastructure: Expertise in cloud platforms (e.g., AWS, GCP, Azure) and container orchestration (e.g., Kubernetes), with experience in infrastructure-as-code (e.g., Terraform).

• Problem-Solving & Decision-Making: Ability to make critical architectural decisions with a focus on business impact, reliability, and system performance.

Why us?

You will be joining a cutting-edge company, where you will tackle extraordinary challenges and work with the very best in the industry.

• Medical, Vision, Dental, 401(k), Commuter, Health and Dependent FSA
• Unlimited PTO
• Industry leading gender-neutral parental leave
• Paid Company Holidays
• Paid Sick Time
• Employee stock purchase program
• Disability and life insurance
• Employee assistance program
• Gym membership reimbursement
• Cell phone reimbursement