Principal Site Reliability Engineer

Okta’s Workforce Identity Cloud Security Engineering group is looking for an experienced and passionate Staff Site Reliability Engineer to join a team focused on designing and developing Security solutions to harden our cloud infrastructure. We embrace innovation and pave the way to transform bright ideas into excellent security solutions that help run large-scale, critical infrastructure. We encourage you to prescribe defense-in-depth measures, industry security standards and enforce the principle of least privilege to help take our Security posture to the next level. Our Infrastructure Security team has a niche skill-set that balances Security domain expertise with the ability to design, implement, rollout infrastructure across multiple cloud environments without adding friction to product functionality or performance. We are responsible for the ever-growing need to improve our customer safety and privacy by providing security services that are coupled with the core Okta product.

This is a high-impact role in a security-centric, fast-paced organization that is poised for massive growth and success. You will act as a liaison between the Security org and the Engineering org to build technical leverage and influence the security roadmap. You will focus on engineering security aspects of the systems used across our services. Join us and be part of a company that is about to change the cloud computing landscape forever.

Bring all the passion and dedication along and there’s no telling what you could accomplish!

You will work on:

• Designing, building, running, and monitoring Okta's production infrastructure
• Be an evangelist for security best practices and also lead initiatives/projects to strengthen our security posture for critical infrastructure
• Responding to production incidents and determining how we can prevent them in the future
• Triaging and troubleshooting complex production issues to ensure reliability and performance
• Identifying and automating manual processes
• Continuously evolving our monitoring tools and platform
• Promoting and applying best practices for building scalable and reliable services across engineering
• Developing and maintaining technical documentation, runbooks, and procedures
• Supporting a 24x7 online environment as part of an on-call rotation
• Be a technical SME for a team that designs and builds Okta's production infrastructure, focusing on security at scale in the cloud.

You are an ideal candidate if you:

• Are always willing to go the extra mile: see a problem, fix the problem.
• Are passionate about encouraging the development of engineering peers and leading by example.
• Have experience automating, securing, and running large-scale production IAM and containerized services in AWS (EC2, ECS, KMS, Kinesis, RDS), GCP (GKE, GCE) or other cloud providers.
• Have deep knowledge of CI/CD principles, Linux fundamentals, OS hardening, networking concepts, and IP protocols.
• Have a deep understanding and familiarity with configuration management tools like Chef and Terraform.
• Have expert-level abilities in operational tooling languages such as Ruby, Python, Go and shell, and use of source control.
• Have experience with industry-standard security tools like Nessus, Qualys, OSQuery, Splunk, etc.
• Have experience with Public Key Infrastructure (PKI) and secrets management
• Lead technical design and architecture decisions, and align project members towards the same goal and standards.

Bonus points for:

• Experience conducting threat assessments, and assessing vulnerabilities in a high-availability setting.
• Understand MySQL, including replication and clustering strategies, and are familiar with data stores such as DynamoDB, Redis, and Elasticsearch.

Minimum Required Knowledge, Skills, Abilities, and Qualities:

• 10 + years of experience architecting and running complex AWS or other cloud networking infrastructure resources
• 6+ years of experience with Chef and Terraform
• Unflappable troubleshooting skills
• Proven experience in collaborating across teams to deliver complex horizontal projects
• Strong leadership skills
• Strong written and verbal communication skills.
• Strong Linux understanding and experience.
• Strong security background and knowledge.
• BS In computer science (or equivalent experience).