Description
Principal Engineer - AI
Team: Engineering
Location: San Jose
Commitment: Full-time
Workplace Type: onsite
Salary:
This position offers a competitive salary range, commensurate with experience and qualifications. The estimated base salary range is between $250,000 - $300,000 annually. In addition to base salary, the compensation package includes benefits such as health, dental, and vision insurance, 401(k), flexible paid time off, life insurance, and opportunities for professional growth. Final compensation will be determined in part by the qualifications of the selected candidate and may be above or below this range. Salary range and benefits are subject to change and may depend on location.
At SAFE Security, our mission is bold and ambitious: We Will Build CyberAGI — a super-specialized system of intelligence that autonomously predicts, detects, and remediates threats. This isn’t just a vision—it’s the future we’re building every day, with the best minds in AI, cybersecurity, and risk. At SAFE, we empower individuals and teams with the freedom and responsibility to align their goals, ensuring we all move towards this goal together.
We operate with radical transparency, autonomy, and accountability—there’s no room for brilliant jerks. We embrace a culture-first approach, offering an unlimited vacation policy, a high-trust work environment, and a commitment to continuous learning. For us, Culture is Our Strategy—check out our Culture Memo to dive deeper into what makes SAFE unique.
As a Principal Engineer - AI, you will define and lead the technical direction of AI systems that power Safe’s CRQ, CTEM, and TPRM products, including agentic workflows, RAG pipelines, LLM orchestration, and AI-native developer tooling. You’ll be the hands-on architect behind Safe’s AI engineering stack, bridging model intelligence with production-grade infrastructure.
You’ll collaborate with product, data, and platform teams to design scalable, explainable, and enterprise-ready systems.
This is a high-impact, technical leadership role that will shape how AI is built, deployed, and governed across Safe.
Core Responsibilities:
- Architect Safe’s AI Systems: Design and scale AI-driven components — LLM orchestration, retrieval-augmented generation (RAG), vector stores, prompt pipelines, and AI microservices. Drive architecture for AI observability, safety, and evaluation (precision, recall, F1, hallucination detection, cost metrics).
- Productionize AI Agents: Build multi-turn, goal-oriented agent systems that automate reasoning across TPRM, CTEM, and CRQ domains (e.g., control reviews, issue RCA, automated responses). Ensure reliability, traceability, and deterministic behavior in production.
- AI Infrastructure & Platform Ownership: Partner with Platform & DevOps teams to operationalize model serving (AWS SageMaker, Bedrock, or self-hosted Llama), build AI APIs, and manage model lifecycle and versioning. Establish feature stores, embedding management, and in-memory retrieval layers.
- Data Pipeline & Knowledge Graph Integration: Work with Data Engineering to design pipelines for structured and unstructured data ingestion, semantic indexing, and context retrieval (Snowflake + Iceberg + LlamaIndex).
- AI Evaluation, Monitoring & Governance: Define internal frameworks for golden dataset validation, LLM evaluation (LangFuse/LangSmith), and safety enforcement policies. Implement human-in-the-loop (HITL) mechanisms and continuous feedback loops.
- Mentor & Multiply: Guide AI and backend engineers on architectural design, experimentation methodologies, and prompt optimization. Collaborate with product leaders to translate abstract AI goals into measurable engineering deliverables.
Minimum Qualifications:
- Experience: 12+ years total experience in software engineering, including 4+ years building AI/ML systems or large-scale data/LLM infrastructure.
- Core Technical Skills:
- MLOps & Infra: Familiar with model versioning, CI/CD for ML, and performance optimization for real-time inference.
- Applied AI Focus: Practical understanding of evaluation metrics, hallucination detection, RAG reliability, and enterprise AI safety.
- Strong programming fundamentals in Python, Go, or TypeScript
- Deep understanding of LLM-based architectures, prompt engineering, and RAG pipelines
- Hands-on experience with LangChain, LlamaIndex, or equivalent orchestration frameworks
- Vector databases (FAISS, Pinecone, Weaviate, Redis Vector, or Milvus)
- Cloud model deployment (AWS SageMaker, Bedrock, Vertex AI, or custom inference APIs)
- Data systems: Snowflake, Iceberg, S3, Postgres/MySQL
Preferred Qualifications:
- Experience integrating AI into cybersecurity or risk management products
- Familiarity with multi-agent systems and autonomous workflows (CrewAI, LangGraph, AutoGen)
- Experience building AI evaluation dashboards and AI observability stacks
- Knowledge of knowledge graphs, semantic search, or retrieval pipelines
- Exposure to data governance, compliance, or SOC2/ISO 27001 environments
- Published research, open-source contributions, or prior leadership of AI teams is a strong plus
If you’re passionate about cyber risk, thrive in a fast-paced environment, and want to be part of a team that’s redefining security—we want to hear from you! 🚀
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say