Sr. Principal Mission Engineer

Location: Columbia, MD

Department: Platform Engineering

Senior Principal Mission Engineer (K8s & Proxmox)

Position Overview

The position is for an on-site, cleared Senior Principal Mission Engineer as part of a team of software and platform engineers building a unified, multi-tenant control plane that abstracts away infrastructure differences across AWS, Azure, and on-premises environments. The platform allows application teams to provision secure, isolated Kubernetes clusters and workloads dynamically. The control plane runs Crossplane and Cluster API (CAPI).

As the primary on-site engineer, you will be dedicated to deploying, managing, and maintaining our hardware and Proxmox-based system within a secure, classified space. In this role, you will be the primary on-site technical authority responsible for the physical and virtual health of our control plane footprint. You will bridge the gap between our remote engineering pipeline and the isolated, classified hardware infrastructure, handling everything from low-level virtualization troubleshooting to direct customer engagement and user training.

The ideal candidate is highly independent, capable of driving projects from inception to completion with minimal supervision.

Key Responsibilities

• System Management & Virtualization Tuning: Maintain, update, and optimize the on-site Proxmox VE hypervisor environments, Ceph distributed storage clusters, and associated underlying server hardware.
• Troubleshooting & Incident Resolution: Diagnose and resolve complex hardware failures, kernel panics, network partitions, virtualization bottlenecks, Kubernetes pod crash loops, CNI networking failures, and node pressure conditions within the isolated classified space.
• Integration & Deployment: Deploy automated system updates, patch air-gapped systems, and integrate new physical compute or storage nodes into the local cluster environment.
• Customer Engagement & Advisory: Serve as the trusted, face-to-face technical representative for the program, managing day-to-day interactions with site stakeholders, mission owners, and local administrators.
• On-Site Training: Conduct technical walkthroughs, operational training, and compliance briefings for customer personnel interacting with the Proxmox execution platform.
• Security & Configuration Control: Partner with the program ISSO/ISSM to enforce local security benchmarks, apply security patches, and support configuration control board reviews for physical hardware modifications.

Qualifications

• Clearance: Active Top Secret / SCI
• Experience: 5+ years of hands-on experience managing enterprise Linux environments and bare-metal hardware infrastructure in secure environments.
• Hypervisor Expertise: Strong, demonstrable experience configuring and troubleshooting Proxmox VE, KVM, QEMU, or equivalent enterprise virtualization technologies (e.g., VMware vSphere, OpenStack).
• Storage & Networking: Practical experience with Ceph storage orchestration, software-defined networking, VLAN tagging, and low-level Linux network troubleshooting.
• Customer-Facing Skills: Excellent interpersonal skills with a proven track record of managing technical relationships and conducting formal training for government or military personnel.

Preferred Qualifications

• Live-Production Incident Response: Hands-on experience managing incident response lifecycles and resolving critical control-plane or etcd failures within live, high-scale production systems.
• Telemetry-Driven Debugging: Advanced capability in utilizing telemetry, API server audit logs, distributed tracing, and metrics to perform root-cause analysis and resolve complex issues in live production environments.

Salary Range: $100,000 - $330,000