Lead Data Engineer

We deliver a connected healthcare experience for over 600,000 members and 70+ companies nationwide who want the best for their employees. Our data engineering team is pivotal in this mission, constructing and managing scalable, high-performance data pipelines and architectures that drive analytics, operational workflows, and machine learning applications.

As a Lead Data Engineer, you will drive the development of robust, scalable, and efficient data solutions, collaborating closely with cross-functional teams. You will provide thought leadership on data architecture, mentor junior engineers, and optimize our data ecosystem for performance and reliability

What you'll do:

• Architect Scalable Data Solutions - Design, develop, and optimize large-scale data pipelines using Spark (PySpark, Scala), Databricks, and distributed data processing frameworks.
• Advance Data Modeling & Architecture - Lead the design and evolution of data models to support analytical, operational, and machine-learning requirements.
• Enhance Data Performance & Reliability - Improve data processing performance, scalability, and reliability, while ensuring data quality and governance.
• Drive Cross-Functional Collaboration - Partner with Product, Engineering, Data Science, and Analytics teams to deliver high-impact data solutions that generate actionable business and clinical insights.
• Mentor & Provide Technical Leadership - Guide junior and mid-level engineers, conduct code reviews, and establish best practices in data engineering.
• Ensure Data Governance & Security - Implement robust security, privacy, and compliance measures for sensitive healthcare data, ensuring adherence to industry regulations.
• Influence Data Strategy - Provide input on data infrastructure decisions, emerging technologies, and process improvements.

To be successful in this role, you'll need:

• 8+ years of data engineering experience in fast-paced, data-driven environments.
• Expertise in building scalable ETL pipelines with Spark (PySpark or Scala) and SQL.
• Deep understanding of data architecture, schema design, and dimensional modeling for analytics and machine learning.
• Proficiency in distributed systems such as Spark, Databricks, or Snowflake.
• Experience with event-driven architectures and streaming platforms like Kafka or Kinesis.
• Excellent communication skills – ability to collaborate cross-functionally and translate complex technical concepts into business impact.
• Mentorship experience – experience guiding engineers and fostering a collaborative, inclusive team culture.
• Security-first mindset – familiarity with data privacy, encryption, and compliance in healthcare or other regulated industries is a plus.

This is a highly visible role within the organization, where your contributions will shape the future of our data ecosystem and drive impactful healthcare insights. If you're passionate about solving complex data challenges while making a real-world difference, we'd love to hear from you!

Pay Transparency Statement 

This is a hybrid position based out of one of our offices, with the expectation of being in office at least two weekdays per week. #LI-hybrid 

The actual pay rate offered within the range will depend on factors including geographic location, qualifications, experience, and internal equity. In addition to the salary, you will be eligible for stock options and benefits like health insurance, 401k, and paid time off. Learn more about our benefits at https://jobs.collectivehealth.com/benefits/.