Lead Backend Engineer - Python

Department: PE-Cloud

Experience: 8+

Skills: Databricks, Elasticsearch, Python, solr, NEMSIS

• Own the lifecycle of data attributes across the technology stack, from definition and modeling through ingestion, indexing, APIs, and presentation.
• Define scalable data models and attribute structures to support evolving healthcare data requirements.
• Develop and maintain data ingestion and enrichment workflows using Python.
• Work closely with Java/Spring services to expose and serve enriched data through APIs.
• Collaborate with front-end teams to ensure data is effectively presented and consumed.

• Design and implement a metadata-driven framework for managing data attributes across the technology stack.
• Create a single attribute definition that can generate the required configurations and artifacts currently maintained across multiple components.
• Establish reusable patterns that simplify the introduction and management of new attributes.
• Improve consistency, maintainability, and operational efficiency across the data platform.

• Design repeatable processes and frameworks for onboarding new data sources.
• Reduce data-source onboarding time from weeks or days to hours wherever practical through automation and metadata-driven configuration.
• Develop scalable ingestion, transformation, enrichment, and indexing workflows.
• Support the initial implementation for one data source and the subsequent onboarding of approximately ten additional sources.

• Redesign the existing Solr architecture to support continued growth beyond the current fixed three-shard layout.
• Define and optimize SolrCloud sharding, routing, indexing, and query strategies.
• Design solutions for high-volume indexing and low-latency search at scale.
• Diagnose and resolve search performance, indexing, and scalability challenges.
• Establish best practices for Solr schema design, collections, replicas, shards, and query optimization.
• Ensure the search architecture scales reliably as data volumes and sources increase.

• Develop scalable, maintainable, and production-ready backend components using Python.
• Contribute confidently to Java/Spring-based services and APIs.
• Design robust data-processing pipelines for ingestion, enrichment, and serving.
• Write clean, testable, and maintainable code with appropriate documentation and engineering practices.
• Take ownership of technical solutions from architecture and design through implementation, testing, and production deployment.

• Work closely with a small, experienced engineering team in an autonomous and fast-moving environment.
• Take ownership of complex technical problems and drive them through to resolution.
• Participate in architecture and design discussions and contribute to technical decision-making.
• Collaborate with engineering, product, and other stakeholders to translate requirements into scalable technical solutions.
• Work through short development and feedback cycles to deliver production-ready improvements quickly.

• Strong production-level experience with Python.
• Deep hands-on experience with Apache Solr/Lucene, preferably SolrCloud 9 or equivalent large-scale deployments.
• Strong understanding of Solr architecture, including sharding, routing, indexing, replication, schema design, and query optimization.
• Strong Elasticsearch experience may be considered for candidates willing and able to transition to Solr.
• Working knowledge of Java and the ability to contribute confidently to Spring-based backend services.
• Strong SQL and data-modeling skills.
• Experience designing scalable data-ingestion and data-processing pipelines.
• Experience with data enrichment, indexing, and search-serving workflows.
• Strong understanding of distributed systems and large-scale data platforms.
• Ability to troubleshoot and optimize high-volume indexing and search workloads.
• Demonstrated ability to take ownership of complex engineering problems from design through production deployment.

• Strong Python development experience.
• Hands-on experience with Solr/Lucene at scale.
• Strong understanding of SolrCloud, sharding, routing, and indexing strategies.
• Experience with Elasticsearch may be considered, provided there is a willingness to transition to Solr.
• Good working knowledge of Java and Spring.
• Strong SQL and data-modeling skills.
• Experience building scalable data-ingestion and processing workflows.
• Strong problem-solving and system-design capabilities.
• Experience working with large-scale data platforms and production environments.

• Experience working with healthcare data.
• Knowledge of NEMSIS data and emergency medical records.
• Experience with Databricks.
• Front-end development experience.
• Experience building metadata-driven platforms or configuration frameworks.
• Experience working with very large-scale datasets and distributed search infrastructure.

• Strong ownership mindset with the ability to independently drive technical initiatives.
• Excellent system-design and architectural thinking.
• Strong analytical and problem-solving skills.
• Comfortable working with ambiguity and evolving requirements.
• Ability to balance long-term architecture with short-term delivery requirements.
• Strong communication and collaboration skills.
• Comfortable working in a small, senior, and highly autonomous engineering team.
• Passion for building scalable, maintainable, and production-ready systems.
• Ability to learn and adapt quickly to new technologies and domain requirements.