Mid-Level Data Engineer, Databricks

Team: IT

Location: US

Commitment: Full-time

Workplace Type: remote

Accountabilities

• Design, develop, test, and maintain data pipelines using Databricks, Python, PySpark, and SQL to ingest, transform, and organize data from source systems into layered, reusable tables.

• Build and maintain data products that support analytics, reporting, and automation across multiple business areas.

• Validate datasets against source systems, investigate discrepancies, troubleshoot data quality issues, and implement corrections before data reaches downstream reporting and analytics processes.

• Apply established ETL/ELT practices, data modeling principles, and quality controls to ensure data accuracy, consistency, and reliability.

• Document data pipelines, table definitions, transformation logic, and business rules to support maintainability, collaboration, and knowledge sharing.

• Take ownership of assigned engineering tasks from initial development through delivery and production support, communicating progress and raising risks or blockers promptly.

• Collaborate with senior engineers, data scientists, government stakeholders, and other team members to clarify requirements and deliver solutions aligned with business needs.

• Adapt to changing project priorities and develop proficiency with additional tools and technologies as new requirements emerge.

• Contribute to continuous improvements in data engineering workflows, pipeline reliability, and the usability of shared datasets.

Requirements:

• Bachelor's degree in computer science, data science, information systems, engineering, or a related discipline, or equivalent professional experience.

• At least 5 years of experience in data engineering, data analysis, or a related technical field.

• Hands-on experience building and supporting data pipelines in Databricks, including notebooks, PySpark or Spark SQL, and Delta tables.

• Proficiency in Python and SQL for data transformation, querying, validation, and troubleshooting.

• Solid understanding of ETL/ELT processes, data modeling, data integration, and data quality practices.

• Ability to investigate data discrepancies, identify root causes, and implement effective solutions.

• Strong documentation, communication, and problem-solving skills, with the ability to explain technical issues and collaborate with both technical and non-technical stakeholders.

• Demonstrated ownership of assigned work, effective time management, and the ability to adapt to shifting priorities while keeping stakeholders informed.

• U.S. citizenship is required.

• Must be eligible to obtain and maintain federal suitability, a government-issued Personal Identity Verification (PIV) credential, and all required system access.

Nice to Have:

• Experience integrating data through APIs.

• Familiarity with Databricks Unity Catalog, Databricks Jobs or Workflows, and CI/CD deployment practices.

• Experience working with Google Cloud Platform (GCP).

• Previous experience supporting federal government clients or projects.

Benefits:

• Salary: Annual compensation range of $100,000–$140,000 USD.

• Work arrangement: Fully remote position within the United States.

• Health coverage: Comprehensive medical, dental, and vision insurance.

• Insurance protection: Short-term disability and life insurance coverage.

• Retirement savings: 401(k) plan with matching employer contributions.

• Paid time off: Paid leave, including 11 holidays and a day off for your birthday.

• Professional development: Opportunities and resources to support ongoing learning, technical growth, and career advancement.

• Fitness benefits: Fitness membership support.

• Remote-work support: Home office allocations to help support your remote working environment.

• Performance incentives: Performance bonuses and profit-sharing opportunities.

• Employee referral program: Referral bonuses for eligible employee referrals.

• Career development and support: A collaborative environment within a growing organization, with an emphasis on employee development, curiosity, authenticity, and helping team members reach their professional goals.

• Inclusive workplace: Commitment to equal employment opportunities and fair consideration for qualified applicants.