Definitive Healthcare

Senior Big Data Engineer

Framingham, MA
Scala SQL Python Spark AWS GCP Unity
Search for More Jobs Talk to a recruiter now 💪
Description

At Definitive Healthcare, our passion is to transform data, analytics and expertise into healthcare commercial intelligence. We help clients uncover the right markets, opportunities and people, so they can shape tomorrow’s healthcare industry. Our SaaS platform creates new paths to commercial success in the healthcare market, so companies can identify where to go next.  

Our employees are kind, collaborative, energetic, approachable and driven. On top of that, we value the unique perspectives, backgrounds and voices of our employees. Why? Because their diverse experiences drive new ideas and help us build a better community. 

For over 10 years, we’ve built a collaborative culture driven by employees who share a passion for improving the healthcare ecosystem, enjoy giving back to the local community and value diversity and inclusion.  

One of the hallmarks of our culture is our commitment to community service. Through the DefinitiveCares program, employees can work with their choice of more than 40 charitable organizations, supporting causes from hunger and homelessness to healthcare, LGBTQ+ issues, racial justice, women’s initiatives and more. 2021 marked the sixth year that we had 100% employee participation in DefinitiveCares. 

We also provide a range of opportunities for employees to connect with each other. Employees can join any of our employee run affinity groups supporting causes such as women’s empowerment, LGBTQ+, Black, indigenous and people of color (BIPOC), disabilities and working parents and potential for many more. Affinity groups often enable greater education companywide through training, events and speaker series. 

We’re also a great place to work. For five years in a row, we’ve been recognized by the Boston Business Journal and the Boston Globe as a best place to work in Massachusetts. In 2022, Energage recognized us for Culture Excellence in Compensation & Benefits, Innovation, Great Leadership, Purpose & Value and Work-Life Flexibility! 

Think you’d be a good addition to our team? Explore our available positions here. We’d love the chance to get to know you.  

Responsibilities:

  • Design and Develop Data Pipelines:
    • Build and maintain scalable data pipelines using Python, Spark, and Databricks.
    • Implement data workflows and ETL processes using Apache Airflow.
  • Data Integration and Management:
    • Integrate data from various sources (AWS, GCP, on-premises) into a unified data warehouse.
    • Handle variety of data formats such as csv, text, xml, parquet, delta etc.,
    • Ensure data quality and integrity through effective data cleansing and curation practices.
    • Manage and optimize data storage solutions, ensuring high availability and performance.
    • Automate observability of data and workloads
  • Metadata Management and Governance:
    • Implement and manage Unity Catalog for metadata management.
    • Ensure data governance policies are followed, including data security, privacy, and compliance.
    • Develop and maintain data documentation and data dictionaries.
    • Automate data observability across pipelines
  • Performance Tuning and Troubleshooting:
    • Optimize Spark jobs for performance and efficiency.
    • Investigate and resolve performance bottlenecks in Spark applications.
    • Utilize JVM tuning techniques to improve application performance.
  • Data Maturity Lifecycle:
    • Implement and manage the Medallion architecture for data maturity lifecycle.
    • Ensure data is appropriately processed and categorized at different stages (bronze, silver, gold) to maximize its usability and value.
  • Collaboration and Continuous Improvement:
    • Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions.
    • Implement CI/CD pipelines to automate deployment and testing of data infrastructure.
    • Stay up to date with the latest industry trends and technologies to continuously improve data engineering practices.

Required Skills and Qualifications:

  • Technical Skills:
    • Hands-on Python or Scala programming.
    • Strong experience with Apache Spark and Databricks.
    • Hands-on experience with Apache Airflow or similar workflow orchestration tools.
    • Data modeling and processing fundamentals with large-scale volume of data
    • Knowledge of data cleansing and curation techniques.
    • Familiarity with Unity Catalog or other metadata management tools.
    • Understanding of data governance principles and best practices.
    • Experience with cloud platforms (AWS and GCP).
    • Strong understanding of normalization and denormalization.
    • Proficiency in CI/CD tools and practices (e.g., Jenkins, GitLab CI, etc.).
    • Experience with JVM tuning and Spark job performance investigation.
    • Experience with Medallion architecture for data maturity lifecycle.
    • Familiarity with containerization
  • Soft Skills:
    • Excellent problem-solving and analytical skills.
    • Strong communication and collaboration skills.
    • Ability to work independently and as part of a team.
    • Detail-oriented with a focus on delivering high-quality work.

Preferred Qualifications:

  • Certification in cloud platforms (AWS Certified Data Analytics, Google Cloud Professional Data Engineer, etc.).
  • Familiarity with SQL and NoSQL databases.
  • Experience in a similar role within a fast-paced, data-driven environment.

Why we love Definitive, and why you will too!

  • Industry leading products
  • Work hard, and have fun doing it
  • Incredibly fast growth means limitless opportunity
  • Flexible and dynamic culture
  • Work alongside some of the most talented and dedicated teammates
  • Definitive Cares, our community service group, gives all of us a chance to give back
  • Competitive benefits package including great healthcare benefits and a 401(k) match

What our Employees are saying about us on Glassdoor: 

 “Great Work atmosphere, great work life balance, excellent company to work for, amazing top notch product, incredible customer service, lots of tools to help you succeed.”

-Business Development Manager

“Great team. Amazing growth. Employees are treated very well.”

-Research Analyst

“I have waited 36 years to work at a dream job for a dream company and I am so happy to have finally got there.”

-Profile Analyst

 

If you don’t fit all of these qualifications, but believe you’re still a great fit, feel free to apply and tell us why in your cover letter.

 

If you are a California, Colorado, New York City or Washington resident and this role is a remote role, you can receive additional information about the compensation and benefits for this role, which we will provide upon request.

 

Definitive Hiring Philosophy

Definitive Healthcare is an equal opportunity employer that celebrates diversity and is committed to creating an inclusive workplace with equal opportunity for all applicants and teammates. Our goal is to recruit the most talented people from a diverse candidate pool regardless of race, color, religion, age, gender, gender identity, sexual orientation or any other status. If you’re interested in working in a fast growing, exciting working environment – we encourage you to apply!

 

Privacy 
Your privacy is important to us. Please review our Candidate Privacy Notice which tells you how we use and process your personal information

 

Please note: All communications regarding the hiring process at Definitive Healthcare will come directly from one of our corporate recruiters or coordinators with an @definitivehc.com email address. We will never request any money transfer or purchase of equipment with a promise of reimbursement. If you receive any suspicious communications, please reach out to careers@definfitivehc.com to confirm your status in the application process. 

Definitive Healthcare
Definitive Healthcare
Analytics Artificial Intelligence (AI) CRM Hospital Information Services Medical SaaS

0 applies

2 views

There are more than 50,000 engineering jobs:

Subscribe to membership and unlock all jobs

Engineering Jobs

60,000+ jobs from 4,500+ well-funded companies

Updated Daily

New jobs are added every day as companies post them

Refined Search

Use filters like skill, location, etc to narrow results

Become a member

🥳🥳🥳 401 happy customers and counting...

Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.

To try it out

For active job seekers

For those who are passive looking

Cancel anytime

Frequently Asked Questions

  • We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
  • Salaries for the engineering jobs on our site range from $100K-$200K. On average, senior engineer positions on our EchoJobs are about $160K.
  • The EchoJobs positions have been sourced and vetted from the top companies to work for in the US as a software engineer, including LinkedIn and other reputable job sites. We also have syndicated jobs from companies that have just raised funding, as well as those that have great unique products and culture. From all of these sources, our founder, Morgan, has also resourced the company's authenticity in terms of their website, public appearance, and more.
  • Yes, our users asked us for just this, so now our search filters allow you to search for your top jobs via location, as well as by onsite, remote, or both. Approximately 30% of our jobs are remote, so you’ve got the best options for you!
  • We have not yet implemented this option, but are considering doing so in the future. For the moment, you would need to cancel your subscription, and resubscribe when you wanted to come back.
  • We add new jobs to EchoJobs every day! We scan our sources for the newest jobs, verify them, and post them to EchoJobs within minutes. We add about 2,000-3,000 new jobs for you each day!
  • From starting your job search to getting hired, the entire job search process can take us software engineers anywhere between 3-6 months. However, at EchoJobs, we’re striving to shorten this duration by finding the best, newest jobs for you, so you can do less job searching, and more applying.
  • We’d recommend checking EchoJobs daily, as we add new jobs to the site each day. Additionally, if you got a chance to read our previous email on “what makes EchoJobs different from any other job search tools,” we also recommended that you set a job alert based on your job filters, so if you get emails on those new jobs, you could be checking more than once per day.
  • If you decide to continue with us after the 1-month trial, we definitely recommend this, as we all know it usually takes 3-6 months to find a quality job as a software engineer these days. So to best support you, we just adjusted our membership options at EchoJobs to monthly, 3 months, or 12 months (this option is more for passive job seekers looking a little bit for the future if they want to come back to work or make a job switch potentially. This lets you see what’s out there in case an even better fit job becomes available.)
  • EchoJobs is truly the only job site of its kind. We want to be THE spot for you to find the best job for you, and haven’t encountered any other company doing this. Other job sites are in niches besides software engineering or focus on a small portion of engineering jobs (like a specific coding language). In the words of Morgan, our founder, “I think what makes EchoJobs different is the amount of jobs, frequency that we add new jobs (we add 2,000-3,000 new jobs daily!), and the powerful search engines to find exactly the job you want more easily and efficiently. We can provide you with the most jobs that are vetted by us, we’ll continually find more new jobs for you, and we make it easier for you to apply and get hired.

What Fellow Engineers Say