Senior Data Engineer (Azure Databricks)
Location: Hyderabad, TS, in
Company Description
Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visit www.blend360.com
Job Description
We are looking for an experienced Senior Azure Databricks Engineer with strong hands-on expertise in Python, SQL, and Apache Spark to design, build, and optimize scalable data pipelines and analytics solutions on the Azure cloud platform. The ideal candidate should have experience working with large datasets, distributed data processing, analytics use cases, and modern data engineering practices.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Databricks
- Implement ETL/ELT workflows using PySpark, Spark SQL, and Python
- Implement pipelines for data ingestion using Azure Data Factory
- Optimize Spark jobs for performance, cost, and scalability
- Work with structured and semi-structured data (Parquet, Delta, JSON, CSV)
- Build and manage Delta Lake tables (ACID, time travel, schema evolution)
- Integrate Databricks with Azure Data Lake Storage (ADLS Gen2)
- Develop complex queries and transformations using SQL
- Collaborate with Data Science teams to prepare data for modelling use cases, ensuring appropriate transformations, feature generation, and storage.
- Follow best practices for security, access control, and governance in Azure
- Ensure data quality, validation, and monitoring using testing tools
- Deployment of solutions to Production environments
Qualifications
- 4+ years of experience in Data Engineering, ideally supporting POS and SKU datasets.
- Handling high volume transactional datasets
- Strong hands-on experience with Azure Databricks
- Understanding of the Medallion Architecture and implementing it within Databricks
- Good understanding of data modelling techniques
- Proficiency in Python for data processing
- Strong knowledge of SQL (joins, window functions, performance tuning)
- Hands-on experience with Apache Spark / PySpark
- Experience working with Delta Lake
- Knowledge of Azure Data Lake Storage (ADLS Gen2)
- Understanding of distributed computing concepts
- Experience with Git version control
- Understanding of ML use cases and data considerations for model development
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say