Description
Senior Data Engineer
Location: Bengaluru, India
Department: Technology
Experience: 3-6 yrs
Skills: ELT, Data Lake, Data Warehouse, EFL
Senior Data Engineer (Databricks) - Contract to Hire (C2H )
Role Overview
We are looking for a highly capable and experienced Senior Data Engineer with strong Databricks expertise to architect, build, and scale our enterprise data platform. This is a Contract-to-Hire (C2H) position.
The role will take ownership of our Data Lake, ETL/ELT frameworks, data migration initiatives, and data integration architecture across multiple systems (ERP, CRM, LMS, Finance, third-party tools, etc.). The ideal candidate should be comfortable working in a scalable cloud-based environment and designing analytics- and AI-ready data infrastructure.
Key Responsibilities
1. Data Platform Architecture & Databricks Implementation
Design and implement enterprise-grade Data Lake architecture.
Build and manage scalable data solutions using Databricks.
Design ingestion frameworks for structured, semi-structured, and unstructured data.
Architect scalable, high-performance, and cost-optimized storage solutions.
Define metadata management, data cataloging, and lineage strategies.
2. ETL / ELT Framework Ownership
Architect and build scalable ETL/ELT pipelines using Databricks and Spark.
Develop reusable pipeline components and transformation logic.
Implement monitoring, logging, alerting, and failure recovery mechanisms.
Optimize performance of large-scale distributed data processing workflows.
Enable batch and near real-time data processing capabilities.
3. Data Migration & Transformation Leadership
Lead large-scale data migration initiatives from legacy systems to modern data platforms.
Plan validation, reconciliation, and rollback strategies.
Ensure data integrity and compliance during migrations.
Work closely with engineering teams for system transitions.
4. Data Modeling & Analytics Enablement
Design dimensional data models (star/snowflake schemas).
Build analytics-ready datasets for BI, dashboards, and executive reporting.
Prepare ML/AI-ready datasets in collaboration with data science teams.
5. Integration & Cross-System Data Flow
Integrate data from ERP, CRM, LMS, payment systems, APIs, and third-party vendors.
Design robust data contracts between services.
Ensure reliability and consistency across distributed systems.
6. Governance, Security & Quality
Establish data governance standards and data quality frameworks.
Implement validation, monitoring, and reconciliation mechanisms.
Ensure compliance with data protection and security policies.
7. Leadership & Collaboration
Provide technical mentorship to junior engineers.
Participate in architecture reviews and strategic data planning.
Drive best practices in documentation and coding standards.
Required Skills & Experience
3–5 years of experience in Data Engineering.
Strong hands-on expertise in Databricks.
Extensive experience with Spark / PySpark.
Strong experience in designing scalable ETL/ELT pipelines.
Proven experience in large-scale data migration projects.
Strong SQL proficiency.
Strong Python programming skills for data engineering.
Experience with cloud platforms (AWS preferred).
Strong understanding of Data Lake concepts (S3, Data Lake, object storage).
Experience with workflow orchestration tools (Airflow/Prefect).
Strong knowledge of data modeling principles.
Good to Have
Experience with Data Lake architecture.
Experience with Kafka or streaming pipelines.
Exposure to CI/CD for data pipelines.
Experience supporting AI/ML data workflows.
Familiarity with BI tools (Power BI, Tableau, etc.).
Experience in SaaS or EdTech domain.
Employment Type
Contract to Hire (C2H)
Reporting Structure
Reports to: AVP – Technology & AI
Location
Indiranagar, Bangalore
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say