Data Engineering
Location: Bengaluru, India
Department: Projects & Delivery
Experience: 2-3 years
Skills: Data Engineering, Ai
- Design, develop, and deploy scalable data pipelines using Databricks (PySpark, Spark SQL), Azure Data Factory, and other Azure data services.
- Implement ETL/ELT processes to ingest, transform, and load data from diverse sources into data lakes and data warehouses.
- Optimize and tune data pipelines for performance, scalability, and cost efficiency.
- Write and optimize complex SQL queries for data extraction, transformation, and analysis.
- Utilize PySpark for large-scale data processing and advanced analytics.
- Implement data partitioning, bucketing, and indexing strategies for efficient data retrieval.
- Integrate data from multiple sources, including structured, semi-structured, and unstructured formats.
- Work with APIs, streaming data, and batch processing to ensure seamless data integration.
- Apply data governance practices to maintain data quality, consistency, and security.
- Monitor and troubleshoot data pipelines to ensure accuracy, availability, and reliability.
- Partner with data scientists, analysts, and other stakeholders to understand requirements and deliver solutions.
- Work closely with DevOps teams to deploy and monitor data pipelines in production environments.
- Document data pipelines, workflows, and processes for knowledge sharing and future reference.
- Maintain up-to-date documentation on data architecture and data models.
- Databricks: Hands-on experience with Databricks for data processing, analytics, and Serverless SQL Warehouse, Unity Catalog, Lakehouse, and Medallion Architecture.
- Azure Data Services: Proficiency in Azure Data Factory, Azure Data Lake Storage, Azure SQL Database, and Key Vault.
- SQL: Strong expertise in writing and optimizing complex SQL queries.
- PySpark: Experience in using PySpark for data processing and transformation.
- ETL/ELT: Solid understanding of ETL/ELT processes and tools.
- Data Modeling: Knowledge of data modeling techniques and best practices.
- Data Governance: Familiarity with data governance, data quality, and data security practices.
- Communication: Ability to clearly articulate technical ideas to both technical and non-technical audiences, especially in client-facing discussions.
- Problem Solving: Strong analytical mindset with the ability to structure ambiguous problems and deliver actionable AI solutions quickly.
- Ownership & Bias for Action: Self-driven, accountable, and proactive in driving outcomes end to end.
- Collaboration: Empathy, active listening, and effective conflict resolution in cross-functional teams.
- Experience with Azure DevOps for CI/CD pipelines.
- Knowledge of Delta Live Tables (DLT) in Databricks.
- Competitive compensation and ESOPs
- Opportunity to work on cutting-edge Generative AI and multimodal systems
- High visibility across teams and leadership
- Support for continuous learning, certifications, and conference participation
There are more than 50,000 engineering jobs:
Subscribe to membership and unlock all jobs
Engineering Jobs
60,000+ jobs from 4,500+ well-funded companies
Updated Daily
New jobs are added every day as companies post them
Refined Search
Use filters like skill, location, etc to narrow results
Become a member
🥳🥳🥳 452 happy customers and counting...
Overall, over 80% of customers chose to renew their subscriptions after the initial sign-up.
To try it out
For active job seekers
For those who are passive looking
Cancel anytime
Frequently Asked Questions
- We prioritize job seekers as our customers, unlike bigger job sites, by charging a small fee to provide them with curated access to the best companies and up-to-date jobs. This focus allows us to deliver a more personalized and effective job search experience.
- We've got over 200,000 jobs from 15,000+ vetted companies. No fake or sleazy jobs here!
- We aggregate jobs from 15,000+ companies' career pages, so you can be sure that you're getting the most up-to-date and relevant jobs.
- We're the only job board *for* software engineers, *by* software engineers… in case you needed a reminder! We add thousands of new jobs daily and offer powerful search filters just for you. 🛠️
- Every single hour! We add 2,000-3,000 new jobs daily, so you'll always have fresh opportunities. 🚀
- Typically, job searches take 3-6 months. EchoJobs helps you spend more time applying and less time hunting. 🎯
- Check daily! We're always updating with new jobs. Set up job alerts for even quicker access. 📅
What Fellow Engineers Say
