Data Engineer-ETL/Snowflake (S04)

Location: Indore, India

Department: ETL & ML

Experience: 8+ years

Key Responsibilities

• Design, develop, and maintain robust ETL/ELT data pipelines for large-scale data processing.
• Develop production-grade data engineering solutions using Python, SQL, Snowflake, and Databricks/PySpark.
• Work with Medallion Architecture (Bronze, Silver, Gold layers) for building scalable and reliable data platforms.
• Design and implement data transformation, cleansing, validation, and enrichment processes.
• Implement Data Completeness, Data Consistency, Data Deduplication, and Data Quality (DCDQ) checks across data pipelines.
• Ensure data accuracy, reliability, consistency, and availability across different stages of the data pipeline.
• Work with Snowflake for data storage, transformation, optimization, and performance tuning.
• Build and optimize data processing pipelines using Databricks and PySpark.
• Develop complex SQL queries and perform data modelling for analytical workloads.
• Integrate data from multiple sources and build scalable data pipelines.
• Troubleshoot pipeline failures, data-quality issues, and performance bottlenecks.
• Collaborate with product, engineering, and business stakeholders to understand requirements and translate them into effective data solutions.
• Independently drive projects from requirement gathering → solution design → development → testing → deployment → production support.
• Follow engineering best practices around code quality, testing, monitoring, documentation, and maintainability.

Required Skills

Data Engineering

• 8+ years of strong hands-on experience in Data Engineering.
• Strong experience designing and developing production-grade ETL/ELT pipelines.
• Strong understanding of data pipelines, data warehousing, data modelling, and data transformation.
• Experience working with Medallion Architecture – Bronze, Silver, and Gold layers.
• Strong understanding of data quality, validation, reconciliation, and DCDQ checks.

Programming

• Strong hands-on Python programming experience.
• Advanced SQL skills.
• Hands-on experience with Apache Spark / PySpark.
• Ability to write clean, reusable, maintainable production code.

Data Platforms

• Strong hands-on experience with Snowflake.
• Experience with Databricks and PySpark.
• Good understanding of modern data warehouse/lakehouse concepts.
• Experience with Delta Lake or similar technologies is a plus.

Data Processing & Orchestration

• Experience building and managing batch data pipelines.
• Experience with data pipeline orchestration tools such as Airflow, Databricks Workflows, or similar.
• Experience with streaming technologies such as Kafka is a plus.

Data Quality

• Strong understanding of data quality frameworks and validation mechanisms.
• Experience implementing checks for:
  • Data completeness
  • Data consistency
  • Data duplication
  • Data accuracy
  • Data reconciliation
• Ability to identify and troubleshoot data-quality issues across different pipeline stages.

Cloud & Other Technologies

• Exposure to AWS, Azure, or GCP is good to have but not mandatory.
• Experience with CI/CD, Docker, Terraform, or Kubernetes is a plus.
• Experience with dbt is a plus.

What We Are Looking For

• Strong Python-first Data Engineering background.
• 8+ years of relevant experience with significant hands-on development.
• Strong Snowflake + ETL/ELT experience.
• Good hands-on experience with Databricks/PySpark.
• Practical understanding of Medallion Architecture.
• Strong focus on data quality and DCDQ checks.
• Ability to independently own data engineering projects from requirements through production.
• Strong problem-solving and analytical skills.
• Good communication and stakeholder-management skills.
• Ability to understand business requirements, ask the right questions, and convert them into scalable technical solutions.