Senior Data Engineer

Location: IN KA Bengaluru, IN MH Mumbai Quantum, IN KL Trivandrum, IN MH Mumbai ISD, IN MH Mumbai Eureka

Time Type: Full time

Job Description

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.


If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Job Description:

Role: Senior Data Engineer - Azure 

Experience Level: 4 to 7 Years 

Work location: Mumbai, Bangalore & Trivandrum 

Notice Period:0-15 days(Only candidate serving NP)

Role & Responsibilities: 

● Support the planning and implementation of data flow design, provide sizing and configuration assistance.

 ● Assisting the Solution Architects on programs for transformations and modernizations of data solutions using Azure cloud data technologies. 

● Write production quality code to deploy ML workflows in a distributed environment. 

● Design and Build Azure Data Pipelines and Data StreamsDesign and build a hybrid data solution using a combination of on-premise and in-cloud services. 

● Diagnose and troubleshoot complex distributed systems problems and develop solutions with a significant impact on our massive scale. 

● Build tools to ingest and process terabyte-scale data on a daily basis. 

● Communicate with a wide set of teams, including product owners, business analysts, data and enterprise architects, platform engineers, and cloud vendors, etc. 

● Compilation of detailed design documents, technical specification documents, and test plans as required.

● Independently manage development tasks as per technical specification documents, prod deployment, post-prod support, etc 

● Managing support to Unit Testing, SIT & UAT. 

● Coordinate with business and support teams on testing and migration activities.

Skills expectation: 

● A strong hands-on knowledge of Spark/Pyspark. 

● Expertise in writing SQL queries and performance optimization. 

● Strong hands-on experience with a programming language like Python(preferred), java, or scala.

● Prior experience with working on cloud platforms like Azure(preferred) , AWS or GCP

● Prior experience of ADF is a plus. 

● DataBricks experience is good to have. 

● Data Warehouse: Snowflake or Synapse 

● Design Dimensional Models for Data Warehouse and Data Marts. 

● Knowledge of version control systems and build automation tools like Azure DevOps and/or MLOps

● Experience with operating system internals, file systems, disk/storage technologies, and storage Protocol.