Senior Data Engineer

The Job in short

We are seeking a Senior Data Engineer to lead the design, development, and maintenance of our Data as a Service (DaaS) platform on Azure. Your expertise in data engineering will enable our clients to perform big data analytics, machine learning, and AI workloads on data ingested from multiple sources, ensuring high security and compliance. The ideal candidate will have a strong background in data engineering, cloud data platforms, ETL processes, and data governance. This role requires deep expertise in Azure Synapse, Spark, Databricks, and Delta Lake, along with a solid understanding of data regulations and compliance.

Meet the job.

● Data Platform Architecture & Implementation: Design, implement, and maintain secure and scalable data infrastructure in Azure.

● Data Pipelines & ETL Processing: Develop and manage data pipelines for extracting, transforming, and loading (ETL) data from various sources into the Data Lakehouse using Azure Data Factory and Databricks.

● Data Governance & Compliance: Ensure data consistency, quality, and governance, adhering to regulatory and compliance requirements.

● Data Security & Compliance: Secure data platforms and ensure compliance with security frameworks (e.g., NIST, CIS, ISO 27001).

● Big Data Processing: Implement and optimize big data processing solutions using Azure Synapse, Spark, and Databricks.

● Data Lakehouse Management: Manage the Data Lakehouse environment, including Delta Lake, to support data analytics and machine learning workloads.

● Streaming Data Processing: Develop and maintain secure streaming data pipelines using Azure Event Hubs and other streaming endpoints.

● Data Sharing & Integration: Implement secure data sharing within and outside the organization using Delta Sharing.

● Data Analytics & Visualization: Enable data scientists and analysts by providing high-quality data for dashboards, AI/ML applications, and other data consumers.

How about you

Required Qualifications:

● 5+ years of experience in data engineering, preferably in Azure Synapse, Spark, or Databricks.

● Strong hands-on experience in data pipeline development and ETL processes.

● Expertise in Data Lakehouse concepts and Delta Lake.

● Deep understanding of data domains, regulations, and compliance requirements.

● Experience in data governance and ensuring data quality.

● Strong scripting and automation skills (Python, PowerShell).

● Solid understanding of cloud data platforms and services in Azure.

Preferred Qualifications:

● Azure data certifications (e.g., DP-200, DP-201).

● Knowledge of secure data practices and compliance standards (e.g., NIST, CIS, ISO 27001).

Why Join Us?

● Work with cutting-edge Azure data technologies.

● Influence and drive data engineering best practices in a fast-paced, cloud-first environment.

● Opportunity for continuous learning and professional development.

● Competitive salary, benefits, and flexible work arrangements.

How to Apply: If you are passionate about data engineering, data security, and automation, we encourage you to apply! Submit your resume and a cover letter detailing your relevant experience.