Data Engineer

Location: DGS India - Bengaluru - Manyata N1 Block

Time Type: Full time

Job Description

Job Description:

Data Engineer
(Azure, Real-Time, Analytics & Data Platforms)

Role Summary

We are seeking a skilled and detail-oriented Data Engineer to design, build, and maintain scalable data systems that power analytics, reporting, and enterprise applications. This role is responsible for implementing robust data pipelines, database solutions, and integration frameworks, ensuring high data quality, reliability, and performance across platforms.

The ideal candidate will bring strong expertise in data architecture, ETL/ELT development, and database engineering, with the ability to work across cross-functional teams to deliver production-grade data solutions.

Core Responsibilities

Data Architecture & Modeling

• Design and maintain enterprise data models supporting SaaS applications and reporting platforms
• Develop logical and physical database architectures
• Design normalized and denormalized schemas based on performance requirements
• Define data standards, naming conventions, and governance frameworks
• Design scalable, multi-tenant data structures

Database Engineering & Performance Optimization

• Develop complex SQL queries, views, stored procedures, and functions
• Analyze query execution plans and implement performance tuning strategies
• Design indexing strategies to optimize data access and performance
• Resolve database bottlenecks, locking, and contention issues
• Implement partitioning and archival strategies for large datasets
• Monitor and tune database performance in production environments

Change Data Capture (CDC) & Incremental Processing

• Design and implement CDC frameworks for near real-time data synchronization
• Build incremental data processing pipelines to optimize performance
• Track inserts, updates, and deletes across source systems
• Implement reconciliation processes to ensure data consistency
• Maintain data lineage and change tracking across systems
• Support event-driven and near real-time data architectures

ETL / ELT Engineering

• Design and develop scalable data ingestion pipelines
• Build automated workflows for integrating APIs, databases, files, and SaaS platforms
• Transform and standardize data across heterogeneous systems
• Implement error handling, retry mechanisms, monitoring, and alerting
• Optimize batch and near real-time processing workflows

API & Data Integration

• Design ingestion frameworks for REST APIs and external data providers
• Build automated synchronization processes across operational systems
• Manage schema evolution and versioning from external APIs
• Implement data validation and quality assurance controls
• Support bidirectional system integrations where required

Data Quality & Governance

• Implement automated data validation and quality frameworks
• Design exception reporting and reconciliation processes
• Establish data quality metrics and stewardship practices
• Monitor completeness, accuracy, and timeliness of datasets
• Support audit, compliance, and regulatory requirements

Reporting & Analytics Infrastructure

• Build backend datasets for Tableau, Power BI, Retool, and custom applications
• Design semantic data layers and reporting models
• Optimize reporting queries for large-scale datasets
• Enable self-service analytics and dashboard capabilities
• Develop reusable data services and APIs for downstream consumers

Data Platform Operations

• Manage production data pipelines and database environments
• Support backup, recovery, and disaster recovery processes
• Monitor pipeline performance, data latency, and system health
• Troubleshoot production data issues and integration failures
• Coordinate schema migrations and data releases across environments

Profile Summary

• Strong experience in SQL and database engineering
• Hands-on expertise with ETL/ELT and data pipeline development
• Good understanding of data modeling and architecture principles
• Experience integrating data from APIs, SaaS platforms, and enterprise systems
• Familiarity with performance tuning and large-scale data systems
• Strong analytical and problem-solving skills
• Ability to work in a highly collaborative, cross-functional environment

Preferred / Good to Have

• Experience with cloud data platforms (Azure preferred)
• Exposure to CDC, real-time streaming, and event-driven architectures
• Familiarity with data observability and monitoring tools
• Experience building data platforms supporting AI/ML use cases
• Knowledge of data governance and compliance frameworks
• Experience with analytics tools (Power BI, Tableau, etc.)

Preferred Media Domain Experience

• Media platforms & workflows – OTT platforms, content lifecycle pipelines, and distribution systems
• Media planning & execution – campaign data flows, ad formats, placements, and delivery pipelines
• Ad-tech ecosystem – programmatic advertising (DSP/SSP), audience targeting, identity resolution, and campaign performance analytics

Location:

DGS India - Bengaluru - Manyata N1 Block

Brand:

Merkle

Time Type:

Full time

Contract Type:

Permanent