Software Engineer- Data Engineer

Location: Pune

Department: Data Platform

Job Title: Data Engineer Role

Summary We are looking for a highly skilled Data Ingestion Engineer to design, build, and maintain scalable, reliable, and secure data ingestion pipelines on AWS. The ideal candidate will have deep experience ingesting data from diverse sources (batch and streaming) into a centralized data platform while ensuring data quality, observability, and performance. 

Key Responsibilities: Data Ingestion & Pipeline Development

• Design and implement batch and real-time data ingestion pipelines using AWS-native services

• Ingest data from databases, APIs, files, event streams, and third-party SaaS systems

• Build fault-tolerant, scalable, and reusable ingestion frameworks

• Handle CDC (Change Data Capture) from transactional systems

AWS & Cloud Engineering 

• Leverage AWS services such as:

• S3, Glue, Lambda, Step Functions 

• Kinesis (Data Streams / Firehose)

• SNS/SQS

• EMR / Spark

• Optimize data ingestion for cost, performance, and scalability

• Implement secure ingestion using IAM, KMS, VPC endpoints, Secrets Manager

Data Quality, Monitoring & Reliability

• Implement data validation, schema enforcement, and error handling

• Build monitoring and alerting using CloudWatch, logs, and metrics

• Ensure idempotency, replayability, and recoverability of ingestion pipelines

• Manage late-arriving data and schema evolution 

DevOps & Automation

• Build CI/CD pipelines for ingestion jobs using Git, Terraform / CloudFormation

• Containerize ingestion services using Docker (ECS/EKS is a plus)

• Follow Infrastructure as Code (IaC) best practices

Collaboration & Documentation

• Work closely with analytics, platform, and downstream data consumers

• Create clear documentation for ingestion patterns, SLAs, and runbooks

• Participate in design reviews and mentor junior engineers

Required Skills & Qualifications:

Core Skills

• 5+ years in Data Engineering with strong focus on data ingestion and cloud-native pipelines 

• Strong expertise in AWS data services

• Proficiency in Python (mandatory); SQL required

• Experience with Spark / PySpark for large-scale ingestion

• Solid understanding of data formats (JSON, Avro, Parquet, CSV) 

Ingestion Patterns

• Batch ingestion (file-based, database extracts)

• Streaming ingestion (event-driven architectures)

• API-based ingestion (REST, OAuth, rate limits)

• CDC tools or patterns (DMS, Debezium, custom CDC)

Data Architecture

• Experience working with data lakes (S3) and lakehouse architectures

• Familiarity with Medallion architecture (Bronze / Silver / Gold)

• Understanding of partitioning, compaction, and metadata management 

Nice-to-Have

• Experience with Snowflake, Redshift, or Databricks

• Knowledge of dbt and downstream transformations

• Banking / FinTech domain experience

• Exposure to data governance, lineage, and catalog tools

• Experience with DBT Cloud, Airflow or AWS Managed Workflows (MWAA)

Soft Skills

• Strong problem-solving and debugging skills

• Ability to work independently in complex environments

• Excellent communication with both technical and non-technical teams

• Ownership mindset for production data pipelines 

Example KPIs for Success

• Reduction in data ingestion failures and latency

• Improved data freshness and SLA adherence

• Scalable ingestion frameworks adopted across teams

• Well-documented, observable, and maintainable pipelines

About Convera

Convera is the largest non-bank B2B cross-border payments company in the world. Formerly Western Union Business Solutions, we leverage decades of industry expertise and technology-led payment solutions to deliver smarter money movements to our customers – helping them capture more value with every transaction. Convera serves more than 30,000 customers ranging from small business owners to enterprise treasurers to educational institutions to financial institutions to law firms to NGOs.

Our teams care deeply about the value we bring to our customers which makes Convera a rewarding place to work. This is an exciting time for our organization as we build our team with growth-minded, results-oriented people who are looking to move fast in an innovative environment.

As a truly global company with employees in over 20 countries, we are passionate about diversity; we seek and celebrate people from different backgrounds, lifestyles, and unique points of view. We want to work with the best people and ensure we foster a culture of inclusion and belonging.

We offer an abundance of competitive perks and benefits including: 

• Competitive salary
• Opportunity to earn an annual bonus.
• Great career growth and development opportunities in a global organization
• A flexible approach to work

There are plenty of amazing opportunities at Convera for talented, creative problem solvers who never settle for good enough and are looking to transform Business to Business payments. Apply now if you’re ready to unleash your potential.

#LI-KP1