Data Engineer Senior (Mandatory-Glue, Athena, Python, ETL) -6 years to 12 years - Bangalore, Pune/Indore

Are you ready to unleash your full potential?  We’re looking for people who are passionate about payments to chart Worldpay’s path to being the largest and most-loved payments company in the world.

About the team:      

We are a part of WorldPay, who invented payment facilitation, enabling technology companies to provide integrated payment solutions, significantly expanding access to modern payment methods to SMBs, and rejuvenating end-user experience.    

Data Engineering Team focusses on maintaining the Data Warehouse, Reporting and Analytical aspects of Payrix. Data Engineers will be directly involved in creating and maintaining the Data Pipelines to ingest, transform, analyze and present the data which exists in multiple SQL, NoSql and file based DBs.

What will you be doing

• Ingesting and managing data in AWS-based data stores including S3, Hudi and RDS.
• Batch ingestion using big data tools like Spark with Python.
• Strong experience in Athena, Glue, Redshift.
• Strong experience with deployment using Serverless framework.
• Interface with other technology teams to extract, transform, and load data from relational data sources such as PostgreSQL, using SQL and AWS big data technologies.
• Experience in deploying applications using Docker and/or Kubernetes.
• Conduct tasks related to feasibility studies, time estimates, risk technology, applications development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas.
• Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users.
•Collaborate with other teams to recognize and help adopt best practices in reporting and analysis: data integrity/quality, pipelines analysis, validation, and documentation.
•Help continually improve ongoing data ingestion, engineering, automating or simplifying self-service support for our partners.

What you bring

• Good experience with PostgreSQL and SQL queries.
•  Hands-on experience working with Spark based ETL design, implementation, and maintenance.
• Experience with Python is a must. Scala and Java experience is a plus.
• DevOps and CICD development experience is a plus.
• Experience with AWS big data platform
            AWS S3, Glue, EMR, Lambda, Redshift, Athena, RDS.
            AWS CLI, SDK, IAM
•Experience with web application, API development and use of JSON/XML/Hypermedia data formats.
•Good communication skills.
•Highly motivated and open-minded with an attitude of learning and solving tough tech problems.
•Experience working in an agile environment with fast-paced changing requirements.
• A team player and a problem solver with a positive mindset.

What we offer