Senior Data Integration Engineer

Main Purpose:

As a highly experienced Data Engineer in our Digital Transformation Team, you will play a crucial role in developing advanced data abstraction layers to facilitate seamless integration between legacy trading systems (built on WPF, .Net (C#, VB), Oracle, Java) and our modern AWS-based data and analytics stack. Your expertise will drive the successful transformation of our data infrastructure, furthering the
use of advanced analytics and data-driven decision-making across the company. You will manage a specialist team of engineers and scientists in order to achieve your objectives.

Knowledge Skills and Abilities, Key Responsibilities:

Abilities, Experience and Qualifications

• Bachelor's degree in Software Engineering, Computer Science or a related field.
• Proven experience in enterprise scale data engineering.
• Expertise in working with legacy systems (Windows, Oracle, Java) and integrating them with
  cloud platforms, preferably AWS (S3, RDS, Lambda, Glue, Redshift).
• Must be able to implement different levels and approaches of CDC from near-real-time, to intraday
  to inter-day.
• Implement data cleansing, data quality, data normalisation and data de-duplication.
• Must be capable of working with multiple data structures, including but not limited to JSON, XML.
• Exposure to data mapping/design tools such as Altova suite (MapForce), Schematron, ERWIN.
• Worked with Apache Spark, Kafka, Hive, Hbase, Flume, Sqoop.
• Proven experience with Object Relational Mapping systems (ORMs) and SOAP / REST APIs.
• Expert level programming skills in Python and Java.
• Implement highly threaded and concurrent Python and Java.
• Expert level knowledge with SQL and NoSQL databases, data modelling, and ETL processes.
• Knowledge of data warehousing principles and architectures.
• Experience with data governance and security.
• Experience in a Commodity Trading, Financial Markets or Logistics Management environment is desirable.

Key Relationships and Department Overview:

• You will report to the head of the Strategic Systems Integration Team who reports to CIO and Chief Data Scientist.
• The Digital Transformation Team is a joint venture between Trading Technology and the Data Science and Engineering Team.

Equal Opportunity Employer

We are an Equal Opportunity Employer and take pride in a diverse workforce. We do not discriminate in recruitment, hiring, training, promotion or other employment practices for reasons of race, colour, religion, gender, sexual orientation, national origin, age, marital or veteran status, medical condition or handicap, disability, or any other legally protected status.