Senior Data Engineer

Oritain, a renowned global data company, is on the lookout for a dynamic Senior Data Engineer to join our talented team. We focus on delivering impactful data insights and scientific traceability to top brands worldwide. Your expertise will contribute to our mission of fostering trust and integrity in supply chains by providing clarity on product origins while tackling critical challenges such as deforestation and modern slavery.

In this role, based in our new Auckland CBD offices you will play a vital role in designing and implementing data solutions and infrastructure that will enable effective collection, storage, processing, and analysis of large-scale datasets. You will work closely with cross-functional teams to understand business requirements and translate them into scalable data pipelines and architectures. You will also be responsible for ensuring data quality, performance, and security throughout the data lifecycle.

This position offers a unique chance to be part of a fast-growing company with a global impact and be a part of a team that leverages cutting-edge technologies to drive meaningful change.

Responsibilities:

• Work closely with data scientists, and software developers to fulfill company requirements in data tooling and cleaning to ensure accurate data reporting.
• Use Python, C#, JavaScript, Terraform, and Azure and best practices of ETL/ELT to automate new and existing data pipelines.
• Drive production changes of the Data platform via change control and code review on GitHub.
• Write and troubleshoot GitHub Actions (CI pipelines).
• Integrate Data Platform with external data providers to enrich core business data such as Google Cloud, Microsoft Bing and Open Supply Hub.
• Propose modifications to the change management team via the Architectural Decision Record (ADR) process.
• Maintain comprehensive documentation and manuals.
• Lead data migration projects via planning and development.
• Manage data backups and test disaster recovery plans.
• Monitor and troubleshoot Cloud infrastructure, services, and applications.
• Collect, analyse, and debug application logs.
• Write unit tests for developments of the Data Platform.
• Find data schema and data entry issues, and work with Software Engineers to improve overall ingestion/egestion flows of the business.
• Monitor the cost of infrastructure and search for cost-effective alternatives when required.

• Bachelor's or Master's degree in Computer Science or a related field.
• Minimum of 5+ years of technical engineering experience in a commercial capacity with proficiency working with coding languages including but not limited to, C#, JavaScript, or Python.
• Experience with Agile methodologies or test-driven development (TDD).
• 3+ years experience administrating or building solutions on a major cloud provider (ideally Azure).
• Strong background in Infrastructure as Code approach (Terraform).
• Experience with configuration management and automation practices.
• Experience with data cleaning, migration processes, and pipeline development (ETL/ELT).
• Familiarity with database technologies such as MSSQL, MySQL, Postgres, Cosmos DB, Dynamo DB, Mongo DB, and Snowflake.
• Experience with search engine technologies, including Azure Cognitive Search and Elastic Search.
• Familiarity with BI/dashboarding technologies such as Power BI, Amazon QuickSight, Kibana, or Grafana.
• Strong troubleshooting and analytical skills.
• Familiarity with DevOps practices, particularly containerisation and continuous integration/continuous deployment (CI/CD).
• Experience with Git version control system.
• Excellent problem-solving and debugging abilities.
• Strong communication and teamwork capabilities.
• Must be legally eligible to work in New Zealand.