Data Solution Engineering Senior Specialist - (Azure)

Employment Type

Permanent

Closing Date

30 July 2024 11:59pm

Job Title

Data Solution Engineering Senior Specialist - (Azure)

Job Summary

Being part of Data Engineering means you'll be part of a team that focuses on extending our network superiority to enable the continued execution of our digital strategy. With us, you'll be working with world-leading technology and change the way we do IT to ensure business needs drive priorities, accelerating our digitisation programme.

We are seeking a highly skilled Senior Data Engineer with expertise in Spark, Python, Scala. The successful candidate will be responsible for will be responsible for designing, developing, and maintaining data pipelines using Spark, Python, Scala, and related technologies. The Senior Data Engineer will also be responsible for ensuring data quality, data security, and optimal performance of the data pipelines.Any new engineer would be mostly into developing reusable data processing and storage frameworks that can be used across data platform. The Senior Data Engineer will also be responsible for mentoring and providing technical guidance to junior team members.

Job Description

Who We Are

Telstra is Australia’s leading telecommunications and technology company spanning over a century with a footprint in over 20+ countries. In India, we’re building a platform for innovative delivery and engagement that will strengthen our position as an industry leader. We’ve grown quickly since our inception in 2019, now with offices in Pune, Hyderabad and Bangalore.

Focus of the Role

Senior Data Engineer role is to coordinate, and execute all activities related to the requirements interpretation, design, and implementation of Data Analytics applications. This individual will apply proven industry and technology experience as well as communication skills, problem-solving skills, and knowledge of best practices to issues related to design, development, and deployment of mission-critical systems with a focus on quality application development and delivery.

What We Offer

Flex appeal: choose where, when and how you work

Performance-related pay

Access to thousands of learning programs so you can level-up

Global presence across 22 countries; opportunities to work where we do business.

Up to 26 weeks maternity leave provided to the birth mother with benefits for all child births

Employees are entitled to 12 paid holidays per calendar year                      

Eligible employees are entitled to 12 days of paid sick / casual leave per calendar year

Relocation support options across India, from junior to senior positions within the company

Receive insurance benefits such as medical, accidental and life insurances

What You’ll Do

Product & Technology - Data & AI team develops and deploys bespoke data-driven solutions, improving Telstra's business and customer experience.

This role is key to the success of the Data Engineering capability at Telstra and will be responsible and accountable for the following:

•            Lead the design, development, and maintenance of data pipelines using Spark, Java/python/scala and related technologies.

•            Design, develop and maintain Java based applications.

•            Working knowledge on cloud data platforms like AWS, (Azure better to have)

•            Work with high volume data and ensure data quality and accuracy.

•            Implement data security and privacy best practices to protect sensitive data.

•            Collaborate with data scientists and business stakeholders to understand data needs and requirements.

•            Prototype solutions for data-driven applications.

•            Develop and maintain documentation on data pipeline architecture, data models, and data workflows.

•            Develop and maintain data systems that facilitate efficient data processing.

•            Mentor and provide technical guidance to junior team members.

•            Monitor and troubleshoot data pipelines to ensure they are performing optimally.

•            Stay up to date with the latest developments in AWS, Spark, Java and related technologies and apply them to solve business problems.

•            Optimize data pipelines for cost and performance.

•            Collaborate with stakeholders to understand business requirements.

•            Align data systems with organizational goals.

•            Automate data processing tasks and workflows to reduce manual intervention.

•            Ability to work in Agile Feature teams.

•            Provide training and educate other team members around core capabilities and helps them deliver high quality solutions and deliverables/documentation.

•            Self-Motivator to perform Design / Develop user requirements, test and deploy the changes into production.

•            Work closely with cross-functional teams on data-related projects

•            Knowledge on AI & GEN AI are better to have

About You

•            Hands-on experience in the following on Spark Core, Spark SQL, SQL/Hive/Impala.

•            Hands on experience on Azure/AWS, Azure Databricks, ADLS, SQL Synapse, Azure Functions, AWS Lambda etc.

•            Experience of working on File formats (Parquet/ORC/AVRO/Delta/Hudi etc.)

•            Experience with high volume data processing and data streaming technologies

•            Experience of using Orchestration tools like Control-m, Azure Data Factory, Airflow, Luigi to schedule jobs.

•            Demonstrated experience leading data engineering projects and mentoring junior team members.

•            Data engineer with expertise of working on Azure Cloud using Databricks, Kinesis/ AzureEvent hub Flume/Kafka/Spark streaming, Azure Data Factory.

•            Strong experience in data modelling, schema design, and ETL/ELT development using SQL and related technologies

•            Familiarity with data security and privacy best practices

•            Good exposure on TDD (Test driven development)

•            Exposure on using CI tools like Git, Bitbucket, Github, Gitlab, Azure DevOps

•            Exposure on using CD tools like Jenkins, Bamboo, Azure DevOps

•            Exposure on Observability tools like Azure Monitor, Graphana etc

•            Prior experience in building or working in team building reusable frameworks.

•            Good understanding of Data Architecture and design principles. (Delta/Kappa/Lambda architecture)

•            Exposure to Code Quality - Static and Dynamic code scans

•            Explore ways to improve data quality and consistency.

•            Ensure seamless integration of data systems.

•            Experience in working on Azure SQL Data warehouse (Synapse) and Azure analysis service or Redshift or Synapse

•            Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS and/or knowledge on NoSQL platforms.

•            Should be able to provide scalable and robust solution architecture depending on the business needs.

•            Propose best practices/standards

If you are looking to take the next step in your career, we want to hear from you!

When you join our team, you become part of a welcoming and inclusive community where everyone is respected, valued and celebrated. We actively seek individuals from various backgrounds, ethnicities, genders and abilities because we know that diversity not only strengthens our team but also enriches our work. We have zero tolerance for harassment of any kind, and we prioritise creating a workplace culture where everyone is safe and can thrive. 

We work flexibly at Telstra. Talk to us about what flexibility means to you. When you apply, you can share your pronouns and / or any reasonable adjustments needed to take part equitably during the recruitment process.