Big Data Engineer - (MTS 2 / MTS 3)

Responsibilities

• Build and test Cloud-based applications for new and existing backend systems to help facilitate development teams to migrate to the cloud with an emphasis on quality, best-practice coding standards, and cost-effectiveness.
• Build platform reusable code and components that could be used by multiple project teams.
• Provide cloud integration development support to various project teams.
• Leverage modern design patterns and architectural principles to build platform reusable code and components that can be used across projects and teams.
• Write both unit and integration tests, and develop automation tools for daily tasks.
• Support product owner in defining future stories and tech lead in defining technical requirements for new initiatives.
• Build rapid technical prototypes for early customer validation of new technologies.
• Collaborate effectively with Data Science to understand, translate, and integrate methodologies into engineering build pipelines.
• Collaborate with cross-functional teams and stakeholders to align development objectives with broader business goals

Key Skills (Domain Expertise)

• 3-5 years of hands-on software development with a bachelor’s degree in computer science, engineering.
• Must have strong cloud Implementation expertise in cloud architecture.
• Must have very good knowledge of storage, network, Compute services. Have sound knowledge in multi-zone, region-based designs.
• Must have the ability to provide solutions utilizing best practices for resilience, scalability, cloud optimization and security.

Technical Skills

• Experience in software development using programming languages & tools/services: Java or Python or Scala and strong in SQL.
• Experience in big data processing tools/languages using Spark Scala.
• Experience with orchestration tools: Apache Airflow or similar tools.
• Strong knowledge on Unix/Linux OS, commands, shell scripting, python, JSON, YAML.
• Agile scrum experience in application development is required. 
• Strong knowledge  in AWS S3, PostgreSQL or MySQL.
• Deployment and automation: Terraform, Cloud Formation would be a plus.
• Strong knowledge  in  Compute: EC2, EMR, AWS Lambda.
• Strong knowledge in Gitlab /Bitbucket.
• AWS Certification is a plus.

Mindset and Attributes

• Strong verbal/written communication and interpersonal skills.
• Must have strong analytical and technical skills in troubleshooting and problem resolution.