Trainee Data Engineer

Key Responsibilities

• Collaborate with customers to gather requirements and to understand their business processes.
• Create and maintain optimal data pipeline architecture.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, SQL and ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Use tools to deploy and monitor the performance of the systems in production.

Required Knowledge

• Some hands-on experience with programming languages such as SQL or Python.
• Base-level understanding of Cloud Data Platform Services and Solutions
• Neural networks, computer vision, language processing, big data processing, or other machine learning sub-domains
• Software configuration management environments and tools such as JIRA, Git, Jenkins, TFS, Shell, PowerShell, Bitbucket, etc 
• Any knowledge of Cloud Computing, AI/Machine Learning, SQL Database Skills, Data Lake Software Skills, Data Modeling Skills, Big Data Software Tools, and Technical Troubleshooting is highly desirable.