Senior Data Platform Engineer

Responsibilities :

• Design and Development of cloud-based microservices to collect data
• Design and Development of REST API to serve Data as a Service
• Design and Development of Data Quality framework to enable automated data quality checks across data pipelines
• Responsible for developing and maintaining CI/CD pipelines
• Maintain and stabilize existing data pipelines and infrastructure to provide timely, quality and accurate data
• Analysis, Performance Improvement, Development, Deployment and Contribute to overall product requirements
• Perform code review and develop best coding practices
• Indulge in brainstorming sessions to help team gain technical knowledge and grow equally
• Define release process improvements along with management of pipeline changes
• Collaborate with peer teams and support each other’s on deliverables

Qualification

• Bachelor’s or Master’s degree in Computer Science or equivalent industry technical skills
• 8+ years of professional development experience
• In-depth knowledge on strong experience in Python programming language and should be an expert Object Oriented Programming
• Strong experience in AWS service - EC2, ECS, Lambda, Step Function, S3, SQS preferred. Experience in other AWS services are added advantage
• Expertise in both SQL and NoSQL databases.
• Experience in Postgres, Redshift, MongoDB and DynamoDB are preferred
• Experience in Linux and Shell scripting preferred
• Experience in Docker, Kubernetes, or MicroService preferred
• Experience in Airflow and Jenkins preferred
• Experience in FAST API, Flask Framework, SQLalchemy are preferred
• Expertise in version control tools Git, Bitbucket preferred
• Experience in creating and implementing a strong data quality system that checks data accuracy at every step, backed by thorough monitoring and observability features