Software Engineer - Data Engineering

RESPONSIBILITIES

• Hands on developer to work on data modeling, code, and test iteratively on technical solutions for monitoring and analytics requirements.
• Implement ETL processes, UI and dashboards for appropriate trends and alerts in BI tools (Superset, Spotfire, QuickSight, etc.) for monitoring.
• Plan, develop, execute and maintain automated unit, functional test cases; develop modular, robust, and maintainable automation scripts.
• Implement automation in the areas of Cloud resource provisioning, unit testing, code coverage, static code analysis and vulnerability scanning with CI/CD process.
• Must have strong analytical and technical skills with passion to deep dive on data  in troubleshooting , devise techniques for problem resolution.
• Promote a culture of best practices with peer code reviews and extreme ownership for continuous incremental delivery.
• Maintain and document standards, best practices, and design/architecture patterns.

QUALIFICATIONS AND SKILLS

• 2-4 years of experience with ETL automation, SQL and BI Tools experience is a must (Superset, TIBCO Spotfire, etc.)
• Programming experience Python/Java and desire to write and review code 70% of time. 
• Familiarity with relational and big data technology stack such as Postgres, Hadoop and columnar Storage formats Parquet
• Familiarity of Linux platforms with knowledge of shell scripting.
• Hands-on experience with Spark, Spark SQL, Spark on K8s, M/R Framework and Big Data Technology Stack.
• Experience developing cloud-hosted (AWS), containerized applications and services is plus.
• AWS Components: EKS, EMR, EC2, S3 storage, Lambda, Relational Database Service, Simple Notification Service (SNS), Elasticache, etc.
• Experience with source code control tools. Gitlab experience is a plus.
• Knowledge of networking and protocols: HTTP/HTTPS, TLS/SSL/certificates, TCP/IP is plus
• Knowledge of CI/CD processes, testing frameworks,  practices and tools (Jenkins, jUnit, Terraform, JFrog, Jacoco, SonarQube, etc.) is a plus