Principal Software Engineer/Developer

Position Description: 

Builds and optimizes big data pipelines, software architecture, and data sets using Azkaban, Luigi, and Apache Airflow. Builds Cloud optimized data solutions and software applications (for software and mobile development) using Amazon Web Services (AWS) Cloud services -- EC2, EMR, Snowflake, and Elastic-Search. Performs Object-Oriented (OO) design and scripts using programming languages (Python, Java, C++, and Scala). Orchestrates data solutions in Cloud using Cloud services -- AWS EMR, EC2, and Jupyter notebooks. Uses business knowledge to translate the vision for divisional initiatives into business solutions by developing complex or multiple software applications and conducting studies of alternatives. Analyzes and recommends changes in project development policies, procedures, standards, and strategies to development experts and management. 

Primary Responsibilities: 

• Participates in architecture design teams. 

• Defines and implements application-level architecture. 

• Develops applications on complex projects, components, and subsystems for the division. 

• Recommends development testing tools and methodologies and reviews and validates test plans. 

• Responsible for QA readiness of software deliverables. 

• Develops comprehensive documentation for multiple applications or subsystems. 

• Establishes full project life cycle plans for complex projects across multiple platforms. 

• Responsible for meeting project goals on-time and on-budget. 

• Advises on risk assessment and risk management strategies for projects. 

• Plans and coordinates project schedules and assignments for multiple projects. 

• Acts as a primary liaison for business units to resolve various project/technology issues. 

• Provides technology solutions to daily issues and technical evaluation estimates on technology initiatives. 

• Advises senior management on technical strategy. 

• Mentors junior team members. 

• Performs independent and complex technical and functional analysis for multiple projects sup-porting several divisional initiatives.  

• Develops original and creative technical solutions to on-going development efforts. 

Education and Experience: 

Bachelor’s degree (or foreign education equivalent) in Computer Science, Engineering, Information Technology, Information Systems, Mathematics, Physics, or a closely related field and five (5) years of experience as a Principal Software Engineer/Developer (or closely related occupation) performing DataMart requirement gathering by designing data pipelines using Snaplogic, Snowflake, and Control-M in a financial services environment.  

Or, alternatively, Master’s degree (or foreign education equivalent) in Computer Science, Engineering, Information Technology, Information Systems, Mathematics, Physics, or a closely related field and three (3) years of experience as a Principal Software Engineer/Developer (or closely related occupation) performing DataMart requirement gathering by designing data pipelines using Snaplogic, Snowflake, and Control-M in a financial services environment. 

Skills and Knowledge: 

Candidate must also possess: 

• Demonstrated Expertise (“DE”) creating and maintaining Extract, Transform, Load/Extract, Load, Transform (ETL/ELT), data ingestion, batch applications, replication, and streaming solutions for data pipelines; building data lake on the Cloud using structured, semi-structured, and unstructured data; and providing actionable insights into customer acquisition, operational efficiency, and Key Performance Indicators (KPIs) by building analytics tools. 

• DE developing, architecting, and designing highly available, secure, scalable fault tolerant applications required for optimal extraction, transformation, and loading of data from data sources using SQL, Python, AWS/Azure Cloud, and big data technologies (Snowflake and Apache Spark). 

• DE supporting Snowflake analytics platform by resource tagging (grouping and monitoring resource consumption and budgeting); collaborating with DBA teams to tag Snowflake compute and storage resources; estimating usage costs and identifying cost control mechanisms using SQL queries on Snowflake Metadata views; and supporting Power BI dashboards with interfacing database views showing resource usage and consumption. 

• DE advising senior management on technical strategy; monitoring tasks and deliverables to complete projects on schedule; building and maintaining day-to-day internal and external relationships to provide high quality service to customers; building and maintaining DevOps practices according to CI/CD pipelines using Jenkins core, GIT, JIRA, and Service Now; and participating and mentoring team during code review and design. 

#PE1M2