Principal Data Engineer

Position Description: 

Designs, develops, and implements data integration solutions and software applications using Java, Python, and JSON. Delivers data-centric data solutions, business metadata, data lineage mappings, and data transformation logic for customers using AWS. Analyzes and manipulates data sets with Oracle, Snowflake, and MS SQL server. Performs Continuous Integration and Deployment (CI/CD) to automate pipelines. Develops software applications and data solutions in a Cloud environment -- AWS and Azure. Collaborates with Data Architects, Business Analysts, Data Modelers and Solution Architects to integrate data with specific development activities. Adheres to Agile best-practices to enhance firm efficiency. 

Primary Responsibilities: 

• Develops complex or multiple software applications and conducts studies of alternatives to translate divisional initiatives into business solutions.  

• Analyzes and recommends changes in project development policies, procedures, standards, and strategies to development experts and management.  

• Participates in architecture design teams.  

• Defines and implements application-level architecture.  

• Develops applications, components, and subsystems to support division-wide complex projects.  

• Recommends development testing tools and methodologies.  

• Reviews and validates test plans, and ensures QA readiness of software deliverables.  

• Develops comprehensive documentation for multiple applications or subsystems.  

• Establishes full project life cycle plans for complex projects across multiple platforms.  

• Advises on risk assessment and risk management strategies for projects.  

• Plans and coordinates project schedules and assignments for multiple projects.  

• Provides technology solutions to daily issues and estimates technical evaluation requirements for technology initiatives. 

Education and Experience: 

Bachelor’s degree (or foreign education equivalent) in Digital Sciences, Computer Science, Engineering, Information Technology, Information Systems, Mathematics, or a closely related field and five (5) years of experience as a Principal Data Engineer (or closely related occupation) designing, developing, and ensuring the reliability of data services in a financial brokerage, retirement, or institutional environment. 

Or, alternatively, Master’s degree (or foreign education equivalent) in Digital Sciences, Computer Science, Engineering, Information Technology, Information Systems, Mathematics, or a closely related field and three (3) years of experience as a Principal Data Engineer (or closely related occupation) designing, developing, and ensuring the reliability of data services in a financial brokerage, retirement, or institutional environment. 

Skills and Knowledge: 

Candidate must also possess: 

• Demonstrated Expertise (“DE”) designing and implementing enterprise scale data management SaaS vendor platforms to implement financial catalogs and lineages using Collibra and Alation on AWS. 

• DE designing and implementing Collibra custom workflows using BPMN, Groovy, APIs, Jenkins Core, and Datadog to enhance data governance and automation capabilities in a financial investments domain. 

• DE designing and developing data processing pipelines using Java, Python, Snaplogic, Spark, Hive, and Jupyterhub to enable data integration, transformation, and analytics in a financial domain. 

• DE building enterprise frameworks to automate and streamline data platform enablement using CloudFormation, IAM, Sagemaker, EMR, DynamoDB, and CloudWatch in a financial domain. 

#PE1M2