Lead Data Engineer

About the role

Require a full-stack big data Lead Data Engineer for platform Data and build activities, responsible for architecting, designing and developing data services solutions using big-data solutions for the Instrument Foundation & infrastructure.  Requires driving both near-term data to satisfy regulatory requirements while strategically creating the data services platform. The person that fills this position will be a vital part of our team and will be responsible for converting technical requirements into outstanding design.

Responsibilities

• Provide technical solutions for Data services that deliver Instrument Data Foundation Platform.
• Work with senior management, business analysts and project managers for requirements and business rules.
• Background and expertise in architecture and designing bigdata solutions and building robust framework at scale.
• Design and implement APIs, abstractions and integration patterns to solve challenging distributed computing problems.
• Understand and own component security analysis, including code and data flow review. Collaborate with security team to implement and verify secure coding techniques.
• Ensure proper metrics instrumentation in software components, to help facilitate real time and remote troubleshooting/performance monitoring.
• Contribute to efficient development process pipeline by leveraging best-in-class tools.
• Experience in defining technical requirements, conceptual, logical and system architectures.
• Identify and utilize best practices in the industry to maximize efficient and elegant solutions while minimizing cost.
• Should be well versed with version control systems and efficiently manage deployment strategy.
• Execute and co-ordinate testing and bug fixing with the Design, Development and DevOps teams in an Agile environment
• Proactively work with the team to ensure the right quality checks are in place at the various stages of the Data Integration, Data Transformation and Data consumption layers.
• Effectively document use cases, test cases, SRS, UATs, best practices, testing policies and other testing documentation.
• Work with the software development team to identify and fix bugs and functional issues of our Big Data products.
• Develop, Review and tests the software components for adherence to the design requirements and document test results.
• Effectively work in a hybrid environment where legacy ETL and Data Warehouse applications and new big-data applications co-exist
• Work with Infrastructure Engineers and System Administrators as appropriate in designing the big-data infrastructure.
• Support ongoing data management efforts for Development, QA and Production environments
• Utilizes a thorough understanding of available technology, tools, and existing designs.
• Leverage knowledge of industry trends to build best in class technology to provide competitive advantage.
• Acts as expert technical resource to programming staff in the program development, testing, and implementation process.

Required Qualifications

• 12+ years of application development and implementation experience using JAVA or python
• 5+ years of experience with Apache Spark using Scala, Java, Python or Data Frames with Resilient Distributed Datasets (RDDs)
• Experience Hadoop ecosystem tools relevant for real-time and batch data ingestion, processing and provisioning using tools such as Apache Flume, Apache Kafka, Apache Sqoop, Apache Flink, Apache Hive or Apache Storm
• Experience delivering ETL(Talend), data warehouse and data analytics capabilities on big-data architecture such as Hadoop
• Reporting experience, analytics experience or a combination of both
• Knowledge and understanding of DevOps principles
• Experience working with NO SQL databases like Mongo DB
• Experience developing Micro Services, RESTFul API using Spring Boot
• Well versed with code deploy tools like Artifactory and UDeploy.
• Should be able to comprehend business flow while appreciating system design and engineering following distributed modern engineering approach
• 5+ years of Agile experience

Desired Qualifications:

• 5+ years of experience in different aspects of quality assurance and quality control via testing, such as End to End testing, integration testing, functional testing, UAT, usability testing, regression testing and automation for Big Data applications.
• Knowledge and understanding of DevOps principles
• Knowledge  on ETL (any flavor) with Cloud engineering
• Strong debugging and problem solving skills with exceptional analytical skills for to perform use case analysis
• Ability to work effectively in virtual environment where key team members and partners are in various time zones and locations
• Knowledge and understanding of project management methodologies: used in waterfall or Agile development projects
• Excellent verbal, written, and interpersonal communication skills
• Ability to interact effectively and confidently with senior management
• A BS/BA degree or higher in information technology

Posting End Date: 

*Job posting may come down early due to volume of applicants.

Applicants with Disabilities

To request a medical accommodation during the application or interview process, visit Disability Inclusion at Wells Fargo.

Drug and Alcohol Policy

Wells Fargo maintains a drug free workplace.  Please see our Drug and Alcohol Policy to learn more.