Data Engineer Senior Consultant

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

Visa Consulting & Analytics (VCA) team is a key part of the Global Solutions organization, a high-performing team of data engineer, scientists, data analysts and statisticians helping major organizations adapt and evolve to meet the changes taking place in technology, finance, and commerce, with cutting-edge, creative and advanced analytic solutions.

Visa is looking for Data Engineer Senior Consultant, who will be the responsible for leading the Data Engineering engagements with our partners and supporting end-to-end delivery. 

He/She will be a member of VCA Data Engineering team in CEMEA region. The position will be based in Visa’s Bangalore office.

The individual will be accountable for supporting and driving the design, development and implementation of analytics-driven strategies as well as high-impact solutions for Visa clients.

RESPONSIBILITIES

• Strong technology and leadership background building enterprise scale applications using Python/ Pandas /Scala, RDBMS. Machine Learning, Data Engineering (Hadoop, Hive, Spark), NoSQL, REST APIs, Kafka, and Data Pipelines desirable.
• Design and deploy data and pipeline management frameworks built on top of open-source components, including Hadoop, Hive, Spark, HBase, Kafka streaming and other Big Data technologies.
• Lead code reviews, ensuring adherence to coding standards, and promoting clean, efficient code within the team.
• Experience with Continuous Integration and Automated Test tools such as Jenkins, Artifactory, Git, Selenium, Chef desirable
• Familiarity or experience with data mining, data science, machine learning and statistical modeling (e.g. regression modeling, clustering techniques, decision trees, etc.) is preferred
• Responsible for the design and implementation of an innovative, scalable, and distributed systems that take advantage of technology to allow standardization, security, timeliness and quality of data.
• Work with product managers in developing a strategy and road map to provide compelling capabilities that helps them succeed in their business goals.
• Work closely with senior engineers to develop the best technical design and approach for new product development.
• Resolve complex technical issues, making technical decisions that impact the project and beyond.
• Project management: prioritization, planning of projects and features, stakeholder management and tracking of external commitments
•  Identify opportunities for further enhancements and refinements to standards and processes.
• Strong Negotiation Skills: You will be a distinguished ambassador for product development, collaborating, negotiating, managing tradeoffs and evaluating opportunistic new ideas with business partners
• Mentor junior data engineers, fostering a culture of continuous learning and innovation.
• Consult with stakeholders to understand their business objectives and translate them into data-driven solutions. Lead data strategy sessions.
• Stay updated with the latest industry trends and technologies in data engineering and adopt them as needed.
• Drive the adoption of data-driven decision making within the organization by promoting the benefits of data engineering solutions.
• Resolve complex technical issues, making technical decisions that impact the project and beyond.
• Mentor junior team members, develop departmental procedures and best practices standards.
• Hire and retain world class talents to deliver data platform projects.

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

Qualifications

Basic Qualifications

• 10+ yrs. work experience with a bachelor’s degree or 8+ years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) in data engineering and analytics field such as computer science, statistics, finance, economics, or relevant area.

Preferred Qualifications

• Working knowledge of Hadoop ecosystem and associated technologies, (For e.g. Apache Spark, Python, Pandas etc.)
• Strong experience in creating large scale data engineering pipelines, data-based decision-making, and quantitative analysis.
• Experience with SQL for extracting, aggregating, and processing big data Pipelines using Hadoop, EMR & NoSQL Databases.
• Experience with complex, high volume, multi-dimensional data, as well as machine learning models based on unstructured, structured, and streaming datasets.
• ETL processes: The role also involves developing and executing large scale ETL processes to support data quality, reporting, data marts, and predictive modeling.
• Spark pipelines: The role requires building and maintaining efficient and robust Spark pipelines to create and access data sets and feature stores for ML models.
• Experience in writing and optimizing spark code and Hive code to process Large Data Sets in Big-Data Environments.
• Strong Development experience in more than one of the following: Python , Scala/java, Golang.
• Knowledge of standard big data and Real Time stack such as Hadoop, Spark, Kafka, Redis, Flink and similar technologies
• Hands on experience in building and maintaining data pipelines, feature engineering pipelines and comfortable with core ML concepts.
• Hands on experience in engineering, testing, validating and productizing AL/ML models for high performance use cases.
• Exposure to model serving engines such as TensorFlow, Triton etc.
• Exposure to model development frameworks like Ml flow.
• Proficient in managing and operating AWS (other cloud) services including EC2, S3, SageMaker etc.
• Proficient in setting up and managing distributed data and computing environments using AWS services.
• Knowledge about DR / HA topologies, Reliability Engineering with hands on experience in implementing the same.
• Knowledge of using and maintaining MLOPS tools and implementing automations for production
• Experience of working with containerized and virtualized environments (Docker, K8s)
• Experience with Unix/Shell or Python scripting and exposure to Scheduling tools like Airflow and Control – M.
• Experience creating/supporting production software/systems and a proven track record of identifying and resolving performance bottlenecks for production systems.
• Exposure to deploying large scale ML/AI models built by the data science teams and experience with development of models is a strong plus.
• Exposure to public cloud equivalents, and ecosystem shall be a plus.
• Strong Experience with Visualization Tools like Tableau, Power BI, is a plus.

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.