Data Engineer (Operations)

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

Technical Responsibilities:

• Manage large scale ETL processes to support publishing of solution DataMarts, Dashboard and report packs to hundreds of Visa's clients.

• Proficient in building, scheduling and managing DAGs in Apache Airflow.

• Monitor data processing tasks in the data pipelines using in-house monitoring tools built on top of Apache Airflow.

• Quality control of data assets to ensure the data loaded across different stages in the data pipeline are reconciled.

• Debugging skills to quickly rectify any execution errors and resume the data pipeline, minimizing potential delays and impact on business operations.

• Strong data analytics skills to identify and address data issues promptly and providing clarifications as needed,

• Identify, design, and develop areas for automation in the data operations processes.

Other responsibilities:

• Collaborating and communicating with stakeholders on recommendations and data clarifications

• Ensure highest level of quality and detail-oriented approach in day-to-day work.

• Adhere to agile principle and actively participate in all the scrum ceremonies.

• Follow the process, standards and guidelines to ensure smooth operations support

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

Qualifications

• 3+ years professional experience in executing and managing data pipelines and writing ETL/ELT code using Hive, PySpark, SQL and Unix.
• Hands-on experience working in the Hadoop eco-system
• Strong experience in writing and optimizing SQL queries on Hive in Big data environment.
• Strong python experience to build automation tools
• Experience working in data related projects

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.