Sr Data Engineer, Partner Experience

- 5+ years of data engineering experience
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with SQL
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
- Experience operating large data warehouses

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status.

Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $139,100/year in our lowest geographic market up to $240,500/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.
Prime Video offers customers a vast collection of movies, series, and sports—all available to watch on hundreds of compatible devices. U.S. Prime members can also subscribe to 100+ channels, including Max, discovery+, Paramount+ with SHOWTIME, BET+, MGM+, ViX+, PBS KIDS, NBA League Pass, MLB.TV, and STARZ, with no extra apps to download and no cable required. The Partner Experience (PX) organization is focused on delivering world-class partner tools to enable partners to launch, grow, and scale their businesses across Prime Video's business lines. The PX organization builds various products to help partners: 1) onboard and make their selection available to customers globally; 2) manage their selection across Prime Video's business lines; and 3) acquire, engage, and retain customers through self-service marketing, merchandising, and promotion tools. By providing partners with best-in-class self-service capabilities throughout their engagement with Amazon, the PX organization powers the flywheel through increased partner satisfaction and expanded selection available to Amazon customers.

The PXBI (Partner Experience BI) team fulfills the data needs of the PX organization, providing big data to power reporting functions across five crucial partner-facing products, meeting contractual obligations, and empowering Prime Video partners with performance data to grow their business. The PXBI team also builds data pipelines to enable data-driven decision-making for the PX organization.

Key job responsibilities
As a Sr. Data Engineer on the team, you will work with very large data sets in one of the world's largest and most complex data lake and data warehouse environments. You will design, build, and maintain robust data infrastructures to process large volumes of data (big data) from various sources. This includes writing clean, maintainable code that adheres to best practices, being familiar with AWS data processing services, possessing broad knowledge of when to utilize different types of data, and understanding data security and compliance standards to ensure alignment with company policies and regulations.

The Sr. Data Engineer will work with the SDE team to build Tahoe, the core data platform that delivers intraday data to power partner-facing products and internal product measurements for 15 products. This includes implementing AWS services to build data pipelines, enhancing the pipelines to cover more business lines in Prime Video, and increasing data service stability to provide accurate and timely data to both external partners and internal stakeholders. The Sr. Data Engineer will also enhance Red/Orange data compliance with Amazon-wide data compliance regulations , manage and monitor the Redshift cluster to ensure high performance, and work with the software engineering team to deprecate legacy data pipelines or rebuild them to handle changes in upstream systems, manage bindles and LDAP groups to ensure proper permissions.

A day in the life
The Sr. Data Engineer will write design documents to propose solutions for challenging data processing and data handling tasks, meet with peers to discuss the solutions, and review the design documents with product managers, senior management, and senior software engineers to communicate their approach and gather feedback. They will then build the data infrastructure and data pipelines using AWS services, and productionalize the solutions into products that will be used by worldwide Prime Video partners.

About the team
We are a centralized data team that supports the Partner Experience organization. The team processes big data using advanced AWS services and works closely with product managers, technical program managers, and software engineers, as well as collaborating with other data teams outside of the Partner Experience organization to deliver products for Prime Video partners. The team currently leverages a mix of NAWS and MAWS data technologies, including but not limited to S3 with Apache Iceberg, Glue / EMR, AWS Step Functions, Redshift, Lambda, CloudWatch, CI/CD, CDK, and internal BDT tools - Datanet, Cradle etc. We use ASR and Kale for data security review.