Sr. Data Engineer

SeatGeek believes live events are powerful experiences that unite humans. With our technological savvy and fan-first attitude we’re simplifying and modernizing the ticketing industry.

We are looking for Data Engineers with varying levels of experience to join SeatGeek’s teams in the Data Science department. As a department, we share common views on experimental rigor, pragmatism, and software quality. Different teams within the data science department play a unique role, and as a result, the responsibilities of data engineers may be tailored to meet those needs. But all data engineers focus on maintaining and expanding our ecosystem of data pipelines. These pipelines provide the basis for many crucial downstream processes: complex analyses, data products, and business performance measurement. We are looking for a Sr. Data Engineer with varying levels of experience to join SeatGeek’s R&D team.

What you'll do

• Build a new stream ingestion process to efficiently send and store currently live listings as users see them
• Build and maintain our streaming data pipelines
• Create tooling and services that support our machine learning applications and data products
• Build a complete ecosystem for our enterprise clients (pro sports teams, Broadway theaters, etc.) to access their data in a more consolidated approach by: exposing API endpoints to pull data continuously, implementing the unified data warehouse and offering dashboards with relevant data for their businesses
• Communicate and collaborate with different stakeholders, like Data Scientists, Analysts, Software Engineers, and our Product team to produce requirements and detailed designs

Who you are

The ideal candidate can relay complex concepts to both technical and non-technical audiences, enjoys partnering with both business users and application developers, and has a proven ability to build systems that allow greater clarity, consistency, and efficiency in utilizing data.  Experience with specific tools is less important than aptitude and drive, but at a minimum, we would expect:

• 5+ experience developing on and maintaining ETL systems, and a high level understanding of different data processing systems
• Experience working cross-functionally with different business units to translate business problems into data problems and solve them
• Comfort turning ideas into code
• Proficiency with analytical and operational SQL queries, and knowledge of how to optimize and troubleshoot
• Have experience with agile methodologies

Bonus points for candidates who have experience with or desire to learn any of the following:

• Data-processing frameworks (Spark, Flink)
• Streaming data/message queues (Akka streams, Kafka)
• AWS infrastructure (we use Redshift, S3, EMR, Kinesis, Lambda, Glue, and RDS)
• Dimensional modeling, especially in e-commerce or consumer tech
• Database internals and tuning, trade-offs and benefits of different datastores (both analytical and transactional)
• Optimization of complex data workflows that reconcile batch and streaming data
• Dependency management at scale across multiple classes of data source

The tools we use

You absolutely do not need experience with all of these, but we thought you might be curious. Tools can be learned, so we care much more about your general engineering skill than knowledge of a particular language.

• Languages: Java and Python for general purpose development
• Frameworks: Airflow for dependency management, Spark on EMR clusters for distributed processing
• Stack: Docker containers running on Nomad on AWS
• Streaming: Flink, Kinesis (+Firehose), Debezium, and Kafka
• Datastores: Postgres, MySQL and SQL Server in production, Redshift and S3
• Other: AWS Lambda, Git, dbt, Docker

Perks: