Senior Data Engineer (HYBRID REMOTE)

About the role

This is a hands-on role for an Senior Data Engineer sitting within the Data Products team. In this role, you will have an enormous impact on the design, implementation, and maintenance of existing and greenfield customer-facing data products. Data is key to Auctane’s strategy. We work at a scale pace and with the latest architecture patterns and tech. We process thousands of events per second, and our massive dataset keeps growing at a staggering pace. We keep improving our data platform and data engineering stack to accommodate growth, enable novel solutions, and provide the best service to our customers. We have a flat and open engineering culture where data & evidence beat opinion and hierarchy, backed by honest discussions. We passionately believe in forming autonomous, cross-functional teams who are empowered to deliver our ambitious strategy. Energy and passion for our business and customers are a part of the Auctane culture – and we love working with like-minded people.

What will you be doing?

• Contribute to the design, build, and operational management of our data pipelines and analytics solution on top of proven AWS data technologies like S3, Athena, Lambda, and Kinesis.
• Collaborate with the Product Owner and other stakeholders to implement the Data Products strategy.
• Develop frameworks and solutions that enable us to acquire, process, monitor and extract value from our massive dataset.
• Drive the design and architecture of Auctane’s data product.
• Contribute directly to the implementation and operations of our systems.
• Be an advocate of data quality and observability principles and use state-of-the-art technologies like Airflow, dbt, and Spark to process data and get our datasets just right.
• Foster engineering excellence by delivering highly reliable software and data pipelines using Software Engineering best practices like automation, version control, continuous integration/continuous delivery, testing, security, etc.

What are we looking for?

• A strong Data Engineering background in a data warehouse or data lake architecture
• Experience working in AWS/GCP cloud infrastructure.
• Experience developing and supporting robust, automated, and reliable data pipelines in Python and SQL.
• Mastered Python and SQL skills.
• Experience with data processing frameworks like Spark, Athena, or Pandas.
• Knowledge of Data Orchestration solutions, preferably Airflow.
• Knowledge of relational database design as well as Snowflake/Redshift or similar

What will make you stand out?

• Production experience working with very large datasets.
• Experience with AWS and big data cloud technologies like EMR, Athena, Glue, and EC2.
• Direct experience with Infrastructure as Code tools (e.g., Terraform, CloudFormation)
• Knowledge and direct experience in using business intelligence and analytics tools (Tableau, Looker, Power BI, etc.)
• A desire to constantly challenge the norm Proven track record of promoting and maintaining high engineering standards.
• Fast learner with a keen interest in evaluating and adopting new technologies: Apache Iceberg, Dagster, DBT, GreatExpectations, Prefect, Snowflake, etc.
• Understanding of core ML concepts like supervised and unsupervised learning, overfitting, and cross-validation as well as LLM concepts like fine-tuning, etc.

Tech Stack

What do we offer?