Senior Data Engineer

What you will do:

We integrate with a variety of external hotel systems using our in-house framework and services. This allows us to gather and process relevant data to power our products. We are looking for a Senior Data Engineer that is passionate about complex data and its quality to assist us in our mission. 

Where you will have impact:

• Design and develop scalable, reliable data pipelines using the Google Cloud stack.
• Ingest, process, and store structured and unstructured data from various sources into our data-lakes and data warehouses.
• Optimise data pipelines for performance and scalability.
• Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
• Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
• Mentor and provide technical guidance to other engineers working with data.

About our team:

The Lighthouse data engineers are the gatekeepers to our vast datasets. Their mission is to reliably transform, integrate and store the numerous data sources that power our BI products. Some datasets are acquired by web scraping or API integrations, others are pushed by external vendors. Our main dataset holds 3 trillion (yes that’s 12 zeros) hotel rates. The engineers work in close collaboration with our Product Integrations Managers, as well as a highly talented group of software engineers and devops engineers to drive initiatives forward.

Who you are:

• 3+ years of professional experience using Python, Java, or Scala for data processing (Python preferred)
• You have experience with writing data processing pipelines
• You stay updated with industry trends, emerging technologies, and best practices in data engineering
• You improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed
• You ship large features independently, generate architecture recommendations with the ability to implement them
• You have great communication skills, both to technical and non-technical parties

We welcome:

• Familiarity with GCP, Kubernetes (GKE preferred),  CI/CD tools (Gitlab CI preferred), familiarity with the concept of Lambda Architecture.
• Experience with Apache Beam or Apache Spark for distributed data processing or event sourcing technologies like Kafka.
• Familiarity with monitoring tools like Grafana & Prometheus.

Technologies you’ll work with:

Google Cloud Pub/Sub, BigTable, BigQuery, Dataflow, Kubernetes, Airflow, Python

 (feel free to ask us why these are the best fit for us)

What’s in it for you:

• A flexible working environment where you can work from home or, if you live nearby, at one of our seven offices around the world
• Flexible time off: we believe in providing our employees with the flexibility and autonomy to manage their work-life balance effectively
• A motivation to deliver your best work as we have built a high-bar and very talented team of individuals that are friendly, creative, open-minded, and passionate about what they do
• Workshops and frameworks that help employees realise their full career potential
• A mandate to operationalize and democratise research through tools, training, and processes
• The opportunity to shape the products that more than 85,000 users rely on worldwide
• The chance to grow and evolve the data culture at a fast-growing scale-up
• A compensation that values your work and which we will proactively keep competitive
  • 160€/month on your Cobee flexible benefits card to be used on either food (restaurants, bars, coffee shops, delivery, etc), Transportation (Monthly bus/metro tickets) or Nursery for children up to 3 years old.
  • 40€/month to learn or improve any language on Preply
  • Alan health insurance: 100% coverage for you, your spouse and your dependants
  • A mobility approach that fits your personal situation, with a mobility budget or company car
  • Extensive health insurance packages for you and your dependents prioritising our employees' holistic well-being and healthcare needs
  • Private pension funding through group insurance
• Spain:
• Belgium: 
• A referral bonus scheme when you bring new talent to our #bestteamever!