Senior Data Engineer

What you will do

We integrate with a variety of external hotel systems and travel industry partners using our in-house framework and services. This allows us to gather and process relevant data to power our products. We are looking for a Senior Data Engineer that is passionate about complex data and its quality to assist us in our mission. 

Where you will have impact

• Design and develop scalable, reliable data pipelines using the Google Cloud stack.
• Ingest, process, and store structured and unstructured data from various sources into our data-lakes and data warehouses.
• Optimise data pipelines for cost, performance and scalability.
• Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
• Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
• Mentor and provide technical guidance to other engineers working with data.
• Partner with Product, Engineering & Data Science teams to operationalise new solutions.

Technologies you’ll work with:

Google Cloud Pub/Sub, BigTable, BigQuery, Dataflow, Kubernetes, Airflow, Python, Dbt, Terraform.
 (feel free to ask us why these are the best fit for us)

About our team

Lighthouse is not only a data-driven company, we are a data company. The heart of all our products is data. It enables hotels to make the right decisions and fuel our analytical solutions. Being a growth company enables us to regularly attract new and interesting datasets, which can unlock new product directions. Today we process billions of data points and +100TB of data on a daily basis, containing hotels' pricing information, search data, hotel bookings, etc. All of that using modern technologies.

The Data Platform lies in the heart of the Engineering Organization of Lighthouse serving a big part of the data needs for the company. We manage state-of-the-art infrastructure, services and products and are constantly innovating to support all our business needs and products at scale.

Lighthouse’s Data Platform team has the following responsibilities and goals:

1. Scalability, reliability, and security: A good data platform should be able to handle increasing volumes of data and user demands, provide reliable access to data, and protect the data from unauthorised access or loss.
2. Enabling data-driven decisions and innovation: The ultimate goal of our team is to support the organisation's strategic objectives, whether that's making better business decisions based on data, developing new data-driven products or services, or improving operational efficiency.
3. User empowerment: A data platform should not just be a tool for data experts, but for all users in the organisation. This includes providing user-friendly tools, clear documentation, and training to help users access and understand the data they need.
4. Data governance and compliance: With the increasing importance and sensitivity of data, it's crucial to have clear policies and procedures for managing data, as well as ensuring compliance with data protection regulations.

What's in it for you?

• Flexible working environment: Work from home or at one of our global offices.
• Flexible time off: Autonomy to manage your work-life balance.
• Collaborative team: High-bar, friendly, creative, and passionate colleagues.
• Career development: Workshops, frameworks, tools, training, and processes to realize your full potential.
• Impactful work: Shape products relied on by 85,000+ users worldwide.
• Data culture growth: Opportunity to evolve data culture in a fast-growing scale-up.
• Competitive compensation: Proactively maintained to value your work.
• Referral bonuses: Earn rewards for bringing in new talent.
• Additional Benefits for Spain:
• Cobee flexible benefits card: 160€/month for food, transportation, or nursery expenses.
• Language learning support: 40€/month on Preply.
• Alan health insurance: 100% coverage for you, your spouse, and dependents.
• Additional Benefits for Belgium:
• Mobility approach: Mobility budget or company car tailored to your personal situation.
• Extensive health insurance: Packages prioritizing holistic well-being and healthcare needs for you and your dependents.
• Private pension funding: Provided through group insurance.

Who you are

• 5+ years of professional experience using Python, Java, or Scala for data processing (Python preferred)
• You have experience with writing data processing pipelines and with cloud platforms like AWS, GCP, or Azure
• You have experience with data pipeline orchestration tools like Apache Airflow (preferred), Dagster or Prefect
• You have a deep understanding of data warehousing strategies
• Experience with transformation tools like dbt to manage data transformation in your data pipelines
• You have some experience in managing infrastructure with IaC tools like Terraform.
• You stay updated with industry trends, emerging technologies, and best practices in data engineering
• You improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed
• You ship large features independently, generate architecture recommendations with the ability to implement them
• You are a strong communicator that can describe complex topics in a simple way to a variety of technical and non-technical stakeholders

We welcome:

• Familiarity with GCP, Kubernetes (GKE preferred),  CI/CD tools (Gitlab CI preferred), familiarity with the concept of Lambda Architecture.
• Experience with Apache Beam or Apache Spark for distributed data processing or event sourcing technologies like Kafka.
• Familiarity with monitoring tools like Grafana & Prometheus.