Senior Data Engineer

Location: Johannesburg, Gauteng, South Africa; Hybrid

• Data Warehouse, Lake and Lakehouse architecture patterns.
• Distributed data processing using frameworks such as Spark or Flink.
• Designing and supporting batch and near-real-time ingestion pipelines.
• Building incremental, idempotent, and fault-tolerant data pipelines.
• Data quality, reconciliation, and observability practices.
• Metadata, lineage, governance, and access control concepts.
• Analytical data modelling and efficient data structures for warehouse and large-scale query workloads.
• Medallion architecture such as Bronze, Silver, and Gold layers.
• Open table formats such as Iceberg.
• Schema evolution, partitioning strategies, file optimisation, and storage layout tuning.
• Event-driven or streaming platforms such as Kafka, Pulsar, or Redpanda.
• Columnar or high-performance analytical platforms such as ClickHouse.
• CI/CD pipelines, deployment automation, and engineering standards for data workloads.
• Experience improving reliability, performance, and scalability across production data platforms.
• Familiarity with monitoring, alerting, observability, and operational support practices.
• Exposure to containerised or clustered environments such as Kubernetes, OpenShift, or similar platforms.
• Strong debugging capability across data, pipeline, compute, and platform layers.
• Strong sense of ownership and accountability.
• Comfortable making technical trade-offs while remaining pragmatic and hands-on.
• Excellent problem-solving and communication skills.
• Able to collaborate effectively with technical and non-technical stakeholders.
• Passionate about building scalable, maintainable, and well-documented systems.
• Committed to sharing knowledge and raising engineering standards across the team.

Play a key hands-on role in designing, building, and improving BET Software's enterprise data platform. This role is suited to an experienced engineer who can own critical data pipelines and platform components, contribute to architectural direction, and help establish strong engineering practices that support trusted, scalable data across the business.