Senior Data Engineer

• Data Warehouse, Lake and Lakehouse architecture patterns.

• Distributed data processing using frameworks such as Spark or Flink.

• Designing and supporting batch and near-real-time ingestion pipelines.

• Building incremental, idempotent, and fault-tolerant data pipelines.

• Data quality, reconciliation, and observability practices.

• Metadata, lineage, governance, and access control concepts.

• Analytical data modelling and efficient data structures for warehouse and large-scale query workloads.

• Medallion architecture such as Bronze, Silver, and Gold layers.

• Open table formats such as Iceberg.

• Schema evolution, partitioning strategies, file optimisation, and storage layout tuning.

• Event-driven or streaming platforms such as Kafka, Pulsar, or Redpanda.

• Columnar or high-performance analytical platforms such as ClickHouse.

• CI/CD pipelines, deployment automation, and engineering standards for data workloads.

• Experience improving reliability, performance, and scalability across production data platforms.

• Familiarity with monitoring, alerting, observability, and operational support practices.

• Exposure to containerised or clustered environments such as Kubernetes, OpenShift, or similar platforms.

• Strong debugging capability across data, pipeline, compute, and platform layers.

• Strong sense of ownership and accountability.

• Comfortable making technical trade-offs while remaining pragmatic and hands-on.

• Excellent problem-solving and communication skills.

• Able to collaborate effectively with technical and non-technical stakeholders.

• Passionate about building scalable, maintainable, and well-documented systems.

• Committed to sharing knowledge and raising engineering standards across the team.