JobCubby JobCubby
Sign up
SSC HR Solutions

Senior Data Engineer (Spark)

SSC HR Solutions Cairo, Cairo, Egypt

Human Resources Services · 51-200 employees

6 h ago
Remote data-engineer Senior (5-10 yrs) Full-time Egypt
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

Builds and maintains large-scale batch data pipelines using Apache Spark to ensure efficient and reliable data processing. Collaborates with platform and product teams to manage end-to-end data flows, data modelling, and lakehouse organization.

What they look for

Apache Spark Data Engineering Performance Tuning Data Modelling Lakehouse Apache Iceberg Trino Batch Data Pipelines Data Quality Automated Scheduling Monitoring Cluster Management

Requirements

Requires strong hands-on experience with Apache Spark, including performance tuning and self-managed cluster environments. Candidates must be proficient in data modelling, automated scheduling, and working with open table formats like Apache Iceberg.

Full description

Description

Builds and runs large scale batch data pipelines, keeping jobs fast and affordable as data volumes grow. Works with Apache Spark to build and manage large scale data processing jobs, including performance tuning to maintain efficient and reliable pipeline execution. Supports data modelling for analytics and organises data in a lakehouse so it is usable downstream. Handles automated scheduling, monitoring, and data quality checks, while working with platform and product teams on end to end data flows.

Requirements

  • Strong hands on Apache Spark including performance tuning, not Spark usage through a managed notebook only.
  • Data modelling for analytics and organising data in a lakehouse so it is usable downstream.
  • Automated scheduling, monitoring, and data quality checks.
  • Works with platform and product teams on end to end data flows.
  • Apache Iceberg or other open table formats.
  • Trino or similar query engines.
  • On premises or self managed cluster experience.

Similar roles