JobCubby JobCubby
Sign up
Valleysoft

Data Engineer - Data Platforms (Databricks)

Valleysoft Cairo, Cairo, Egypt

Software Development · 51-200 employees

6 h ago
data-engineer Senior (5-10 yrs) Full-time Egypt
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

Design, develop, and maintain scalable data pipelines and enterprise data platforms using Databricks and Apache Spark. Collaborate with cross-functional teams to support analytics and AI initiatives while ensuring data security, quality, and performance.

What they look for

Databricks Apache Spark Python SQL ETL ELT Delta Lake Data Engineering Data Pipelines Data Governance Data Architecture Cloud Platforms Automation Data Quality Performance Optimization

Requirements

Requires a bachelor's degree in a relevant field and at least 6 years of experience in data engineering. Candidates must possess strong proficiency in Python, SQL, Databricks, Apache Spark, and cloud-based data architecture.

Full description

Valleysoft is a regional IT services provider delivering enterprise technology, application development, process management, and IT support services to clients around the world. Working across the Information Technology and Services sector, the company helps organizations build reliable, scalable digital solutions that support complex business and operational needs.

This role is an opportunity to contribute to modern data platform initiatives that power analytics and AI use cases across the enterprise. As a Data Engineer – Data Platforms (Databricks), you will play a central role in shaping data infrastructure, supporting cross-functional teams, and helping ensure data systems are performant, secure, and built to scale.

Responsibilities• Design, develop, and maintain scalable data pipelines for batch and real-time processing.

  • Build and optimize enterprise data platforms using Databricks and Apache Spark.
  • Develop ETL/ELT processes to ingest, transform, and integrate data from multiple sources.
  • Implement Delta Lake architecture and maintain data lake/lakehouse solutions.
  • Collaborate with Data Scientists, AI Engineers, Business Analysts, and application teams to support analytics and AI initiatives.
  • Optimize data storage, partitioning, and query performance.
  • Develop reusable data engineering frameworks and automation solutions.
  • Implement data quality, validation, lineage, and governance processes.
  • Ensure data security, privacy, and compliance with enterprise standards.
  • Monitor platform performance, troubleshoot production issues, automate deployment and infrastructure, contribute to architecture reviews, and maintain technical documentation and runbooks.
  • Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
  • 6+ years of data engineering experience working with Databricks and Apache Spark.
  • Strong experience with Python and SQL, along with hands-on involvement in ETL/ELT processes.
  • Experience working with Delta Lake and cloud platforms.
  • Fluent in English.
  • Eligibility to work in Egypt.

Similar roles