M
Mid (2-5 yrs)
Full-time
Egypt
Log in
to apply, save this posting, or score it against your profile with AI.
About the role
Build and maintain scalable data pipelines to support data transformation, integration, and loading across the platform. Collaborate with data scientists and stakeholders to ensure data flows meet processing and analytical requirements.
What they look for
Python
SQL
Java
Spark
Hadoop
SSIS
Informatica
Data Engineering
ETL
ELT
Data Pipelines
Data Integrity
Data Quality
Performance Optimization
Informatica BDM
IDQ
Requirements
Requires a bachelor's degree in computer science or a related field with 4-5 years of experience in data engineering. Proficiency in Python, SQL, Java, and big data tools like Spark, Hadoop, and ETL software is essential.
Full description
- Data Pipeline Development: Build and maintain scalable data pipelines that support data transformation, integration, and loading across the data platform.
- Data Integrity & Quality: Ensure the integrity, quality, and security of data across the platform by implementing and developing appropriate data validation, monitoring, and error handling mechanisms.
- Performance Optimization: Optimize data workflows for performance, reliability, and scalability to meet business and technical requirements.
- Collaboration: Collaborate with data scientists, analysts, and other stakeholders to ensure data flows meet their processing and analytical needs.
- Continuous Improvement: Stay updated with the latest big data technologies and best practices, and implement improvements in data processing pipelines.
- Collaborate with data scientists, analysts, and business stakeholders to align data pipelines with business needs.
- Work with third-party vendors or consultants related to ETL tools, big data technologies, or platform services.
Requirements
- Education: Bachelor’s degree in computer science, Data Engineering, or a related field.
- Experience (3-5 years): Proven experience with data engineering and building scalable data pipelines for data transformation, integration, and storage.
- Technical Skills: Proficiency in programming languages such as Python, SQL, or Java. Experience with big data technologies like Spark, Hadoop, and familiarity with ETL tools like SSIS and Informatica.
- Soft Skills: Strong problem-solving, analytical, and communication skills, with the ability to collaborate across teams.
- Expertise in building and maintaining scalable data pipelines and architectures.
- Proficient in Python, SQL, Java, and big data tools such as Spark and Hadoop.
- Experience with ETL tools (SSIS, Informatica) and managing data integration processes. Develop and optimize ETL/ELT processes using (Informatica BDM , SSIS , IDQ ..etc)
- Strong understanding of data integrity, quality, and security BDM, SSIS, Ability to optimize workflows for performance and reliability.
