Senior/Lead Data Engineer
Sabenza IT & Recruitment · City of Cape Town, Western Cape, South Africa
IT Services and IT Consulting · 11-50 employees
About the role
Design, build, and optimize scalable enterprise data pipelines and cloud-native data platforms. Lead technical initiatives, mentor engineering staff, and drive DevSecOps and CI/CD best practices across the organization.
What they look for
Requirements
Requires a Bachelor's degree and over 8 years of experience in Data or Software Engineering. Essential expertise includes Python, PySpark, SQL, AWS cloud services, and experience within the Retail or FMCG industry.
Full description
Our client is looking for an experienced Senior / Lead Data Engineer to join their growing Data Engineering team. This role is responsible for designing and delivering scalable cloud-based data solutions, leading technical initiatives, and mentoring engineers while driving best practices across enterprise data platforms.
Requirements
Key Responsibilities
- Design, build and optimise enterprise data pipelines and data platforms.
- Develop scalable cloud-native data engineering solutions.
- Collaborate with business and technical stakeholders to deliver data-driven solutions.
- Define and maintain data architecture and engineering standards.
- Drive DevSecOps and CI/CD best practices.
- Mentor and coach Data Engineers.
- Ensure data quality, governance and platform reliability.
- Participate in Agile ceremonies and technical planning.
- Drive continuous improvement and innovation across the data environment.
Requirements
- Bachelor's Degree or Diploma in Computer Science, Engineering, Mathematics, Information Systems or similar.
- 8+ years' experience in Data Engineering or Software Engineering.
- Strong experience with Business Intelligence and Data Warehousing.
- Advanced SQL skills.
- Expert knowledge of Python and PySpark.
- Experience with Hadoop and Apache Spark.
- Strong AWS experience, including EMR, S3, Athena, RDS, Lambda and API Gateway.
- Good understanding of SDLC, Solution Architecture and DevSecOps.
- Experience with Azure, Google Cloud Platform and Snowflake is advantageous.
- Previous experience leading or mentoring engineering teams.
- Retail or FMCG industry experience is essential.
Technical Skills
- Python
- PySpark
- SQL
- AWS
- Hadoop
- Apache Spark
- Data Warehousing
- Business Intelligence
- ETL / ELT
- Snowflake
- Azure
- Google Cloud Platform
- DevSecOps
- CI/CD
- Solution Architecture
