Data Engineer

Data Engineer

Contract: Full-time; 2-year fixed term, renewable

Remuneration: Competitive, based on experience and competence

Location: Any AP office location, though South Africa is preferred; hybrid

Reports to: Partner, Product Innovation   

About us

Africa Practice is a mission-led strategic consulting firm advising investors and development partners across the continent. Our mission is to shape a prosperous Africa that inspires the world. For over 20 years, we have been partnering with global investors, policy-makers, development partners and other changemakers to drive transformation on the continent. 

Our insights and advocacy solutions are rooted in detailed systems and stakeholder understanding. We identify, generate, and analyse human-centred systems data to design impactful advocacy interventions. We deliver our world-class insights and advocacy solutions through our local presence.

We operate as one company, with a diverse team working on a hybrid basis in Abuja, Cape Town, Dakar, Dar es Salaam, Gaborone, Johannesburg, Lagos, Nairobi, and London. We are risk analysts, development economists, advocates, storytellers, data scientists, policy experts,  systems thinkers and creative minds. Over 80% of our staff are African and over 50% are women. We have been a signatory to the UN Global Compact since 2021 and achieved BCorp certification in 2024. 

About the role  

This is a technical, hands-on role requiring strong execution in data pipeline engineering, database management, and system optimisation. The successful candidate will support the evolution of our data infrastructure by building, maintaining, and expanding our hybrid data lakehouse.

You will focus on integrating robust open-source applications alongside our existing Google Cloud Platform (GCP) environment to enhance scalability and flexibility. Working closely with our analytics and consulting teams, you will ensure our data assets are reliable, accessible, and secure, directly enabling the delivery of superior insights to our clients.

Responsibilities:

Data lakehouse development and optimisation:

• Build, maintain, and optimise reliable data pipelines and ETL/ELT processes to ingest data from diverse networks and sources.

• Integrate open-source tools and frameworks into our existing GCP infrastructure to enhance pipeline efficiency and flexibility.

• Manage and optimise data storage, schemas, and query performance within Google BigQuery and cloud storage systems (Cloud SQL, Amazon RDS).

• Support data scientists and analysts by structuring, cleaning, and preparing datasets for advanced analytics and machine learning models.

• Capable of creating, deploying, logging, monitoring and maintaining cloud jobs on GCP for daily data ingestions 

Data governance and compliance:

• Implement and adhere to robust data governance policies, ensuring all data engineering activities comply with relevant regulations (e.g. GDPR, POPIA) and ethical best practices.

• Build data quality checks, monitoring systems, and testing frameworks to safeguard the accuracy and reliability of our data pipelines.

• Document data architectures, workflows, and schemas to support internal data literacy and maintain the company's technical knowledge base.

Operational support:

• Monitor daily data operations, troubleshooting pipeline failures, handling data ingestion issues, and resolving system bottlenecks promptly.

• Support the cost-efficient use of cloud resources and open-source deployments through continuous performance tuning.

Stakeholder engagement:

• Collaborate closely with internal stakeholders (consultants, analysts, and data scientists) to understand their data requirements and translate them into efficient technical solutions.

• Communicate technical data pipeline concepts and limitations clearly to non-technical team members.

What we are looking for:

• Proven experience working as a data engineer or in a similar technical data infrastructure role, preferably within a professional services or fast-paced consulting environment.

• Strong hands-on experience with cloud technologies, specifically Google Cloud Platform (GCP) services relevant to data handling (e.g. BigQuery, Cloud Storage).

• Deep technical proficiency with open-source data engineering tools, particularly for orchestration (e.g. Apache Airflow), transformation (e.g. dbt), and processing (e.g. Apache Spark or Kafka).

• Solid understanding of data modelling concepts, ETL/ELT development, and data lakehouse architectures.

• High proficiency in SQL and Python for data manipulation and pipeline construction.

• Experience with DevOps and CI/CD tools, including Git, GitHub Actions, and Docker containerisation.

• Be AI Native - have considerable experience using AI coding tools such as Codex, Claude, Cursor 

• Familiarity with handling both structured and unstructured data streams.

• A bachelor's degree in computer science, information technology, engineering, or a related quantitative field.

What we offer

We offer the successful candidate an opportunity to develop a rewarding career within a dynamic pan-African consultancy advising a prestigious portfolio of clients. This position offers:

• Competitive remuneration

• Immediate opportunities for growth

• Collaboration with a diverse and experienced team of colleagues

Don’t meet every single requirement? Studies have shown that women and people of colour are much less likely to apply for a position unless they meet every single qualification. So this notwithstanding, if you are excited about this role, we would love to hear from you. At Africa Practice, we are dedicated to building an inclusive, diverse and authentic workplace. You might just be the new team member we are looking for - for this or other roles. 

Do you want to contribute to an influential and prosperous Africa, inspiring the world? We’re looking for Africa’s changemakers – smart people with strong analytical, advocacy, digital and problem-solving skills who can reconcile diverse perspectives to deliver sustainable change. 

Please send in your application by clicking on the “apply” button. 

Do include a cover letter explaining why you think you are suited to this position and indicating your availability. Your documents must be entitled with “*your name* Data Engineer”. 

The vacancy is immediately open for suitable applications and will be closed as soon as a suitable candidate is identified. Early applications are advised.