About the role
You will design, develop, and maintain scalable data pipelines and ETL/ELT processes while modernizing legacy infrastructure. Additionally, you will collaborate with the analytics team to deliver effective data solutions and ensure the reliability and performance of the data platform.
What they look for
Requirements
Candidates must have at least 2 years of professional experience in data engineering with strong proficiency in SQL and Python. Hands-on experience with AWS services, Apache Airflow, and data warehousing concepts is required.
Full description
What to expect of your role:
We are looking for a Data Engineer to help us develop and maintain our data platform. Our infrastructure is primarily AWS-based, with Airflow, S3 and Redshift at its core, alongside a legacy stack based on MS SQL Server, SSIS and custom Python scripts. You will work on building reliable data pipelines, improving our data infrastructure, and gradually modernising legacy processes
You will be expected to:
- Design, develop and maintain scalable and reliable data pipelines and ETL/ELT processes;
- Develop and optimise data warehouse solutions;
- Develop and maintain data infrastructure, primarily on AWS, while also supporting parts of our legacy infrastructure;
- Develop APIs and data integrations;
- Contribute to the development and maintenance of real-time data pipelines using Kafka;
- Automate and optimise data processes and workflows;
- Monitor, troubleshoot and improve the reliability, performance, security and data quality of pipelines and infrastructure;
- Work closely with the analytics team and other stakeholders to understand their data needs and deliver effective solutions;
- Maintain clear and up-to-date documentation for data pipelines, infrastructure and workflows;
- Contribute to the improvement and modernisation of existing data processes and architecture.
What you need to succeed in this role:
- 2+ years of professional experience in data engineering;
- Strong SQL and Python skills;
- Hands-on experience with AWS, including S3, Redshift, Glue and Lambda;
- Hands-on experience with Apache Airflow;
- Strong understanding of data warehousing concepts and data modelling;
- Hands-on experience designing, developing and maintaining production data pipelines, with a focus on reliability, monitoring and data quality;
- Solid understanding of relational and non-relational databases;
- Familiarity with Git and CI/CD practices, preferably GitLab;
- Experience developing APIs and data integrations;
- Kafka and real-time data processing experience is a plus;
- MS SQL Server and SSIS experience is a plus;
- Good written and verbal communication skills;
- Upper-intermediate English or higher.
Similar roles
-
Senior Data Engineer R&D
MSD Prague, Prague, Czechia
-
Sr Data Engineer
Porch Group India
-
FBS SAP Informatica Power Center Data Engineer
Capgemini Mexico
-
Forward Deployed Data Engineer
Total Performance Consulting Georgia, United States
-
Senior Data Engineer - Algolia
Perficient Bogota, RAP (Especial) Central, Colombia
-
Senior Data Engineer - Tableau
Perficient Bogota, RAP (Especial) Central, Colombia