Data Engineer — PySpark + AWS Glue
Yadimen Consulting Limited Pune, Maharashtra, India
IT Services and IT Consulting · 51-200 employees
About the role
Design, develop, and maintain scalable ETL pipelines using PySpark and AWS Glue. Orchestrate data workflows and ensure pipeline reliability through monitoring and performance optimization.
What they look for
Requirements
Requires 4-8 years of experience with strong hands-on skills in PySpark, AWS Glue, and ETL development. Proficiency in AWS services like Step Functions and Lambda is essential for this role.
Full description
Data Engineer — PySpark + AWS Glue
ETL & Data Pipelines | 2 Openings
Location: Remote (UK)
Employment Type: Contract / Permanent
Experience Level: 4–8 Years
Openings: 2
About the Role
We are looking for a skilled Data Engineer with solid hands-on experience in PySpark, AWS Glue, and ETL development to build and maintain scalable, production-grade data pipelines on AWS. You will be part of a delivery-focused team working on complex data engineering challenges, contributing to data lake architecture and end-to-end pipeline development.
Requirements
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines
using PySpark and AWS Glue
- Build and optimise data ingestion, transformation, and
loading workflows
- Orchestrate data workflows using AWS Step Functions
- Develop serverless functions using AWS Lambda (Python)
- Work with data lake architectures on AWS to support
analytical use cases
- Ensure pipeline reliability, monitoring, and
performance optimisation
Required Skills & Experience
- Strong hands-on experience with PySpark and AWS Glue
- Proven track record in ETL pipeline development and
optimisation
- Experience orchestrating workflows with AWS Step
Functions
- Proficiency in serverless development using AWS Lambda
(Python)
- Good SQL skills and a solid understanding of data
processing principles
Nice to Have
- Exposure to Java-based microservices
- Understanding of REST APIs and backend service
integrations
- Experience working with AWS-based data lakes
Similar roles
-
Cloud Data Engineer
Poolia Stockholm, Sweden
-
Lead Data Engineer
Avaron AB Solna, Sweden
-
Senior Data Engineer
Poland and Eastern Europe Bulgaria
-
Associate Consultant - Data Engineer(Python/Pyspark)
KPMG India Bangalore, Karnataka, India
-
Data Engineer
FanDuel Edinburgh, Scotland, United Kingdom
-
Data Engineer
PA Consulting Bristol, England, United Kingdom