Data Engineer — PySpark + AWS Glue
Yadimen Consulting Limited · Pune, Maharashtra, India
About the role
Design and maintain scalable ETL pipelines and data ingestion workflows using PySpark and AWS Glue. Orchestrate data workflows with AWS Step Functions and develop serverless functions using AWS Lambda.
What they look for
Requirements
Requires 4-8 years of experience with a strong track record in ETL development and AWS data services. Proficiency in Python, SQL, and data processing principles is essential.
Full description
Data Engineer — PySpark + AWS Glue
ETL & Data Pipelines | 2 Openings
Location: Remote (UK)
Employment Type: Contract / Permanent
Experience Level: 4–8 Years
Openings: 2
About the Role
We are looking for a skilled Data Engineer with solid hands-on experience in PySpark, AWS Glue, and ETL development to build and maintain scalable, production-grade data pipelines on AWS. You will be part of a delivery-focused team working on complex data engineering challenges, contributing to data lake architecture and end-to-end pipeline development.
Requirements
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines
using PySpark and AWS Glue
- Build and optimise data ingestion, transformation, and
loading workflows
- Orchestrate data workflows using AWS Step Functions
- Develop serverless functions using AWS Lambda (Python)
- Work with data lake architectures on AWS to support
analytical use cases
- Ensure pipeline reliability, monitoring, and
performance optimisation
Required Skills & Experience
- Strong hands-on experience with PySpark and AWS Glue
- Proven track record in ETL pipeline development and
optimisation
- Experience orchestrating workflows with AWS Step
Functions
- Proficiency in serverless development using AWS Lambda
(Python)
- Good SQL skills and a solid understanding of data
processing principles
Nice to Have
- Exposure to Java-based microservices
- Understanding of REST APIs and backend service
integrations
- Experience working with AWS-based data lakes
