JobCubby JobCubby
Sign up
Y

Data Engineer — PySpark + AWS Glue

Yadimen Consulting Limited Pune, Maharashtra, India

IT Services and IT Consulting · 51-200 employees

Jun 01
Remote data-engineer Senior (5-10 yrs) Contractor Full-time India
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

Design, develop, and maintain scalable ETL pipelines using PySpark and AWS Glue. Orchestrate data workflows and ensure pipeline reliability through monitoring and performance optimization.

What they look for

PySpark AWS Glue ETL Development AWS Data Pipelines AWS Step Functions AWS Lambda Python SQL Data Lake Architecture Java REST APIs

Requirements

Requires 4-8 years of experience with strong hands-on skills in PySpark, AWS Glue, and ETL development. Proficiency in AWS services like Step Functions and Lambda is essential for this role.

Full description

Data Engineer — PySpark + AWS Glue

ETL & Data Pipelines | 2 Openings

Location: Remote (UK)

Employment Type: Contract / Permanent

Experience Level: 4–8 Years

Openings: 2

About the Role

We are looking for a skilled Data Engineer with solid hands-on experience in PySpark, AWS Glue, and ETL development to build and maintain scalable, production-grade data pipelines on AWS. You will be part of a delivery-focused team working on complex data engineering challenges, contributing to data lake architecture and end-to-end pipeline development.

Requirements

Key Responsibilities

  • Design, develop, and maintain scalable ETL pipelines

using PySpark and AWS Glue

  • Build and optimise data ingestion, transformation, and

loading workflows

  • Orchestrate data workflows using AWS Step Functions
  • Develop serverless functions using AWS Lambda (Python)
  • Work with data lake architectures on AWS to support

analytical use cases

  • Ensure pipeline reliability, monitoring, and

performance optimisation

Required Skills & Experience

  • Strong hands-on experience with PySpark and AWS Glue
  • Proven track record in ETL pipeline development and

optimisation

  • Experience orchestrating workflows with AWS Step

Functions

  • Proficiency in serverless development using AWS Lambda

(Python)

  • Good SQL skills and a solid understanding of data

processing principles

Nice to Have

  • Exposure to Java-based microservices
  • Understanding of REST APIs and backend service

integrations

  • Experience working with AWS-based data lakes

Similar roles