Senior AWS Data Engineer

We are seeking an experienced Senior AWS Data Engineer to lead and execute a large-scale migration from Matillion ETL to a modern AWS-native data platform. The ideal candidate will possess deep expertise in Apache Airflow, AWS Glue, AWS Lambda and Snowflake, with a strong background in designing scalable data pipelines, cloud-based data integration, workflow orchestration, and data warehouse modernization.

The candidate will work closely with business stakeholders, data architects, and engineering teams to migrate existing ETL workloads, optimize data processing frameworks, and establish best practices for cloud-native data engineering.

Requirements

Key Responsibilities

• Lead

the migration of existing Matillion ETL pipelines to AWS-native solutions.

• Participate

in Requirements Gathering and Analysis of existing data processing jobs.

• Design,

develop, and promote scalable data pipelines using AWS Glue and Apache Airflow (AWS Service).

• Build

robust data ingestion, transformation, and orchestration frameworks on AWS.

• Create

reusable and modular ETL/ELT components for enterprise data platforms.

• Data

modeling and warehousing concepts – must have

• Data

migration and modernization projects – good to have

• Implement

data quality, monitoring, alerting, security, governance, and compliance standards.

• Collaborate

with data architects and business teams to understand migration requirements and translate them into technical solutions.

• Optimize

data processing performance, cost, and scalability across AWS services.

• Participate

in CI/CD best practices and DevOps processes for data engineering workflows.

• Perform

code reviews, mentor junior engineers, and drive engineering excellence.

Mandatory Skills

Core Technologies

• Apache

Airflow (DAG development, workflow orchestration, scheduling, monitoring)

• AWS

Glue (ETL jobs, Glue Studio, Crawlers, Data Catalog, Spark-based transformations)

• Snowflake (Data Warehousing, SnowSQL, Snowpipe, Streams & Tasks, Performance

Tuning)

• AWS Lambda

and Step Functions (preferred)

• Git

experience

Programming languages: Python, SQL, Spark (must)

PySpark, Matillion, Shell scripting (good to have)

Experience Requirements

• 8+

years of overall data engineering experience.

• 4+

years of hands-on experience with AWS data services.

• Proven

experience migrating ETL platforms such as Matillion, Informatica, Talend, or DataStage to AWS-native architectures (Good to have).

• Strong

experience developing workflows in Apache Airflow.

• Extensive

experience building and optimizing AWS Glue jobs.

• Hands-on

experience with Snowflake administration and performance tuning.

• Strong

understanding of cloud architecture, scalability, deployments.