About the role

Develop and maintain ETL pipelines on AWS cloud while overseeing data lakes and warehouses. Lead projects by collaborating with clients to provide technical suggestions and establish data quality standards.

What they look for

Python Advanced SQL PySpark AWS ETL Pipelines Data Modeling Spark Architecture Airflow SSIS Data Governance CI/CD Terraform Power BI Kafka Kinesis DBT

Requirements

Requires a bachelor's degree and over 8 years of experience in data migration, specifically with SSIS and AWS. Proficiency in Python, advanced SQL, and PySpark is mandatory.

Full description

Why Join Iris?Are you ready to do the best work of your career at one of India’s Top 25 Best Workplaces in IT industry? Do you want to grow in an award-winning culture that truly values your talent and ambitions?Join Iris Software — one of the fastest-growing IT services companies — where you own and shape your success story.

About Us At Iris Software, our vision is to be our client’s most trusted technology partner, and the first choice for the industry’s top professionals to realize their full potential.

With over 4,300 associates across India, U.S.A, and Canada, we help our enterprise clients thrive with technology-enabled transformation across financial services, healthcare, transportation & logistics, and professional services.

Our work covers complex, mission-critical applications with the latest technologies, such as high-value complex Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation.

Working with UsAt Iris, every role is more than a job — it’s a launchpad for growth.

Our Employee Value Proposition, “Build Your Future. Own Your Journey.” reflects our belief that people thrive when they have ownership of their career and the right opportunities to shape it.

We foster a culture where your potential is valued, your voice matters, and your work creates real impact. With cutting-edge projects, personalized career development, continuous learning and mentorship, we support you to grow and become your best — both personally and professionally.

Curious what it’s like to work at Iris? Head to this video for an inside look at the people, the passion, and the possibilities. Watch it here.

Job Description

Must Have -

Python & Advanced SQL
Spark / distributed processing / PySpark
Cloud platforms (AWS) – Modules like Lambda, Glue, EMR, Dynamo DB, S3, Redshift, EC2, RDS, DMS, Step Functions, Cloud Watch, Systems Manager, Event Bridge, DMS, Boto3 API’s
Data pipeline automation

Data governance & quality. Preferred if worked on AWS Data Zone.

Any orchestrator like Airflow, Step Functions

Knowledge of any git repository like GitHub, GitLab, Bitbucket.

Nice to have -

Kafka/Kinesis streaming
Basic knowledge of AI
Power BI

Postgre SQL

DBT

Experience with CI/CD implementation using Terraform or TypeScript.

Job Summary:

We are seeking a Lead Data Engineer who is able to develop and maintain ETL Pipelines preferably on AWS cloud. Should be able to lead a project that includes working with client and come up with suggestions as and when required. Oversee data pipelines, data lakes, data warehouses, and ETL/ELT processes. Establish data quality standards, monitor system health and drive continuous improvements. Should be proficient in Python, advanced SQL and PySpark. Sound understanding of Spark architecture. Basic understanding of AI is essential. Knowledge of DBT, Postgre database and Power BI would be an added advantage.

Key Responsibilities:

Hands on in Python and advanced SQL.
Data modelling with good understanding of dimensions and facts.
Good understanding of Spark eco-system and PySpark programming.
Able to convert existing SSIS workflows to ETL pipelines running on AWS. That includes knowledge of Lambda, Glue or EMR, Systems Manager, Event Bridge, EC2, DMS
Knowledge of Dynamo DB, Redshift, RDS (Preferably Postgre)
Sound knowledge of AWS Boto3 API’s
Awareness of reporting tool like Power BI.
Monitor and troubleshoot migration processes, resolving issues in a timely manner.
Work closely with business stakeholders, developers, and administrators to understand data requirements.
Good understanding of a Data Lakehouse
Understanding of CI/CD process with exposure to TypeScript and any Git repository like GitHub, GitLab or Bitbucket
Document migration procedures, processes, and best practices.

Required Skills and Qualifications:

Bachelor’s degree in Computer Science, Information Technology, or related field.
8+ years of experience in data migration or similar roles.
Strong proficiency in SSIS.
Proven experience in ETL tools like SSIS and data migration pipelines build on AWS.
Familiarity with data modeling and schema design principles.
Ability to analyze and resolve complex data-related issues.
Strong analytical and problem-solving abilities.
Excellent communication and collaboration skills.
Ability to work independently and in a team-oriented environment.

Preferred:

Familiarity with Agile project management methodologies.

Mandatory Competencies

Data Science and Machine Learning - Data Science and Machine Learning - Python

Data Science and Machine Learning - Data Science and Machine Learning - Apache Spark

Data & AI - Data Engineering - Data Quality & Validation

Big Data - Big Data - Pyspark

Database - Database Programming - SQL

Beh - Communication and collaboration

Perks and Benefits for IrisiansIris provides world-class benefits for a personalized employee experience. These benefits are designed to support financial, health and well-being needs of Irisians for a holistic professional and personal growth. Click here to view the benefits.