Data Engineer Senior Consultant

NTT DATA Services Bangalore, Karnataka, India

IT Services and IT Consulting · 10,001+ employees

2 d ago

Senior (5-10 yrs) Full-time India

Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.

About the role

Translate Informatica workflows into Python extract-load pipelines and dbt models, and orchestrate end-to-end processing with Airflow. Validate data quality and functional equivalence, troubleshoot performance and data discrepancies, contribute to CI/CD and code reviews, and prepare technical documentation and handover materials.

What they look for

Informatica Python Extract-Load Pipelines dbt Apache Airflow SQLAlchemy cx_Oracle pyodbc Oracle Data Pump SQL Server BCP GitLab CI/CD Data Quality Testing Data Warehousing SQL Performance Troubleshooting Code Review

Requirements

The role calls for experience assessing complex Informatica workflows and implementing equivalent Python and dbt solutions for high-volume data. It also requires skills in Airflow orchestration, automated testing, GitLab CI/CD, performance troubleshooting, and source-to-warehouse reconciliation.

Full description

Key Responsibilities

Assess complex Informatica workflows including sessions, mappings, and parameterizations and translate them into equivalent Python EL scripts and dbt transformation models.
Develop and maintain Python EL pipelines to land high-volume data, including billion-row tables, into the target warehouse using SQLAlchemy, cx_Oracle, pyodbc, and bulk export tools such as Oracle Data Pump and SQL Server BCP.• Design and develop dbt transformation models based on Informatica mapping logic, incorporating dbt best practices including model layering, macros, incremental strategies, and snapshot patterns.
Develop dbt-native tests as well as custom Python unit tests to validate transformation correctness and data quality.
Develop Airflow DAGs to orchestrate Python EL and dbt scripts end-to-end, producing output that is functionally equivalent to the source Informatica workflows.
Contribute to GitLab CI/CD pipeline for dbt and Airflow code, including lint gates, automated testing, and deployment to shared NAS.
Perform peer code reviews and provide constructive technical feedback to fellow engineers.
Troubleshoot performance issues and data discrepancies during SIT and UAT, including row-count reconciliation between source Oracle/SQL Server systems and the target warehouse.
Contribute to technical documentation, runbooks, and handover materials.