Data Senior Engineer

NTT DATA Services Bangalore, Karnataka, India

IT Services and IT Consulting · 10,001+ employees

5 d ago

Senior (5-10 yrs) Full-time India

Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.

About the role

The role involves migrating complex Informatica workflows to Python EL scripts and dbt transformation models. It also includes orchestrating data pipelines using Airflow and ensuring data quality through rigorous testing and performance troubleshooting.

What they look for

Informatica Python Dbt SQLAlchemy Oracle SQL Server Airflow GitLab CI/CD Data Warehousing ETL Data Quality Unit Testing Performance Tuning Data Migration SQL

Requirements

Candidates must have extensive experience in data engineering, specifically with Informatica, Python, and dbt. Proficiency in SQL, database management systems like Oracle and SQL Server, and CI/CD practices is required.

Full description

Key Responsibilities

Assess complex Informatica workflows including sessions, mappings, and parameterizations and translate them into equivalent Python EL scripts and dbt transformation models.
Develop and maintain Python EL pipelines to land high-volume data, including billion-row tables, into the target warehouse using SQLAlchemy, cx_Oracle, pyodbc, and bulk export tools such as Oracle Data Pump and SQL Server BCP.• Design and develop dbt transformation models based on Informatica mapping logic, incorporating dbt best practices including model layering, macros, incremental strategies, and snapshot patterns.
Develop dbt-native tests as well as custom Python unit tests to validate transformation correctness and data quality.
Develop Airflow DAGs to orchestrate Python EL and dbt scripts end-to-end, producing output that is functionally equivalent to the source Informatica workflows.
Contribute to GitLab CI/CD pipeline for dbt and Airflow code, including lint gates, automated testing, and deployment to shared NAS.
Perform peer code reviews and provide constructive technical feedback to fellow engineers.
Troubleshoot performance issues and data discrepancies during SIT and UAT, including row-count reconciliation between source Oracle/SQL Server systems and the target warehouse.
Contribute to technical documentation, runbooks, and handover materials.