EXL

Associate - Project Manager

EXL Haryana, India

Business Consulting and Services · 10,001+ employees

5 d ago
project-manager Senior (5-10 yrs) Full-time India
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

The role involves building and maintaining data pipelines to ingest sources into the Fabric platform while implementing standardization and transformation logic. Additionally, the associate will manage data quality checks, monitoring, and performance optimization for the entity resolution engine.

What they look for

PySpark SQL Data Engineering Delta Lake Lakehouse Architecture Microsoft Fabric Change Data Capture Data Quality Pipeline Development Spark Notebooks Data Transformation Performance Tuning Great Expectations Master Data Management Data Ingestion

Requirements

Candidates must have hands-on experience in data engineering with strong proficiency in PySpark and SQL. Experience with Delta Lake, medallion architecture, and implementing incremental data loads is required for this position.

Full description

Build and operate the data pipelines that feed the Entity Hub. This role lands all six in-scope sources into Fabric, implements standardization and transformation logic, and maintains the data quality checks and monitoring that the entity resolution engine depends on. Reliable, observable ingestion is the foundation the entire programme rests on.

Responsibilities

  • Ingestion development — build and maintain pipelines to land the six in-scope sources (Secretary of State, D&B, ARROW, E1, hCue, DocCentral) into the Fabric Bronze/raw layer.
  • Mirroring & CDC — implement Fabric Mirroring for supported structured sources and establish change-data-capture patterns; implement watermark/incremental load logic where mirroring is unavailable.
  • Raw layer management — maintain one Delta table per source on an append-only basis, retaining evidence records and full source provenance.
  • Standardization & transformation — implement name normalization, address parsing and attribute standardization logic in Spark notebooks; support identifier-spine construction.
  • Data quality — implement data quality checks, validation rules, threshold alerts and exception handling; support reconciliation against source.
  • Pipeline operations — schedule, monitor and troubleshoot pipeline runs; investigate failures and performance issues; maintain run documentation.
  • Performance tuning — optimise Spark jobs, Delta file sizes, partitioning and pipeline efficiency to manage Fabric capacity consumption.
  • Documentation — produce and maintain source-to-target mappings, transformation logic documentation and lineage records.

Qualifications

  • + years hands-on data engineering with strong PySpark and SQL
  • Production experience building ingestion pipelines from multiple heterogeneous sources
  • Working knowledge of Delta Lake and medallion/lakehouse architecture
  • Experience implementing incremental loads and CDC-style processing
  • Experience implementing data quality checks and troubleshooting pipeline failures

Nice-to-Have

  • Microsoft Fabric hands-on experience (Mirroring, Copy Jobs, Environments)
  • Exposure to entity/master data standardization (name and address parsing)
  • Familiarity with libraries such as Great Expectations for data quality
  • Experience optimising for Fabric capacity/CU consumption

Key Deliverables Owned

  • Operational ingestion pipelines for all agreed sources
  • Bronze/raw layer with one Delta table per source and CDC retained
  • Standardization and parsing transformation logic
  • Data quality checks, monitoring and exception handling
  • Source-to-target mapping and run documentation

Similar roles