Pearson

Quality Assurance Engineer III– ETL & Data Test Automation

Pearson Pallavaram, Tamil Nadu, India

Education · 10,001+ employees

17 h ago
Senior (5-10 yrs) Full-time India
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

The role involves designing and maintaining automated validation solutions for complex ETL data pipelines and cloud-based platforms. You will collaborate with cross-functional teams to perform root-cause analysis and ensure data integrity across various storage layers.

What they look for

Python SQL ETL Testing Data Validation BigQuery API Automation GCP Data Pipelines Pytest Pandas CI/CD Jenkins GitHub Actions Data Quality REST API dbt

Requirements

Candidates must possess strong expertise in Python and SQL-based test automation frameworks for data validation. Experience with GCP data services and API automation is required to support end-to-end testing and CI/CD integration.

Full description

Job Description: Quality Assurance Engineer III– ETL & Data Test Automation

Role Overview

We are looking for a hands-on Quality Engineering Engineer with strong expertise in ETL/Data Testing Automation and working knowledge of API Automation. The primary responsibility is to build scalable automated validation solutions for data pipelines, data transformations, and cloud-based data platforms.

Key Responsibilities

  • Design, develop, and maintain Python and SQL-based ETL/Data Testing Automation frameworks.
  • Automate source-to-target validation across complex ETL/ELT data pipelines.
  • Validate data across Bronze/Raw, Silver/Curated, and Gold/Consumption layers.
  • ETL & Data Validation: Validate source-to-target data, transformations, schema, record counts, nulls, duplicates, and business rules using BigQuery and SQL.
  • Automation: Develop Python/dbt-based automation for ETL validation, data-quality checks, regression testing, and automated test reporting.
  • GCP: Validate end-to-end data pipelines across BigQuery, GCS, Dataflow, Pub/Sub, and Cloud Composer, including full and incremental data loads.
  • Build reusable automation for:• Source-to-target reconciliation
  • Row-count and aggregate validation
  • Schema and data-type validation
  • Null, duplicate, and mandatory-field checks
  • Data completeness and integrity checks
  • Transformation and business-rule validation
  • Incremental, delta, and historical load validation
  • Data freshness and pipeline validation
  • Develop solutions for validating high-volume datasets using Python, SQL, Pandas, and cloud data platforms.
  • Perform complex SQL validation and troubleshoot data discrepancies across source and target systems.
  • Validate structured and semi-structured data, including JSON and nested datasets.
  • Develop reusable, configuration-driven or metadata-driven automation components to reduce manual testing.
  • Integrate automated ETL/data-quality test suites with CI/CD pipelines such as Jenkins, GitHub Actions, or Azure DevOps.
  • Perform defect investigation and root-cause analysis by collaborating with developers, data engineers, and business teams.
  • Maintain automation code using Git/GitHub, including appropriate branching and pull-request practices.
  • Support API validation as a secondary responsibility, including REST API request/response, status code, JSON payload, authentication, schema, and basic integration validation.
  • Develop and maintain basic API automation using Python Requests/Pytest, Postman/Newman, or REST Assured when required.
  • Contribute to test strategy, regression coverage, release readiness, and overall quality engineering practices.