Data Quality Engineer (Data QE)

Job Summary

We are seeking an experienced Data Quality Engineer (Data QE) to ensure the quality, accuracy, reliability, and integrity of data across enterprise data platforms. The ideal candidate will have strong experience in Data Warehousing, Medallion Architecture, ETL/ELT testing, Data Governance, Data Quality frameworks, Data Lineage, and test automation. This role will be responsible for validating data pipelines from multiple source systems such as CDP, GA4, BigQuery, and Azure Blob Storage into Snowflake, and ensuring accurate reporting in Power BI.

Required Experience

• __

years of experience in Data Quality Engineering, Data Warehouse Testing, or Data Validation.

• Strong

hands-on experience with PostgreSQL, Microsoft SQL Server, Snowflake and BigQuery.

• Experience

validating enterprise Data Warehouse solutions and Medallion Architecture implementations.

• Proven

experience performing Source-to-Target validation from CDP, GA4, BigQuery, Blob Storage, PostgreSQL, and MSSQL into Snowflake.

• Experience

in validating Power BI reports, dashboards, semantic models, and KPIs.

• Strong

understanding of ETL/ELT data processing, data transformations, and data lineage.

• Experience

working with Data Governance, Data Quality, Metadata Management, Data Lineage, Data Dictionary, and Data Catalog solutions.

• Experience

with Data Quality tools such as Soda or equivalent.

• Experience

designing, executing, and automating test cases for large-scale data platforms using SQL, Python, and PySpark.

• Experience

integrating automated testing into CI/CD pipelines using Azure DevOps or GitHub Actions.

Key Responsibilities

Data Validation & Testing

• Perform

end-to-end data validation across the data lifecycle, including ingestion, transformation, storage, and reporting.

• Validate

data movement from source systems including:

• Customer

Data Platforms (CDP)

• Google

Analytics 4 (GA4)

• Google

BigQuery

• Azure

Blob Storage

• PostgreSQL

• MSSQL

• Snowflake

• Perform

comprehensive Source-to-Target (S2T) data validation between source systems and Snowflake.

• Validate

data accuracy, completeness, consistency, uniqueness, and timeliness across datasets.

• Execute

data reconciliation and data profiling activities.

• Conduct

source-to-target validation across databases, data lakes, and data warehouses.

• Validate

schema changes, constraints, indexes, stored procedures, functions, triggers, and views.

Medallion Architecture Validation

• Validate

data across Bronze, Silver, and Gold layers within the Medallion Architecture.

• Verify

data transformations, cleansing, aggregations, and business rules between layers.

• Ensure

data quality controls are enforced throughout the Medallion framework.

ETL/ELT Testing

• Test

and validate ETL/ELT pipelines and data transformations.

• Validate

business mappings, transformation logic, and data lineage.

• Verify

incremental and full-load processes.

• Perform

regression, integration, functional, and end-to-end testing of data pipelines.

• Identify

and troubleshoot data discrepancies across source and target systems.

Snowflake Data Validation

• Validate

data ingestion, transformations, and storage within Snowflake.

• Perform

large-scale data validation using SQL and automation scripts.

• Validate

Snowflake views, tables, materialized views, stored procedures, and data sharing mechanisms.

Power BI Testing

• Validate

datasets, data models, measures, KPIs, and dashboards.

• Ensure

Power BI reports accurately reflect Snowflake data.

• Verify

report-level calculations, filters, row-level security, and business metrics.

• Perform

end-to-end testing from source systems through Snowflake into Power BI.

Data Governance & Data Management

• Validate

implementation of Data Governance policies and standards.

• Ensure

compliance with enterprise data quality and governance requirements.

• Validate

and maintain:

• Data

Lineage

• Data

Dictionary

• Metadata

Management

• Business

Glossary

• Data

Catalogs

• Collaborate

with Data Governance and Business teams to improve data quality processes.

Data Quality Frameworks

• Implement

and execute data quality checks and monitoring solutions.

• Define

and monitor Data Quality KPIs and metrics.

• Develop

and automate validation rules for:

• Completeness

• Accuracy

• Consistency

• Uniqueness

• Validity

• Timeliness

• Support

root cause analysis and remediation of data quality issues.

Test Automation

• Develop

automated data validation frameworks and reusable test suites.

• Automate

source-to-target reconciliation and regression testing.

• Integrate

automated data quality tests into CI/CD pipelines.

• Build

automated validation using SQL, Python, PySpark, and Data Quality tools.

Required Technical Skills

Databases & Data Warehousing

• PostgreSQL

• Microsoft

SQL Server (MSSQL)

• Snowflake

• BigQuery

• Data

Warehouse Testing

• Data

Modeling Concepts

• Star

Schema & Snowflake Schema

• Slowly

Changing Dimensions (SCD)

• Fact

& Dimension Validation

• CDC

Validation

• Data

Migration Testing

SQL Expertise

• Advanced

SQL Query Writing

• Complex

Joins

• Window

Functions

• Aggregations

• Query

Optimization

• Data

Reconciliation Queries

• Data

Profiling and Data Analysis

Modern Data Platform Validation

• Medallion

Architecture Validation (Bronze, Silver, Gold)

• Source-to-Target

Testing

• ETL/ELT

Testing

• Data

Lake & Data Warehouse Validation

• Power

BI Report & Dashboard Validation