Senior Data Engineer (Data Harmonization)

🧠 Tech Level: Senior 🧾 Employment type: Full time 🌍 Candidate Location: EU, Georgia, Uzbekistan 🚀 Start: ASAP ⚙️ Project Phase: Ongoing

👥 Customer Description:

The client is a global e-commerce and marketplace platform operating across North America and multiple international regions. The company manages a complex ecosystem of merchant transactions, payments, promotions, and financial operations, supporting millions of customers and partners worldwide.

🧩 Project Description:

Over time, the client's financial data landscape evolved into regionally fragmented platforms with separate data models, pipelines, and reporting logic across geographies. This created challenges in data consistency, reconciliation, auditability, and operational efficiency — particularly for revenue recognition, tax compliance (VAT), and month-end close.

The Data Engineer will be responsible for designing, developing, and optimizing ETL/ELT processes and scalable data solutions on BigQuery, ensuring efficient integration, transformation, and delivery of financial and operational data across North America and International regions.

The role ensures standardized and reliable data pipelines and data models, consistent financial logic (revenue, GL, VAT) across reporting domains, and high-quality, scalable, and auditable data processing workflows.

💡 Hard Skills / Must Have:

Experience designing and building data pipelines (Bronze → Silver → Gold) using SQL-based ELT patterns.

Data ingestion from multiple sources (SFTP, APIs, databases) into BigQuery.

Dimensional modeling: fact and dimension tables, SCD handling, surrogate key logic.

Data quality checks integrated with pipeline execution (row counts, reconciliation, validations).

Apache Airflow DAG development for orchestration, scheduling, and monitoring.

BigQuery performance and cost optimization (partitioning, clustering, query tuning).

CI/CD experience with Git, Flyway, Jenkins.

Schema evolution and deployment practices.

Experience with data integrations with downstream systems (ERP, reporting tools).

📌 Responsibilities:

Develop and maintain data pipelines (Bronze → Silver → Gold) using SQL-based ELT patterns.

Implement data ingestion from multiple sources into BigQuery (raw, append-only).

Build and optimize transformation logic for the curated Mart (Silver) layer.

Ensure reprocessing and idempotency — pipelines must be reliably rerun from Bronze.

Support Airflow DAG development for orchestration and monitoring.

Optimize BigQuery performance and cost.

Contribute to metadata, lineage, and documentation in the data catalog.

Support testing, reconciliation, and parallel run validation during migration from legacy systems.

Troubleshoot pipeline failures and support monitoring and alerting processes.

🧪 Technology Stack:

Cloud & DWH: Google BigQuery, GCP.

Orchestration: Apache Airflow (Cloud Composer).

Databases: BigQuery, AlloyDB / PostgreSQL, Teradata (legacy).

Data Integration: SQL-based ELT, SFTP, APIs (NetSuite, PSPs).

Governance: OpenMetadata (lineage, catalog), IAM/RBAC.

CI/CD & DevOps: GitHub, Jenkins, Flyway, Terraform.

Data Quality: Metadata-driven DQ framework (BigQuery + Airflow).

📩 Ready to Join? We look forward to receiving your application and welcoming you to our team!