About the role
Design and execute end-to-end ELT pipelines to migrate structured data from SQL sources into Snowflake. Orchestrate bulk file transfers and ensure data integrity through reconciliation and validation checks.
What they look for
Requirements
Requires hands-on experience with ELT/ETL pipelines, SQL performance tuning, and Snowflake data loading strategies. Proficiency in Azure Data Lake Storage and file processing patterns is essential for this role.
Full description
About Job
Primary Focus: SQL-to-Snowflake Data Transfer, ELT Pipelines & File-to-ADLS Ingestion
Role Overview
We are seeking an experienced Data Engineer to support a high-priority data integration and cloud enablement initiative. The primary focus of this role is designing and executing end-to-end ELT pipelines and data transfer workflows—migrating structured data from SQL sources into Snowflake and orchestrating bulk file transfers into Azure Data Lake Storage (ADLS).
Responsibilities
- Design, build, and maintain scalable ELT/ETL pipelines and data workflows for ingestion and transformation.
- Execute structured data extraction, movement, and landing from relational SQL databases directly into Snowflake staging and core layers.
- Build, execute, and monitor file movement tasks to efficiently transfer flat files, logs, or unstructured formats into Azure Data Lake Storage (ADLS).
- Write and tune high-performance SQL queries for data modeling, validation, data verification, and staging transformations.
- Conduct data reconciliation and completeness checks to ensure zero loss across data pipelines during bulk migration.
- Collaborate closely with the Lead Data Integration Expert and client engineering teams to align with platform connectivity, security, and governance protocols.
Required Skills
- ELT / Data Pipelines: Proven, hands-on experience building, optimizing, and monitoring production ELT/ETL pipelines and data workflows.
- SQL & Relational Databases: Strong proficiency in SQL (writing complex queries, performance tuning, indexing) for extracting and validating large datasets across relational engines.
- Snowflake: Hands-on experience loading and modeling data in Snowflake using staging strategies, COPY commands, or bulk loading utilities.
- Azure Cloud Storage: Solid background working with Azure Data Lake Storage (ADLS Gen2), Blob Storage, and associated file transfer/ingestion patterns.
- File Processing: Practical experience with bulk file ingestion formats (CSV, Parquet, JSON) and file movement tooling.
- Agile Execution: Ability to deliver rapid, high-quality results within structured project timelines.
Preferred Skills
- Experience with orchestration and transformation tools (e.g., Airflow, dbt, Azure Data Factory, or Python-driven pipeline movers).
- Familiarity with enterprise or industrial data integration platforms.
- Version control using Git.
Similar roles
-
Associate Data Engineer
TriHealth Cincinnati, Ohio, United States
-
Kaipūhanga Raraunga Mātāmua Senior Data Engineer
Ampol Wellington, Wellington, New Zealand · NZ$117K–NZ$156K/yr
-
Senior Salesforce Data Engineer
Jobgether India
-
Data Engineer
Accent Group Careers Melbourne, Victoria, Australia
-
2027 Summer Internship Program - Global Technology & Solutions - Data Engineer
TD Mount Laurel, New Jersey, United States · $64K/yr
-
Data Engineer
Intertec Skopje, North Macedonia