Data Engineer

We are looking for a Data Engineer with strong hands-on experience in Microsoft Fabric to design, build, optimize, and support scalable data engineering solutions. The role focuses on data ingestion and transformation, Lakehouse and Warehouse development, data pipelines, and reliable delivery of curated, analytics-ready data. The successful candidate will work closely with data analysts, Power BI developers , application teams, and business stakeholders to translate business needs into secure, maintainable, high-performing data solutions.

Requirements

Key Responsibilities

Microsoft Fabric & Data Platform Development

· Design, develop, and maintain end-to-end data engineering solutions using Microsoft Fabric.

· Build and manage Fabric Data Factory pipelines for batch, scheduled, and incremental data ingestion.

· Develop and maintain Fabric Lakehouse and Warehouse solutions, using OneLake for centralized data storage and management.

· Use Fabric notebooks, Spark, SQL analytics, and Dataflows Gen2 to process and transform data.

· Apply Medallion Architecture (Bronze, Silver, and Gold) where appropriate for data organization, quality, and reuse.

Data Ingestion, Transformation & Modeling

· Develop robust ETL/ELT processes to ingest data from databases, REST APIs, files, and cloud/SaaS applications.

· Implement incremental loads, change data capture (CDC), slowly changing dimensions (SCD), partitioning, and other suitable processing strategies.

· Develop and optimize analytical data models, including dimensional models and star/snowflake schemas.

· Use SQL, Python, PySpark, and Spark SQL to transform, validate, and prepare data for reporting and analytics.

· Implement data quality checks, validation, reconciliation, error handling, and appropriate recovery mechanisms.

Performance, Reliability & Operations

· Monitor pipeline execution and data processing; troubleshoot failures and perform root-cause analysis.

· Optimize Spark jobs, SQL queries, pipeline design, and storage usage for performance, scalability, and cost efficiency.

· Implement logging, monitoring, alerting, and production-support practices to improve platform reliability.

· Support issue resolution and continuous improvement across data pipelines and platform workloads.

Security, Governance & Deployment

· Apply appropriate security controls, workspace access, identity and access management, and data access practices with platform and security teams.

· Follow data governance, documentation, and engineering standards; support data lineage and governance requirements where applicable.

· Use Git-based source control and support CI/CD and deployment processes for Fabric solutions through Azure DevOps or GitHub.

· Document data architecture, pipelines, data models, dependencies, operational procedures, and technical decisions.

Collaboration & Business Alignment

· Work with Power BI teams to provide curated, trusted, analytics-ready datasets and support semantic-model requirements.

· Collaborate with analysts, data scientists, application teams, and business stakeholders to understand requirements and deliver practical solutions.

· Contribute to design reviews, estimation, sprint planning, testing, release activities, and continuous improvement in an Agile delivery environment.

Required Qualifications

Experience & Education

· 3+ years of experience in data engineering, with mandatory hands-on experience developing solutions in Microsoft Fabric.

· Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, Data Science, or a related field; equivalent professional experience may be considered.

· Experience building end-to-end data pipelines and supporting data solutions through development, deployment, and production.

Microsoft Fabric Skills

· Hands-on experience with Fabric Data Factory / Data Pipelines, Lakehouse, Warehouse, OneLake, notebooks, Spark, SQL analytics, and Dataflows Gen2.

· Understanding of Microsoft Fabric architecture, workspaces, and end-to-end data processing patterns.

· Ability to develop, troubleshoot, and optimize production-ready data pipelines in Microsoft Fabric.

· Familiarity with Fabric data agents and their potential use in data discovery or conversational access to data.

Core Data Engineering Skills

· Strong SQL skills, including complex queries, joins, CTEs, window functions, and query optimization.

· Strong programming knowledge in Python and/or PySpark, with practical experience in data transformation and processing.

· Solid understanding of ETL/ELT concepts, batch and incremental processing, data warehousing, and dimensional modeling.

· Experience with data quality, validation, reconciliation, error handling, and reliable pipeline design.

· Working understanding of cloud security, identity, access management, and secure handling of enterprise data.

DevOps & Engineering Practices

· Experience with Git and source control, and familiarity with Azure DevOps or GitHub.

· Understanding of CI/CD pipelines, automated deployment, environment management, and release practices.

· Experience with monitoring, logging, troubleshooting, and production support.

Preferred Qualifications

· Microsoft Certified: Fabric Data Engineer Associate (DP-700) is preferred; Microsoft Certified: Fabric Analytics Engineer Associate (DP-600) is also relevant.

· Experience migrating data solutions from Azure Synapse Analytics, Azure Data Factory, Databricks, or traditional data warehouses to Microsoft Fabric.

· Experience with Power BI semantic models and DAX.

· Knowledge of Microsoft Purview, data governance, data lineage, and compliance practices.

· Exposure to real-time analytics, Eventstream, or Real-Time Intelligence in Microsoft Fabric.

· Experience integrating REST APIs and SaaS data sources.

· Familiarity with Microsoft Azure AI Foundry / Microsoft Foundry and its data or AI-related capabilities.

· Experience working in Agile/Scrum teams.

Preferred Candidate Profile

· Strong analytical thinking, problem-solving, and attention to data quality.

· Able to work independently while collaborating effectively with cross-functional teams.

· Clear communication and technical documentation skills.

· Able to translate business requirements into scalable and maintainable data solutions.

· Takes ownership of data engineering solutions from design and development through deployment and production support.

· Shows initiative in learning and applying evolving Microsoft Fabric and modern data engineering capabilities.

Key Success Measures

· Reliable, maintainable, and well-documented Fabric pipelines delivered to agreed timelines.

· Improved data quality, reconciliation, and availability of trusted analytics-ready datasets.

· Effective pipeline and query performance, with appropriate cost and resource optimization.

· Timely resolution of pipeline issues and adherence to security and engineering standards.

· Effective collaboration with reporting, analytics, application, and business teams.​