Lead Data Engineer

Mandatory Skill Set: SQL + Python + PySpark + Azure Synapse

Role Overview

We are seeking a highly experienced Lead Data Engineer to drive the strategy, architecture, delivery, and optimization of enterprise-scale data platforms on Microsoft Azure. The ideal candidate will combine deep technical expertise with strong leadership, stakeholder management, and delivery governance capabilities to build scalable, secure, and cost-efficient data solutions.

Must-Have Skills

• Strong

expertise in SQL, Python, and PySpark

• Extensive

experience with Azure Data Factory (ADF) and Azure Synapse Analytics.

• Hands-on

experience with Delta Lake architecture and modern data lakehouse implementations.

• Advanced

knowledge of Data Modeling and Data Warehousing concepts.

• Strong

exposure to the Microsoft Fabric ecosystem.

• Proven

expertise in designing cloud-native data architectures and enterprise-scale solutions.

• Strong design thinking and problem-solving capabilities.

• Working

knowledge of CI/CD pipelines and DevOps practices.

• Experience

in cloud cost optimization, performance tuning, and resource governance.

Key Responsibilities

• Lead

and own the end-to-end delivery of enterprise data platforms, including architecture, development, deployment, and optimization.

• Define

and drive the data platform roadmap, ensuring alignment with business objectives and strategic priorities.

• Architect

and govern scalable data pipelines, data models, data warehouses, and lakehouse solutions using Azure services.

• Provide

technical leadership to Data Engineering teams through design reviews, code reviews, and implementation of engineering best practices.

• Drive

the adoption and implementation of Microsoft Fabric, APIs, Azure Functions, and Python-based solutions.

• Ensure

platform scalability, reliability, security, and operational excellence while optimizing cloud costs.

• Establish

and enforce data governance, data quality, security, and compliance standards.

• Implement

and drive CI/CD, DevOps, and Agile delivery practices across the data engineering landscape.

• Collaborate

with business stakeholders, architects, and technology teams to translate business requirements into scalable technical solutions.

• Mentor,

coach, and develop Data Engineering talent, fostering a culture of continuous learning and innovation.

• Leverage GitHub Copilot and AI-assisted engineering tools to improve productivity

and solution quality.

• Drive

innovation by evaluating emerging technologies and identifying opportunities for platform modernization.

Experience

• 10+

years of overall IT experience, including 10+ years in Data Engineering.

• Minimum 5+ years of leadership experience managing large-scale Data

Engineering programs and teams.

• Proven

track record of leading cross-functional teams and delivering complex enterprise data transformation initiatives.

• Experience

building, hiring, and scaling high-performing Data Engineering teams.

• Experience

working within multi-vendor and globally distributed delivery environments.

• Strong

experience in stakeholder management, delivery governance, and executive-level communication.

Preferred Skills

• Experience

building and managing enterprise APIs and Azure Function Apps.

• Exposure

to framework-based data engineering development approaches.

• Experience

in solution architecture, stakeholder engagement, and delivery governance.

• Azure

certifications such as:

• Azure

Data Engineer Associate

• Azure

Solutions Architect Expert

• Familiarity

with MLOps, Data Science, and AI-driven analytics solutions.

• Hands-on

experience with Power BI and enterprise reporting platforms.

• Experience

leveraging AI and Generative AI capabilities within Data Engineering ecosystems.

Key Leadership Traits

• Strong

ownership mindset with a focus on accountability and business outcomes.

• Excellent

communication, stakeholder management, and influencing skills.

• Ability

to balance technical leadership with delivery and people management responsibilities.

• Strategic

thinker with a strong focus on execution excellence.

• Strong

conflict resolution and cross-functional collaboration skills.

• Adaptability

and resilience in fast-paced, evolving environments.

• Passion

for mentoring teams and fostering a culture of innovation and continuous improvement.