Senior Data Engineer (Technical Lead)

We are seeking an experienced and hands-on Technical Lead (Data Engineering) to lead a team of Data Engineers responsible for delivering enterprise data solutions. The successful candidate will possess strong technical expertise in Python, Microsoft SQL Server, Data Engineering, Data Quality, Database Performance Optimization, and DevOps practices.

This role requires managing a team of 5 to 8 engineers while actively participating in architecture, development, troubleshooting, and project delivery activities. The ideal candidate should be able to mentor team members, drive technical excellence, and step into hands-on development activities whenever project timelines demand additional support.

Role and Responsibilities

Technical Leadership

• Lead, mentor, and manage a team of 5 to 8 Data Engineers.

• Provide technical leadership and guidance throughout the software development lifecycle.

• Conduct design reviews, code reviews, and technical solution reviews.

• Establish and enforce development standards, coding best practices, and quality controls.

• Support team members in resolving complex technical and production issues.

Data Engineering & Development

• Design, develop, and support enterprise-scale data engineering solutions using Python and Microsoft SQL.

• Develop and maintain robust ETL/ELT processes for data ingestion, transformation, and validation.

• Build scalable and maintainable data pipelines to support business reporting, analytics, and operational needs.

• Develop reusable frameworks and utilities to improve development efficiency and maintainability.

Data Quality Management

• Design and implement data quality frameworks and controls across data pipelines.

• Establish automated validation, reconciliation, completeness, consistency, and accuracy checks.

• Investigate and resolve data quality issues through root cause analysis and remediation plans.

• Continuously improve data reliability and governance practices.

SQL Performance Tuning & Optimization

• Analyze and optimize complex SQL queries, stored procedures, views, and database objects.

• Perform execution plan analysis, indexing reviews, partitioning reviews, and query optimization.

• Troubleshoot production performance issues and implement long-term performance improvements.

• Ensure database solutions are designed for scalability, stability, and efficiency.

DevOps & Release Management

• Implement and manage source code repositories using Git.

• Drive DevOps best practices across the development lifecycle.

• Manage and enforce branching strategies

• Support CI/CD deployment pipelines, release management, and change implementation processes.

• Ensure application deployments follow governance, quality, and security standards.

Documentation & Governance

• Prepare and maintain comprehensive technical documentation including: Technical Specifications, Source-to-Target Mapping Documents, Deployment Guides, Operational Runbooks, and Support Procedures.

• Ensure documentation is complete, accurate, and maintained throughout the project lifecycle.

• Support audit, compliance, and governance requirements.

Delivery & Stakeholder Management

• Work closely with Business Analysts, Solution Architects, Project Managers, QA teams, and business stakeholders.

• Participate in project planning, estimation, and resource allocation activities.

• Manage technical risks, dependencies, and delivery challenges proactively.

• Provide regular status updates and recommendations to management.

Hands-On Development

• Actively participate in development activities when required.

• Support the team during critical project phases and aggressive delivery timelines.

• Take ownership of complex development tasks to ensure successful project delivery.

• Lead by example through hands-on problem solving, troubleshooting, and implementation

QUALIFICATIONS AND EDUCATION REQUIREMENTS

• Bachelor's Degree in Computer Science, Information Technology, Engineering, or a related discipline.

• 10 to 12 years of relevant IT experience in Data Engineering and Application Development.

• Minimum 4 years of experience in leading technical teams.

• Proven experience delivering enterprise-scale data engineering solutions.

REQUIRED TECHNICAL SKILLS

Data Engineering

• Strong experience designing and implementing Data Engineering solutions.

• Extensive experience developing ETL/ELT processes and data integration frameworks.

• Experience with large-scale data processing and enterprise data platforms.

Python

• Strong hands-on experience developing Python applications, utilities, and data processing frameworks.

• Experience using Python for data transformation, automation, and workflow development.

Microsoft SQL Server

• Advanced SQL and T-SQL programming expertise.

• Strong knowledge of: Stored Procedures, Views, Functions, Query Optimization, Performance Tuning, Indexing Strategies, Database Design, Execution Plan Analysis, and Partitioning

Data Quality

• Experience implementing data quality controls and monitoring frameworks.

• Strong understanding of data validation, reconciliation, and governance principles.

DevOps

• Experience with Git-based source control and branching strategies.

• Experience implementing CI/CD deployment processes.

• Knowledge of automated build, deployment, and release management practices.