Databricks - Principal Data Engineer

Req ID: 376003

NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.

We are currently seeking a Databricks - Principal Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).

Experience: 10+ years overall | 5+ years hands-on DBT experience Function: Data & Analytics / Modern Data Engineering Role Type: Architecture, Consulting, CoE Leadership & Delivery Enablement

About the Role

We are looking for an experienced DBT Architect / DBT Center of Excellence (CoE) Lead to establish and lead our DBT capability and provide technical leadership across the organization's modern data engineering and analytics initiatives.

The role will be responsible for building the DBT practice, standards, reference architectures, reusable accelerators, estimation frameworks, delivery methodologies, and engineering best practices while supporting global delivery teams across client engagements.

The ideal candidate will combine deep hands-on expertise in DBT, cloud data platforms, modern data engineering, data architecture, and DevOps with strong consulting and stakeholder-management capabilities.

This is a strategic and hands-on leadership role requiring the ability to work across pre-sales, solution architecture, delivery, capability development, modernization programs, and technology roadmaps.

Key Responsibilities

1. DBT Center of Excellence Leadership

• Establish and lead the DBT Center of Excellence across the Data & Analytics practice.

• Define the DBT capability strategy, operating model, governance structure, and roadmap.

• Establish enterprise-wide DBT development standards, coding guidelines, design patterns, naming conventions, testing standards, and deployment practices.

• Define reusable reference architectures and implementation patterns for different DBT adoption scenarios.

• Build a community of DBT architects, engineers, SMEs, and practitioners across delivery organizations.

• Conduct technical forums, knowledge-sharing sessions, workshops, and DBT enablement programs.

• Define competency frameworks and career paths for DBT practitioners.

• Evaluate emerging DBT capabilities and recommend adoption strategies.

2. Solution Architecture & Technical Leadership

• Architect scalable DBT-based modern data platforms and transformation frameworks.

• Design DBT solutions across platforms such as Snowflake, Databricks, Redshift, BigQuery and other cloud data platforms.

• Define DBT project structures, environments, deployment models, CI/CD architecture, orchestration, security, and governance.

• Design solutions for:

• Enterprise data warehouses

• Lakehouse architectures

• Data marts

• Data products

• ELT modernization

• Legacy ETL modernization

• Informatica/SSIS/Ab Initio and other ETL-to-DBT migrations

• Define strategies for incremental processing, snapshots, SCD implementations, data quality, testing, lineage, observability, and performance optimization.

• Provide technical governance and architecture reviews for major DBT implementations.

• Troubleshoot complex performance, scalability, deployment, and architectural challenges.

3. Pre-Sales & Client Consulting

• Support the sales organization in DBT-related pre-sales and client engagements.

• Participate in client discussions, discovery workshops, technical presentations, demonstrations, and solutioning sessions.

• Develop DBT solution proposals, architecture diagrams, migration approaches, and implementation strategies.

• Lead technical responses for RFPs/RFIs/RFQs involving modern data engineering and DBT.

• Identify opportunities to modernize legacy ETL platforms using DBT and cloud-native data platforms.

• Develop solution narratives and differentiators for DBT-based offerings.

• Support account teams in identifying and shaping new DBT opportunities.

• Present technical solutions and roadmaps to senior client stakeholders.

4. Estimation & Delivery Planning

• Define standardized DBT estimation frameworks and productivity benchmarks.

• Develop estimation models based on:

• Number of source systems

• Number of ETL workflows

• Transformation complexity

• SQL complexity

• Number of dependencies

• Data volumes

• Testing requirements

• Migration patterns

• Create effort estimation models for Simple / Medium / Complex DBT development and migration workloads.

• Define team composition, skill requirements, productivity assumptions, and delivery capacity.

• Support delivery teams in developing ROM, indicative, and detailed estimates.

• Define delivery timelines and realistic productivity targets.

5. DBT Migration & Modernization

• Define migration strategies for converting legacy ETL technologies into modern DBT architectures.

• Establish migration assessment frameworks and factory-style delivery models.

• Define migration waves, prioritization criteria, dependency management, and rollout strategies.

• Develop approaches for automated or semi-automated migration of legacy ETL code.

• Establish validation, reconciliation, parallel-run, and production cutover strategies.

• Define approaches for migrating large-scale environments with thousands of ETL workflows and data objects.

• Provide technical oversight for migration programs and resolve complex migration challenges.

6. Accelerators & Automation

• Identify opportunities to build DBT accelerators, frameworks, utilities, and automation tools.

• Lead development of accelerators for:

• ETL-to-DBT conversion

• SQL conversion

• DBT model generation

• Dependency analysis

• Complexity assessment

• Code quality assessment

• Automated testing

• Documentation generation

• Lineage analysis

• Migration assessment

• Effort estimation

• Deployment automation

• DBT project scaffolding

• Explore the use of Generative AI and Agentic AI for DBT development, migration, testing, documentation, and code remediation.

• Establish reusable assets that can improve delivery productivity and reduce implementation effort.

• Define metrics to measure accelerator adoption and productivity improvements.

7. Delivery Enablement & Rollout

• Develop standardized DBT implementation and rollout methodologies.

• Define end-to-end delivery frameworks covering:

Assessment → Architecture → Development → Testing → Deployment → Production → Optimization.

• Create delivery playbooks, templates, checklists, standards, and reference implementations.

• Support large delivery teams during DBT adoption and implementation.

• Establish governance checkpoints and architecture review processes.

• Define production readiness criteria and operational support models.

• Mentor senior architects and engineering leads working on DBT programs.

8. Performance, Cost & Optimization

• Establish DBT performance optimization standards and best practices.

• Optimize DBT workloads for cloud data warehouse/lakehouse platforms.

• Analyze query performance, warehouse utilization, concurrency, workload patterns, and data volumes.

• Develop strategies for optimizing Snowflake/Databricks compute consumption and DBT execution performance.

• Define best practices around materializations, incremental models, clustering/partitioning, query optimization, and workload orchestration.

• Establish frameworks for monitoring DBT performance and cost.

9. Governance, Quality & Engineering Standards

• Define enterprise DBT governance standards covering:

• Code quality

• Testing

• Documentation

• Naming conventions

• Version control

• CI/CD

• Environment management

• Security

• Access control

• Data lineage

• Observability

• Deployment governance

• Define quality gates and automated checks for DBT projects.

• Establish reusable testing frameworks and engineering standards.

• Drive adoption of best practices across delivery teams.

Required Technical Skills

Core DBT

• 5+ years of hands-on DBT experience with strong architectural understanding.

• Strong experience with dbt Core and/or dbt Cloud.

• Expert knowledge of:

• DBT models

• Sources

• Seeds

• Snapshots

• Macros

• Packages

• Tests

• Exposures

• Documentation

• Tags

• Model contracts

• Incremental models

• Materializations

• Hooks

• Variables

• Jinja

• Project configurations

• Strong understanding of DBT DAGs, dependencies, lineage, and orchestration.

• Experience designing enterprise-scale DBT projects.

Cloud Data Platforms

Strong experience with one or more:

• Snowflake

• Databricks

• Redshift

• BigQuery

• Azure Synapse

Strong understanding of cloud data warehouse/lakehouse architecture and optimization.

Data Engineering

• Strong SQL expertise.

• Advanced data modeling skills.

• Dimensional modeling and Data Vault knowledge.

• Strong understanding of ELT/ETL architecture.

• Experience with large-scale data transformation pipelines.

• Understanding of batch and incremental processing.

• SCD Type 1/Type 2 implementation.

• Data quality and reconciliation frameworks.

• Data lineage and metadata management.

DevOps & Engineering

• Git and Git-based development workflows.

• CI/CD implementation for DBT.

• Azure DevOps, GitHub Actions, GitLab, Jenkins, or similar.

• Environment and release management.

• Automated testing and quality gates.

• Infrastructure/configuration management concepts.

• Experience integrating DBT with enterprise orchestration tools.

Legacy Modernization

Experience with one or more legacy ETL technologies such as:

• Informatica

• Informatica PowerCenter / IDMC

• SSIS

• Ab Initio

• DataStage

• Talend

• Pentaho

Experience leading ETL modernization or migration to DBT/cloud data platforms is highly desirable.

AI & Emerging Technology Skills

The candidate should demonstrate awareness of how AI can accelerate modern data engineering.

Experience with or understanding of:

• Generative AI for code generation and conversion

• LLM-based SQL/code modernization

• Agentic AI for data engineering workflows

• Automated ETL-to-DBT conversion

• AI-assisted testing and remediation

• AI-based code quality and complexity analysis

• Metadata-driven automation

• RAG-based enterprise engineering assistants

Experience building or architecting AI-powered DBT migration accelerators will be a strong advantage.

Consulting & Leadership Skills

• Excellent communication and presentation skills.

• Ability to communicate complex technical concepts to both technical and business stakeholders.

• Strong client-facing consulting experience.

• Ability to lead architecture discussions with senior client stakeholders.

• Strong problem-solving and analytical skills.

• Ability to influence technical decisions across distributed/global teams.

• Experience mentoring architects and senior engineers.

• Ability to operate effectively in a matrixed delivery organization.

• Strong commercial awareness and understanding of delivery economics.

• Ability to translate business requirements into scalable technical solutions.

Preferred Certifications

One or more of the following would be desirable:

• dbt Analytics Engineering Certification

• Snowflake certifications

• Databricks certifications

• AWS / Azure / GCP certifications

• Data engineering or cloud architecture certifications

Experience Profile

Must Have

• 10+ years of overall IT/data engineering experience

• 5+ years of hands-on DBT experience

• Strong SQL and data modeling expertise

• Strong experience with Snowflake, Databricks, or another modern cloud data platform

• Experience designing enterprise-scale data transformation solutions

• Experience leading technical teams or architecture initiatives

• Experience with DBT standards, governance, CI/CD, testing, and deployment

• Strong client-facing and consulting experience

Preferred

• Experience establishing or leading a DBT CoE

• Experience in large-scale Informatica/legacy ETL-to-DBT migration

• Experience supporting RFP/RFI/pre-sales activities

• Experience creating estimation frameworks and delivery methodologies

• Experience developing DBT accelerators and automation

• Experience with GenAI/Agentic AI for data engineering

• Experience working across global delivery organizations

• Experience managing large-scale DBT modernization programs

About NTT DATA

NTT DATA is a $30 billion business and technology services leader, serving 75% of the Fortune Global 100. We are committed to accelerating client success and positively impacting society through responsible innovation. We are one of the world's leading AI and digital infrastructure providers, with unmatched capabilities in enterprise-scale AI, cloud, security, connectivity, data centers and application services. our consulting and Industry solutions help organizations and society move confidently and sustainably into the digital future. As a Global Top Employer, we have experts in more than 50 countries. We also offer clients access to a robust ecosystem of innovation centers as well as established and start-up partners. NTT DATA is a part of NTT Group, which invests over $3 billion each year in R&D.

Whenever possible, we hire locally to NTT DATA offices or client sites. This ensures we can provide timely and effective support tailored to each client’s needs. While many positions offer remote or hybrid work options, these arrangements are subject to change based on client requirements. For employees near an NTT DATA office or client site, in-office attendance may be required for meetings or events, depending on business needs. At NTT DATA, we are committed to staying flexible and meeting the evolving needs of both our clients and employees. NTT DATA recruiters will never ask for payment or banking information and will only use @nttdata.com, @nttdatafed.com and @talent.nttdataservices.com email addresses. If you are requested to provide payment or disclose banking information, please submit a contact us form, https://us.nttdata.com/en/contact-us.

NTT DATA endeavors to make https://us.nttdata.com accessible to any and all users. If you would like to contact us regarding the accessibility of our website or need assistance completing the application process, please contact us at https://us.nttdata.com/en/contact-us. This contact information is for accommodation requests only and cannot be used to inquire about the status of applications. NTT DATA is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status. For our EEO Policy Statement, please click here. If you'd like more information on your EEO rights under the law, please click here. For Pay Transparency information, please click here.