JobCubby JobCubby
Sign up
Unison Group

Databricks Data Engineer (Spark AND Unity)

Unison Group Bengaluru, Karnataka, India

Business Consulting and Services · 11-50 employees

Yesterday
data-engineer Mid (2-5 yrs) Full-time India
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.

About the role

Design, build, and optimize scalable data pipelines using Azure Databricks and Medallion Architecture. Collaborate with stakeholders to refine requirements and implement data quality, security, and orchestration workflows.

What they look for

Azure Databricks PySpark SQL Python Delta Lake Unity Catalog Azure DevOps ETL Data Pipelines Medallion Architecture Delta Live Tables ADLS Gen2 Azure SQL Database Bicep CI/CD Data Modeling

Requirements

Requires 3-4 years of experience in data engineering with a focus on Azure Databricks and ETL processes. Proficiency in PySpark, SQL, Python, and cloud-based data orchestration is essential.

Full description

Interview - Face to face

Position Overview: Build and maintain scalable data pipelines using Azure Databricks to support DATA & AI ACDP projects. Apply 3-4 years of experience to deliver reliable ETL processes and collaborate on data-driven insights.

Key Responsibilities:

  • Design, build, and optimize data pipelines in Azure Databricks for ingestion, ETL/ELT, and transformations, implementing Medallion Architecture (Bronze, Silver, Gold) with Delta Lake for data quality and versioning.
  • Utilize Databricks Spark (PySpark, SQL), Delta Live Tables, and Unity Catalog for pipeline development, governance, and basic streaming.
  • Integrate data from ADLS Gen2, Azure SQL Database, and SQL Pools; support orchestration via Databricks workflows.
  • Implement CI/CD in Azure DevOps for Databricks deployments, including Bicep templates and ADF integration.
  • Apply data quality measures (expectations, constraints), security (RBAC, encryption), and monitoring for performance/cost efficiency using auto-scaling and Photon.
  • Work with analysts and stakeholders to refine requirements into functional workflows; handle data cleansing, modeling, and Python/SQL scripting for structured/unstructured data.
  • Troubleshoot pipelines and contribute to documentation/best practices.

Required Qualifications:

  • 3-4 years in data engineering, focused on Azure Databricks pipelines and ETL
  • Proficiency in PySpark, SQL, Python; experience with cloud storage and basic orchestration
  • Familiarity with Azure fundamentals and collaborative tools

Similar roles