JobCubby JobCubby
Sign up
Unison Group

Senior Data Engineer

Unison Group Kuala Lumpur, Kuala Lumpur, Malaysia

Business Consulting and Services · 11-50 employees

Oct 14
data-engineer Senior (5-10 yrs) Full-time Malaysia
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.

About the role

You will design, develop, and operationalize scalable Delta Lakehouse architectures while optimizing data pipelines and machine learning workloads. Additionally, you will manage platform governance, mentor engineering teams, and collaborate with stakeholders to deliver AI and LLM solutions.

What they look for

Databricks Delta Lake Spark SQL Python Scala Unity Catalog Terraform CI/CD DevOps MLOps Data Engineering Cloud Computing ETL Machine Learning Data Governance Infrastructure as Code

Requirements

The role requires extensive hands-on experience with Databricks, Delta Lake, and Unity Catalog, alongside proficiency in Python or Scala and SQL. Candidates must also possess strong knowledge of cloud ecosystems, infrastructure-as-code tools like Terraform, and DevOps best practices.

Full description

Role Overview

  • We are seeking a hands-on Databricks engineer with deep technical expertise in building, operating, and optimizing Lakehouse-based data and AI solutions. In this role, you will design, develop, and operationalize Delta Lakehouse architectures that deliver measurable outcomes for enterprise customers. You will own implementation activities, contribute to technical delivery leadership, and mentor engineering teams in best practices across data engineering, platform governance, and AI.

Key Responsibilities

  • Design, build, and maintain scalable data pipelines using Delta Live Tables (DLT) / Lakeflow Spark Declarative Pipelines, Spark SQL, Python, or Scala.
  • Optimize batch ETL, streaming, and machine-learning workloads for performance, cost efficiency, scalability, and reliability.
  • Configure and administer Databricks workspaces, Unity Catalog, cluster policies, SQL Warehouses, access controls, and secure governed environments.
  • Automate infrastructure provisioning and application deployments using Terraform, Git, and CI/CD pipelines.
  • Establish platform observability, monitoring, alerting, and cost-management practices using tools such as Splunk, Prometheus, or AWS CloudWatch and equivalent cloud-native tools.
  • Collaborate with customers and cross-functional teams to deliver AI/LLM solutions using MLflow, Mosaic AI, and relevant Databricks model-serving and evaluation capabilities.
  • Troubleshoot production issues, improve operational reliability, and document architecture, engineering standards, and runbooks.
  • Provide technical guidance, conduct code reviews, and mentor engineers on Databricks, data engineering, governance, and DevOps best practices.

Required Skills & Experience

  • Strong hands-on experience with Databricks, including workspace configuration, notebooks, compute, jobs, and orchestration.
  • Practical expertise in Delta Lake, DLT / Lakeflow Spark Declarative Pipelines, Unity Catalog, and Databricks SQL Warehouses.
  • Proficiency in Python or Scala, with strong SQL skills for data engineering and/or machine-learning workflows.
  • Good understanding of at least one major cloud ecosystem: AWS, Microsoft Azure, or Google Cloud Platform (GCP).
  • Experience with Terraform or similar infrastructure-as-code tools, Git-based workflows, CI/CD, and DevOps/MLOps practices.
  • Familiarity with monitoring, observability, security, and governance frameworks for large-scale data platforms.
  • Strong problem-solving skills and the ability to communicate technical concepts clearly to customer and engineering stakeholders.

Similar roles