JobCubby JobCubby
Sign up
iLink Digital

Sr Data Engineer -

iLink Digital · Toronto, Ontario, Canada

IT Services and IT Consulting · 1,001-5,000 employees

Jun 09
Senior (5-10 yrs) Full-time Canada
Log in to apply, save this posting, or score it against your profile with AI.

About the role

Design, build, and optimize scalable data and machine learning platforms using Databricks and Spark. Manage end-to-end data pipelines, ML workflows, and production AI systems while ensuring data quality and security.

What they look for

Databricks Apache Spark Python PySpark Delta Lake Lakehouse MLflow MLOps AWS Azure GCP Git CI/CD Generative AI RAG Vector Databases

Requirements

Requires 5+ years of experience with Databricks and Apache Spark along with strong Python and PySpark skills. Proficiency in Delta Lake, MLOps, and cloud platforms is mandatory, with preference for experience in Generative AI and RAG pipelines.

Full description

Job Summary

We are seeking a highly skilled Databricks Engineer with AI/ML experience to design, build, and optimize scalable data and machine learning platforms on Databricks. The role involves end-to-end ownership of data pipelines, ML workflows, and production AI systems.

Key Responsibilities

  • Design

and implement scalable ETL pipelines using Databricks & Spark

  • Build

Lakehouse architecture using Delta Lake

  • Develop

and deploy ML models using MLflow

  • Implement

MLOps pipelines for training, testing, and serving models

  • Optimize

cluster performance and reduce compute cost

  • Build

RAG and LLM-based solutions using Mosaic AI

  • Integrate

analytics with BI tools (Power BI, Tableau)

  • Implement

data governance using Unity Catalog

  • Collaborate

with Data Scientists and Business teams

  • Ensure

data quality, security, and compliance

Required Skills

Mandatory

  • 5+

years of Databricks & Apache Spark

  • Strong

Python & PySpark

  • Experience

with Delta Lake & Lakehouse

  • MLflow

& MLOps experience

  • Cloud

platform (AWS/Azure/GCP)

  • Git

& CI/CD

Preferred

  • Experience

with LLMs & Generative AI

  • RAG

pipelines & Vector Databases

  • Deep

Learning frameworks

  • Databricks

certifications

  • Power

BI integration

Requirements

Job Summary

We are seeking a highly skilled Databricks Engineer with AI/ML experience to design, build, and optimize scalable data and machine learning platforms on Databricks. The role involves end-to-end ownership of data pipelines, ML workflows, and production AI systems.

Key Responsibilities

  • Design

and implement scalable ETL pipelines using Databricks & Spark

  • Build

Lakehouse architecture using Delta Lake

  • Develop

and deploy ML models using MLflow

  • Implement

MLOps pipelines for training, testing, and serving models

  • Optimize

cluster performance and reduce compute cost

  • Build

RAG and LLM-based solutions using Mosaic AI

  • Integrate

analytics with BI tools (Power BI, Tableau)

  • Implement

data governance using Unity Catalog

  • Collaborate

with Data Scientists and Business teams

  • Ensure

data quality, security, and compliance

Required Skills

Mandatory

  • 5+

years of Databricks & Apache Spark

  • Strong

Python & PySpark

  • Experience

with Delta Lake & Lakehouse

  • MLflow

& MLOps experience

  • Cloud

platform (AWS/Azure/GCP)

  • Git

& CI/CD

Preferred

  • Experience

with LLMs & Generative AI

  • RAG

pipelines & Vector Databases

  • Deep

Learning frameworks

  • Databricks

certifications

  • Power

BI integration