Data Engineer – Databricks / BigQuery / Snowflake

We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data solutions that support advanced analytics and machine learning initiatives. The ideal candidate will have strong expertise in data pipeline development, cloud platforms, and big data technologies, with a focus on delivering reliable and high-performance data systems.

In this role, you will collaborate closely with data scientists, analysts, and engineering teams to enable data-driven decision-making. You will be responsible for building robust ETL pipelines, integrating machine learning workflows, and ensuring data quality, governance, and scalability across enterprise data platforms.

This position offers the opportunity to work on innovative data solutions that drive business insights, sustainability initiatives, and digital transformation.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using SQL, Python, and PySpark.

• Build and maintain ETL workflows to transform raw data into structured, analytics-ready datasets.

• Collaborate with data scientists to deploy and integrate machine learning models into production environments.

• Work with cloud platforms such as AWS and Azure to manage and scale data infrastructure.

• Utilize Databricks and Snowflake for big data processing and data warehousing solutions.

• Implement workflow orchestration using tools like Apache Airflow to automate data pipelines.

• Manage infrastructure as code using Terraform for scalable and reliable deployments.

• Optimize data storage, retrieval, and performance across data warehouse systems.

• Ensure data quality, governance, and compliance using tools like Unity Catalog or Hive Metastore.

• Troubleshoot and resolve data pipeline issues while continuously improving system performance.

Requirements

Basic Requirements

• 8+ years of experience in data engineering or related roles.

• Strong proficiency in SQL and Python for data processing and pipeline development.

• Hands-on experience with PySpark and large-scale data processing.

• Experience with ETL frameworks and data pipeline architecture.

• Working knowledge of cloud platforms such as AWS and Azure.

• Experience with Databricks and Snowflake environments.

• Familiarity with workflow orchestration tools like Apache Airflow.

• Experience with Terraform or similar infrastructure-as-code tools.

• Strong understanding of data warehousing concepts and best practices.

• Excellent problem-solving, analytical, and communication skills.

Preferred Qualifications

• Experience integrating machine learning or AI models into data pipelines.

• Exposure to data governance and cataloging tools such as Unity Catalog or Hive Metastore.

• Experience working in large-scale enterprise or consulting environments.

• Knowledge of performance tuning and optimization for big data systems.

• Familiarity with real-time data processing frameworks is a plus.