Data Engineer GCP AWS & Databricks

Data Engineer – GCP, AWS & Databricks

Location: Remote – United States Employment Type: Contract Experience Required: 6+ Years

Position Overview

We are seeking an experienced Data Engineer to design, develop, and maintain scalable cloud-based data pipelines and data platforms. The ideal candidate will have strong hands-on experience with GCP, AWS, Databricks, Apache Spark, PySpark, Python, and SQL.

This role requires expertise in building reliable ETL/ELT pipelines, integrating data from multiple sources, and optimizing cloud data solutions for performance, security, scalability, and cost.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.

• Build cloud-based data solutions using GCP and AWS services.

• Use Databricks, Apache Spark, and PySpark for large-scale data processing.

• Integrate structured, semi-structured, and unstructured data from multiple sources.

• Develop and optimize batch and real-time data-processing workflows.

• Improve pipeline performance, reliability, scalability, and cost efficiency.

• Implement data-quality checks, monitoring, security, and governance standards.

• Design and support cloud data warehouses and data lakes.

• Troubleshoot production issues and perform root-cause analysis.

• Collaborate with data architects, analysts, application teams, and business stakeholders.

• Create and maintain technical documentation for pipelines, data models, and workflows.

Required Qualifications

• 5+ years of professional data engineering experience.

• Strong hands-on experience with both GCP and AWS.

• Expertise in Databricks, Apache Spark, and PySpark.

• Strong programming skills in Python and SQL.

• Proven experience developing ETL/ELT pipelines and cloud data platforms.

• Experience with data warehouses, data lakes, and dimensional data modeling.

• Experience with orchestration tools such as Apache Airflow or Google Cloud Composer.

• Understanding of data security, governance, monitoring, and quality frameworks.

• Strong analytical, troubleshooting, and problem-solving skills.

• Excellent communication and cross-functional collaboration skills.

Preferred Qualifications

• Experience with GCP services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, and Cloud Composer.

• Experience with AWS services such as S3, Glue, EMR, Redshift, Lambda, and Kinesis.

• Familiarity with Delta Lake and Databricks Lakehouse architecture.

• Experience with streaming technologies such as Apache Kafka.

• Familiarity with CI/CD, Git, Terraform, and cloud infrastructure automation.

• Experience working in Agile development environments.

Must-Have Skills

GCP | AWS | Databricks | Apache Spark | PySpark | Python | SQL | ETL/ELT | Airflow/Cloud Composer | Data Warehousing | Data Lakes

This is a remote position.