GCP Data Engineer

EXL Gurugram, Haryana, India

Business Consulting and Services · 10,001+ employees

4 h ago

data-engineer Senior (5-10 yrs) Full-time India

Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

The role involves designing and maintaining scalable data pipelines using GCP big data tools like BigQuery, Dataproc, and Airflow. Responsibilities also include implementing ETL/ELT solutions and ensuring robust CI/CD integration for production workflows.

What they look for

GCP BigQuery Dataproc Airflow Dataflow Pub/Sub Data Modeling ETL/ELT Python SQL Java Scala Kubernetes CI/CD Cloud Build GitHub Actions

Requirements

Candidates must hold a degree in Computer Science or a related field and possess at least 6 years of experience in data engineering. Proficiency in Python, SQL, and GCP cloud infrastructure is required to build and manage complex data solutions.

Full description

Technical Skills:-

1. Core Data Engineering Skills

Big Data Technologies: Proficiency in using GCP's big data tools like:

BigQuery: For data warehousing and SQL analytics.

Dataproc: For running Spark and Hadoop clusters.

Airflow: For pipeline Orchestration

Dataflow: For stream and batch data processing.(High level Idea)

Pub/Sub: For real-time messaging and event ingestion.(High level Idea)

Data Modeling: Experience/Knowledge on designing scalable, efficient data models for OLAP and OLTP use cases.

ETL/ELT Pipelines:

Expertise in building automated, scalable, and reliable pipelines using custom Python/Scala solutions or Cloud Data Fusion.

2. Programming and Scripting

Strong coding skills in Python, SQL, and optionally Java/Scala.

Familiarity with APIs and SDKs for GCP services to build custom data solutions.

3. Cloud Infrastructure

Understanding of GCP services such as Cloud Storage, Compute Engine, and Cloud Functions.

Familiarity with Kubernetes (GKE) and containerisation for deploying data pipelines. (Optional but Good to have)

4. DevOps and CI/CD

Experience setting up CI/CD pipelines using Cloud Build, GitHub Actions, or other tools.

Monitoring and logging tools like Cloud Monitoring and Cloud Logging for production workflows.

Soft Skills: -

1. Innovation and Problem-Solving

Ability to think creatively and design innovative solutions for complex data challenges.

Experience in prototyping and experimenting with cutting-edge GCP tools or third-party integrations.

Strong analytical mindset to transform raw data into actionable insights.

2. Collaboration

Teamwork: Ability to collaborate effectively with data analysts, and business stakeholders.

Communication: Strong verbal and written communication skills to explain technical concepts to non-technical audiences.

3. Adaptability and Continuous Learning

Open to exploring new GCP features and rapidly adapting to changes in cloud technology

Responsibilities