JobCubby JobCubby
Sign up
XPT Software Australia Pty Ltd

Data Engineer with GCP

XPT Software Australia Pty Ltd Sydney, New South Wales, Australia

IT Services and IT Consulting · 51-200 employees

7 h ago
data-engineer Senior (5-10 yrs) Contractor Australia
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.

About the role

Design, develop, and maintain scalable data pipelines and ETL/ELT workflows using GCP services. Collaborate with cross-functional teams to ensure data security, governance, and optimal performance of data processing solutions.

What they look for

Google Cloud Platform BigQuery Python SQL ETL ELT Apache Beam Cloud Dataflow Cloud Composer Pub/Sub Cloud Storage Data Warehousing Data Modeling Data Pipelines Batch Processing Real-time Processing

Requirements

Requires 4-8 years of experience with strong proficiency in Google Cloud Platform, Python, and SQL. Candidates must have hands-on experience with BigQuery, data modeling, and developing batch or real-time data pipelines.

Full description

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using GCP services.
  • Build and optimise ETL/ELT workflows for batch and real-time data processing.
  • Work with Google BigQuery for data warehousing, SQL development, and query optimisation.
  • Develop data processing solutions using Python, PySpark, and Apache Beam.
  • Implement data integration using Cloud Dataflow, Cloud Composer, and Pub/Sub.
  • Build and manage data lakes and data warehouses on Google Cloud.
  • Implement data quality checks, data validation, and monitoring.
  • Optimise pipeline performance, scalability, and cost efficiency.
  • Collaborate with data analysts, data scientists, architects, and business teams.
  • Ensure data security, governance, and compliance best practices.

Mandatory Technical Skills

  • Strong experience in Google Cloud Platform (GCP).
  • Hands-on experience with BigQuery.
  • Strong programming skills in Python and SQL.
  • Experience developing ETL/ELT data pipelines.
  • Experience with Cloud Dataflow and/or Apache Beam.
  • Knowledge of Cloud Storage (GCS).
  • Experience with data modelling and data warehousing.
  • Experience with batch and real-time data processing.

Similar roles