Data Engineer

Data Engineer

Purpose: Build, automate, secure, and operate the Google Cloud data platform, focusing on Infrastructure as Code, CI/CD, environment management, security, observability, and FinOps across Development, Test, and Production environments.

Responsibilities

• Build and maintain reusable Terraform modules for GCP infrastructure, including VPC, VPC-SC, BigQuery, Cloud Storage, Composer, Datastream, Dataplex, IAM, KMS, and related services.

• Provision and maintain Dev, Test, and Production environments using consistent Infrastructure as Code patterns.

• Implement Terraform remote state, policy-as-code controls, and automated drift detection.

• Build and manage Cloud Build CI/CD pipelines for data ingestion, transformation, data quality rules, and semantic models.

• Implement automated unit, integration, infrastructure, and data-quality testing gates.

• Automate non-production environment refreshes with appropriate PII masking/tokenisation.

• Configure Cloud Logging, Cloud Monitoring, dashboards, SLA monitoring, and failure alerts.

• Implement FinOps controls, including billing exports to BigQuery, budgets, alerts, resource labelling, and cost reporting.

• Support cloud security hardening and remediation of infrastructure-level penetration-test findings.

Required skills and experience

• 6+ years of experience in Cloud Infrastructure, Platform Engineering, DevOps, DataOps, or similar roles.

• Minimum 3+ years of hands-on GCP experience.

• Proven experience supporting production-grade cloud or enterprise data platforms.

• Strong understanding of cloud infrastructure, networking, security, CI/CD, and observability.

• Experience managing multiple environments such as Dev, Test, and Production.

• Hands-on expertise with Google Cloud Platform (GCP).

• Proficiency in Terraform, including module design, remote state, environment management, and policy-as-code.

• Experience with Cloud Build / CI/CD.

• Strong Python skills for automation and tooling.

• Experience with BigQuery and Cloud Storage.

• Knowledge of GCP networking: Shared VPC, Private Service Connect, Private Google Access.

• Understanding of security: IAM, KMS, Secret Manager, VPC Service Controls.

• Experience with Cloud Logging and Cloud Monitoring.

• Ability to implement automated testing and deployment gates for data pipelines.

• Familiarity with Git workflows, code reviews, branch protection, tagged releases, and rollback.

• Container fundamentals: Docker, Cloud Run, GKE.

• Good-to-have skills include Cloud Composer, Dataplex, Datastream, GKE/Kubernetes, Cloud Interconnect and hybrid networking, data masking/tokenisation, FinOps, GCP Billing Export to BigQuery, Power BI, penetration-test remediation, and GitHub Actions or GitLab CI/CD.

Certifications

• Required (at least one): Google Cloud Professional Cloud DevOps Engineer; Google Cloud Associate Cloud Engineer.

• Preferred: HashiCorp Terraform Associate; Google Cloud Professional Cloud Security Engineer.