JobCubby JobCubby
Sign up
T

Technology Architect Python Big Data

Timebox Pro Charlotte, North Carolina, United States

IT Services and IT Consulting · 51-200 employees

7 h ago
python Principal (10+ yrs) Full-time United States
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

Lead delivery coordination and manage complex software application development projects while building a containerized API control plane for Spark-based data pipelines. Ensure observability, policy enforcement, and metadata management while collaborating with onsite and offshore teams.

What they look for

Python PySpark API Development Microservices CI/CD GitHub Actions GCP ETL Docker Kubernetes FastAPI ARQ Pytest Software Engineering System Architecture

Requirements

Requires 10+ years of IT experience with strong proficiency in Python, PySpark, and GCP platform services. Candidates must demonstrate expertise in CI/CD automation, containerization, and the ability to manage offshore resources in a client-facing role.

Full description

Visa-independent candidates only

About the Role

We are seeking a senior Technology Architect / Python Platform Engineer to design and build a containerized API layer that abstracts and governs interactions with Apache Spark through a well-defined API contract. This is a platform-building role focused on creating consistent, secure, and scalable access to Spark-based data pipelines—not simply consuming existing tools.

You will develop production-grade APIs in Python, package them as reusable containerized services (Docker/Kubernetes), and operate them as platform capabilities. Strong hands-on experience with GCP, CI/CD automation (especially GitHub Actions), and solid software engineering practices around testing, versioning, and deployment are essential. The role also involves client coordination, delivery ownership, and managing offshore resources while collaborating with onsite teams.

Key Responsibilities

  • Lead delivery coordination and manage complex software application development & support projects
  • Design and build a containerized API control plane that securely governs access to Spark-based data pipelines
  • Develop production-grade Python APIs and platform services using modern design patterns
  • Drive CI/CD automation, testing, versioning, and deployment practices
  • Coordinate with clients, work closely with onsite team members, and manage offshore resources
  • Ensure observability, policy enforcement, metadata management, and scalable job execution

Must-Have Skills

• Python (strong production experience)

  • PySpark

• API development & Microservices

• CI/CD (GitHub Actions preferred)

  • GCP
  • ETL
  • Experience building control planes / platform services
  • Containerization (Docker / Kubernetes)

Preferred Technical Background

  • FastAPI, ARQ, Kubernetes, Pytest
  • Design patterns: modular containerized services, asynchronous processing, job-based work queues, immutability & idempotency, metadata management, policy enforcement, semantic modeling, observability/telemetry
  • Functional and behavioral testing

Requirements

  • 10+ years of IT experience
  • Proven ability to interact and coordinate with clients
  • Experience managing offshore teams and collaborating with onsite resources
  • Strong software engineering fundamentals (testing, versioning, deployment)
  • No certifications required

Additional Details

  • Local candidates preferred (Charlotte, NC area)
  • Hybrid onsite arrangement
  • Visa-independent candidates only

If you are a hands-on Python platform engineer with deep experience building APIs and control planes around Spark/GCP, and you thrive in a client-facing architect role with delivery ownership, we’d like to hear from you.

Apply now or share your updated resume.

Flexible work from home options available.

Similar roles