Sr. Data Platform Engineer

A World-Class Team

BJ’s Wholesale Club is powered by more than 30,000 team members who make a real impact every day. Whether you're stocking shelves, solving problems or shaping strategy, your work helps families save on what matters most.

We’re a team built on purpose and opportunity. Join us and be part of something meaningful.

Why You’ll Love Working at BJ’s

At BJ’s Wholesale Club, our team members are at the heart of everything we do. That’s why we offer a comprehensive benefits package designed to support your health, well-being and future – both on and off the job. When you grow, we grow.

Here’s just some of what you can look forward to:

• Weekly Pay: Get paid every week so that you can manage your money on your terms.
• Free BJ’s Memberships: Enjoy a complimentary The Club Card Membership, plus a free Supplemental Membership for someone in your household.*
• Generous Paid Time Off: Take the time you need with vacation, personal, sick days, holidays, bereavement, and jury duty leave.*
• Flexible and Affordable Health Benefits: Choose from three medical plans, and access optional dental, vision, Health Savings Account (HSA), and flexible spending account options to fit your lifestyle.*
• 401(k) Retirement Savings Plan: Build your financial future with a company match (available to team members 18 and older).*
• Employee Stock Purchase Plan:  Accumulate funds through after-tax payroll deductions that can be used to purchase shares of BJ’s common stock at a 15% discount.*

*Eligibility requirements vary by position.

Role Summary

We are seeking a Senior Data Platform Engineer to support the administration, automation, governance, security, and optimization of our enterprise Databricks Lakehouse platform on AWS.

This is a hands-on technical role responsible for maintaining and enhancing the Databricks platform, implementing infrastructure automation, enforcing compute and security standards, supporting cost-governance initiatives, and enabling reliable Data Engineering and Data Science workloads.

The ideal candidate will have strong hands-on experience with Databricks platform administration, AWS cloud services, Terraform-based infrastructure automation, and DevOps practices. The individual will evaluate emerging Databricks capabilities through proof-of-concept implementations and contribute to reusable platform patterns and standards.

This role will collaborate closely with Data Engineering, Data Science, Cloud Platform, Security, and Governance teams to ensure the Databricks environment remains secure, reliable, scalable, and cost-efficient.

Key Responsibilities

Databricks Platform Administration

• Support and maintain enterprise Databricks workspace architecture, including networking, security, and compute governance.
• Implement and maintain cluster policies, job policies, compute guardrails, and platform governance standards.
• Manage and optimize Databricks compute environments including job clusters, interactive clusters, and serverless compute.
• Implement Unity Catalog for centralized data governance and access management.
• Manage workspace configurations, platform integrations, and operational reliability.

AWS Cloud Architecture for Databricks

Design and maintain secure and scalable Databricks infrastructure leveraging AWS services including:

• IAM roles and cross-account access architecture
• VPC networking, subnets, and secure connectivity
• PrivateLink and secure endpoint configuration
• S3 data lake architecture
• AWS KMS encryption and key management
• CloudWatch monitoring and logging integration

Ensure the platform aligns with enterprise security, compliance, and networking standards.

Infrastructure as Code & DevOps Automation

• Build and maintain Databricks infrastructure using Terraform.
• Implement infrastructure-as-code frameworks for workspace provisioning, cluster policies, and platform configurations.
• Develop automation using:• Databricks CLI
• Databricks REST APIs
• Python and shell scripting

• Design and maintain CI/CD pipelines for Databricks workloads and platform infrastructure.
• Integrate deployments with Git-based development workflows (GitHub, GitLab, Azure DevOps).
• Enable standardized environment promotion frameworks across Dev, Test, and Production environments.

Platform Cost Governance & Optimization

• Monitor and analyze DBU consumption and compute utilization across workloads.
• Provide recommendations on cost optimizations
• Implement guardrails such as:• Cluster policies
• Budget alerts
• Resource tagging
• Usage monitoring dashboards

• Identify and implement cost optimization strategies across:• Compute configurations
• Job scheduling patterns
• Serverless vs classic compute usage
• Storage lifecycle management

• Build dashboards for cost transparency.

Data Platform Enablement

Provide technical guidance and platform best practices for data engineering and data science workloads, including:

Data Engineering

• Delta Lake architecture and performance optimization
• Medallion architecture (Bronze/Silver/Gold)
• Batch and streaming data ingestion
• Large-scale Spark workloads

Data Science & AI

• MLflow experimentation and model lifecycle management
• Feature Store usage
• Machine learning pipelines and model deployment frameworks.

Innovation & Proof of Concept Development

Evaluate and operationalize new Databricks capabilities by building Proof of Concepts (POCs) and evaluate POCs and support adoption into platform standards.

Areas of focus include:

• Databricks Serverless Compute
• Delta Live Tables
• Auto Loader
• Lakehouse AI capabilities
• Databricks Feature Store
• Vector Search and LLM integrations
• Lakehouse monitoring and observability frameworks

Support and maintain reference architectures and reusable design patterns based on POC outcomes.

Required Technical Skills

Databricks Platform

• Databricks workspace administration
• Cluster and job policy design
• Unity Catalog governance
• Delta Lake architecture
• Databricks REST APIs and CLI automation
• Performance optimization and troubleshooting

AWS Cloud Platform

Strong hands-on experience with:

• AWS IAM
• VPC networking and security
• S3 data lake architecture
• AWS KMS encryption
• CloudWatch monitoring
• PrivateLink connectivity
• Cross-account access architecture
• Enterprise cloud security practices

Infrastructure & Automation

• Terraform for Databricks and AWS infrastructure
• Infrastructure as Code design patterns
• Databricks CLI and API automation
• Python / Bash scripting
• Platform automation frameworks

DevOps & DataOps

• CI/CD pipeline development
• Git-based version control workflows
• Automated deployment and release processes
• Monitoring, logging, and observability frameworks
• Incident management and root cause analysis

Data Engineering & Analytics Skills

• Apache Spark / PySpark
• SQL optimization
• Delta Lake performance tuning
• Streaming frameworks
• Data pipeline orchestration
• Data modeling and ETL best practices

Preferred Qualifications

• Databricks Certified Data Engineer / Databricks Platform Architect
• AWS Certified Solutions Architect
• Experience designing enterprise-scale Lakehouse platforms
• Experience supporting large-scale ML workloads
• Experience implementing data governance frameworks

This is a hybrid role. Tuesday through Thursday are in-office days at BJ's Club Support Center in Marlborough, MA and Monday and Friday are remote days.

In accordance with the Pay Transparency requirements, the following represents a good faith estimate of the compensation range for this position. At BJ’s Wholesale Club, we carefully consider a wide range of non-discriminatory factors when determining salary. Actual salaries will vary depending on factors including but not limited to location, education, experience, and qualifications. The pay range for this position is $100,000.00 - $131,500.00 

 

We recognize the growing role of AI tools, including ChatGPT, and value familiarity with them. That said, we want to hear from your authentic self. Your application should reflect your own skills, experiences, and insights rather than AI-generated responses.