Sr. Data Engineer

Overview

Manages and evolves the enterprise data lake and data warehouse while ensuring the reliable, secure, and efficient flow of high-quality data. Implements data processes, managing data architecture, designing ETL processes, and analyzing data for business insights.

The base salary range for this position is $99,937-$157,044.

Actual pay will be determined based upon a candidate’s job-related knowledge, skills, education, experience, geographic location, and may include other job-related factors such as certification(s), professional licensure, or internal equity considerations.

Responsibilities

• Design, implement and maintain scalable data pipnes on WS using S3, DMS, Glue, lambda, step function/MWAA & Redshift.

• Develop robust batch and near-real-time ETL/ELT workflow to ingest, cleanse, transform and load data from databases, legacy applications and event streams using Python & Pyspark.

• Design incremental/CDC mechanism, including restart ability, idempotency, duplicate handling and recovery.

• Implement automated controls for completeness, accuracy, reconciliation, schema changes & lineage.

• Optimize Glue/Spark, Athena, Redshift & S3 workload through partitioning, columnar formats, query tuning and appropriate storage/compute design.

• Design near real time/event-driven pipelines using Kinesis/Kafka where required, covering ordering, retry, idempotency and failure recovery.

• Implement AWS data security, least privilege access, data classification and governance controls.

• Monitor pipelines such as CloudWatch, troubleshoot failure and resolving production data incidents.

• Enforce Git/version control, code review, automated testing and CI/CD practices.

• Work with product owners, architect, reporting and business stakeholders to translate requirements into scalable data solutions.

• Document pipelines and operational procedures.

Qualifications

Qualifications Required

• Bachelor's degree in a computer-related field from an accredited college or university and five (5) or more years of experience in data engineering, building scalable and distributed ETL data pipelines in enterprise environments.

• Experience building and operating scalable AWS-based data platforms and pipelines using services including Lambda, Glue, Athena, S3, Redshift, DMS, MWAA (Airflow), and Step Functions, supporting batch, CDC, and near real-time data processing.

• Advanced proficiency in Python, SQL, and PySpark with hands-on experience developing reusable ETL/ELT frameworks, data warehouses, data marts, and integrations across databases, APIs, event streams, and analytics environments.

• Experience implementing data quality, governance, and optimization best practices, including automated validation frameworks, Lake Formation and Glue Data Catalog, performance tuning, and cost optimization across AWS data services.

• Strong communication skills with the ability to translate complex data concepts for business stakeholders; experience in healthcare, life sciences, and other highly regulated environments with HIPAA, GDPR, FDA, or similar compliance requirements preferred.

• Experience with metadata management, data lineage, data observability, master data management, or enterprise data catalog solutions.

• Knowledge with data modeling & analytical data model, schema design, schema evolution, and data structure optimized for reporting and analytics.

• Knowledge of data lake and data warehouse architecture include data partitioning and columnar storage format such as Parquet.

• Relevant AWS certification, such as AWS Certified Data Engineer – Associate, or an equivalent cloud or data engineering certification.

WORKING CONDITIONS

• Flexible work hours in fun collaborative environment

• Working remote requires a reliable internet connection

• Must have the ability to travel, as needed for company meetings