Senior Data Engineer

Job Title: Senior Data Engineer Job Code: PA 815653 Client: Commonwealth of Pennsylvania – EISO/PALDS Location: Harrisburg, Pennsylvania (Remote) Duration: 11/23/2026 – 12/31/2026 Work Hours: Monday–Friday, 8:00 AM–4:30 PM Work Arrangement: Remote Closing Date: 11/10/2026 at 4:00 PM Interview Process: Two rounds – First round via Teams; second round in-person in Harrisburg for finalists.

Position Overview

The Commonwealth of Pennsylvania is seeking a Senior Data Engineer & Analytics Consultant to support the Pennsylvania Longitudinal Data System (PALDS).

The candidate will transform complex data into reliable, actionable insights to support program delivery, operational improvements, and executive decision-making across state government agencies.

The role involves designing and developing data pipelines, integrating and transforming data from multiple sources, addressing data quality issues, and building analytics solutions. The candidate will work with large, complex, and confidential datasets while supporting data modernization, data conversions, and enterprise data architecture initiatives.

Required Skills

• 5+ years of hands-on experience with cloud-based data services, particularly AWS (S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, DynamoDB).

• 5+ years of programming/scripting experience with Python, R, Java, C++, or Scala.

• 5+ years of experience with relational and NoSQL databases, including Oracle, MS SQL Server, PostgreSQL, and Cassandra.

• 5+ years of experience with Git, version control systems, and CI/CD pipelines.

• 5+ years of experience in data modeling, including dimensional modeling, Data Vault, and Star/Snowflake schemas.

• 5+ years of experience with data cataloging and metadata management tools such as AWS Glue Data Catalog.

• 5+ years of knowledge of data governance, data lineage, and data quality frameworks.

• 5+ years of experience with REST API design and integration patterns.

• 5+ years of experience with big data technologies such as Hadoop, Spark, and Kafka.

• 5+ years of experience with data pipeline and workflow management tools such as Azkaban, Luigi, and Airflow.

• 5+ years of experience with data integration solutions such as Informatica, MuleSoft, Talend, and TIBCO.

• 5+ years of experience with stream-processing technologies such as Storm and Spark Streaming.

• Advanced SQL skills and experience developing and optimizing complex queries.

• Experience building and optimizing large-scale data pipelines and analytical datasets.

• Experience developing dashboards, reports, analytical models, and executive decision-support products.

• Strong analytical, problem-solving, project management, and organizational skills.

Key Responsibilities

• Design, build, and maintain scalable data pipeline architectures.

• Develop cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.

• Build and maintain data orchestration workflows using Apache Airflow, AWS Step Functions, or similar tools.

• Integrate, transform, and process large datasets from multiple sources.

• Develop analytical data environments supporting statistical analysis, predictive modeling, and machine learning.

• Partner with data scientists, research analysts, and policy analysts to support research and statistical modeling.

• Implement ETL processes, data quality checks, validation, monitoring, and alerting frameworks.

• Develop dashboards, reports, analytical models, and decision-support tools.

• Optimize data infrastructure, automate manual processes, and improve scalability.

• Support data modernization, system replacement, and data conversion initiatives.

• Ensure compliance with data governance, privacy, security, and Commonwealth data-sharing requirements.

• Collaborate with executive, data, design, and support teams to address data infrastructure needs.

Primary Skills

Python | Java | AWS | S3 | EC2 | Glue | EMR | RDS | Redshift | Athena | Lambda | DynamoDB | SQL | Oracle | MS SQL Server | PostgreSQL | Cassandra | R | C++ | Scala | Data Modeling | Informatica | MuleSoft | Talend | TIBCO | REST APIs | Git | CI/CD | Hadoop | Spark | Kafka | Airflow | Azkaban | Luigi | Storm | Spark Streaming

This is a remote position.