Lead Data Engineer / Integration Specialist

Company Description

 A best-in-class city that attracts best-in-class talent, Philadelphia is an incredible place to build a career. From our thriving arts scene and rich history to our culture of passion and grit, there are countless reasons to love living and working here. With a workforce of over 30,000 people, and more than 1,000 different job categories, the City of Philadelphia offers boundless opportunities to make an impact.

As an employer, the City of Philadelphia values inclusion, integrity, innovation, empowerment, and hard work above all else. We offer a vibrant work environment, comprehensive health care and benefits, and the experience you need to grow and excel. If you’re interested in working with a passionate team of people who care about the future of Philadelphia, start here.

What We Offer:

Impact - The work you do here matters to millions.

Growth - Philadelphia is growing, why not grow with it?

Diversity & Inclusion - Find a career in a place where everyone belongs.

Benefits - We care about your well-being.

Job Description

Data Engineering at the City is a unique opportunity for meaningful, technically challenging, and service-oriented work. The Lead Data Engineer / Integration Specialist will play a key role in designing, building, documenting, and improving enterprise data integrations across diverse City systems. This position will focus on complex integrations that require strong data analysis, software development practices, and reusable engineering patterns. The role will support integrations across systems of record, centralized data platforms, SaaS environments, departmental applications, APIs, file-based systems, and systems that may not provide native APIs or standardized data access methods. In those cases, the Lead Data Engineer / Integration Specialist may help design custom API access patterns, extraction strategies, data contracts, or other integration approaches that allow the City to share and reuse data in a reliable and maintainable way.

The Lead Data Engineer / Integration Specialist will help standardize integration patterns, improve data consistency and performance, reduce one-off integration work, and strengthen the City’s ability to build scalable data pipelines and APIs. The position will support priority departmental and enterprise integration initiatives, while also contributing to the broader enterprise integration portfolio managed by CityGeo.

We are looking for a candidate who is not only technically skilled, but also passionate about leveraging data to improve City operations and community life. This team is fundamental to the innovation of City services that directly affect residents. If you are eager to contribute to a team that values innovation, efficiency, service- oriented work, and data-driven decision-making, we encourage you to apply. Some prominent services the team is responsible for include DataBridge, OIT’s enterprise data warehouse, AIS, the City’s custom address information systems which supports public address search and internal geocoding, and Databridge-Airflow, a custom workflow generator and orchestrator for departmental integrations (https://metadata.phila.gov/) and open data publishing. Some examples of OIT’s commitment to serving the public through the effective and innovative integration of City open data include:

https://Atlas.phila.gov

https://property.phila.gov

https://Openmaps.phila.gov

https://streetsmartphl.phila.gov/

www.phila.gov/solarmap

www.phila.gov/stressmap

Essential Functions

· Design, develop, test, deploy, and maintain complex enterprise data pipelines and integrations.

· Work with departments, product teams, vendors, and technical partners to understand how data is

created, stored, updated, accessed, and used.

· Design integrations that connect systems of record, centralized databases, SaaS platforms, APIs, file-

based systems, and custom applications.

· Develop robust ETL/ELT workflows using Python, SQL, Apache Airflow, DBT, Postgres, PostGIS, AWS,

MuleSoft, and related tools.

· Develop or support custom API, data access, or extraction patterns for systems that do not provide

native APIs or standard integration methods.

· Establish reusable integration templates, code patterns, data contracts, naming conventions, and

documentation practices.

· Improve data consistency, transformation logic, performance, reliability, and maintainability across

enterprise integrations.

· Support critical systems and platforms such as DataBridge, DataBridge-Airflow, AIS, open data publishing

workflows, enterprise APIs, and department-facing data products.

· Collaborate with data architects, systems engineers, application teams, and department stakeholders to

align integration designs with business processes and enterprise data standards.

· Contribute to data governance, metadata, master data management, data quality, and integration

standardization efforts.

· Participate in technical design reviews, code reviews, and integration planning to improve

maintainability, scalability, and quality.

· Troubleshoot production data pipelines and integration issues, including failures related to source-

system changes, schema drift, data quality, performance, scheduling, or access.

· Write clear technical documentation for systems, pipelines, APIs, data models, integration logic, and

operational procedures.

· Communicate complex technical concepts in understandable terms to technical and non-technical

stakeholders.

Competencies, Knowledge, Skills and Abilities

Competencies:

· Clear and concise written and verbal communication.

· Strong documentation habits and ability to explain technical decisions.

· Creative problem-solving and critical thinking.

· Ability to work with data from diverse domains and systems.

· Ability to manage multiple priorities and workflows simultaneously.

· Ability to guide technical implementation in a collaborative team environment.

· Ability to identify opportunities for standardization, reuse, automation, and continuous improvement. · Service-oriented approach to working with departments and stakeholders.

Knowledge and Skills:

· Advanced Python skills for data engineering, automation, API integration, and pipeline development.

· Strong SQL skills and experience working with relational databases, preferably Postgres and PostGIS.

· Experience designing, building, and maintaining ETL/ELT pipelines.

· Experience with Apache Airflow, DBT, Git/GitHub, CI/CD practices, and modern data engineering workflows.

· Experience integrating with REST APIs, SaaS systems, databases, file-based systems, and other data sources.

· Familiarity with API design, data contracts, schema validation, and integration documentation. · Familiarity with Docker, Bash, and Linux command-line tools.

· Familiarity with cloud services, preferably AWS.

· Understanding of data governance, metadata, master data management, data quality, and data standardization concepts.

· Familiarity with MuleSoft or other integration-platform technologies is preferred. · Experience working with spatial datasets, addressing data, or geospatial systems is a plus.

· Experience working in a public-sector, civic technology, or large enterprise environment is a plus.

Abilities:

· To reason about, model, transform, and validate complex datasets. · To translate business processes and stakeholder needs into technical integration designs.

· To build maintainable, reusable, and well-documented integration patterns. · To troubleshoot data pipeline, API, and database issues across multiple systems.

· To maintain clean, secure, and reliable data environments.

· To work effectively in a hybrid environment that includes cloud, on-premises, open source, custom, and commercial systems.

· To contribute to shared engineering standards and improve team practices over time.

Qualifications

• Bachelor’s degree in Computer Science, Information Systems, Data Science, GIS, Engineering, or a related field. Equivalent work experience may be considered.
• Five or more years of progressively responsible experience in data engineering, data integration, ETL/ELT development, API integration, or a related role.
• Demonstrated experience designing and maintaining production data pipelines or enterprise integrations.
• Proven experience with Python, SQL, relational databases, and version-controlled development workflows.
• Experience working with modern data engineering tools and practices, including orchestration, automated testing, CI/CD, documentation, and production support.
• Experience working directly with stakeholders to understand business processes, data needs, and system constraints.
• Dedication to making Philadelphia municipal government more robust, efficient, equitable, and data-informed.

Previous work in a public-sector or large-organization setting is a plus.

Additional Information

Salary Not to Exceed: $100,000

Starting salary to be determined based on experience and qualifications. 

Important: To apply, candidates must provide a cover letter and a resume.

Did you know? 

• We offer Comprehensive health coverage for employees and their eligible dependents 

• Our wellness program offers eligibility into the discounted medical plan 

• Employees receive paid vacation, sick leave, and holidays 

• Generous retirement savings options are available 

• Pay off your student loans faster - As a qualifying employer, City of Philadelphia employees are eligible to participate in the Public Service Loan Forgiveness program. Join the ranks of hundreds of employees who have already benefited from this program and achieved student loan forgiveness. 

• Enjoy a Free Commute on SEPTA - Starting September 1, 2023, eligible City employees will no longer have to worry about paying for SEPTA public transportation. Whether you're a full-time, part-time, or provisional employee, you can seize the opportunity to sign up for the SEPTA Key Advantage Program and receive free Key cards for free rides on SEPTA buses, trains, trolleys, and regional rails. 

• Unlock Tuition Discounts and Scholarships - The City of Philadelphia has forged partnerships with over a dozen esteemed colleges and universities in the area, ensuring that our employees have access to a wide range of tuition discounts and scholarships. Experience savings of 10% to 40% on your educational expenses, extending not only to City employees but in some cases, spouse and dependents too! 

 

*The successful candidate must be a city of Philadelphia resident within six months of hire 

 

Effective May 22, 2023, vaccinations are no longer required for new employees that work in non-medical, non-emergency or patient facing positions with the City of Philadelphia. As a result, only employees in positions providing services that are patient-facing medical care (ex: Nurses, doctors, emergency medical personnel), must be fully vaccinated. 

The City of Philadelphia is an Equal Opportunity employer and does not permit discrimination based on race, ethnicity, color, sex, sexual orientation, gender identity, religion, national origin, ancestry, age, disability, marital status, source of income, familial status, genetic information or domestic or sexual violence victim status. If you believe you were discriminated against, call the Philadelphia Commission on Human Relations at 215-686-4670 or send an email to . For more information, go to: Human Relations Website: 

http://www.phila.gov/humanrelations/Pages/default.asp 

 

• Minimum Education Required: College degree