Lead Data Engineer - Python, Databricks, React
JPMorgan Chase & Co. Glasgow, Scotland, United Kingdom
Financial Services · 10,001+ employees
About the role
The Lead Data Engineer will design and maintain scalable data pipelines and analytics solutions to support AI initiatives within the Corporate Technology team. They will also lead data quality improvements, implement platform controls, and drive the adoption of AI-assisted development tools to enhance productivity.
What they look for
Requirements
Candidates must have formal software engineering training and extensive experience in data management, profiling, and analysis using tools like Python, Spark, and Databricks. Strong communication skills and experience with data governance, lineage, and AI-assisted development workflows are essential for this role.
Full description
Join us as a Lead Data Engineer and help shape the future of data-driven innovation at JPMorganChase. You will collaborate with talented colleagues to deliver impactful solutions that power AI and analytics initiatives across the firm. We value your expertise, encourage growth, and support your journey to make a lasting impact. Experience a culture that celebrates diverse perspectives and fosters continuous learning.
Job Summary: As a Lead Data Engineer in the Corporate Technology team supporting CIO, Treasury & Corporate Risk Management, you will enhance, build, and deliver data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. You will maintain critical data pipelines and architectures across multiple technical areas, supporting the firm’s business objectives. Your work will drive innovation, operational excellence, and team success, contributing to a collaborative culture that values your ideas and technical leadership.
Job Responsibilities:
- Make data available for AI and analytics initiatives, working closely with use case owners to define requirements, manage product dependencies, and support agile routines that oversee cross-product data dependencies and prioritize delivery
- Collaborate with business, technology, and operations partners to understand data requests and accelerate provisioning through deployment of "AI for Data"
- Drive adoption of AI-assisted development tools (e.g., Claude Code, Copilot) to accelerate delivery and improve developer productivity
- Partner with business and technology teams to rapidly prototype and deploy analytics and tooling, leveraging AI/ML and innovative approaches
- Implement and manage platform controls, including access, security, and compliance, ensuring all data and AI solutions meet firmwide SDLC standards
- Provide transparency and drive executive visibility into bottlenecks, progress, performance metrics, and adoption tracking in making AI-ready and critical data sources available for innovation
- Identify the lineage and provenance of critical data assets to support governance, regulatory, and business requirements; embed evergreen controls on data flows to improve safety, transparency, and traceability
- Drive insight into areas of efficiency and risk through consolidation and reengineering of data flows
- Lead data quality issue root cause analysis using deep data profiling and advanced analytics techniques, then fix the cause and embed uplifted evergreen controls to prevent future failures
- Develop proactive controls to reduce the time from data quality issue identification to resolution, improving client experience and driving operational efficiency
- Demonstrate control environment improvements and reduction in toil through common tooling and frameworks; uplift the metadata (semantic layer) of existing data to support AI and Natural Language Query (NLQ) usage, accelerate adoption of Mesh data architecture, reduce consumer friction, and deliver data product prototypes
- Uses enterprise-authorized AI capabilities within the work environment to accelerate data platform and model design analysis and documentation, validating outputs and handling data according to sensitivity and security requirements
- Applies reuse-first, AI-assisted practices within delivery and operational routines (e.g., backup/recovery validation and access control review support), ensuring traceability/auditability and alignment to resiliency and security expectations
Required Qualifications, Capabilities, and Skills:
- Formal training or certification on software engineering concepts and applied experience
- Experience and awareness in working within Risk Analytics space; preferred knowledge of corporate bond investment assets and structured credit products such as securitisation (e.g., CLO, CMBS, RMBS, ABS)
- Proven experience integrating AI-assisted development tools (e.g., Claude Code, Copilot, or similar) into engineering workflows
- Experience in strategic or transformational change initiatives, including data governance, data quality, or analytics transformation programs
- Strong technical skills in data profiling, analysis, and data management using modern tools and environments (Python, R, SQL, Spark, DataBricks, cloud platforms)
- Understanding of data lineage concepts and experience with lineage analysis, metadata management, and data cataloguing
- Excellent communication skills with the ability to convey complex technical concepts to diverse audiences, including executive leadership
- Experience with data quality frameworks, including profiling, rule development, issue remediation, and preventative controls
- Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity
- Ability to review and validate AI-assisted outputs (e.g., code, model/design summaries or operational checklists) before use, escalating when uncertain and following data handling requirements
- Hands-on practical experience delivering system design, application development, testing, and operational stability
Preferred Qualifications, Capabilities, and Skills:
- Hands-on experience with data lineage tools and techniques, including graph & vector databases and metadata management platforms
- Hands-on experience with LLM Ops, MLOps, and AI/ML platform deployment at scale
- Familiarity with business-led analytics delivery models and rapid prototyping frameworks
- Experience with AI/ML governance, prompt engineering, and integrating AI tools into SDLC
- Experience with AI/ML technologies and their application to data management challenges (e.g., automated data profiling, metadata enrichment)
- Understanding of agile and product management methodologies and experience working in agile teams
- Ability to multi-task in a fast-paced environment and operate independently with minimal supervision; strong judgment with the ability to balance strategic vision with pragmatic, incremental delivery
- Experience building and growing capabilities and developing talent in data science or data management teams
- Excellent interpersonal skills and ability to build strong working relationships with business, technology, and control stakeholders across global teams
J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent corporations, governments, wealthy individuals and institutional investors. Our first-class business in a first-class way approach to serving clients drives everything we do. We strive to build trusted, long-term partnerships to help our clients achieve their business objectives.
We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants’ and employees’ religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.
Our professionals in our Corporate Functions cover a diverse range of areas from finance and risk to human resources and marketing. Our corporate teams are an essential part of our company, ensuring that we’re setting our businesses, clients, customers and employees up for success.
Similar roles
-
Software Engineer (Python) | eBike (f/m/div.)
Bosch Group Braga, Portugal
-
Semi Senior / Senior Data Engineer (AWS, Python & SQL)
Devsu Honduras
-
Python Developer
Enteractive Il-Gżira, Siggiewi (Citta' Ferdinand), Malta
-
[8PV] Automation Engineer (Python + Power Automate)
Software Mind San José, Costa Rica
-
Backend Engineer with Python, PHP along with web scrapping
NielsenIQ Chennai, Tamil Nadu, India
-
Lead Principal Site Reliability Engineer(Python, Kubernetes & Cloud Infra)
Oracle Bengaluru, Karnataka, India