Senior Data Engineer
Shell · London, England, United Kingdom
Automotive · 501-1,000 employees
About the role
You will design, build, and optimize complex data pipelines and management layers to transform structured and unstructured data into business insights. Additionally, you will collaborate with project teams to troubleshoot pipeline issues and leverage agentic workflows to enhance data processing efficiency.
What they look for
Requirements
The role requires strong experience in the Azure ecosystem, Apache Spark, and Python, along with proficiency in data warehousing and cloud-native pipeline development. Candidates must possess excellent analytical skills and a proven ability to handle large volumes of data within complex IT environments.
Benefits
Full description
, United Kingdom Job Family Group:
Information Technology (IT) Worker Type:
Regular Posting Start Date:
August 4, 2026 Business Unit:
Projects and Technology Experience Level:
Experienced Professionals Job Description:
What’s the role
Are you a Senior Data Engineer passionate about building scalable data solutions and turning data into business value?
In this role, you'll work on data engineering projects within IT – T&S, delivering complex data management solutions by leveraging industry best practices. You will collaborate closely with project teams to design, build, and optimize efficient data pipelines and data platforms that enable the business to unlock insights and maximize the value of its data.
If you thrive in solving complex data challenges and enjoy creating high-quality, reliable data solutions, then this role is for you.
What you’ll be doing
- Understanding the business vision from the data solutions and work with all project stakeholders to enable that.
- Working alongside multiple other project team members to drive towards the common goal.
- Helping the business analysts to create data mapping documents that define the end to end pipeline.
- Building and delivering data pipelines that ingests, transform & load data (structured and unstructured) from multiple sources into the target data lake/database.
- Building the data management layer in line with the business needs and standards defines.
- Triaging problems in the pipelines or data layers and effectively fix them based on the priority.
- Automating data loading during initial or incremental loads.
- Supporting the operations team during critical issues and helps them in faster resolution.
- Leveraging agentic workflows to generate data pipelines
- Deep query analysis and optimization ability using Azure SQL Databases
What you bring
- IT industry experience, with at least good exposure on Big data technologies.
- Interest and passion in Big Data technologies and appreciates the value that can be brought in with an effective data management solution.
- Has worked on real data challenges and handled huge volume, velocity and variety of data.
- Understands Infrastructure concepts and has worked with multiple cloud tenants.
- Proficient with data warehousing and traditional ETL technologies.
- Experience in building cloud native Data Pipelines with modularization, containerization & unit test frameworks.
- Strong experience in writing stored procedure, functions and triggers in SQL.
- Strong experience on Azure ecosystem and has worked with data management on Databricks, Delta Lake, ADF, Blob Store, ADLS etc.
- Strong coding experience in Apache Spark – primarily PySpark. Must be able to effectively churn out complex transformation requirements with ease.
- Strong coding experience in Polars dataframe technology. Must be able to effectively churn out complex transformation requirements with ease.
- Solid programming skills primarily in Python.
- Proficient with agentic workflows using GitHub Copilot.
- Implement complex data processing algorithms in real time in and efficient manner using Spark and other technologies.
- Worked with both structured and unstructured data
- Knowledge on data visualisation and modelling tools.
- Excellent analytical & problem-solving skills, willingness to take ownership and resolve technical challenges.
- Excellent communication and stakeholder management skills
- Industry certification in Data Engineering is an added advantage.
- Knowledge on latest database technologies like MongoDB, Cosmos, Cassandra etc. would be an advantage
After you submit your application, you will receive an email from Shell providing a link to an online assessment (HireVue) that is a mandatory part of the process and has a validity period – please check your spam/junk mailbox if you do not see the email. Once completed, your application will be reviewed to determine next steps. Thank you for your interest in Shell.
What we offer
You bring your skills and experience to Shell and in return you work with talented, committed people on one of the most important challenges facing our planet. You’ll have the opportunity to develop the skills you need to grow in an environment where we value honesty, integrity, and respect for one another. You’ll be able to balance your priorities as you become the best version of yourself.
- Continuously grow the transferable skills you need to get ahead.
- Work at the forefront of technology, trends, and practices.
- Collaborate with experienced colleagues with unique expertise.
- Achieve your balance in a values-led culture that encourages you to be the best version of yourself.
- Make an impact in one of the world’s leading data-driven businesses.
- Benefit from flexible working hours, and the possibility of remote/mobile working.
- Perform at your best with a competitive starting salary and annual performance related salary increase – our pay and benefits packages are considered to be among the best in the world.
- Take advantage of paid parental leave, including for non-birthing parents.
- Join an organisation working to become one of the most diverse and inclusive in the world. We strongly encourage applicants of all genders, ages, ethnicities, cultures, abilities, sexual orientation, and life experiences to apply.
- Grow as you progress through diverse career opportunities in national and
- international teams.
- Gain access to a wide range of training and development programmes.
Shell is working to advance an inclusive, psychologically safe and accessible environment where people with disabilities can excel. If you require any accommodations or accessibility adjustments (e.g. assistive technology, communication support, any other) during the application or interview process, please let us know directly via careers@shell.com . We strive to ensure that our process and workplace is accessible to everyone and are dedicated to making reasonable adjustments to support your needs.
Shell in The United Kingdom
Shell is a vital contributor to the UK, supporting energy security, jobs and economic value. We provide energy to fuel homes, hospitals, schools, vehicles, machinery and factories. Our history here dates back over 125 years. A UK-headquartered global energy leader, and leading FTSE multinational, we are active across the country’s energy system.
In the years ahead, as the UK looks to strengthen energy security and deliver its 2050 net-zero goal, Shell UK aims to play a crucial role. We aim to be a major investor in the UK energy system by helping our customers decarbonise with a focus on transport and industry.
-
DISCLAIMER:
Please note: We occasionally amend or withdraw Shell jobs and reserve the right to do so at any time, including prior to the advertised closing date. Before applying, you are advised to read our data protection policy. This policy describes the processing that may be associated with your personal data and informs you that your personal data may be transferred to Shell/Shell Group companies around the world. The Shell Group and its approved recruitment consultants will never ask you for a fee to process or consider your application for a career with Shell. Anyone who demands such a fee is not an authorised Shell representative and you are strongly advised to refuse any such demand. Shell is an Equal Opportunity Employer.