Senior Data Engineer
Public Sector Resourcing London Britain Township, Pennsylvania, United States
Staffing and Recruiting · 201-500 employees
Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.
About the role
The Senior Data Engineer will develop, test, and maintain data ingestion and transformation pipelines using Microsoft Fabric and Azure services. They will also design optimized data models and integrate data from enterprise applications like Dynamics 365.
What they look for
Requirements
Candidates must have strong commercial experience as a Data Engineer and hands-on expertise with Microsoft Fabric and Azure data technologies. Essential requirements include SQL proficiency, experience with ETL/ELT pipelines, and the ability to obtain SC/DV security clearance.
Full description
Description & Requirements
On behalf of BEIS, we are looking for a Senior Data Engineer Inside IR35 for a 4 months contract based REMOTELY.
As a Senior Data Engineer, your main responsibilities will be:
PLEASE NOTE: SC/DV Clearance is an essential requirement for this role, as a minimum you must be willing & eligible to undergo checks. Please note, due to the exceptional requirements of this position (short-term nature of this role and speed at which we require a postholder in situ) preference may be given to candidates who meet all of the essential criteria and hold active security clearance.
Key Responsibilities
- Develop, test and maintain data ingestion and transformation pipelines using Microsoft Fabric, OneLake and Azure services
- Build and support reusable data engineering components and data integration patterns
- Design and optimise data models, datasets and data products for reporting, analytics and AI use cases
- Integrate data from Dynamics 365, Dataverse and other enterprise applications
- Develop API-based and event-driven data integration solutions
- Implement data quality, validation and reconciliation processes within pipelines
- Monitor, troubleshoot and optimise data platform performance
- Support CI/CD, automated testing and DataOps practices
- Maintain technical documentation, metadata and data lineage information
- Collaborate with technical teams, business stakeholders and third-party suppliers
- Investigate and resolve data quality, integration and platform issues
- Contribute to continuous improvement of engineering standards and platform capabilities
Essential Skills & Experience
- Strong commercial experience as a Data Engineer or Senior Data Engineer
- Hands-on experience with Microsoft Fabric and OneLake, or closely related Azure data technologies
- Strong SQL development and database optimisation skills
- Experience building and supporting ETL/ELT pipelines
- Experience working with Azure data services and cloud-based data platforms
- Knowledge of Dynamics 365, Dataverse or similar enterprise business systems
- Experience with APIs, event-driven integrations and change-data capture techniques
- Strong understanding of data modelling, transformation and integration best practices
- Experience implementing data quality, governance and security controls
- Knowledge of CI/CD, version control and DataOps methodologies
- Ability to troubleshoot, investigate and resolve production data issues
Desirable Experience
- Azure Data Factory
- Azure Synapse Analytics
- Azure Data Lake
- Microsoft Fabric Lakehouse and Warehouse solutions
- Fabric Notebooks
- Python or PySpark
- AI and advanced analytics platforms
- Enterprise-scale reporting and management information environments
We're Looking For Someone Who
- Takes ownership of technical delivery and engineering quality
- Enjoys solving complex data integration challenges
- Can work effectively with both technical and non-technical stakeholders
- Has a proactive approach to identifying risks, issues and improvement opportunities
- Thrives in collaborative, delivery-focused environments
- Is passionate about modern data engineering and cloud technologies
Key Technologies
- Microsoft Fabric
- OneLake
- Azure Data Services
- Dynamics 365
- Dataverse
- SQL
- APIs
- CI/CD
- DataOps
- Azure DevOps / Git
Please be aware that this role can only be worked within the UK and not Overseas.
In applying for this role, you acknowledge the following "this role falls in scope of the Off Payroll Working in the Public Sector legislation. Any rates of payment quoted will reflect the gross rate per day for the assignment and will be subject to appropriate taxes and statutory costs. As such the payment to the intermediary and your income resulting from this contract will be different".
General information
Posting ID
PIP12296
Type of employment
Contract
Organisation
Department for Business, Energy & Industrial Strategy
Contract duration
4 months
Location
London
Workplace Type
Remote
Similar roles
-
R0173157 Data Engineer (A)
Aerodyne Industries, LLC Houston, Texas, United States · $84K–$100K/yr
-
Senior Data Engineer
AIR Communities Denver, Colorado, United States · $125K–$145K/yr
-
Senior Data Engineer
ZoomInfo Technologies LLC Toronto, Ontario, Canada
-
Lead Data Engineer
AIR Communities Denver, Colorado, United States · $135K–$160K/yr
-
Data Engineer
Shaker Recruitment Marketing Oak Park, Illinois, United States
-
Lead Data Engineer
PetSmart Phoenix, Arizona, United States