Senior Data & AI Engineer

About the role

Cloudbox is building a unified multi-tenant data and reporting platform on Microsoft Fabric, aggregating 10+ vendor APIs (Autotask PSA, Microsoft 365/Graph, Azure Monitor, Microsoft Defender, Huntress, Proofpoint, Cato SASE, BambooHR, Xero, and internal systems) into a medallion lakehouse for hedge fund and financial-services clients. You'll own the data platform end-to-end, from raw vendor ingestion through to a query-ready gold layer and an AI-assisted reporting layer. This is a greenfield build: you will design the medallion architecture, build it, and then run it as its long-term owner.

What you'll own

• Metadata-driven OneLake medallion lakehouse architecture (bronze/silver/gold)

• Azure Functions-based vendor ingestion pipelines across the full vendor set, fanned out per tenant from a central metadata registry, starting with an Autotask-first vertical slice

• Dimensional modelling and semantic model design - the semantic layer serving dashboards and reporting, with row-level security (RLS) for client-level access

• The AI proof-of-concept - grounded, provenance-backed natural-language querying over platform data, with anti-fabrication controls

• Data quality, reconciliation, and observability for the pipeline layer - structured logging, SLOs, and alerting, with reconciliation proven against synthetic vendor data before production

• Data governance with Microsoft Purview - lineage, classification, and sensitivity labelling across the medallion layers

• Documentation and knowledge transfer - runbooks, architecture decision records, and pairing with the wider team, so the platform never carries a bus factor of one

What we're looking for

• Senior-level experience with Microsoft Fabric, OneLake, or equivalent modern lakehouse platforms (Databricks, Synapse acceptable if willing to specialise into Fabric)

• Strong dimensional modelling and semantic model design background (star schemas, conformed dimensions, Fabric/Power BI semantic models)

• Experience building production data pipelines from third-party/vendor APIs - ideally within a metadata-driven ingestion framework

• Hands-on with the Azure services surrounding Fabric - Function Apps, Key Vault, Blob Storage, and Static Web Apps

• Source control and CI/CD experience with both Azure DevOps and GitHub, including Fabric Git integration; Bicep or equivalent infrastructure-as-code a plus

• Security-first engineering habits - managed identities and Key Vault for all secrets; no credentials in code, ever

• Working knowledge of Microsoft Purview (or equivalent) for governance, lineage, and sensitivity labelling - this platform serves regulated financial-services clients

• Familiarity with grounding techniques for LLM-based querying over structured data (RAG, citation/provenance patterns) is a plus - the core screen for this role is lakehouse and modelling depth, and the AI layer can be developed in-role with specialist support

• Comfortable being the technical anchor for the platform's most critical dependency chain

Nice to have: Exposure to Azure AI Foundry (model deployment, evaluation, prompt flow), Fabric IQ and its ontology layer, Fabric data agents, MCP-style agent tool-calling, Azure Key Vault / Managed HSM and customer-managed key (CMK) patterns, and Fabric capacity planning (F-SKU sizing) - these support later platform tiers and none is a screening requirement