Senior Data Engineer (Databricks) - London
Auquan Ltd London, England, United Kingdom
Financial Services · 11-50 employees
Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.
About the role
You will build and maintain end-to-end data pipelines to transform raw customer data into clean, secure, and well-modeled inputs for AI agents. Additionally, you will collaborate with engineering pods to turn bespoke data work into reusable templates and ensure high data quality through monitoring and on-call support.
What they look for
Requirements
The role requires 5+ years of experience in data engineering with strong proficiency in Python, SQL, and distributed processing engines like Spark. Candidates must have a solid grasp of data modeling, orchestration tools, and cloud platforms, along with the ability to communicate complex data trade-offs to stakeholders.
Benefits
Full description
ALFA builds autonomous AI agents for finance workflows (credit, investment, ESG and risk) for banks, private equity and private credit firms, and asset managers. Agents are only as good as the data they run on, and that data arrives messy: filings, portfolio packs, CRM exports, spreadsheets and PDFs.
The Data Engineer owns the path from a customer's raw data to clean, secure, well-modeled data that our agentic workflows and deliverables can trust. You will work in pods alongside Outcome Engineers and Product Engineers, and turn one-off customer data work into reusable pipelines and primitives as our platform grows. What we’re looking for• 5+ years in data engineering, building production pipelines end to end.
- Strong Python and SQL.
- Hands-on experience with Spark or another distributed processing engine.
- Experience with orchestration tools (Airflow, Dagster, Prefect or similar) and cloud data platforms (AWS, GCP or Azure).
- Solid grasp of data modelling, partitioning, and incremental or change-data-capture (CDC) patterns.
- You take ownership, communicate clearly, and like working closely with the people who use the data.
Nice to have• Databrtalog,Workflows, DLT) or similar lakehouse work on Snorg/Hudi.
- Experience with financial or market data.
- Experi or AIsystems: document parsing, vector stores, retrie
- Streaming (Kafka, Kinesis) and dbt.
- Customer empathy: you think like an analyst or risk officer and know which numbers drive their decisions
- Structured thinker: you turn noisy data into clear schemas, steps and validation rules
- Comfortable in ambiguity: you make progress with imperfect data and bring structure where there is none
- Proactive problem-solver: you spot a broken feed before the client does
- Hustle and ownership: you own outcomes end to end, not just tickets
- Curious, fast learner: you keep up with data tooling, AI and finance developments
- Clear communicator: you explain data trade-offs to senior customer stakeholders
- Commercial instinct: you see where a pipeline built for one customer can serve the next
- Build ingestion pipelines for customer financial data from APIs, spreadsheets, documents and third-party sources.
- Design and maintain data models that keep each customer's data secure and separate.
- Supply agentic workflows with reliable inputs, and keep outputs traceable back to source.
- Use LLMs to extract structured data from unstructured documents, with evals and validation checks that catch bad extractions before customers do.
- Plan and run data migrations as our platform evolves.
- Work with Outcome Engineers on live customer deliveries, then turn bespoke data work into reusable connectors, schemas and templates.
- Own data quality, monitoring and runbooks for the pipelines you ship, and join on-call for incidents that touch data.
We are an office-first company and expect regular in-person collaboration as part of this role. You should also be comfortable collaborating across time zones (US, UK, India). We offer a competitive package designed to support you both inside and outside of work:
- 5 days in-person in our Paddington offices, with occasional client travel
- Base Salary: Factors that may impact the agreed upon salary within the range for a particular candidate include years of experience, level of education obtained, skill set, and other external factors.
- Performance Bonus: bonus target
- Equity: Meaningful stock options so you share in our growth
- Time Off: Flexible PTO
- Health & Wellbeing: Private Medical Insurance (UK)
- Retirement: Pension
Similar roles
-
Senior Data Engineer II
Oscar Health New York, New York, United States · $181K–$237K/yr
-
Data Engineer
AcuityMD $113K–$159K/yr
-
Staff Data Engineer, Analytics
Afresh San Francisco, California, United States · $168K–$250K/yr
-
Data Engineer - Agentic AI
Defense Unicorns Springfield, Virginia, United States · $149K–$201K/yr
-
Senior Data Engineer
Southend Pharmacy Houston, Texas, United States · €45K–€65K/yr
-
Senior Data Engineer (Talent Pool)
ioet San José de Alluriquín, Santo Domingo de los Tsáchilas, Ecuador