JobCubby JobCubby
Sign up
CMC Markets

Senior Data Engineer

CMC Markets London, England, United Kingdom

Financial Services · 501-1,000 employees

13 h ago
data-engineer Senior (5-10 yrs) Full-time United Kingdom
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

Design and operate scalable ETL/ELT pipelines to process high-frequency market data for quantitative research and trading systems. Collaborate with traders and researchers to implement data quality frameworks and deliver production-grade datasets.

What they look for

Python SQL ETL/ELT Pipelines Market Data Parquet Apache Spark Kafka Time-series Analysis Data Quality Docker Linux CI/CD PostgreSQL ClickHouse Snowflake Databricks

Requirements

Requires expert-level proficiency in Python and SQL with extensive experience in building scalable data pipelines for market data. Candidates must possess deep knowledge of time-series data, distributed processing technologies, and modern data storage formats.

Full description

Senior Data Engineer – Market Data

We are looking for a Senior Data Engineer to build and operate a high-performance market data platform supporting quantitative research and production trading systems. You'll design scalable data pipelines, process billions of market data records, and deliver reliable, production-quality datasets for research, analytics and real-time applications.

Key Responsibilities

  • Build scalable ETL/ELT pipelines for batch and streaming market data.
  • Develop production-grade Python and SQL solutions.
  • Ingest, normalise and validate market data, including trades, ticks, Level 1 & 2 quotes, order books, reference data and corporate actions.
  • Implement automated data quality, reconciliation and monitoring frameworks.
  • Optimise data storage using Parquet, Arrow and modern data lake technologies.
  • Partner with quantitative researchers, traders and engineering teams to deliver trusted datasets.

Essential Skills

  • Strong Python and SQL.
  • Experience building scalable, production data pipelines.
  • Expertise with Parquet and modern data storage formats.
  • Deep understanding of high-frequency market data, including tick data, order books and market data normalisation.
  • Experience handling time-series challenges such as timestamp precision, sequence gaps, duplicate and out-of-order events.
  • Experience with distributed processing (Spark, Polars, Dask, Ray or similar).
  • Experience with Kafka or similar streaming technologies.
  • Familiarity with PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar.
  • Docker, Linux, Git and CI/CD experience.

Highly Desirable

  • Familiarity with OneTick, PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar.
  • Order book reconstruction
  • Tick-to-bar aggregation
  • Airflow, Dagster, Prefect or MLflow
  • Experience supporting quantitative research or trading systems
  • Multi-asset market data (Equities, Futures, FX, Options, ETFs, CFDs)

What We're Looking For

The successful candidate will have a strong background in market data engineering with proven experience building reliable, scalable data platforms.

Our non-negotiable requirements are:

  • Expert Python
  • Strong SQL
  • Scalable data pipeline development
  • Apache Parquet
  • Market data normalisation
  • Automated data quality controls
  • Deep tick and market data expertise

Experience in MLOps is advantageous but not essential. We welcome candidates with strong experience in either market data engineering or MLOps, provided they have the skills to build and operate production-grade data platforms.

Similar roles