Data-driven Innovation in Financial Markets
Successful electronic engagement with Financial Markets requires a foundation of meaningful and productive handling of noisy and high dimensional real-time and historical data. Techniques must be informed by established methods, while also exploiting appropriate technology to generate innovative custom outcomes.
Effective approaches to making large data manageable and understandable involve:
- deriving and calculating of robust summary metrics,
- dimensionality reduction,
- modelling time-dependence,
- calculating correlation & co-integration of processes.
Interacting with meaningful tabular and graphical representations guides this process, and provides groundwork for signals and alerting, as well as AI/model-assisted multi-factor data exploration.
Rapid advances in technology are drastically changing the data-analysis landscape, enabling greater utility and deeper exploration, but a solid theoretical basis and careful testing remains paramount for accuracy and effectiveness.
Technological foundations include:
- high performance time-series databases,
- FPGAs for real-time responsiveness to streaming data,
- cloud for reliable, access-anywhere deployment,
- modern web browsers for ease of use and portability.
In our ongoing research, we are applying the philosophy and techniques described above to problems of interest to algorithmic traders, exchange providers, and both buy and sell side market participants.
Specifically:
- evolutionary algorithms built on analytic-driven micro-strategies,
- simulation of exchanges from the behaviour of multiple interacting, data- and goal-driven actors, with a matching engine exposing and managing the resulting order book,
- formal quantification & representation of data relationships and trading behaviours for execution and/or market monitoring.
These are each built upon a common data management and analysis core, and combine to create a financial data ecosystem providing power and insight across the entire problem domain.