Observability, drift and model decay

A trading model that worked when it was deployed will not necessarily keep working. Markets change, data sources change, and the system around the model changes. Observability is the discipline of seeing those changes before they become losses.

The three signals

• Metrics: numbers tracked over time, such as latency, error rates, prediction distributions and live performance against expectation.
• Logs: detailed records of what happened and why, kept so any decision can be reconstructed later.
• Traces: the path of a single request through every component, used to find where time or errors come from.

Two kinds of drift

Data drift means the inputs have changed: volatility regimes shift, volume patterns move, a data vendor changes a field. Concept drift means the relationship the model learned has changed: the same inputs no longer lead to the same outcomes. Data drift can often be detected directly. Concept drift usually shows up first as live results moving away from what testing suggested.

Safer releases

• Shadow deployment: a new model runs on live data and records its decisions without acting on them, so it can be compared with the current one.
• Canary release: a new version handles a small share of activity first, with automatic rollback if its metrics degrade.
• Versioned rollback: every deployed model and configuration can be restored to a known previous state.

The goal is not a model that never decays. It is a system that notices decay quickly and responds in a controlled way.

Educational content only. Not financial advice.