Continuous learning: how a football prediction model adapts without data leakage

One-sentence summary

Foresportia updates its predictions rigorously: new data → checks → after-the-fact evaluation → drift detection → league-level recalibration → automatic adjustments (thresholds/weights) with safeguards.

3 definitions (to stay oriented)

• Model drift: the league changes, so yesterday’s stats no longer fully describe today.
• Calibration: a “60%” should behave like ~6 cases out of 10 over a sufficiently long period.
• Auto-configuration: bounded automatic adjustments (small steps, no abrupt changes).

Why continuous learning is crucial in football

Football is a non-stationary system: squads evolve, playing styles shift, and some leagues are naturally more volatile. Without adaptation, a model can remain globally consistent while losing reliability in a specific league or period.

This is especially true at: season starts, congested schedules, transfer windows, or after coaching changes.

The pipeline (overview)

1. Collection & checks: match data, consistency, timestamps.
2. Pre-processing: aggregations, form indicators, schedule context.
3. Prediction: raw probabilities + reliability indicators.
4. Temporal validation: evaluation always on later matches, never on the future (data leakage prevention).
5. League-level calibration: adjusting probability reliability on recent history.
6. Auto-configuration: micro-adjustments (thresholds/weights) with limits and cooldowns.
7. Monitoring: metrics and alerts (drift).
8. Publication: export of results to the site.

Auto-calibration: probabilities that hold over time

Calibration is very concrete: if a model frequently announces ~60%, we want to observe ~60% corresponding outcomes over time.

This is measured using metrics (and especially reliability curves):

• Brier Score: penalizes probabilities far from reality.
• LogLoss: strongly penalizes overly confident errors.
• Reliability curve: compares announced probabilities to observed frequencies.

To go deeper (with a clear “60% = 6/10” example): Probability calibration.

Auto-configuration: adjusting usage without breaking reliability

Reliable probabilities are not enough: one must also decide how to use them (e.g., which matches to highlight). Auto-configuration adjusts operational parameters cautiously:

• League-specific thresholds: micro-adjustments based on the volume ↔ reliability trade-off.
• Temporal weighting: if drift appears, more weight is given to recent data (without overreaction).
• Regularization: with limited data (season start), extreme corrections are avoided.
• Safeguards: min/max bounds, limited speed, cooldown before further changes.

Complementary article: Thresholds: coverage vs accuracy.

Data leakage prevention: strict temporal validation

This is the most important point: a “continuously learning” system can become misleading if it indirectly sees the future. Here, evaluation uses chronological splits: training on the past, testing on the future.

Automatic adjustments (thresholds, weights) are applied only to upcoming matches, never by rewriting the past.

Concretely, what changes for you?

• More coherent probabilities: fewer “high percentages” that fail to hold over time.
• Better stability: fewer shocks when a league becomes temporarily chaotic.
• Clearer reading: uncertainty is better reflected instead of being hidden.

You can follow after-the-fact evaluation here: Past results.

FAQ

Conclusion

Continuous learning is not about “learning faster”, but about learning properly: temporal validation (data leakage prevention), drift monitoring, league-level recalibration, and cautious auto-configuration. The goal is simple: more reliable probabilities and clearer uncertainty.