Explaining Machine Learning-Based Forecasts Through Historical Analogies

Overview

Debates about the future trajectory of economic conditions often revolve around historical analogies—how past events compare to present conditions. Policymakers and analysts frequently frame their assessments in these terms, while forecasting models are typically explained through the contributions of individual variables.

In Goulet Coulombe, Göbel, and Klieber (2024), we reconcile these two perspectives, showing that black-box machine learning (ML) algorithms inherently operate in the language of historical analogies. We demonstrate that they can be understood as weighted combinations of past economic episodes, offering a transparent, intuitive way to interpret machine learning forecasts.

This perspective builds on the observation that each out-of-sample prediction can be rewritten as a linear combination of in-sample values of the target variable. The weights in this combination reflect pairwise proximity scores between current and past economic conditions. Importantly, these proximity weights emerge naturally from a wide class of models—including (kernel) ridge regression, random forests, boosted trees, and neural networks—without any additional computation. The mathematical foundation for this view lies in the dual formulation of least squares-based estimators, which applies not only to complex ML models but also to familiar linear forecasting tools such as factor-augmented autoregressions.

Forecast Weights

Intuitively, the weights indicate how closely the information in the test data–used to make one specific prediction—resembles the information from each historical period in the training sample. While some models (e.g., random forest) constrain weights to be positive, capturing only similarities, others accommodate both similarities and dissimilarities with the past. Regardless of the approach, the weights represent the attention assigned to each period covered by the in-sample data when formulating an out-of-sample prediction.

Below we show the time-series of said (smoothed) forecast weights for several ML models and forecast horizons of one and four quarters.

Forecast Contributions

Applying the above forecast weights to the training observations of the target variable allows us to decompose a prediction into contributions from specific historical points. Summing these contributions creates a time series that precisely matches the prediction, offering a clear and intuitive explanation of how ML forecasts link past and present.

Forecast Performance

What makes for a good forecasting model is obviously not interpretability alone, but its actual out-of-sample (OOS) forecasting performance. This is not to degrade the importance of interpretability at all, but rather the opposite: interpretability is key to understanding the root cause of the performance of the model — be it out- or underperformance.

In the figures below, we thus show the root-mean-square-error (RMSE) of several models, relative to the RMSE of an autoregressive process of order four, AR(4) — for two sub periods, analogous to Table 4 in Goulet Coulombe et al. (2024). A number below 1 indicates that the model is on average doing better at forecasting the observed inflation print during the corresponding OOS period, than the AR(4) model does. A number above 1 signals the opposite: the model’s prediction were less accurate than those of the AR(4) benchmark.