Improved Discrimination of Influenza Forecast Accuracy Using Consecutive Predictions

Discussion Our findings indicate that use of forecast streak, in addition to forecast lead and ensemble variance, further discriminates the accuracy of our influenza predictions. This discrimination can be applied in real-time so that operational forecasts of influenza incidence are more precisely segregated. By better distinguishing good and bad forecasts, our predictions are not only well calibrated (i.e. reliable), but they are also ‘sharp’–that is, outcomes are associated with more precise certainties. This idea of sharpness is important. It is the ability to distinguish high and low probability events. As a counter example, one can predict a 50% chance that a coin flip will yield heads. Over many predictions, those forecasts will prove reliable but not sharp (nor particularly informative). For influenza prediction we want to avoid this regression to the mean. That is, rather than assign average probabilities over all predictions, our post-processing aims to distinguish accurate and inaccurate forecasts. While we still want overall forecast accuracy to improve, we also want to be able to reliably discriminate high certainty predictions (e.g. 70%, 90%) from lower certainty events (e.g. 10%, 30%). Such discrimination allows individuals and public health officials to act on forecast information with greater confidence. For example, a forecast of, e.g., a 10% chance that influenza will peak in 5 ±1 weeks indicates that the best model estimate is a peak in 5...
Source: PLOS Currents Outbreaks - Category: Epidemiology Authors: Source Type: research