Flu Forecast Game

About Flu Forecast Game

2026–27 season · weekly influenza hospital admissions

The science

The goal of the Flu Forecast Game is to investigate the aggregation of both crowd and expert forecasts through a novel "community forecast" interface.

Influenza hospitalizes hundreds of thousands of people in the US each season, and its timing and size vary widely from year to year. Each week, scientific experts submit statistical and mechanistic model forecasts to the CDC's FluSight (Reich et al., 2019a; Mathis et al., 2024), whose ensemble combines them into a single forecast. But prior research has found that these model forecasts can benefit from being combined with human judgment and aggregation (McAndrew et al., 2022; Bosse et al., 2022).

Clinicians, public health staff, and members of the public can observe flu trends before they become apparent in the data available to scientists. This project seeks their input to complement the scientific models. We believe that influenza forecasting is a domain where both "wisdom of the crowd" (Galton, 1907) and expert analyses have key roles to play.

What we want to learn

  • Whether a crowd forecast like this is accurate and well calibrated.
  • Whether it adds something to the FluSight models in terms of accuracy, early warning, or otherwise.
  • Whether it is useful to public health in producing actionable insights into trends for officials and hospitals.

The community forecast for every place and week is public and updates continuously. The forecast and trade charts here can show the CDC's FluSight ensemble beside the community forecast: turn on “CDC forecast” by the chart. After the season we will compare these forecasts with the FluSight ensemble and with individual models, measuring accuracy against influenza hospital admissions reported to NHSN. We will also look for evidence of early warning trends or other information revelation in the community forecasting activity.

Under the hood: market dynamics

A market is one way to pool many people's beliefs into a "community forecast", which in traditional markets is a price. In this project, each participant "spends" their points by proposing updates to the community forecasts and earns reward points based on accuracy later. The specific mechanism, compatible with forecasting the median and the 10th and 90th percentiles through the theory of proper scoring rules, is based on scoring rule markets for general statistics (Frongillo & Waggoner, 2018; Lambert et al., 2008). For the mechanics, with worked examples, see How it works.

Points have no cash value and cannot be bought or sold. The forecast infrastructure is run on a public test blockchain with test tokens, which makes every prediction and settlement publicly verifiable, but no monetary value is involved.

Principal investigators

Supported by Gensyn.

Relationship to prior work

Combining many models' forecasts is now standard in US influenza forecasting: in CDC's FluSight challenge the ensemble has been among the most accurate forecasts (Reich et al., 2019b; Mathis et al., 2024), as it was for COVID-19 deaths (Cramer et al., 2022). Human judgment can add to this accuracy. Aggregated forecasts from people have been competitive with statistical models for influenza and COVID-19 cases, and adding them to model ensembles has improved accuracy (Farrow et al., 2017; McAndrew et al., 2022; Bosse et al., 2022). The Flu Forecast Game collects this judgment through a market, an approach tried for influenza since the mid-2000s (Polgreen et al., 2007; Tung et al., 2015), and uses play money, which has forecast about as well as real money in some settings (Servan-Schreiber et al., 2004; but see Rosenbloom & Notz, 2006).

There is also evidence against markets. Low-volume markets on Polymarket were recently dominated by the FluSight ensemble for US influenza hospitalizations (Dudley & Magdaleno, 2026), though the two forecasts compared had different targets and formats. Statistically aggregated polls have outperformed markets (Atanasov et al., 2017), and play-money markets can be manipulated, including by traders using multiple accounts (Blume et al., 2010; Rasooly & Rozzi, 2025).

Our design responds to these findings. The standard approach Polymarket used in the study above, of binning the outcome into a small number of bins, introduces a significant forecast error. By contrast, we employ a novel median market framework. In our markets, the state (community prediction) is directly a prediction of the same median and 0.1 and 90th percentiles as FluSight, so the two forecasts can be evaluated with the same weighted interval score (Bracher et al., 2021). Liquidity is scheduled and sized to each place, participation is free, and points are spread across many markets.

References

  • Atanasov, P., Rescober, P., Stone, E., Swift, S. A., Servan-Schreiber, E., Tetlock, P., Ungar, L., & Mellers, B. (2017). Distilling the wisdom of crowds: Prediction markets vs. prediction polls. Management Science, 63(3), 691–706. doi.org/10.1287/mnsc.2015.2374
  • Blume, M., Luckner, S., & Weinhardt, C. (2010). Fraud detection in play-money prediction markets. Information Systems and e-Business Management, 8(4), 395–413. doi.org/10.1007/s10257-008-0103-3
  • Bosse, N. I., Abbott, S., Bracher, J., Hain, H., Quilty, B. J., Jit, M., van Leeuwen, E., Cori, A., & Funk, S. (2022). Comparing human and model-based forecasts of COVID-19 in Germany and Poland. PLoS Computational Biology, 18(9), e1010405. doi.org/10.1371/journal.pcbi.1010405
  • Bracher, J., Ray, E. L., Gneiting, T., & Reich, N. G. (2021). Evaluating epidemic forecasts in an interval format. PLoS Computational Biology, 17(2), e1008618. doi.org/10.1371/journal.pcbi.1008618
  • Cramer, E. Y., Ray, E. L., Lopez, V. K., et al. (2022). Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States. Proceedings of the National Academy of Sciences, 119(15), e2113561119. doi.org/10.1073/pnas.2113561119
  • Dudley, C., & Magdaleno, R. (2026). Prediction markets underperform simple baselines for infectious disease forecasting. Preprint, arXiv:2605.11220. arxiv.org/abs/2605.11220
  • Farrow, D. C., Brooks, L. C., Hyun, S., Tibshirani, R. J., Burke, D. S., & Rosenfeld, R. (2017). A human judgment approach to epidemiological forecasting. PLoS Computational Biology, 13(3), e1005248. doi.org/10.1371/journal.pcbi.1005248
  • Frongillo, R., & Waggoner, B. (2018). An axiomatic study of scoring rule markets. 9th Innovations in Theoretical Computer Science Conference (ITCS 2018), LIPIcs 94, 15:1–15:20. doi.org/10.4230/LIPIcs.ITCS.2018.15
  • Galton, F. (1907). Vox populi. Nature, 75(1949), 450–451. doi.org/10.1038/075450a0
  • Lambert, N. S., Pennock, D. M., & Shoham, Y. (2008). Eliciting properties of probability distributions. Proceedings of the 9th ACM Conference on Electronic Commerce (EC '08), 129–138. doi.org/10.1145/1386790.1386813
  • Mathis, S. M., Webber, A. E., León, T. M., et al. (2024). Evaluation of FluSight influenza forecasting in the 2021–22 and 2022–23 seasons with a new target laboratory-confirmed influenza hospitalizations. Nature Communications, 15, 6289. doi.org/10.1038/s41467-024-50601-9
  • McAndrew, T., Codi, A., Cambeiro, J., Besiroglu, T., Braun, D., Chen, E., Urtubey De Cesaris, L. E., & Luk, D. (2022). Chimeric forecasting: Combining probabilistic predictions from computational models and human judgment. BMC Infectious Diseases, 22, 833. doi.org/10.1186/s12879-022-07794-5
  • Polgreen, P. M., Nelson, F. D., & Neumann, G. R. (2007). Use of prediction markets to forecast infectious disease activity. Clinical Infectious Diseases, 44(2), 272–279. doi.org/10.1086/510427
  • Rasooly, I., & Rozzi, R. (2025). How manipulable are prediction markets? Preprint, arXiv:2503.03312. arxiv.org/abs/2503.03312
  • Reich, N. G., Brooks, L. C., Fox, S. J., et al. (2019a). A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States. Proceedings of the National Academy of Sciences, 116(8), 3146–3154. doi.org/10.1073/pnas.1812594116
  • Reich, N. G., McGowan, C. J., Yamana, T. K., et al. (2019b). Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. PLoS Computational Biology, 15(11), e1007486. doi.org/10.1371/journal.pcbi.1007486
  • Rosenbloom, E. S., & Notz, W. (2006). Statistical tests of real-money versus play-money prediction markets. Electronic Markets, 16(1), 63–69. doi.org/10.1080/10196780500491303
  • Servan-Schreiber, E., Wolfers, J., Pennock, D. M., & Galebach, B. (2004). Prediction markets: Does money matter? Electronic Markets, 14(3), 243–251. doi.org/10.1080/1019678042000245254
  • Tung, C.-Y., Chou, T.-C., & Lin, J.-W. (2015). Using prediction markets of market scoring rule to forecast infectious diseases: A case study in Taiwan. BMC Public Health, 15, 766. doi.org/10.1186/s12889-015-2121-7