A joint calibration model for combining predictive distributions

Patrizia Agati, Daniela Giovanna Calò, Luisa Stracqualursi

Abstract


In many research fields, as for example in probabilistic weather forecasting, valuable predictive information about a future random phenomenon may come from several, possibly heterogeneous, sources. Forecast combining methods have been developed over the years in order to deal with ensembles of sources: the aim is to combine several predictions in such a way to improve forecast accuracy and reduce risk of bad forecasts.
In this context, we propose the use of a Bayesian approach to information combining, which consists in treating the predictive probability density functions (pdfs) from the individual ensemble members as data in a Bayesian updating problem. The likelihood function is shown to be proportional to the product of the pdfs, adjusted by a joint “calibration function” describing the predicting skill of the sources (Morris, 1977). In this paper, after rephrasing Morris’ algorithm in a predictive context, we propose to model the calibration function in terms of bias, scale and correlation and to estimate its parameters according to the least squares criterion. The performance of our method is investigated and compared with that of Bayesian Model Averaging (Raftery, 2005) on simulated data.

Full Text:

PDF (English)

References


M. COLLINS (2007), Ensemble and probabilities: a new era in the prediction of climate change, “Philosophical Transactions of the Royal Society”, A, 365, pp. 1957-1970.

R. M. COOKE (1991), Experts in uncertainty. Opinion and subjective probability in science, Oxford University Press, Oxford.

A.P. DEMPSTER, N.M. LAIRD, D.B. RUBIN (1977), Maximum likelihood from incomplete data via the EM algorithm, “Journal of the Royal Statistical Society”, B, 39, pp. 1-39.

T. GNEITING, A.H. WESTVELD, A.E. RAFTERY, T. GOLDMAN (2004), Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation, Technical report n. 449, Department of Statistics, University of Washington.

J.A. HOETING, D.M. MADIGAN, A.E. RAFTERY, C.T. VOLINSKY (1999), Bayesian Model Averaging. A tutorial, “Statistical Science”, 14, pp. 382-401.

E.E. LEAMER (1978), Specification Searches, Wiley, New York.

D.V. LINDLEY (1990), The 1988 Wald Memorial Lectures: the present position in Bayesian Statistics, “Statistical Science”, 5, pp. 44-65.

G.J. McLACHLAN, T. KRISHNAN (1997), The EM algorithm and Extensions, Wiley, New York.

P. MONARI, P. AGATI (2001), Fiducial inference in combining expert judgements, “Statistical Methods and Applications”, 10, pp. 81-97.

P. A. MORRIS (1977), Combining expert judgements: a Bayesian approach, “Management Science”, 23, pp. 679-693.

A.E. RAFTERY, T. GNEITING, F. BALABDAOUI, M. POLAKOWSKI (2005), Using Bayesian model averaging to calibrate forecast ensembles, “Monthly Weather Review”, 133, pp. 1155-1174.

J.R. RHOME (2007), Technical summary of the National Hurricane Center track and intensity models, National Oceanic and Atmospheric Administration, National Weather Service, available at http://www.nhc.noaa.gov/modelsummary.shtml

R.L. WINKLER, R.T. CLEMEN (2004), Multiple experts vs multiple methods: combining correlation assessments, “Decision Analysis”, 1, pp. 167-176.




DOI: 10.6092/issn.1973-2201/3505