<div>L. S. Jackson<sup>1</sup>, A. C. Maycock<sup>1</sup>, T.
Andrews<sup>2</sup>, C. J. Smith<sup>1</sup>, and P. M.
Forster<sup>3</sup></div><div><sup>1</sup> School of Earth and Environment, University of
Leeds, Leeds, UK. <sup>2</sup> Met Office Hadley Centre, Exeter,
UK. <sup>3</sup> Priestley International Centre for Climate,
University of Leeds, Leeds, UK.</div><div>Corresponding author: L. S. Jackson
(l.s.jackson@leeds.ac.uk)</div><div>Key Points:</div><ul><li>Emulators of global surface temperature calibrated to individual
climate models can generate large errors in past and future
predictions.</li><li>Emulation errors are not systematically related to model parameters
meaning they cannot be predicted.</li><li>Rigorous out-of-sample evaluation is necessary to characterize
emulator performance.</li></ul><div>Abstract</div><div>Climate model emulators are widely used to generate temperature
projections for climate scenarios, including in the recent IPCC Sixth
Assessment Report. Here we evaluate the performance of a two-layer
energy balance model in emulating historical and future temperature
projections from CMIP6 models. We find that prediction errors can be
large (greater than 0.5<sup>o</sup>C in a given year) and differ
markedly between climate models, forcing scenarios and time periods.
Errors arise in emulating the near-surface temperature response to both
greenhouse gas and aerosol forcing; in some periods the errors due to
these forcings oppose one another, giving the spurious impression of
better emulator performance. Time-varying and state-dependent feedbacks
may contribute to prediction errors. Close emulations can be produced
for a given period but, crucially, this does not guarantee reliable
emulations of other scenarios and periods. Therefore, rigorous
out-of-sample evaluation is necessary to characterize emulator
performance.</div><div>Plain Language Summary</div><div>Complex climate models are state-of-the-art tools used to produce
projections of future climate but they are expensive and take a long
time to run. Climate model emulators are simple statistical or
physically based models which can reproduce projections from complex
climate models at lower cost and more quickly. In this study, we use a
climate model emulator to reproduce projections of twentieth and
twenty-first century temperatures for eight complex climate models. We
show that close emulations can be produced for pre-defined climate
scenarios and time periods. Close emulations are not guaranteed,
however, when the emulator is used for other climate scenarios or other
periods. This is important because climate model emulators are
frequently used to produce projections that are not available from
complex climate models. Evaluation of climate model emulators and
characterization of their uncertainties, therefore, should use data not
used in the calibration of the emulator.</div>