\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage{natbib}




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}



\usepackage{float}








\begin{document}

\title{Machine-learned uncertainty quantification is not magic: Lessons learned
from emulating radiative transfer with ML}



\author[1,2]{Ryan Lagerquist}%
\author[1,2,3]{Imme Ebert-Uphoff}%
\author[2]{David D Turner}%
\author[2]{Jebb Q. Stewart}%
\affil[1]{Cooperative Institute for Research in the Atmosphere (CIRA), Colorado State University}%
\affil[2]{National Oceanic and Atmospheric Administration (NOAA) Global Systems Laboratory (GSL)}%
\affil[3]{Department of Electrical and Computer Engineering, Colorado State University}%


\vspace{-1em}



  
  \date{June 12, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup





\selectlanguage{english}
\begin{abstract}
Machine-learned uncertainty quantification (ML-UQ) has become a hot
topic in environmental science, especially for neural networks.~
Scientists foresee the use of ML-UQ to make better decisions and assess
the trustworthiness of the ML model.~ However, because ML-UQ is a new
tool, its limitations are not yet fully appreciated.~ For example, some
types of uncertainty are fundamentally unresolvable, including
uncertainty that arises from data being out of sample,~\emph{i.e.},
outside the distribution of the training data.~ While it is generally
recognized that ML-based point predictions (predictions without UQ) do
not extrapolate well out of sample, this awareness does not exist for
ML-based uncertainty.~ When point predictions have a large error,
instead of accounting for this error by producing a wider confidence
interval, ML-UQ often fails just as spectacularly.~ We demonstrate this
problem by training ML with five different UQ methods to predict
shortwave radiative transfer.~ The ML-UQ models are trained with real
data but then tasked with generalizing to perturbed data
containing,~\emph{e.g.}, fictitious cloud and ozone layers.~ We show
that ML-UQ completely fails on the perturbed data, which are far outside
the training distribution.~ We also show that when the training data are
lightly perturbed -- so that each basis vector of perturbation has
\emph{a little} variation in the training data -- ML-UQ can extrapolate
along the basis vectors with some success, leading to much better (but
still somewhat concerning) performance on the validation and testing
data.~ Overall, we wish to discourage overreliance on ML-UQ, especially
in operational environments.%
\end{abstract}%



\sloppy




\selectlanguage{english}
\FloatBarrier
\end{document}

