Abstract

With the advent of high-resolution forecasts came new challenges for analyzing their performance. It is no longer satisfactory to report only the root mean square error or similar grid-point-by-grid-point accumulative summaries. Often such scores give misleading information caused by either doubl... Show moreWith the advent of high-resolution forecasts came new challenges for analyzing their performance. It is no longer satisfactory to report only the root mean square error or similar grid-point-by-grid-point accumulative summaries. Often such scores give misleading information caused by either double penalties (when a small error in timing or spatial placement leads to a miss and a false alarm accumulation for this same error) or over accumulation of small-scale errors. When compared against coarser models, the coarser models generally perform better because of their insensitivity to these types of errors. Nevertheless, most forecasts find the high-resolution models to be superior. Other challenges include the need for providing more diagnostic information about forecast performance so that a researcher or user can know how the forecast did well or poorly. These issues have brought on a rapid increase in new forecast verification methodology since the late nineties and early naughts. In many cases, the methods came directly from other related fields, such as, spatial statistics, image analysis, and computer vision, which led to a wide array of new methodologies that were not well understood by most researchers. In order to make some order out of the chaos, an inter-comparison project was formed, which enabled a better understanding of the various methods and how they relate to each other. The project focused on single snapshot spatial forecasts of fairly basic precipitation patterns over the central United States. Now, a new inter-comparison project is underway that aims to bring more complexity into the meta-verification project. It is called the Mesoscale Verification Intercomparison over Complex Terrain (MesoVICT; https://ral.ucar.edu/projects/icp), and it involves various variables including precipitation and wind, ensembles of forecasts, ensembles of observations, and is centered of complex terrain in central Europe. This talk will give an overview of spatial forecast verification methods and introduce the new project. Show less