Summarising the difference in the relative economic value of severe fire weather forecasts in a single number metric  — Australian Meteorological and Oceanographic Society

Summarising the difference in the relative economic value of severe fire weather forecasts in a single number metric  (#214)

Nicholas Loveday 1 , Deryn Griffiths 2 , Michael Foley 2 , Alexei Hider 2
  1. Science and Innovation Group, Bureau of Meteorology, Darwin, Northern Territory, Australia
  2. Science and Innovation Group, Bureau of Meteorology, Melbourne, Victoria, Australia

The Bureau of Meteorology routinely issues gridded fire weather forecasts and warnings. Important decisions need to be made around what inputs to use for these forecasts and how much operational meteorologists should alter automated forecasts with the aim of improving the forecast for users. Verification of forecasts of grassland and forest fire danger indices can inform these decisions to ensure that the Bureau delivers as much value as possible to the Australian community. 

Relative Economic Value (REV) curves provide an idealised way to compare the value provided by different forecast systems for particular decision thresholds. REV curves have highlighted the additional value that operational meteorologists provide over automated guidance in forecasting whether or not severe fire danger thresholds (Fire Danger Index ≥ 50) will be reached. In contrast, mean error metrics have in some cases hidden the value that operational meteorologists provide with forecasting severe thresholds. 

Relative economic value curves provide a great deal of information but using them to summarise a large number of results can be difficult. In contrast, a single number that captures the difference in value has the advantage of being usable in scorecards as a summary statistic. It provides a simple way to highlight which parts of Australia and which lead days and seasons a particular forecast system (either fully automated, or a blend of human and machine) is providing the most value for more extreme fire weather forecasts. 

We propose a metric that, in a single number, summarises the difference in value between two forecast sources. We demonstrate how this can be used to summarise both 'potential' (where the user selects the optimal forecast decision thresholds based on their own sensitivities) and 'actual' REV curves for severe fire danger forecasts. We explore the contrasting insights that the 'potential' and 'actual' REV summaries provide.

#amos2020