How accurate is our weather forecast?
A recent Audit report states that the forecast accuracy as assessed by the public is different from that by the Observatory. In gist, the public considers that the Observatory's forecast is accurate around 75-80% of the time, while the Observatory's self evaluation puts it at about 90%. Why is that? Which one is true?
Public assessment of forecast accuracy is obtained from twice yearly surveys conducted by an independent company. Such surveys have been carried out since 1989. Years of experience looking at the survey results tells us that the public base their impression mostly on our performance in forecasting significant changes in the weather, such as a typhoon passage or the arrival of cold weather. They tend to give a higher mark if the change is well anticipated or a lower mark if otherwise. This is fully understandable as it is next to impossible asking somebody to recollect how the Observatory performs during a series of days when the weather hardly changes.
On the other hand, at the Observatory we use an objective verification scheme to evaluate the accuracy of a weather forecast. In essence, 5 weather elements are evaluated: temperature, rainfall, wind strength, cloud cover, and visibility. If these elements carry equal marks, then each of them carry 20 marks, contributing to a total of 100 marks if the forecast is perfect. However, in real life it would not be realistic to allocate equal marks to them, because their importance to daily life tends to vary with the seasons. For instance, the more important element for different times of the year would be: rain in summer, visibility in spring, and temperature in winter or the cooler months. Thus, a higher mark will be allotted to a correct rain forecast in summer, a correct temperature forecast in winter, so on and so forth.
This way, the weather forecast issued each day is evaluated. At this point you will immediately notice that while the public cares about weather changes, the Observatory's verification is carried out every day, irrespective. This is essentially where the difference between public perception and the Observatory's objective verification comes from. There is no conflict between the two as they are looking at different things.
The Observatory verifies its forecast each day for the very reason that the forecaster has to take into account all relevant weather information each time before issuing the forecast. For instance, if today is a fine day, before arriving at a forecast for yet another fine day tomorrow the forecaster has to convince himself and people at the Observatory that there are sound reasons for him/her to do so. In this instance, the forecast has to scientifically discount the various factors that may bring a cloudy or rainy day, including clouds that are lurking upstream or rain that is round the corner and is already affecting Hong Kong's neighbouring areas. In other words, the forecaster goes through the same processes each day come rain or shine. This is an aspect of the forecaster's work that the public may not be immediately aware of.
Another aspect that the public does not readily recognize is their ever increasing expectations about the weather forecast. It is widely accepted that the accuracy of weather forecast world-wide has slowly but consistently improved, thanks to the work and co-operation of meteorologists, the emergence of new equipment, and the advent of powerful and affordable computers. With such improvement, it is only natural for people to expect the weather forecast to be better and better. The end result is that the gap between public expectations and the accuracy of weather forecast will remain.
We are looking into whether or not the discrepancy between public perception and objective evaluation of weather forecasts can be narrowed.
Figure 1 Observatory weather forecasting centre
The Observatory will not stop its pursuit of continuous improvement. For instance, its short-term forecasting system has been featured many times in international arenas in support of important events, including the Beijing Olympics in 2008, the East Asian Games in 2009, as well as the Shanghai EXPO, the Asian Games in Guangzhou, and the Commonwealth Games in New Delhi, all in 2010. We will continue our effort at seeking perfection.
B.Y. Lee and Terence Kung