数值天气预报检验方法研究进展

doi:10.11867/j.issn.1001-8166.2014.03.0327

数值天气预报检验是改进及应用数值模式的重要环节。近年来,模式检验中的观念不断更新,适用于不同预报产品及不同用户需求的模式检验方法也不断涌现。首先简单回顾了以列联表为基础的传统的模式检验方法。其次重点总结了伴随高分辨率数值预报而出现的空间诊断检验技术,按照检验目的的不同,诊断方法可以归纳为:①基于滤波技术的分辨模式在不同时空尺度上预报能力的邻域法、尺度分离法;②利用位移偏差诊断模式预报位置、面积、方位、轴角等与观测差异的属性判别法、变形评估法。然后阐述了集合样本成员的概率分布函数(PDF)、集合预报与观测概率分布函数相似程度、事件发生的概率预报等集合预报检验方法。最后论述了空间诊断技术、集合预报检验方法的适用领域,并讨论了模式检验中存在的一些问题及未来的发展方向。

关键词: 空间诊断, 集合预报, 概率预报

The numerical weather prediction verification is a key step to improving applied numerical models. In recent years, the concepts used in model verification have been updated constantly and the model verification approaches fit for different prediction products and meeting the needs of different customers have proposed continuously. First, this research gives a brief overview of traditional model verification methods based on contingency table. Second, the spatial verification technology along with high resolution numerical prediction is illustrated. According to different purposes, the spatial verification methods can mostly be classified into two general categories: filtering methods and displacement methods. The filtering methods can be further delineated into neighborhood and scale separation, and the displacement methods can be divided into features-based ones and field deformations. Third, the probability distribution function of ensemble prediction sample members, the similar degrees between probability distribution function of ensemble prediction and the observation distribution function, the probability of an event occurring and such ensemble prediction verification methods are described. Finally, the fields where spatial verification technology and ensemble prediction verification methods can be used are analyzed and some problems concerning model verification and the direction in which these technologies will go are discussed.

Key words: Spatial forecast verification, Ensemble forecasts, Probability forecasts.

中图分类号:

P456.7

图1 邻域法匹配示意图 ^{[

30
]} (a)观测场;(b)传统方法检验的匹配模式;(c)邻域法匹配

Fig.1 The schematic of neighborhood method matching ^{[

30
]} (a) The observation in the domain;(b)The traditional verification matches the same grid box in the forecast;(c)Fuzzy verification considers a neighbourhood surrounding the observations

图2 形变方法示意图 (a)预报场;(b)观测场;(c)通过预报场、观测场构造的形变矢量^{[

4
]}

Fig.2 The schematic of field deformation methods (a)The forecast field;(b) The observation field;(c)The vector field is formed by field-deformation approaches^{[

4
]}

[1]	Chen Haishan,Sun Zhaobo. Design of a comprehensive land surface model and its validation Part II: Model validation[J].Chinese Journal of Atmospheric Sciences, 2005, 29(2):272-282.
	[陈海山,孙照渤.陆面模式CLSM的设计及性能检验II.模式检验[J].大气科学,2005,29(2):272-282.]
[2]	Pan Liujie, Zhang Hongfang, Zhu Weijun, et al. Forecast performance verification of the ECMWF model over the Northeast Hemisphere[J].Climatic and Environmental Research, 2013, 18(1):112-123.
	[潘留杰, 张宏芳, 朱伟军, 等. ECMWF模式对东北半球气象要素场预报能力的检验[J]. 气候与环境研究, 2013, 18(1) :112-123.]
[3]	Wilks D S. Statistical Methods in the Atmospheric Sciences[M]. Oxford,UK:Academic Press, 2005: 234-251.
[4]	Gilleland E, David A, Brown G. Intercomparison of spatial forecast verification methods[J]. Weather and Forecasting, 2009, 24(6):1 416-1 429.
[5]	Baldwin M E, Kain J S. Sensitivity of several performance measures to displacement error, bias, and event frequency[J].Weather and Forecasting, 2006, 21(5):636-648.
[6]	Casati B, Wilson L J, Stephenson D B, et al. Review forecast verification: Current status and future directions[J].Meteorological Applications, 2008, 15(3):3-18.
[7]	Zepeda A J, Foufoula G E, Droegemeier K.Space-time rainfall organization and its role in validating quantitative precipitation forecasts[J]. Journal of Geophysical Research, 2000, 105(D8):10 129-10 146.
[8]	Tustison B, Foufoula-Georgiou E, Harris D.Scale-recursive estimation for multisensor quantitative precipitation forecast verification: A preliminary assessment[J].Journal of Geophysical Research, 2003, 108(3):11 775-11 784.
[9]	Casati B, Ross G, Stephenson D B. A new intensity-scale approach for the verification of spatial precipitation forecasts[J]. Meteorological Applications, 2006, 11:141-154.
[10]	Davis C, Brown B, Bullock R. Object-based verification of precipitation forecasts. Part I: Methods and application to mesoscale rain areas[J]. Monthly Weather Review, 2006, 134(10):1 772-1 784.
[11]	Davis C A, Brown B G, Bullock R G. Object-based verification of precipitation forecasts. Part II: Application to convective rain systems[J].Monthly Weather Review, 2006, 134(7):1 785-1 795.
[12]	Liu Jianguo, Xie Zhenghui, Zhao Linna, et al. BMA probabilistic forecasting for the 24-h TIGGE multi-model ensemble forecasts of surface air temperature[J]. Chinese Journal of Atmospheric Sciences, 2013, 37(1):43-53.
	[刘建国,谢正辉,赵琳娜,等.基于多模式集合的小时气温概率预报[J]. 大气科学, 2013, 37(1):43-53.]
[13]	Zhang Hongfang, Pan Liujie. A comparison analysis of precipitation forecast ability between ECMWF and Japan[J]. Meteorological Monthly, 2014,in press.[张宏芳,潘留杰. ECMWF、日本高分辨率模式降水预报能力的对比分析[J].气象, 2014,待刊.]
[14]	Li Weiping, Liu Xin, Nie Suping, et al. Comparative studies of snow cover parameterization schemes used in climate models[J]. Advances in Earth Science, 2009, 24(5): 512-522.
	[李伟平,刘新,聂肃平, 等. 气候模式中积雪覆盖率参数化方案的对比研究[J]. 地球科学进展, 2009, 24(5): 512-522.]
[15]	Wang Chenghai. Evaluating the progress of the CMIP and its application prospect in China[J]. Advances in Earth Science, 2009, 24(5): 461-468.
	[王澄海. CMIP研究计划的进展及其在中国地区的检验和应用前景[J]. 地球科学进展, 2009, 24(5): 461-468.]
[16]	Kuang Xueyuan, Liu Jian, Wang Hongli, et al. Comparison of simulated and reconstructed precipitation in China during the last millennium[J]. Advances in Earth Science,2009, 24(2): 159-171.
	[况雪源,刘健,王红丽, 等. 近千年来中国区域降水模拟与重建资料的对比分析[J]. 地球科学进展, 2009, 24(2): 159-171.]
[17]	Wang Bin, Xie Xin, Li Lijuan. A review on aspects of climate simulation assessment[J]. Advances in Atmospheric Sciences, 2009, 26(4):736-747.
[18]	Liu Yanhua,Zhang Shuwen,Mao Lu, et al. An evaluation of simulated and estimated land surface states with two different models[J]. Advances in Earth Science, 2013,28(8):913-922.
	[刘彦华,张述文,毛璐, 等. 评估两类模式对陆面状态的模拟和估算[J].地球科学进展,2013,28(8):913-922.]
[19]	Brownlee K A. Statistical Theory and Methodology in Science and Engineering[M]. New York: John Wiley & Sons Wiley, 1965: 26-30.
[20]	Schaefer J T. The critical success index as an indicator of warning skill[J].Weather and Forecasting, 1990, 5(4):570-575.
[21]	Mason I. Dependence of the critical success index on sample climate and threshold probability[J]. Australian Meteorological Magazine, 1989, 37:75-81.
[22]	Doswell C A, Jones R D, Keller D L. On summary measures of skill in rare event forecasting based on contingency tables[J]. Weather and Forecasting, 1990, 5(4):576-585.
[23]	Brier G W. Verification of forecasts expressed in terms of probability[J].Monthly Weather Review, 1950, 78(1):1-3.
[24]	Stephenson D B. Use of the “odds ratio” for diagnosing forecast skill[J].Weather and Forecasting, 2000, 15(7):221-232.
[25]	Brill K, Mesinger F. Applying a general analytic method for assessing bias sensitivity to bias—Adjusted threat and equitable threat scores[J]. Weather and Forecasting, 2009, 24(6):1 748-1 754.
[26]	Ding Jincai. Review of weather prediction verifying techniques[J]. Journal of Nanjin Institute of Meteorology, 1995,18(1):143-150.
	[丁金才. 天气预报评分方法评述[J]. 南京气象学院学报,1995, 18(1):143-150.]
[27]	Jolliffe I T, Stephenson D B. Forecast Verification: A Practitioner’s Guide in Atmospheric Science[M]. Chichester: Wiley and Sons, 2003.
[28]	Hoffman R N, Liu Z, Louis J, et al. Distortion representation of forecast errors[J].Monthly Weather Review, 1995, 123(5):2 758-2 770.
[29]	Ebert E, McBride J L.Verification of precipitation in weather systems: Determination of systematic errors[J]. Journal of Hydrology, 2000, 239(6):179-202.
[30]	Ebert E, McBride J L. Neighborhood verification: A strategy for rewarding close forecasts[J].Weather and Forecasting, 2009, 18(4):632-638.
[31]	Baldwin M E, Kain J S. Sensitivity of several performance measures to displacement error, bias, and event frequency[J].Weather and Forecasting, 2006, 21(5):636-648.
[32]	Marzbzn C, Sandgathe S. Cluster analysis for object-oriented verification of fields: A variation[J].Monthly Weather Review, 2008, 136(8):1 013-1 025.
[33]	Micheas A C, Fox N I, Lack S A. Cell identification and verification of QPF ensembles using shape analysis techniques[J].Journal of Hydrology, 2007, 343(4):105-116.
[34]	Lack S, Limpert G L, Fox N I. An object-oriented multiscale verification scheme[J].Monthly Weather Review, 2009, 131(8):1 001-1 005.
[35]	Davis C A, Brown B G, Bullock, et al. The method for object-based diagnostic evaluation (MODE) applied to numerical forecasts from the 2005 NSSLSPC spring program[J].Weather and Forecasting, 2009, 24(8):1 252-1 267.
[36]	Sock S, Tribbia J, Rakovec J. Object-based analysis and verification of WRF model precipitation in the low- and midlatitude Pacific Ocean[J].Monthly Weather Review, 2010, 138(11):4 561-4 575.
[37]	Johnson A, Wang X, Kong F, et al. Object-based evaluation of the impact of horizontal grid spacing on convection—Allowing forecasts[J].Monthly Weather Review, 2013, 141(8):3 413-3 425.
[38]	Briggs W M, Levine R A. Wavelets and field forecast verification[J]. Monthly Weather Review, 1997, 125(3):1 329-1 341.
[39]	Harris D, Foufoula G E, Droegemeier K, et al. Multiscale statistical properties of a high-resolution precipitation forecast[J]. Journal of Hydrometeorology, 2001, 2: 406-418.
[40]	Roberts N M, Lean H W. Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events[J]. Monthly Weather Review, 2008, 136(1):78-97.
[41]	Wang N, Lu C. Two-dimensional continuous wavelet analysis and its application to meteorological data[J].Journal of Atmospheric and Oceanic Technology, 2010, 27(2):652-666.
[42]	Liu Y, Brown J, Demargne J, et al. A wavelet based approach to assessing timing errors in hydrologic predictions[J]. Journal of Hydrology, 2011, 397: 210-224.
[43]	Mittermaier M P. Using an intensity-scale technique to assess the added benefit of high-resolution model precipitation forecasts[J]. Atmospheric Science Letters, 2006, 7: 35-42.
[44]	Casati B, Wilson L J. A new spatial-scale decomposition of the brier score: Application to the verification of lightning probability[J]. Monthly Weather Review, 2007, 133(4):81-101.
[45]	Casati B. New developments of the intensity-scale technique within the spatial verification methods intercomparison project[J]. Weather and Forecasting, 2010, 138(8):121-128.
[46]	Theis S E, Hense A, Damrath U. Probabilistic precipitation forecasts from a deterministic model: A pragmatic approach[J]. Meteorological Applications, 2005, 12: 257-268.
[47]	Roberts N M, Lean H W. Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events[J]. Monthly Weather Review, 2007, 136(8):78-97.
[48]	Ebert E. Fuzzy verification of high-resolution girded forecasts: A review and proposed framework[J]. Meteorological Applications, 2010, 15: 51-64.
[49]	Mittermaier M, Roberts N. Intercomparison of spatial forecast verification methods: Identifying skillful spatial scales using the fractions skill score[J].Weather and Forecasting, 2010, 25(5):343-354.
[50]	Weusthoff T, Ament F, Aroagaus M, et al. Assessing the benefits of convection permitting models by neighborhood verification: Examples from MAP D-PHASE[J].Monthly Weather Review, 2010, 138(8):3 418-3 433.
[51]	Gilleland E, David A, Brown B G, et al. Verifying forecasts spatially[J]. American Meteological Society, 2010, 8:1 365-1 373.
[52]	Alexander G D, Weinman J A, Karyampudi V M, et al. The effect of assimilating rain ratesv derived from satellites and lightning on forecasts of the 1993 Superstorm[J].Monthly Weather Review, 1999, 127(6):1 433-1 457.
[53]	Nehrkorn T, Hoffman R, Grassotti C, et al. Feature calibration and alignment to represent model forecast errors: Empirical regularization[J]. Quarterly Journal of the Royal Meteorological Society, 2003, 129: 195-218.
[54]	Keil C, Craig G C.A displacement-based error measure applied in a regional ensemble forecasting system[J].Monthly Weather Review,2007, 135(9):3 248-3 259.
[55]	Keil C, Craig G C. A displacement and amplitude score employing an optical flow technique[J].Weather and Forecasting,2009, 24(6):1 297-1 308.
[56]	Marzban C, Sandgathe S, Lyons H, et al. Three spatial verification techniques: Cluster analysis, variogram, and optical flow[J].Weather and Forecasting, 2009, 24(11):1 457-1 471.
[57]	Marzban C, Sandgathe S. Optical flow for verification[J].Weather and Forecasting, 2010, 25(10):1 479-1 494.
[58]	Mesinger F. Bias adjusted precipitation threat scores[J]. Advances in Geosciences, 2008, 16: 137-142.
[59]	Lakshmanan V, Kain J. A gaussian mixture model approach to forecast verification[J].Weather and Forecasting, 2010, 25(6):908-920.
[60]	Venugopal V, Basu S, Georgiou E F. A new metric for comparing precipitation patterns with an application to ensemble forecasts[J]. Journal of Geophysical Research, 2005, 110: D08111,doi:10.1029/2004JD005395.
[61]	Gilleland E. Testing competing precipitation forecasts accurately and efficiently: The spatial prediction comparison test[J].Monthly Weather Review, 2013, 141(1):340-355.
[62]	Du Jun, Chen Jing. The corner stone in facilitating the transition from deterministic to probabilistic forecasts—Ensemble forecasting and its impact on numerical weather prediction[J].Meteorological Monthly, 2010, 36(11):1-11.
	[杜均,陈静.单一值预报向概率预报转变的基础——谈谈集合预报及其带来的变革[J].气象, 2010, 36(11):1-11.]
[63]	Murphy A H. A new vector partition of the probability score[J].Journal of Applied Meteorology, 1973, 12: 595-600.
[64]	Stephenson D B, Coelho C, Jolliffe I T. Two extra components in the Brier score decomposition[J].Weather and Forecasting, 2008, 23(4):708-716.
[65]	Ferro C. Comparing probabilistic forecasting systems with the Brier score[J].Weather and Forecasting, 2007, 22(3):1 076-1 089.
[66]	Ferro C, Richardson D S, Weigel A. On the effect of ensemble size on the discrete and continuous ranked probability scores[J]. Meteorological Applications, 2008, 15: 19-24.
[67]	Hamill T M, Colucci S J. Verification of Eta-RSM short range ensemble forecasts[J]. Monthly Weather Review, 1997, 125: 711-724.
[68]	Bröcker J, Smith L A. Increasing the reliability of reliability diagrams[J].Weather and Forecasting, 2007, 22(3):651-661.
[69]	Wilks D S. The calibration simplex: A generalization of the reliability diagram for three-category probability forecasts[J].Weather and Forecasting, 2013, 28(5):1 210-1 218.
[70]	Kharin K, Zwiers F W. Improved seasonal probability forecasts[J]. Journal of Climate, 2007, 16(11):1 684-1 701.
[71]	Murphy A H. What is a good forecast? An essay on the nature of goodness in weather forecasting[J].Weather and Forecasting, 1993, 8(4):281-293.
[72]	Wilson L J. Comments on “Probabilistic predictions of precipitation using the ECMWF ensemble prediction system”[J].Weather and Forecasting, 2000, 15(3):361-364.
[73]	Bowler N E, Pierce C E, Seed A W. STEPS: A probabilistic forecasting scheme which merges an extrapolation nowcast with downscaled NWP[J].Quarterly Journal of the Royal Meteorological Society, 2008, 132(6):2 127-2 155.
[74]	Du Jun, Deng Guo. The utility of the transition from deterministic to probabilistic weather forecasts—Verification and application of probabilistic forecasts[J]. Meteorological Monthly, 2010, 36(12):10-18.
	[杜均,邓国.单一值预报向概率预报转变的价值——谈谈概率预报的检验和应用[J].气象, 2010, 36(12):10-18.]
[75]	Wilson L J, Burrows W R, Lanzinger A. A strategy for verification of weather element forecasts from an ensemble prediction system[J]. Monthly Weather Review, 1999, 127(3):956-970.
[76]	Bröcker J, Smith L A. Scoring probabilistic forecasts: On the importance of being proper[J].Weather and Forecasting, 2007, 22(2):382-388.
[77]	Hersbach H. Decomposition of the continuous rank probability score for ensemble prediction systems[J].Weather and Forecasting, 2000, 15(6):559-570.
[78]	Hamill T M. Interpretation of rank histograms for verifying ensemble forecasts[J].Monthly Weather Review, 2001, 129(2):550-560.
[79]	Jolliffe I T, Cristina P. Evaluating rank histograms using decompositions of the chi-square test statistic[J].Monthly Weather Review, 2008, 136(6):2 133-2 139.
[80]	Dawid A P. Statistical theory: The prequential approach[J]. Journal Royal Statistical Society, 1984, 147:278-292.
[81]	Candille G, Talagrand O. Evaluation of probabilistic prediction systems for a scalar variable[J]. Quarterly Journal of the Royal Meteorological Society, 2005, 131: 2 131-2 150.
[82]	Wilks D S. The Minimum Spanning Tree (MST) histogram as a verification tool for multidimensional ensemble forecasts[J].Monthly Weather Review,2004, 132(7):1 329-1 340.
[83]	Gombos D, Hansen J A. Theory and applications of the minimum spanning tree rank histogram[J].Monthly Weather Review, 2007, 135(4):1 490-1 505.
[84]	Roulston M S, Smith L A. Evaluating probabilistic forecasts using information theory[J]. Monthly Weather Review, 2002, 130(6):1 653-1 660.
[85]	Gneiting T, Raftery A E. Strictly proper scoring rules prediction and estimation[J].Journal of the American Statistics Association, 2007, 102(3):359-378.

[1]	高丽,陈静,郑嘉雯,陈权亮. 极端天气的数值模式集合预报研究进展[J]. 地球科学进展, 2019, 34(7): 706-716.
[2]	朱月佳,邢蕊,朱明佳,王东勇,邱学兴. 联合概率方法在安徽强对流潜势预报中的应用和检验[J]. 地球科学进展, 2019, 34(7): 731-746.
[3]	杨秋明. 10~30 d延伸期天气预报方法研究进展与展望[J]. 地球科学进展, 2015, 30(9): 970-984.
[4]	熊春晖，张立凤，关吉平，陶恒锐，苏佳佳. 集合—变分数据同化方法的发展与应用[J]. 地球科学进展, 2013, 28(6): 648-656.
[5]	吴增茂;谢红琴;张志南;万小芳. 海洋生态预报的复杂性与研究方法的讨论[J]. 地球科学进展, 2004, 19(1): 81-086.

阅读次数

全文

摘要

Progress on Verification Methods of Numerical Weather Prediction