收稿日期: 2002-07-01
修回日期: 2003-02-08
网络出版日期: 2003-08-01
A NEW SCATHELESS COMPRESSION ENCODING SCHEME FOR METEOROLOGICAL GRID DATA BASED ON STATISTICAL MODEL
Received date: 2002-07-01
Revised date: 2003-02-08
Online published: 2003-08-01
对当前广泛使用的气象格点数据结构进行了统计分析,通过分析常用气象要素格点资料相邻格点之间的相关性及计算要素场的符号熵和信息冗余度,认为气象格点数据中存在着明显的信息冗余,具有很高的可压缩性,且相关性越好,可压缩性越高。在此分析基础上建立了气象格点数据的二维线性预测统计模型,剔除冗余信息,并结合Huf fman编码,提出了一种气象格点数据无损压缩新方法。该方法可极大提高气象格点数据的压缩率,且能保证在有效精度内数据完全无损。最后对常用气象格点资料进行了压缩对比试验,结果表明,该方案压缩效果明显优于当前国际通用的气象数据压缩编码格式(如GRIB和netCDF码),从而能够大大提高气象以及地球科学中海量数据采集、存储和传输交换的业务应用效率。
关键词: 气象数据; 格点资料; 无损压缩; 预测编码; Huf fman编码
张韧 , 罗坚 , 黄峰 , 王继光 . 基于统计模型的气象数据无损压缩新方法[J]. 地球科学进展, 2003 , 18(4) : 637 -642 . DOI: 10.11867/j.issn.1001-8166.2003.04.0637
In this paper, the general statistical structure and characteristics of meteorology grid data were analyzed. By analyzing the correlation between neighbor grid data, calculating its relevant symbol entropy and information redundancy, it was found that there is more information redundancy in meteorology grid data, so the grid data set can be better potentially compressed. The higher the correlation is, the more the condensability is. Based on above analyses, a 2-dimensions liner predictive statistical model for meteorology grid data was established to reduce information redundancy, and by combining of Huffman encoding which is efficient information source encoding scheme, a new scatheless compression scheme was designed to deal with meteorological grid data. By using the new scheme, the compression ratio for general meteorology grid data can be effectively promoted, and the compressing and uncompressing process of data is complete scatheless within available precision. Finally, a set of contrast experiment based on the usual meteorology grid data were presented and carried out. It is proved that the compression efficiency of the new scheme is evidently superior to that of GRIB and netCDF. The new compression encoding scheme may be widely used to treat with the vast data in meteorology and other earth sciences to improve the efficiency of data collecting, storing, translating and exchanging.
[1] World Meteorological Organization. Guide to WMO Binary Code Forms, Part I: A Guide to the Code form FM-94 Bufr[R].World Weather Watch Technical Report No.17,1994.
[2] World Meteorological Organization. Guide to WMO Binary Code Forms, Part II: A Guide to the Code form FM 92-IX Ext Grib[R]. World Weather Watch Technical Report No.17,1994.
[3] Rew R K, Davis G P. NetCDF: An Interface for Scientific Data Access, Computer Graphics and Applications[R]. IEEE, 1990.76-82.
[4] Raymond D J. A C language-based modular system for analyzing and displaying gridded numerical data[J]. Journal of Atmospheric and Oceanic Technology,1998,5: 501-511.
[5] Hong Xiaoda. Multimedia Computer and Technology of Data Compress[M]. Beijing: China International Broadcast Press, 1999.102-104 [洪小达.多媒体计算机与数据压缩技术[M].北京:中国国际广播出版社,1999.102-104.]
[6] Rong Guan’ao. Processing of Computer Images [M]. Beijing: Tsinghua University Press,2000.26-28,221-223.[容观澳.计算机图像处理[M].北京:清华大学出版社,2000. 26-28,221-223.]
/
〈 |
|
〉 |