地球科学进展 ›› 2008, Vol. 23 ›› Issue (2): 206 -213. doi: 10.11867/j.issn.1001-8166.2008.02.0206

“土地利用/覆盖变化与综合减灾”专辑 上一篇    下一篇

基于BP神经网络的气象格点数据无损压缩方法
赵苏璇,罗 坚,杨成荫   
  1. 中国人民解放军理工大学气象学院,江苏 南京 211101  
  • 收稿日期:2007-10-09 修回日期:2008-01-08 出版日期:2008-02-10
  • 通讯作者: 赵苏璇 E-mail:zhaowenxiao1982@126.com

Scatheless Compression Encoding for Meteorological Grid Data Based on BP Neural Network

Zhao Suxuan Luo Jian Yang Chengyin   

  1. 中国人民解放军理工大学气象学院,江苏 南京 211101
  • Received:2007-10-09 Revised:2008-01-08 Online:2008-02-10 Published:2008-02-10

格点资料是目前气象数据存储、传输和应用的主要形式,爆炸性增长的资料给数据的存储、传输带来了巨大压力,设计适合格点资料特点的压缩方案越来越重要。大气是一种连续介质,格点资料是反映其动力学及热力学性质的连续物理量的采样和量化,相邻格点间存在很大的相关性。在二维线性预测基础上引入BP神经网络,建立了基于神经网络的二次预测模型,有效剔除气象格点数据的冗余信息,结合熵编码,提出了一种高效无损压缩新方案。该方案具有极高的压缩比,并能保证在有效精度内数据完全无损。与广泛应用的netCDF格式压缩效率对比试验表明,该方案的编码效率是netCDF格式的3~5倍,接近压缩的理论下限,能极大地减少存储空间及传输时间,适合大气以及地球科学中海量数据的存储和传输,有很好的实际应用前景。

Grid data is the main form used in meteorological data storing, transmitting and applying. Along with wide application of global high resolution numerical models, data size is increasing explosively. It brings huge pressure to data storing, transmitting and managing. Designing a compression scheme which is suitable for grid data is becoming more and more important. Presently, most widely used grid data coding schemes are GRIB and netCDF, which are established by WMO. GRIB is specially used in coding meteorological grid data, and it is standard encoding for the form-driven format promoted by WMO. Although GRIB possesses a certain compression ability, less attention is paid to storing efficiency, and the compression ratio is unsatisfactory.
Atmosphere is a kind of continuous medium, the physical quantity describing its dynamic and thermodynamic properties are also continuous. Grid data are results of sampling and quantization of continuous variables, so the correlation between neighboring grid data is very high. In other words, there exists much information redundancy, and the data is compressible. 
In this paper, BP neural network is introduced based on the idea of 2-dimensions liner prediction. Using its great ability to simulate nonlinear information, a secondary predictive model for meteorology grid data is established to reduce information redundancy to the minimum. This model can eliminate most correlation. By combining entropy encoding, a new scatheless and highly efficient compression scheme is designed to deal with meteorological grid data.
By using the new scheme, the compression ratio for general meteorology grid data can be effectively promoted, and the compressing process of data is completely scatheless within available precision. A set of contrast experiment between netCDF and the new scheme proved that the compression efficiency of the new scheme is evidently superior to that of netCDF. Its compression ratio is 3 to 5 times as large as netCDF's and is close to the lower limit in theory. The storage space and transmission time all can be extremely reduced by using this scheme. The new compression encoding scheme may be widely used to treat with the vast data in meteorology and other earth sciences.

中图分类号: 

[1] World Meteorological Organization. Guide to WMO Binary Code Forms: PART I: A Guide to the Code Form FM-94 Bufr [R].World Weather Watch Technical Report No.171994.

[2] World Meteorological Organization. Guide to WMO Binary Code Forms: PART II:A Guide to the Code Form FM 92-IX Ext. GRIB [R].World Weather Watch Technical Report No.171994.

[3] Rew R KDavis G P. NetCDF: An interface for scientific data access. Computer Graphics and Applications [J]. IEEE1990104: 76-82.

[4] Raymond D J. A C Language-Based modular system for analyzing and displaying gridded numerical data [J]. Journal of Atmospheric and Oceanic Technology19985: 501-511. 

[5] Wu GuoqingChen Hong. A lossless compression scheme for scientific data from simulation [J]. Computer Engineering and Applications20065: 172-175.[吴国清,陈虹. 一种科学数据无损压缩方法[J].计算机工程与应用,20065: 172-175.]

[6] Lou Jian. A new scatheless compression encoding scheme for meteorological grid data based on statistical model [J].  Advances in Earth Science2003184:637-642.[罗坚. 基于统计模型的气象数据无损压缩新方法[J] 地球科学进展,2003184:637-642.]

[7] Rong Guang'ao. Processing of Computer Images [M]. Beijing: Tsinghua University Press2000:26-28221-223.[容观澳. 计算机图像处理[M]. 北京:清华大学出版社,2000: 26-28221-223.]

 

[1] 摆玉龙, 路亚妮, 刘名得. 基于变分模态分解的机器学习模型择优风速预测系统[J]. 地球科学进展, 2021, 36(9): 937-949.
[2] 刘文聪, 张春菊, 汪陈, 张雪英, 朱月琴, 焦守涛, 鲁艳旭. 基于 BiLSTM-CRF的中文地质时间信息抽取[J]. 地球科学进展, 2021, 36(2): 211-220.
[3] 陈科贵, 李进, 黄长兵, 陈愿愿, 王刚, 刘阳. BP神经网络在富钾卤水中的应用研究[J]. 地球科学进展, 2018, 33(6): 614-622.
[4] 陈科贵, 陈旭, 张家浩. 复合渗透率测井评价方法在砂砾岩稠油油藏的应用 *——以克拉玛依油田某区八道湾组为例[J]. 地球科学进展, 2015, 30(7): 773-779.
[5] 罗坚,姜勇强,戴彩悌. 提升小波变换在气象格点数据无损压缩中的应用[J]. 地球科学进展, 2012, 27(4): 460-465.
[6] 李启权,王昌全,岳天祥,张文江,余勇. 基于神经网络模型的中国表层土壤有机质空间分布模拟方法[J]. 地球科学进展, 2012, 27(2): 175-184.
[7] 朱子先, 臧淑英. 基于遗传神经网络的克钦湖叶绿素反演研究[J]. 地球科学进展, 2012, 27(2): 202-208.
[8] 柴琳娜,屈永华,张立新,梁顺林,王锦地. 基于自回归神经网络的时间序列叶面积指数估算[J]. 地球科学进展, 2009, 24(7): 756-768.
[9] 王红丽,顾亚进,刘健,况雪源,提汝媛. BP神经网络在古气候序列重建中的应用[J]. 地球科学进展, 2009, 24(11): 1275-1279.
[10] 李天斌,肖学沛. 地下工程岩爆预测的综合集成方法[J]. 地球科学进展, 2008, 23(5): 533-540.
[11] 吴财芳;曾勇;秦勇. 神经网络分析方法在瓦斯预测中的应用[J]. 地球科学进展, 2004, 19(5): 860-866.
[12] 罗坚,黄峰,张韧,王继光. 基于统计模型的气象数据无损压缩新方法[J]. 地球科学进展, 2003, 18(4): 637-642.
[13] 哈斯巴干,马建文,李启青. ASTER数据的自组织神经网络分类研究[J]. 地球科学进展, 2003, 18(3): 345-350.
[14] 李双成,郑度. 人工神经网络模型在地学研究中的应用进展[J]. 地球科学进展, 2003, 18(1): 68-076.
阅读次数
全文


摘要