“土地利用/覆盖变化与综合减灾”专辑

基于BP神经网络的气象格点数据无损压缩方法

  • 赵苏璇 ,
  • 杨成荫 ,
  • 罗坚
展开
  • 中国人民解放军理工大学气象学院,江苏 南京 211101  
赵苏璇(1982-),女,江苏南通人,硕士研究生,主要从事热带天气学和气象信息处理研究.E-mail:zhaowenxiao1982@126.com

收稿日期: 2007-10-09

  修回日期: 2008-01-08

  网络出版日期: 2008-02-10

Scatheless Compression Encoding for Meteorological Grid Data Based on BP Neural Network

Expand
  • 中国人民解放军理工大学气象学院,江苏 南京 211101

Received date: 2007-10-09

  Revised date: 2008-01-08

  Online published: 2008-02-10

摘要

格点资料是目前气象数据存储、传输和应用的主要形式,爆炸性增长的资料给数据的存储、传输带来了巨大压力,设计适合格点资料特点的压缩方案越来越重要。大气是一种连续介质,格点资料是反映其动力学及热力学性质的连续物理量的采样和量化,相邻格点间存在很大的相关性。在二维线性预测基础上引入BP神经网络,建立了基于神经网络的二次预测模型,有效剔除气象格点数据的冗余信息,结合熵编码,提出了一种高效无损压缩新方案。该方案具有极高的压缩比,并能保证在有效精度内数据完全无损。与广泛应用的netCDF格式压缩效率对比试验表明,该方案的编码效率是netCDF格式的3~5倍,接近压缩的理论下限,能极大地减少存储空间及传输时间,适合大气以及地球科学中海量数据的存储和传输,有很好的实际应用前景。

本文引用格式

赵苏璇 , 杨成荫 , 罗坚 . 基于BP神经网络的气象格点数据无损压缩方法[J]. 地球科学进展, 2008 , 23(2) : 206 -213 . DOI: 10.11867/j.issn.1001-8166.2008.02.0206

Abstract

Grid data is the main form used in meteorological data storing, transmitting and applying. Along with wide application of global high resolution numerical models, data size is increasing explosively. It brings huge pressure to data storing, transmitting and managing. Designing a compression scheme which is suitable for grid data is becoming more and more important. Presently, most widely used grid data coding schemes are GRIB and netCDF, which are established by WMO. GRIB is specially used in coding meteorological grid data, and it is standard encoding for the form-driven format promoted by WMO. Although GRIB possesses a certain compression ability, less attention is paid to storing efficiency, and the compression ratio is unsatisfactory.
Atmosphere is a kind of continuous medium, the physical quantity describing its dynamic and thermodynamic properties are also continuous. Grid data are results of sampling and quantization of continuous variables, so the correlation between neighboring grid data is very high. In other words, there exists much information redundancy, and the data is compressible. 
In this paper, BP neural network is introduced based on the idea of 2-dimensions liner prediction. Using its great ability to simulate nonlinear information, a secondary predictive model for meteorology grid data is established to reduce information redundancy to the minimum. This model can eliminate most correlation. By combining entropy encoding, a new scatheless and highly efficient compression scheme is designed to deal with meteorological grid data.
By using the new scheme, the compression ratio for general meteorology grid data can be effectively promoted, and the compressing process of data is completely scatheless within available precision. A set of contrast experiment between netCDF and the new scheme proved that the compression efficiency of the new scheme is evidently superior to that of netCDF. Its compression ratio is 3 to 5 times as large as netCDF's and is close to the lower limit in theory. The storage space and transmission time all can be extremely reduced by using this scheme. The new compression encoding scheme may be widely used to treat with the vast data in meteorology and other earth sciences.

参考文献

[1] World Meteorological Organization. Guide to WMO Binary Code Forms: PART I: A Guide to the Code Form FM-94 Bufr [R].World Weather Watch Technical Report No.171994.

[2] World Meteorological Organization. Guide to WMO Binary Code Forms: PART II:A Guide to the Code Form FM 92-IX Ext. GRIB [R].World Weather Watch Technical Report No.171994.

[3] Rew R KDavis G P. NetCDF: An interface for scientific data access. Computer Graphics and Applications [J]. IEEE1990104: 76-82.

[4] Raymond D J. A C Language-Based modular system for analyzing and displaying gridded numerical data [J]. Journal of Atmospheric and Oceanic Technology19985: 501-511. 

[5] Wu GuoqingChen Hong. A lossless compression scheme for scientific data from simulation [J]. Computer Engineering and Applications20065: 172-175.[吴国清,陈虹. 一种科学数据无损压缩方法[J].计算机工程与应用,20065: 172-175.]

[6] Lou Jian. A new scatheless compression encoding scheme for meteorological grid data based on statistical model [J].  Advances in Earth Science2003184:637-642.[罗坚. 基于统计模型的气象数据无损压缩新方法[J] 地球科学进展,2003184:637-642.]

[7] Rong Guang'ao. Processing of Computer Images [M]. Beijing: Tsinghua University Press2000:26-28221-223.[容观澳. 计算机图像处理[M]. 北京:清华大学出版社,2000: 26-28221-223.]

 

文章导航

/