Grid data is the main form used in meteorological data storing, transmitting and applying. Along with wide application of global high resolution numerical models, data size is increasing explosively. It brings huge pressure to data storing, transmitting and managing. Designing a compression scheme which is suitable for grid data is becoming more and more important. Presently, most widely used grid data coding schemes are GRIB and netCDF, which are established by WMO. GRIB is specially used in coding meteorological grid data, and it is standard encoding for the form-driven format promoted by WMO. Although GRIB possesses a certain compression ability, less attention is paid to storing efficiency, and the compression ratio is unsatisfactory.
Atmosphere is a kind of continuous medium, the physical quantity describing its dynamic and thermodynamic properties are also continuous. Grid data are results of sampling and quantization of continuous variables, so the correlation between neighboring grid data is very high. In other words, there exists much information redundancy, and the data is compressible.
In this paper, BP neural network is introduced based on the idea of 2-dimensions liner prediction. Using its great ability to simulate nonlinear information, a secondary predictive model for meteorology grid data is established to reduce information redundancy to the minimum. This model can eliminate most correlation. By combining entropy encoding, a new scatheless and highly efficient compression scheme is designed to deal with meteorological grid data.
By using the new scheme, the compression ratio for general meteorology grid data can be effectively promoted, and the compressing process of data is completely scatheless within available precision. A set of contrast experiment between netCDF and the new scheme proved that the compression efficiency of the new scheme is evidently superior to that of netCDF. Its compression ratio is 3 to 5 times as large as netCDF's and is close to the lower limit in theory. The storage space and transmission time all can be extremely reduced by using this scheme. The new compression encoding scheme may be widely used to treat with the vast data in meteorology and other earth sciences.