大数据环境下卫星对地观测数据集成系统的关键技术

建立卫星对地观测数据集成系统是遥感卫星数据信息资源有效管理与应用的重要手段。从我国对地观测重大需求以及前沿科学问题入手,提出大数据环境下卫星对地观测数据集成系统建立中亟待解决的关键技术,包括大容量异构对地观测数据集成的语义技术、基于网格的遥感图像快速处理技术、遥感大数据深度分析技术、多数据中心协同处理及云平台技术,为实现集成卫星图像、地面观测数据和模拟模型的元数据管理、几何精度纠正和卫星数据质量评价、海量卫星图像数据的空间分析与知识发现、分布式高性能卫星图像数据管理和归档等基本功能,为解决海量卫星数据分布式存储与计算、数据集成与互操作、空间数据分析与地学知识发现提供新思路、新技术与新方法。

关键词: 卫星观测数据集成系统, 语义技术, 网格计算, 大数据深度分析, 云平台

The establishment of satellite earth observation system is an important means for effective management and application of satellite information resources. From significant demands for earth observation in China as well as cutting-edge scientific issues, we propose some key technologies of developing earth observation satellite data integration system under big data environment, including semantic integration of large heterogeneous earth observation data, fast data processing of satellite remote sensing imagery based on grid, in-depth analysis and knowledge discovery of big satellite data, and collaboration processing of multiple data centers and cloud-platform.It is hoped to provide with new technologies and methods for satellite big data management, analysis and archiving.

Key words: Satellite observation data integration system, Semantic technologies, Grid computing, Big data deep analysis, Cloud platform

中图分类号:

P208

图1 卫星观测数据集成系统及其大数据技术应用的总体技术框架

Fig.1 Overall technical framework of satellite data integration system based on big data technologies

图2 基于语义技术的大容量异构对地观测数据集成

Fig.2 Massive heterogeneous Earth observation data integration based on semantic technologies

[1]	Wang Yi.The development of the Earth observation system[J]. Advances in Earth Science, 2005, 20(9): 980-989.
	[王毅. 新一代对地观测系统的发展[J].地球科学进展,2005,20(9):980-989.]
[2]	Li Deren, Tong Qingxi, Li Rongxing, et al.Current issues in high-resolution Earth observation technology[J]. Science in China (Series D), 2012, 42(6): 805-813.
	[李德仁,童庆禧,李荣兴,等.高分辨率对地观测的若干前沿科学问题[J].中国科学:D辑,2012,42(6):805-813.]
[3]	Max Craglia, Kees de Bie, Martino Pesaresi, et al. Digital Earth 2020: Towards the vision for the next decade[J]. International Journal Digital Earth, 2012, 5: 4-21.
[4]	Li Deren.Opportunities for geomatics[J]. Geomatics and Information Science of Wuhan University, 2004, 29(9): 753-756.
	[李德仁. 地球空间信息学的机遇[J].武汉大学学报:信息科学版,2004,29(9):753-756.]
[5]	Li Deren, Shen Xin.Intelligent Earth observing system[J]. Science of Surveying and Mapping, 2005, 30(4): 9-11.
	[李德仁,沈欣.论智能化对地观测系统[J].测绘科学,2005,30(4):9-11.]
[6]	Rong Xie, Ryosuke Shibasaki.Creating a satellite-data integration system[J]. GIM International, 2006, 20(12): 21-23.
[7]	Gong Jianya.Advances in Earth Observation Data Processing and Analysis[M]. Wuhan: Wuhan University Press, 2007.
	[龚健雅. 对地观测数据处理与分析研究进展[M].武汉: 武汉大学出版社,2007.]
[8]	James Manyika, Michael Chui, Brad Brown, et al.Big Data: The Next Frontier for Innovation, Competition, and Productivity[R]. McKinsey Global Institute,2011.
[9]	Daniel Sui, Sarah Elwood, Michael Goodchild.Introduction: Volunteered geographic information, the exaflood, and the growing digital divide[M]∥Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI) in Theory and Practice. New York: Springer, 2013: 1-14.
[10]	Big Data Expert Committee of China Computer Federation. White Paper of Big Data Technologies and Industry Development in China[Z]. Beijing: China Computer Federation, 2013.
	[中国计算机学会大数据专家委员会.中国大数据技术与产业发展白皮书[Z].北京:中国计算机学会,2013.]
[11]	Yu Xing. The bigdata tool for geochemical study of seabed rocks—PetDB and its application in geoscience[J]. Advances in Earth Science, 2014, 29(2): 306-314.
	[余星. 海底岩石地球化学研究中的“大数据”——PetDB及其应用[J].地球科学进展,2014,29(2):306-314.]
[12]	Li Guoqing, Wu Yanhui.Scientific research of Earth observation in era of big data[J]. China Computer Society Newsletter, 2013, 9(9): 27-31.
	[李国庆,邬延辉.大数据时代的对地观测科学研究[J].中国计算机学会通讯,2013,9(9):27-31.]
[13]	Zhou Xiaofang, Lu Jiaheng, Li Cuiping, et al.Challenges of big data from the perspective of data management[J]. China Computer Society Newsletter, 2012, 8(9): 16-20.
	[周晓方,陆嘉恒,李翠平,等.从数据管理视角看大数据挑战[J].中国计算机学会通讯,2012,8(9):16-20.]
[14]	Qin Xiongpai, Wang Huiju, Du Xiaoyong, et al.Big data analysis—Competition and symbiosis of RDBMS and MapReduce[J]. Journal of Software, 2012, 23(1): 32-45.
	[覃雄派,王会举,杜小勇,等.大数据分析——RDBMS 与MapReduce 的竞争与共生[J]. 软件学报,2012,23(1):32-45.]
[15]	Yang Bingxin.Brief introduction to Xiangshan science conferences of nos[J]. China Basic Science, 2012, 14(4): 22-29.
	[杨炳忻. 香山科学会议第420-424 次学术讨论会简述[J].中国基础科学,2012,14(4):22-29.]
[16]	Gong Xueqing, Jin Cheqing, Wang Xiaoling,et al.Data-intensive science and engineering: Requirements and challenges[J]. Chinese Journal of Computers, 2012, 35(8): 1 563-1 578.
	[宫学庆,金澈清,王晓玲,等.数据密集型科学与工程:需求和挑战[J].计算机学报,2012,35(8):1 563-1 578.]
[17]	Wang Shan, Wang Huiju, Qin Xiongpai, et al.Architecting big data: Challenges, studies and forecasts[J]. Chinese Journal of Computers, 2011, 34(10): 1 741-1 752.
	[王珊,王会举,覃雄派,等.架构大数据:挑战、现状与展望[J].计算机学报,2011,34(10):1 741-1 752.]
[18]	Ma Shuai, Li Jianxin, Hu Chunming.Challenging and thinking of big data science and engineering[J]. China Computer Society Newsletter, 2012, 8(9): 22-30.
	[马帅,李建欣,胡春明.大数据科学与工程的挑战与思考[J].中国计算机学会通讯,2012,8(9):22-30.]
[19]	Li Guojie.The scientific value of big data research[J]. China Computer Society Newsletter, 2012, 8(9): 8-15.
	[李国杰. 大数据研究的科学价值[J].中国计算机学会通讯,2012,8(9):8-15.]
[20]	International Organization for Standardization. ISO/TC 211-Geographic information/Geomatics[S/OL].(2005-07-28)[2015-01-24]. URL
[21]	OGC. The OpenGIS Abstract Specification[S/OL].(2005-07-28)[2015-01-24]. URL
[22]	Gu Xingfa.Development Status and Prospects of China Earth Observation System[R]. Wuhan: Wuhan University, 2010.
	[顾行发. 中国空间对地观测体系发展现状与前瞻[R].武汉:武汉大学,2010.]
[23]	State Administration of Science, Technology and Industry for National Defense[EB/OL]. China Major Projects of High-Resolution Earth Observation System, 2007.[2015-04-06]. .[国家国防科技工业局[EB/OL]. 中国高分辨率对地观测系统重大专项网, 2007.[2015-04-06]. .] URL
[24]	Gu Xingfa.Thinking of Industry Development of China’s Satellite Applications[R]. Wuhan: Wuhan University, 2010.
	[顾行发. 我国卫星应用产业发展思考[R].武汉:武汉大学,2010.]
[25]	Dai Weimin.Techniques and Methods of Semantic Web Information organization[M]. Shanghai: Academia Press, 2008.
	[戴维民. 语义网信息组织技术与方法[M].上海:学林出版社,2008.]
[26]	Rong Xie, Ryosuke Shibasaki.Imagery metadata development based on ISO/TC 211 standards[J]. CODATA Data Science Journal, 2007, 6(3): 28-45.
[27]	Hua Yu, Jiang Hong, Zhu Yifeng, et al.SmartStore: A new metadata organization paradigm with semantic-awareness for next-generation file systems[C]∥Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2009: 1-12.
[28]	Zeng Shaobin, Xie Chuanjie, Li Jiaqi, et al.Parallel fusion for remote sensing images based on grid services[J]. Journal of Geo-Information Science,2010, 12(2): 269-274.
	[曾少斌,谢传节,李佳琪,等.基于网格服务的遥感图像并行融合[J].地球信息科学学报,2010,12(2):269-274.]
[29]	Li Shengyang, Zhang Aijun, Zhu Chongguang, et al.Fast processing of grid-based remote sensing images[J]. Computer Engineering, 2007, 33(6): 35-37.
	[李盛阳,张爱军,朱重光,等.基于网格的遥感图像快速处理[J].计算机工程,2007,33(6):35-37.]
[30]	Dhruba Borthakur.The Hadoop Distributed File System: Architecture and Design[R]. The Apache Software Foundation, 2007.
[31]	Jeffrey Dean, Sanjay Ghemawat.MapReduce: Simplified data processing on large clusters[J].Communications of the ACM, 2008, 51(1): 107-113.
[32]	Kyuseok Shim.MapReduce Algorithms for Big Data Analysis, Proceedings of 8th International Workshop[C]. Berlin: Springer-Verlag Heidelberg, 2013: 44-48.
[33]	Thomas Erl, Ricardo Puttini, Zaigham Mahmood.Cloud computing: Concepts, Technology & Architecture[M]. New York: ServiceTech Press, 2013.
[34]	Navid Golpayegani, Milton Halem.Cloud computing for satellite data processing on high end compute clusters[C]//IEEE International Conference on Cloud Computing. Bangalore, India, 2009: 88-92.
[35]	Lu Junjie, Hou Weizhen.Construction of e-government cloud platform oriented information resources integration[J]. Library Studies, 2012, 13: 36-40.
	[鲁俊杰,侯卫真.面向信息资源整合的电子政务云平台构建研究[J].图书馆学研究,2012,13:36-40.]

[1]	谢榕, 刘亚文, 李翔翔. 大数据环境下卫星对地观测数据集成系统的关键技术[J]. 地球科学进展, 2015, 30(8): 855-862.

阅读次数

全文

摘要

Key Technologies of Earth Observation Satellite Data Integration System under Big Data Environment