With the rapid development of China's sea observation technologies and data collection methods, we have an ever expanding amount of observed data with more and more complexity. Therefore, the traditional data storage and processing techniques cannot effectively manage and analyze the big ocean data. Meanwhile, the regionalization and the distribution of these data become an challenge for a unified data management, during the data mining process. From the perspective of the big data life cycle, this paper reviews the technical principles of the existing open source big data framework in areas of data acquisition, storage, processing, intelligent analysis, security protection and data governance, focusing on their applications for the big ocean data.
[1] 宋坤. 大数据理念在海洋环境观测数据共享中的应用研究[J]. 海洋开发管理, 2015(6):43-45. Song Kun. Application of the concept of big data in the marine environ-ment observation in data sharing[J]. Marine Development and Manage-ment, 2015(6):43-45.
[2] 徐文, 鄢社锋, 季飞, 等. 海洋信息获取、传输、处理及融合前沿研究评述[J]. 中国科学(信息科学), 2016, 46(8):1053. Xu Wen, Yan Shefeng, Ji Fei, et al. Review of frontier research on ma-rine information acquisition, transmission, processing and fusion[J]. Sci-entia Sinica Informations, 2016, 46(8):1053.
[3] 解鹏飞, 刘玉安, 赵辉, 等. 基于大数据的海洋环境监测数据集成与应用[J]. 海洋技术学报, 2016, 35(1):93-101.Xie Pengfei, Liu Yu'an, Zhao Hui, et al. Integration and application of marine environmental monitoring data based on large data[J]. Ocean Technology, 2016, 35(1):93-101.
[4] 黄冬梅, 邹国良. 海洋大数据[M]. 上海:上海科学技术出版社, 2016. Huang Dongmei, Zou Guoliang. Ocean big data[M]. Shanghai:Shanghai Scientific & Technical Publishers, 2016.
[5] 崔松雪, 刘艳艳, 陈戈. 数据仓库技术在海洋大气地理信息系统平台中的应用[J]. 中国海洋大学学报(自然科学版), 2009(增刊1):455-458. Cui Songxue, Liu Yanyan, Chen Ge. Application of data warehouse tech-nology in marine atmospheric geographic information system platform[J]. Periodical of Ocean University of China, 2009(Suppl 1):455-458.
[6] 王立峰. HBase数据库中大对象存储方案的研究[J]. 电脑知识与技术, 2014(23):5401-5402. Wang Lifeng. Research on storage scheme of large object in HBase da-tabase[J]. Computer Knowledge and Technology, 2014(23):5401-5402.
[7] Dimiduk N, Khurana A. HBase in action[M]. Grand Forks, ND:Man-ning Publications, 2012.
[8] 王博千, 于齐, 刘辛, 等. 面向Cassandra数据库的高效动态数据管理机制[J]. 计算机科学, 2016, 43(7):197-202. Wang Boqian, Yuqi, Liuxin, et al. Efficient dynamic data management mechanism for Cassandra database[J]. Computer Science, 2016, 43(7):197-202.
[9] 马豫星. Redis数据库特性分析[J]. 物联网技术, 2015(3):105-106. Ma Yuxing. Analysis of Redis database characteristics[J]. Internet of Things Technologies, 2015(3):105-106.
[10] Carlson J. Redis in action[M]. Grand Forks, ND:Manning Publica-tions, 2013.
[11] 王余蓝. 图形数据库NEO4J与关系据库的比较研究[J]. 现代电子技术, 2012, 35(20):77-79. Wang Yulan. A comparative study of graphic database NEO4J and re-lational database[J]. Modern Electronics Technique, 2012, 35(20):77-79.
[12] 刘驰, 符积高, 徐闻春. Spark原理、机制及应用[M]. 北京:机械工业出版社, 2016. Liu Chi, Fu Jigao, Xu Wenchun. Principle, mechanism and applica-tion of Spark[M]. Beijing:China Machine Press, 2016.
[13] 陈小龙, 关键, 何友. 微多普勒理论在海面目标检测中的应用及展望[J]. 雷达学报, 2013, 2(1):123-134. Chen Xiaolong, Guan Jian, He You. Application and prospect of mi-cro Doppler theory in sea surface target detection[J]. Journal of Ra-dars, 2013, 2(1):123-134.
[14] 黄文坚, 唐源. TensorFlow实战[M]. 北京:电子工业出版社, 2017. Huang Wenjian, Tang Yuan. TensorFlow practice[M]. Beijing:Publish-ing House of Electronics Industry, 2017.
[15] 王冬海, 卢峰, 方晓蓉, 等. 海洋大数据关键技术及在灾害天气下船舶行为预测上的应用[J]. 大数据, 2017, 3(4):81-90. Wang Dongmei, Lu Feng, Fang Xiaorong, et al. Key technology of ocean big data and its application in prediction of ship behavior un-der severe weather[J]. Big Data Research, 2017, 3(4):81-90.
[16] Chen X L, Guan J, Bao Z H, et al. Detection and extraction of target with micro-motion in spiky sea clutter via short-time fractional Fouri-er transform[J]. IEEE Transactions on Geoscience and Remote Sens-ing, 2014, 52(2):1002-1018.
[17] 陈小龙, 关键, 黄勇, 等. 雷达低可观测目标探测技术[J]. 科技导报, 2017, 35(11):30-38. Chen Xiaolong, Guan Jian, Huang Yong, et al. Radar low observable target detection technique[J]. Science & Technology Review, 2017, 35(11):30-38.
[18] 周志华, 王珏. 机器学习及其应用[M]. 北京:清华大学出版社, 2007. Zhou Zhihua, Wang Yu. Machine learning and its applications[M]. Bei-jing:Tsinghua University Press, 2007.
[19] 黄冬梅, 赵丹枫, 魏立斐, 等. 大数据背景下海洋数据管理的挑战与对策[J]. 计算机科学, 2016, 43(6):17-23. Huang Dongmei, Zhao Danfeng, Wei Lifei, et al. Challenge and coun-termeasure of ocean data management in big data background[J]. Com-puter Science, 2016, 43(6):17-23.
[20] 刘驰, 胡柏青, 谢一, 等. 大数据治理与安全:从理论到开源实践[M]. 北京:机械工业出版社, 2017. Liu Chi, Hu Baiqing, Xie Yi, et al. Big data governance and security:From theory to open source practice[M]. Beijing:China Machine Press, 2017.
