文献类型：期刊文献

英文题名：GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark

作者：Huang, Zhou[1,2,3];Chen, Yiran[1,2];Wan, Lin[4];Peng, Xia[5,6]

第一作者：Huang, Zhou

通讯作者：Peng, X[1];Peng, X[2]

机构：[1]Peking Univ, Inst Remote Sensing, Beijing 100871, Peoples R China;[2]Peking Univ, GIS, Beijing 100871, Peoples R China;[3]Beijing Adv Innovat Ctr Future Internet Technol, Beijing 100124, Peoples R China;[4]China Univ Geosci, Fac Informat Engn, Wuhan 430074, Peoples R China;[5]Beijing Union Univ, Inst Tourism, Collaborat Innovat Ctr eTourism, Beijing 100101, Peoples R China;[6]Chinese Acad Sci, Inst Geog Sci & Nat Resources Res, State Key Lab Resources & Environm Informat Syst, Beijing 100101, Peoples R China

第一机构：Peking Univ, Inst Remote Sensing, Beijing 100871, Peoples R China

通讯机构：[1]corresponding author), Beijing Union Univ, Inst Tourism, Collaborat Innovat Ctr eTourism, Beijing 100101, Peoples R China;[2]corresponding author), Chinese Acad Sci, Inst Geog Sci & Nat Resources Res, State Key Lab Resources & Environm Informat Syst, Beijing 100101, Peoples R China.|[1141732]北京联合大学旅游学院;[11417]北京联合大学;

年份：2017

卷号：6

期号：9

外文期刊名：ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION

收录：;Scopus(收录号：2-s2.0-85029311142);WOS:【SCI-EXPANDED(收录号:WOS:000416386100026)】；

基金：This research was supported by grants from the National Key Research and Development Program of China (2017YFB0503602), the National Natural Science Foundation of China (41401449, 41501162, 41771425), the Scientific Research Key Program of Beijing Municipal Commission of Education (KM201611417004), the Beijing Philosophy and Social Science Foundation, the Talent Optimization Program of Beijing Union University and State Key Laboratory of Resources and Environmental Information System.

语种：英文

外文关键词：big data; GeoSpark SQL; Spark; spatial query processing; spatial database

摘要：In the era of big data, Internet-based geospatial information services such as various LBS apps are deployed everywhere, followed by an increasing number of queries against the massive spatial data. As a result, the traditional relational spatial database (e.g., PostgreSQL with PostGIS and Oracle Spatial) cannot adapt well to the needs of large-scale spatial query processing. Spark is an emerging outstanding distributed computing framework in the Hadoop ecosystem. This paper aims to address the increasingly large-scale spatial query-processing requirement in the era of big data, and proposes an effective framework GeoSpark SQL, which enables spatial queries on Spark. On the one hand, GeoSpark SQL provides a convenient SQL interface; on the other hand, GeoSpark SQL achieves both efficient storage management and high-performance parallel computing through integrating Hive and Spark. In this study, the following key issues are discussed and addressed: (1) storage management methods under the GeoSpark SQL framework, (2) the spatial operator implementation approach in the Spark environment, and (3) spatial query optimization methods under Spark. Experimental evaluation is also performed and the results show that GeoSpark SQL is able to achieve real-time query processing. It should be noted that Spark is not a panacea. It is observed that the traditional spatial database PostGIS/PostgreSQL performs better than GeoSpark SQL in some query scenarios, especially for the spatial queries with high selectivity, such as the point query and the window query. In general, GeoSpark SQL performs better when dealing with compute-intensive spatial queries such as the kNN query and the spatial join query.