文献类型：期刊文献

中文题名：面向国家高性能计算环境的虚拟数据空间系统

英文题名：Virtual data space system for national highperformance computing environment

作者：秦广军[1];肖利民[2,3];张广艳[4];牛北方[5,6];陈志广[7]

第一作者：秦广军

机构：[1]北京联合大学智慧城市学院,北京100101;[2]北京航空航天大学计算机学院,北京100191;[3]软件开发环境国家重点实验室,北京100191;[4]清华大学计算机科学与技术系,北京100084;[5]中国科学院计算机网络信息中心,北京100190;[6]中国科学院大学,北京100190;[7]中山大学计算机学院,广东广州510006

第一机构：北京联合大学智慧城市学院

年份：2021

卷号：7

期号：2

起止页码：101-122

中文期刊名：大数据

外文期刊名：Big Data Research

收录：CSTPCD;;国家哲学社会科学学术期刊数据库

基金：国家重点研发计划资助项目(No.2018YFB0203901)。

语种：中文

中文关键词：高性能计算环境;大型计算问题;虚拟数据空间;广域分布式存储;统一命名空间

外文关键词：high-performance computing environment;large-scale computing problem;virtual data space;wide-area distributed storage;unified namespace

摘要：高性能计算环境是支撑国家科技创新、经济发展、国防建设的核心信息基础设施,世界高性能计算强国纷纷建设基于多超算中心资源的广域高性能计算环境。然而,高性能计算环境中资源种类繁多且地域分布广,无法有效发挥资源的聚合效应,难以满足大型应用对广域分布数据的统一管理和高效访问需求。为此,提出了一套可用于构建广域全局虚拟数据空间的完整技术体系,包括虚拟数据空间模型、跨域虚拟数据空间构建、广域环境中数据高效迁移、广域环境中存算协同调度、跨域高并发数据聚合处理等技术,并研发了一个可运行于国家高性能计算环境的虚拟数据空间系统,可有效支撑广域分散异构存储资源的统一高效访问,实现广域环境中分布数据的跨域共享和协同处理。目前,该软件系统已在国家高性能计算环境实验性部署,并验证了分子对接、全基因组关联分析、天气预报模式3类典型大型应用。验证结果表明,所研虚拟数据空间构建方法和系统可有效聚合广域分散的存储资源,满足大型应用的数据空间需求。
High-performance computing(HPC)environment is the core information infrastructure supporting national scientific and technological innovation,economic development and national defense construction.High-performance computing powers around the world have been building wide-area HPC environments based on multi-supercomputing center resources.However,in the high-performance computing environment,there are many kinds of resources and wide geographical distribution,which cannot effectively exert the aggregation effect of resources,and it is difficult to meet the requirements of large-scale applications for unified management and efficient access to wide-area distributed data.To this end,a complete set of technologies were proposed,which could be used to build wide-area global virtual data space,including virtual data space model,cross-domain virtual data space constructing,efficiently migrating data in a wide-area environment,co-scheduling of storage resources and computing job and cross-domain high concurrency data aggregation processing,etc.Based on the above,a virtual data space system has been developed for the national high-performance computing environment(NHPCE),which can effectively support the unified and efficient access to the wide area distributed heterogeneous storage resources,and the distributed data in the wide-area environment can be shared and cooperative processed in a cross-domain manner.At present,the system was experimental deployed in NHPCE and three typical large-scale applications,such as molecular docking,genome-wide association study and weather forecasting model,have been verified.The verification results show that the developed technology and software system can effectively aggregate the wide area distributed storage resources and meet the data space requirements of large-scale applications.