文献类型：期刊文献

英文题名：An improved hadoop data load balancing algorithm

作者：Liu, Kun[1,2]; Xu, Gaochao[1]; Yuan, Jun'e[2]

第一作者：Liu, Kun;刘琨

通讯作者：Xu, G.

机构：[1] College of Computer Science and Technology, Jilin University, Jilin Changchun, China; [2] College of Applied Science and Technology, Beijing Union University, Beijing, China

第一机构：College of Computer Science and Technology, Jilin University, Jilin Changchun, China

年份：2013

卷号：8

期号：12

起止页码：2816-2822

外文期刊名：Journal of Networks

收录：EI(收录号：20135117100742);Scopus(收录号：2-s2.0-84890109234)

语种：英文

外文关键词：Big data - Digital storage - File organization - Resource allocation

摘要：Data load balancing is one of the key problems of big data technology. As a big data application, Hadoop has had many successful applications. HDFS is Hadoop Distributed File System and has the load balancing procedure which can balance the storage load on each machine. However, this method cannot balance the overload rack preferentially, and so it is likely to cause the breakdown of overload machines. In this paper, we focus on the overload machines and propose an improved algorithm for balancing the overload racks preferentially. The improved method constructs Prior Balance List list which includes overload machines, For Balance List list and NextForBalanceList list by many factors and balances among the racks selected from these lists firstly. Experiments show that the improved method can balance the overload racks in time and reduce the possibility of breakdown of these racks. ? 2013 ACADEMY PUBLISHER.