文献类型：会议论文

英文题名：Improved Vanishing Gradient Problem for Deep Multi-layer Neural Networks

作者：Wang, Di[1]; Liu, Xia[1]; Zhang, Jingqiu[2]

第一作者：Wang, Di

机构：[1] School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, 100876, China; [2] College of Applied Arts and Science, Beijing Union University, Beijing, 100191, China

第一机构：School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, 100876, China

会议论文集：Cognitive Systems and Information Processing - 7th International Conference, ICCSIP 2022, Revised Selected Papers

会议日期：December 17, 2022 - December 18, 2022

会议地点：Fuzhou, China

语种：英文

外文关键词：Backpropagation - Deep neural networks - Gradient methods - Intelligent robots - Network layers

摘要：Deep learning technologies have been broadly utilized in theoretical research and practical application of intelligent robots. Among numerous paradigms, BP neural network attracts wide attentions as an accurate and flexible tool. However, there always exists an unanswered question centering around gradient disappearance when using back-propagation strategy in multi-layer BP neural network. Moreover, the situation deteriorates sharply in the context of sigmoid transfer functions employed. To fill this research gap, this study explores a new solution that the relative magnitude of gradient descent is estimated, and neutralized via a new developed function with increasing properties. As a result, the undesired gradient disappearance problem is alleviated while reserving the traditional merits of the gradient descent method. The validity is verified by an actual case study of subway passenger flow, and the simulation results elucidate a superior convergence speed compared with the original algorithm. ? 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.