文献类型：期刊文献

英文题名：Spatial-temporal hypergraph based on dual-stage attention network for multi-view data lightweight action recognition

作者：Zhixuan, Wu[1];Nan, Ma[2,3];Cheng, Wang[1];Cheng, Xu[1];Genbao, Xu[2,3];Mingxing, Li[1]

第一作者：Zhixuan, Wu

通讯作者：Nan, M[1]

机构：[1]Beijing Union Univ, Beijing Key Lab Informat Serv Engn, Beijing 100101, Peoples R China;[2]Beijing Univ Technol, Fac Informat & Technol, Beijing 100124, Peoples R China;[3]Beijing Univ Technol, Engn Res Ctr Intelligence Percept & Autonomous Con, Minist Educ, Beijing 100124, Peoples R China

第一机构：北京联合大学北京市信息服务工程重点实验室

通讯机构：[1]corresponding author), Beijing Univ Technol, Fac Informat & Technol, Beijing 100124, Peoples R China.

年份：2024

卷号：151

外文期刊名：PATTERN RECOGNITION

收录：;EI(收录号：20241315815993);WOS:【SCI-EXPANDED(收录号:WOS:001222662000001)】；

基金：This work is supported by the Beijing Natural Science Foundation (No. 4222025) , the National Natural Science Foundation of China (No. 61931012) , the Beijing Municipal Science and Technology (No. Z221100000222016) , and QIYUAN LAB Innovation Foundation (Inno-vation Research) Project (No. S20210201107) .

语种：英文

外文关键词：Dual-stage attention network; Salient region; Spatial-temporal hypergraph neural network; Multi-view; Action recognition

摘要：For the problems of irrelevant frames and high model complexity in action recognition, we propose a Spatial- Temporal Hypergraph based on Dual -Stage Attention Network (STHG-DAN) for multi-view data lightweight action recognition. It includes two stages: Temporal Attention Mechanism based on Trainable Threshold (TAMTT) and Hypergraph Convolution based on Dynamic Spatial-Temporal Attention Mechanism (HG-DSTAM). In the first stage, TAM-TT uses a learning threshold to extract keyframes from multi-view videos, with the multiview data serving as a guarantee for providing more comprehensive information subsequently; In the second stage, HG-DSTAM divides the human joints into three parts: trunk, hand and leg to build spatial-temporal hypergraphs, extracts high -order features from spatial-temporal hypergraphs constructed of multi-view human body joints, inputs them into the dynamic spatial-temporal attention mechanism, and learns the intra frame correlation of multi-view data between the joint features of body parts, which can obtain the significant areas of action; We use multi-scale convolution operation and depth separable network, which can realize efficient action recognition with a few trainable parameters. We experiment on the NTU-RGB+D, NTU-RGB+D 120 and the imitating traffic police gesture dataset. The performance and accuracy of the model are better than the existing algorithms, effectively improving the machine and human body language interaction cognitive ability.