文献类型：期刊文献

英文题名：Learning spatiotemporal features with 3D DenseNet and attention for gesture recognition

作者：Liu, Honegzhe[1,2]; Deng, Zhifang[1,2,3]; Xu, Cheng[1,2]

第一作者：Liu, Honegzhe

通讯作者：Xu, Cheng

机构：[1] Beijing Key Laboratory of Information Service Engineering, Beijing Union University, Beijing, China; [2] College of Robotics, Beijing Union University, Beijing, China; [3] Institute of Network Technology, Beijing University of Posts and Telecommunications, Beijing, China

第一机构：北京联合大学北京市信息服务工程重点实验室

年份：2019

外文期刊名：International Journal of Electrical Engineering and Education

收录：EI(收录号：20200207986030);Scopus(收录号：2-s2.0-85077378935)

基金：Liu Honegzhe 1 2 https://orcid.org/0000-0002-1172-6590 Deng Zhifang 1 2 3 https://orcid.org/0000-0003-4913-5371 Xu Cheng 1 2 1 Beijing Key Laboratory of Information Service Engineering, Beijing Union University, Beijing, China 2 College of Robotics, Beijing Union University, Beijing, China 3 Institute of Network Technology, Beijing University of Posts and Telecommunications, Beijing, China Cheng Xu, Beijing Union University, Beijing 100101, China. Email: xucheng@buu.edu.cn 2019 0020720919894196 ? The Author(s) 2019 2019 SAGE Publications Gesture recognition aims at understanding dynamic gestures of the human body and is one of the most important ways of human–computer interaction; to extract more effective spatiotemporal features in gesture videos for more accurate gesture classification, a novel feature extractor network, spatiotemporal attention 3D DenseNet is proposed in this study. We extend DenseNet with 3D kernels and Refined Temporal Transition Layer based on Temporal Transition Layer, and we also explore attention mechanism in 3D ConvNets. We embed the Refined Temporal Transition Layer and attention mechanism in DenseNet3D, named the proposed network “spatiotemporal attention 3D DenseNet.” Our experiments show that our Refined Temporal Transition Layer performs better than Temporal Transition Layer and the proposed spatiotemporal attention 3D DenseNet in each modality outperforms the current state-of-the-art methods on the ChaLearn LAP Large-Scale Isolated gesture dataset. The code and pretrained model are released in https://github.com/dzf19927/STA3D . Spatiotemporal features gesture recognition DenseNet Refined Temporal Transition Layer attention mechanism the National Key Technology R&D Program 2015BAH55F03 Supporting Plan for Cultivating High Level Teachers in Colleges and Universities in Beijing IDHT20170511 Premium Funding Project for Academic Human Resources Development in Beijing Union University BPHR2019AZ01 the National Natural Science Foundation of China 61871039 edited-state corrected-proof typesetter ts2

语种：英文

外文关键词：Convolutional neural networks - Human computer interaction - Large dataset

摘要：Gesture recognition aims at understanding dynamic gestures of the human body and is one of the most important ways of human–computer interaction; to extract more effective spatiotemporal features in gesture videos for more accurate gesture classification, a novel feature extractor network, spatiotemporal attention 3D DenseNet is proposed in this study. We extend DenseNet with 3D kernels and Refined Temporal Transition Layer based on Temporal Transition Layer, and we also explore attention mechanism in 3D ConvNets. We embed the Refined Temporal Transition Layer and attention mechanism in DenseNet3D, named the proposed network "spatiotemporal attention 3D DenseNet." Our experiments show that our Refined Temporal Transition Layer performs better than Temporal Transition Layer and the proposed spatiotemporal attention 3D DenseNet in each modality outperforms the current state-of-the-art methods on the ChaLearn LAP Large-Scale Isolated gesture dataset. The code and pretrained model are released in https://github.com/dzf19927/STA3D. ? The Author(s) 2019.