文献类型：期刊文献

英文题名：Designing an Interactive Communication Assistance System for Hearing-Impaired College Students Based on Gesture Recognition and Representation

作者：Zhu, Yancong[1,2];Zhang, Juan[2,3];Zhang, Zhaoxi[4];Clepper, Gina[5];Jia, Jingpeng[6];Liu, Wei[2]

第一作者：Zhu, Yancong

通讯作者：Liu, W[1];Jia, JP[2]

机构：[1]Beihang Univ, Sch Biol Sci & Med Engn, Beijing 100191, Peoples R China;[2]Beijing Normal Univ, Fac Psychol, Beijing 100875, Peoples R China;[3]China Reform Hlth Management & Serv Grp Co Ltd, Human Resources Dept, Beijing 100028, Peoples R China;[4]Univ Buffalo, Sch Engn & Appl Sci, Buffalo, NY 14260 USA;[5]Purdue Univ, Coll Engn, W Lafayette, IN 47907 USA;[6]Beijing Union Univ, Coll Special Educ, Beijing 100075, Peoples R China

第一机构：Beihang Univ, Sch Biol Sci & Med Engn, Beijing 100191, Peoples R China

通讯机构：[1]corresponding author), Beijing Normal Univ, Fac Psychol, Beijing 100875, Peoples R China;[2]corresponding author), Beijing Union Univ, Coll Special Educ, Beijing 100075, Peoples R China.|[1141759]北京联合大学特殊教育学院;[11417]北京联合大学;

年份：2022

卷号：14

期号：7

外文期刊名：FUTURE INTERNET

收录：EI(收录号：20222812338914);Scopus(收录号：2-s2.0-85133622102);WOS:【ESCI(收录号:WOS:000832298200001)】；

基金：We would like to express our gratitude to the hearing-impaired students for their meaningful support of this work. We would like to thank the China–US Young Maker Competition organized by the Ministry of Education of People’s Republic of China.

语种：英文

外文关键词：hearing-impaired college student; multimedia-equipped classroom; qualitative analysis; gesture recognition and representation; tracking processing

摘要：Developing a smart classroom can make the modern classroom more efficient and intelligent. Much research has been conducted pertaining to smart classrooms for hearing-impaired college students. However, there have been few significant breakthroughs in mobilizing students' learning efficiency as measured by information transmission, communication, and interaction in class. This research collects data through nonparticipatory observation and in-depth interviews and analyzes available data on classroom interaction needs of these students. We found that diversified explanations, recordable interactive contents, and interaction between teachers and students could improve the learning effects in the classroom. We also propose a tracking-processing method based on gesture recognition and representation and present a design for a processing system based on AT89C52 microcontroller and Kinect. In this way, sign language can be translated into text and all students can receive the information and participate in the interaction, which greatly improves students' autonomy and enthusiasm of learning. This design enables deaf students to fully use classroom learning resources, reduces learning time costs, and improves learning efficiency. It can assist teachers in teaching and tutoring students to enhance their experience.