A user evaluation of speech/phrase recognition software in critically ill patients: a DECIDE-AI feasibility study

Abstract

Objectives: Evaluating effectiveness of speech/phrase recognition software in critically ill patients with speech impairments. Design: Prospective study. Setting: Tertiary hospital critical care unit in the northwest of England. Participants: 14 patients with tracheostomies, 3 female and 11 male. Main outcome measures: Evaluation of dynamic time warping (DTW) and deep neural networks (DNN) methods in a speech/phrase recognition application. Using speech/phrase recognition app for voice impaired (SRAVI), patients attempted mouthing various supported phrases with recordings evaluated by both DNN and DTW processing methods. Then, a trio of potential recognition phrases was displayed on the screen, ranked from first to third in order of likelihood. Results: A total of 616 patient recordings were taken with 516 phrase identifiable recordings. The overall results revealed a total recognition accuracy across all three ranks of 86% using the DNN method. The rank 1 recognition accuracy of the DNN method was 75%. The DTW method had a total recognition accuracy of 74%, with a rank 1 accuracy of 48%. Conclusion: This feasibility evaluation of a novel speech/phrase recognition app using SRAVI demonstrated a good correlation between spoken phrases and app recognition. This suggests that speech/phrase recognition technology could be a therapeutic option to bridge the gap in communication in critically ill patients. What is already known about this topic: Communication can be attempted using visual charts, eye gaze boards, alphabet boards, speech/phrase reading, gestures and speaking valves in critically ill patients with speech impairments. What this study adds: Deep neural networks and dynamic time warping methods can be used to analyse lip movements and identify intended phrases. How this study might affect research, practice and policy: Our study shows that speech/phrase recognition software has a role to play in bridging the communication gap in speech impairment.