Using Object Detection to Select Grasp Type & Control Functional Electrical Stimulation for Hand Rehabilitation

Unilateral muscle weakness and hand paralysis is the most common outcome after a stroke. Functional electrical stimulation (FES) is effective in assisting hand prehension, but conventional stimulators require users to manually select the grasp type (e.g. by pressing a button), which is challenging for patients with severe paralysis. Also, patients need to frequently divert attention from their task to operate the stimulator, which is cumbersome and reduces their engagement in the therapy. In this study, we develop a novel deep learning-based object detection approach to select multiple grasp types and control an electrical stimulator, in order to assist grasping. Object detection was performed using a state-of-the-art YOLOv5 algorithm, which achieved above 93% mean average precision. The algorithm tracked the positions of the hand and objects and selected a grasp type based on the object nearest to the hand. Once the grasp type was selected, a custom-built FES stimulator was activated to execute pre-defined stimulation sequences and allow a person to grasp the nearest object. This contactless, vision-based solution is beneficial for patients opting for homebased rehabilitation since it doesn’t require additional setup time or help from caregivers. The future scope of this work includes testing the object detection-based FES on stroke patients and determining its efficacy in restoring hand movements.