Facial Blendshapes for Custom Action: An Experimental Evaluation Using Facial Expressions for Human-Computer Interaction

Abstract

This study presents a novel human-computer interaction approach that uses facial blendshapes and facial expressions to define custom actions. By using predefined facial expressions and their combinations, users can trigger custom actions in a simulated text editor. This approach can provide an accessible interaction method for individuals with upper limb disabilities and potentially offer productivity benefits for regular users. The suggested approach was implemented and evaluated by conducting an experiment involving six participants. The study compared task completion time using traditional mouse input and a combination of mouse and facial expressions. Preliminary results showed that while using facial expressions requires more time to master, participants valued the hands-free interaction potential of this approach.