Google has announced an exciting new project called Project Gameface, aimed at improving accessibility in PC gaming. The new tool uses facial gestures and head movements to replace traditional physical controllers. This will greatly help folks who have trouble playing their favourite games using a mouse or gamepad.
As PC gaming continues to expand worldwide, many users find that traditional gaming peripherals do not meet their needs. For instance, some people have difficulty naturally using a keyboard and mouse. To address this issue, Google has developed a new tool that relies solely on facial movements.
Project Gameface uses standard webcams without facial recognition hardware to detect facial gestures and head movements. The tool then translates these movements into cursor movements on the user’s computer. This new tool was co-created by Lance Carr, who has muscular dystrophy that makes fundamental muscle tasks impossible. Carr had to use specialized gear to play games that fit his condition.
To develop this technology, Google created a tool to detect 468 points on a user’s face using the company’s open-source machine learning library, MediaPipe. These dynamic points allow the software to detect and distinguish between basic facial gestures and dramatic movements. The user can assign different actions to each facial expression using this program. For example, assigning a smiling face to a left mouse click and raising the left eyebrow to a right mouse click is possible. The tool can be tuned according to the user’s preferences, so they can set their comfort zone.
Project Gameface is still in the works and can potentially serve many people in the future. The project is now open source and can be downloaded from GitHub. Hopefully, this tool will be realized soon, and other projects derived from it will be born. This new tool is a great step toward making PC gaming more inclusive for all users.
