Palm+Act: Operation by Visually Captured 3D Force on Palm

The goal of this research is to detect multi-dimensional touch vectors on the body, especially the palm with fingers. Compared to the traditional on-body interaction, this research enriches the available information. For example, 3D touching force and torque are some of them. Although simple sensing of these multi-dimensional touch vectors is difficult, our technical innovation helps to detect them. The proposed technology enables us to estimate force by tracking 2D skin displacement using a single RGB camera. This simple sensing method of multi-dimensional touch vectors is expected to enable users to control various complex systems intuitively. As a social impact with this technology, on-palm interaction becomes a universal controller for various systems in daily life, such as mobile robots, home electronics and digital contents.

Ryosuke Ono
Osaka University

Shunsuke Yoshimoto
Osaka University

Kosuke Sato
Osaka University