Multi-Gesture based Football Game in Smart Phones

We demonstrate a hand and foot based immersive multimodal interaction approach for smart phone. A smart phone based real-time multimodal immersive football game is designed as a proof of concept.

The application renders the game graphics and game status information using smart phone’s screen, audio and vibration. Augmented reality image rendering technology is employed for an immersive football game experience. The players interact with the game using both foot gestures and finger touch on screen, which trigger the interaction event and generate activity sequences for interactive buffers. The trajectory of the football is changed by interactive buffers. In our system, the players use foot to kick the ball on the smart phone screen, meanwhile, they guard the goal post area using hand-finger. The combination of CTM (Contour based Template Matching) and TLD (Tracking-Learning-Detection) provides a core support for foot-gesture interaction by accurately tracking the shoes or feet of various sizes, appearances, and colors.

Our algorithm uses dynamic programming based CTM algorithm to localize the foot-contour and employ TLD frame work to strengthen the tracking process in every frame. TLD algorithm combines elements from tracking, learning and detection in the 2D image space to make it a long-term tracker.

CTM approach is an indispensable step for identifying a variety of shoes in the beginning. CTM will give TLD the region of interest (ROI) for initialization. The proposed combined algorithm successfully solves all of the unexpected detection results of foot/shoe look-likes. Our algorithm execution time is faster and is accurate even when the foot/shoe is moving fast and in image blur cases. It’s worth mentioning that the method can detect almost obscured foot and also all kinds of the foot-gesture in the football game

Zhihan Lv
Shenzhen Institutes of Advanced Technology, Chinese Academy of Science

Shafiq Ur Réhman
i2Lab, Institutionen för Tillämpad Fysik och Elektronik, Umeå University