Evolving human-device interaction in the context of ubiquitous computing

Abstract

Sensing, computing and communicating are three basic elements of the Internet of Things (IoT). Benefited from hardware innovation, present devices have been enhanced to such a extent that even a palm-sized device owes powerful sensing, computing and communicating capabilities. This consequently promotes the broad applications of ubiquitous computing and blurs the boundary between human and device. In such a situation, the human-device interaction (HDI) is becoming increasingly pervasive around us and covers a wide range of applications including environment sensing and human dynamics sensing. In this thesis, we follow a line of exploring different approaches of designing human-device interactive systems for the mentioned applications. Specifically, we propose four novel systems in this thesis covering two aspects of human-device interaction from application perspective, namely, environment sensing and human dynamics sensing. In the first work, we propose a novel system to aid users distinguish in-wall objects and map hidden pipeline layout, using off-the-shelf sensors embedded in smartphones. In the second work, we design a objects distinguishing system named TagFree with commercial Wi-Fi infrastructure which differentiates a single object at a time or up to three objects simultaneously with favorable performance. Compared with conventional methods, this system removes the need of additional devices attached to objects. Following the above work in environment sensing, we also conduct research in gesture recognition and text entry with commodity devices. In the third work, we develop GRfid, a novel device-free gesture recognition system based on phase information output by COTS RFID devices which is potentially applied in smart homes, museums and art galleries where RFID technology is widely utilized. In the last work, we present a novel text-entry system, AcouTexts, aiming at dealing with problem of inputting texts on tiny devices. With AcouTexts, users can enter texts to a device by just using a finger even without touching the device. We prototype the above systems with commodity devices/infrastructure and evaluate their performances with real-world experiments.

Date of Award	2017
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology