Vision-based tactile sensor development and application in robotics

Abstract

As Robots’ application field extends beyond the traditional hard-coding assembly line into domestic serving and modern industrial scenarios, more intelligent interaction between robots, environments, objects, and human users is often required. The challenges faced by robots in adapting to various situations have posted an urgent demand for more accurate, higher frequency, and multimodality perception methods. Inspired by the human nature of using the sense of touch, tactile sensors are developed to provide necessary contact feedback information for robotic systems and complete control tasks. A vision-based tactile sensor emerges outstandingly with its unique advantages. In this thesis, we illustrate the development of a high-performance, lowcost, easy-deployable vision-based tactile sensor with its working principle, design, and fabrication process. Contact information obtained by the sensor provides control feedback for downsteaming robotic applications. In particular, we first show the hardware design of the sensor, named DelTact, with the modular configuration, fabrication process, and evaluation. The sensor uses an optimized random color pattern to capture the deformation of a soft elastomer under contact via a camera and extracts contact information from images through machine vision algorithms. Sensor and robot hardware are integrated with a parallel and dexterous gripper. The sensor structure is also reconstructed to combine with other robot components, i.e., robot arm and robot foot, to provide full-body tactile measurement across the robot surface. The sensor achieves better technical parameters than other state-of-the-art sensors in terms of higher spatial resolution and smaller size. Regarding application, a high-frequency dense optical flow first tracks the deformation of the random color pattern to provide fundamental level contact information. Based on the optical flow, accurate, computationally effective, real-time estimations of 3-axis contact shape and force are finished respectively using Gaussian density and Helmholtz-Hodge decomposition. Then, a human-robot interaction task is performed on the tactile-sensing robot arm, and a proprioceptive control task is demonstrated on the tactile-sensing robot leg. The tactile sensing dexterous gripper completes a higher level of tool manipulation task of blind object picking up. Comprehensive expp[periments and evaluation of the results are presented. Finally, the thesis is concluded by discussion, limitations, and future work directions.

Date of Award	2023
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Hongyu YU (Supervisor) & Michael Yu WANG (Supervisor)