Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task

Haofei Wang; Xujiong Dong; Zhaokang Chen; Bertram E. Shi

doi:10.1109/EMBC.2015.7318649

Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task

Haofei Wang, Xujiong Dong, Zhaokang Chen, Bertram E. Shi

HKUST Fok Ying Tung Graduate School

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

53 Citations (Scopus)

Abstract

We describe a hybrid brain computer interface that integrates gaze information from an eye tracker with brain activity information measured by electroencephalography (EEG). Users explicitly control the end effector of a robot arm to move in one of four directions using motor imagery to perform a pick and place task. Measurements of the natural eye gaze behavior of subjects is used to infer the instantaneous intent of the users based on the past gaze trajectory. This information is integrated with the output of the EEG classifier and contextual information about the environment probabilistically using Bayesian inference. Our experiments demonstrate that subjects can achieve 100% task completion within three minutes and that the integration of EEG and gaze information significantly improves performance over either cue in isolation.

Original language	English
Title of host publication	2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1476-1479
Number of pages	4
ISBN (Electronic)	9781424492718
DOIs	https://doi.org/10.1109/EMBC.2015.7318649
Publication status	Published - 4 Nov 2015
Event	37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015 - Milan, Italy Duration: 25 Aug 2015 → 29 Aug 2015

Publication series

Name	Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
Volume	2015-November
ISSN (Print)	1557-170X

Conference

Conference	37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015
Country/Territory	Italy
City	Milan
Period	25/08/15 → 29/08/15

Bibliographical note

Publisher Copyright:
© 2015 IEEE.

Keywords

EEG
Hybrid Brain Computer Interface (BCI)
assistive technology
eye tracker
gaze control
human computer interaction
motor imagery
robotics

Access to Document

10.1109/EMBC.2015.7318649

Cite this

Wang, H., Dong, X., Chen, Z., & Shi, B. E. (2015). Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task. In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015 (pp. 1476-1479). Article 7318649 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS; Vol. 2015-November). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/EMBC.2015.7318649

Wang, Haofei ; Dong, Xujiong ; Chen, Zhaokang et al. / Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task. 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 1476-1479 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS).

@inproceedings{78f4ca19b19f46d29c5583d260b76257,

title = "Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task",

abstract = "We describe a hybrid brain computer interface that integrates gaze information from an eye tracker with brain activity information measured by electroencephalography (EEG). Users explicitly control the end effector of a robot arm to move in one of four directions using motor imagery to perform a pick and place task. Measurements of the natural eye gaze behavior of subjects is used to infer the instantaneous intent of the users based on the past gaze trajectory. This information is integrated with the output of the EEG classifier and contextual information about the environment probabilistically using Bayesian inference. Our experiments demonstrate that subjects can achieve 100\% task completion within three minutes and that the integration of EEG and gaze information significantly improves performance over either cue in isolation.",

keywords = "EEG, Hybrid Brain Computer Interface (BCI), assistive technology, eye tracker, gaze control, human computer interaction, motor imagery, robotics",

author = "Haofei Wang and Xujiong Dong and Zhaokang Chen and Shi, \{Bertram E.\}",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.; 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015 ; Conference date: 25-08-2015 Through 29-08-2015",

year = "2015",

month = nov,

day = "4",

doi = "10.1109/EMBC.2015.7318649",

language = "English",

series = "Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1476--1479",

booktitle = "2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015",

}

Wang, H, Dong, X, Chen, Z & Shi, BE 2015, Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task. in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015., 7318649, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, vol. 2015-November, Institute of Electrical and Electronics Engineers Inc., pp. 1476-1479, 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015, Milan, Italy, 25/08/15. https://doi.org/10.1109/EMBC.2015.7318649

Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task. / Wang, Haofei; Dong, Xujiong; Chen, Zhaokang et al.
2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 1476-1479 7318649 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS; Vol. 2015-November).

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

TY - GEN

T1 - Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task

AU - Wang, Haofei

AU - Dong, Xujiong

AU - Chen, Zhaokang

AU - Shi, Bertram E.

PY - 2015/11/4

Y1 - 2015/11/4

N2 - We describe a hybrid brain computer interface that integrates gaze information from an eye tracker with brain activity information measured by electroencephalography (EEG). Users explicitly control the end effector of a robot arm to move in one of four directions using motor imagery to perform a pick and place task. Measurements of the natural eye gaze behavior of subjects is used to infer the instantaneous intent of the users based on the past gaze trajectory. This information is integrated with the output of the EEG classifier and contextual information about the environment probabilistically using Bayesian inference. Our experiments demonstrate that subjects can achieve 100% task completion within three minutes and that the integration of EEG and gaze information significantly improves performance over either cue in isolation.

AB - We describe a hybrid brain computer interface that integrates gaze information from an eye tracker with brain activity information measured by electroencephalography (EEG). Users explicitly control the end effector of a robot arm to move in one of four directions using motor imagery to perform a pick and place task. Measurements of the natural eye gaze behavior of subjects is used to infer the instantaneous intent of the users based on the past gaze trajectory. This information is integrated with the output of the EEG classifier and contextual information about the environment probabilistically using Bayesian inference. Our experiments demonstrate that subjects can achieve 100% task completion within three minutes and that the integration of EEG and gaze information significantly improves performance over either cue in isolation.

KW - EEG

KW - Hybrid Brain Computer Interface (BCI)

KW - assistive technology

KW - eye tracker

KW - gaze control

KW - human computer interaction

KW - motor imagery

KW - robotics

UR - https://openalex.org/W2293069699

UR - https://www.scopus.com/pages/publications/84953240858

U2 - 10.1109/EMBC.2015.7318649

DO - 10.1109/EMBC.2015.7318649

M3 - Conference Paper published in a book

C2 - 26736549

T3 - Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS

SP - 1476

EP - 1479

BT - 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015

Y2 - 25 August 2015 through 29 August 2015

ER -

Wang H, Dong X, Chen Z, Shi BE. Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task. In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 1476-1479. 7318649. (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS). doi: 10.1109/EMBC.2015.7318649

Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this