3D motion decomposition for RGBD future dynamic scene synthesis

Xiaojuan Qi; Zhengzhe Liu; Qifeng Chen; Jiaya Jia

doi:10.1109/CVPR.2019.00786

3D motion decomposition for RGBD future dynamic scene synthesis

Xiaojuan Qi, Zhengzhe Liu, Qifeng Chen, Jiaya Jia

Department of Computer Science and Engineering

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

20 Citations (Scopus)

Abstract

A future video is the 2D projection of a 3D scene with predicted camera and object motion. Accurate future video prediction inherently requires understanding of 3D motion and geometry of a scene. In this paper, we propose a RGBD scene forecasting model with 3D motion decomposition. We predict ego-motion and foreground motion that are combined to generate a future 3D dynamic scene, which is then projected into a 2D image plane to synthesize future motion, RGB images and depth maps. Optional semantic maps can be integrated. Experimental results on KITTI and Driving datasets show that our model outperforms other state-of-the-arts in forecasting future RGBD dynamic scenes.

Original language	English
Title of host publication	Proceedings - 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019
Publisher	IEEE Computer Society
Pages	7665-7674
Number of pages	10
ISBN (Electronic)	9781728132938
DOIs	https://doi.org/10.1109/CVPR.2019.00786
Publication status	Published - Jun 2019
Event	32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019 - Long Beach, United States Duration: 16 Jun 2019 → 20 Jun 2019

Publication series

Name	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Volume	2019-June
ISSN (Print)	1063-6919

Conference

Conference	32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019
Country/Territory	United States
City	Long Beach
Period	16/06/19 → 20/06/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

Image and Video Synthesis
RGBD sensors and analytics

Access to Document

10.1109/CVPR.2019.00786

Cite this

Qi, X., Liu, Z., Chen, Q., & Jia, J. (2019). 3D motion decomposition for RGBD future dynamic scene synthesis. In Proceedings - 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019 (pp. 7665-7674). Article 8953473 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Vol. 2019-June). IEEE Computer Society. https://doi.org/10.1109/CVPR.2019.00786

@inproceedings{75a1730cad674bc78ed476c0eb243c20,

title = "3D motion decomposition for RGBD future dynamic scene synthesis",

abstract = "A future video is the 2D projection of a 3D scene with predicted camera and object motion. Accurate future video prediction inherently requires understanding of 3D motion and geometry of a scene. In this paper, we propose a RGBD scene forecasting model with 3D motion decomposition. We predict ego-motion and foreground motion that are combined to generate a future 3D dynamic scene, which is then projected into a 2D image plane to synthesize future motion, RGB images and depth maps. Optional semantic maps can be integrated. Experimental results on KITTI and Driving datasets show that our model outperforms other state-of-the-arts in forecasting future RGBD dynamic scenes.",

keywords = "Image and Video Synthesis, RGBD sensors and analytics",

author = "Xiaojuan Qi and Zhengzhe Liu and Qifeng Chen and Jiaya Jia",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019 ; Conference date: 16-06-2019 Through 20-06-2019",

year = "2019",

month = jun,

doi = "10.1109/CVPR.2019.00786",

language = "English",

series = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

publisher = "IEEE Computer Society",

pages = "7665--7674",

booktitle = "Proceedings - 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019",

}

Qi, X, Liu, Z, Chen, Q & Jia, J 2019, 3D motion decomposition for RGBD future dynamic scene synthesis. in Proceedings - 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019., 8953473, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2019-June, IEEE Computer Society, pp. 7665-7674, 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, United States, 16/06/19. https://doi.org/10.1109/CVPR.2019.00786

3D motion decomposition for RGBD future dynamic scene synthesis. / Qi, Xiaojuan; Liu, Zhengzhe; Chen, Qifeng et al.
Proceedings - 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019. IEEE Computer Society, 2019. p. 7665-7674 8953473 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Vol. 2019-June).

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

TY - GEN

T1 - 3D motion decomposition for RGBD future dynamic scene synthesis

AU - Qi, Xiaojuan

AU - Liu, Zhengzhe

AU - Chen, Qifeng

AU - Jia, Jiaya

PY - 2019/6

Y1 - 2019/6

N2 - A future video is the 2D projection of a 3D scene with predicted camera and object motion. Accurate future video prediction inherently requires understanding of 3D motion and geometry of a scene. In this paper, we propose a RGBD scene forecasting model with 3D motion decomposition. We predict ego-motion and foreground motion that are combined to generate a future 3D dynamic scene, which is then projected into a 2D image plane to synthesize future motion, RGB images and depth maps. Optional semantic maps can be integrated. Experimental results on KITTI and Driving datasets show that our model outperforms other state-of-the-arts in forecasting future RGBD dynamic scenes.

AB - A future video is the 2D projection of a 3D scene with predicted camera and object motion. Accurate future video prediction inherently requires understanding of 3D motion and geometry of a scene. In this paper, we propose a RGBD scene forecasting model with 3D motion decomposition. We predict ego-motion and foreground motion that are combined to generate a future 3D dynamic scene, which is then projected into a 2D image plane to synthesize future motion, RGB images and depth maps. Optional semantic maps can be integrated. Experimental results on KITTI and Driving datasets show that our model outperforms other state-of-the-arts in forecasting future RGBD dynamic scenes.

KW - Image and Video Synthesis

KW - RGBD sensors and analytics

UR - https://www.webofscience.com/wos/woscc/full-record/WOS:000542649301028

UR - https://openalex.org/W2979544965

UR - https://www.scopus.com/pages/publications/85078754188

U2 - 10.1109/CVPR.2019.00786

DO - 10.1109/CVPR.2019.00786

M3 - Conference Paper published in a book

T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SP - 7665

EP - 7674

BT - Proceedings - 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019

PB - IEEE Computer Society

T2 - 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019

Y2 - 16 June 2019 through 20 June 2019

ER -

3D motion decomposition for RGBD future dynamic scene synthesis

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this