SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks

Simiao Yu; Hao Dong; Felix Liang; Yuanhan Mo; Chao Wu; Yike Guo

doi:10.1109/ICIP.2019.8804285

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks

Simiao Yu, Hao Dong, Felix Liang, Yuanhan Mo, Chao Wu, Yike Guo

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

8 Citations (Scopus)

Abstract

Semantic image manipulation (SIM) aims to generate realistic images from an input source image and a target text description, such that the generated images not only match the content of the description, but also maintain text-irrelevant features of the source image. It requires to learn a good mapping between visual features and linguistic features. Previous works on SIM can only generate images of limited resolution that typically lack of fine and clear details. In this work, we aim to generate high-resolution photo-realistic images for SIM. Specifically, we propose SIMGAN, a generative adversarial networks (GAN) based architecture that is capable of generating images of size 256 × 256 for SIM. We demonstrate the effectiveness of SIMGAN and its superiority over existing methods via qualitative and quantitative evaluation on Caltech-200 and Oxford-102 datasets.

Original language	English
Title of host publication	2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings
Publisher	IEEE Computer Society
Pages	734-738
Number of pages	5
ISBN (Electronic)	9781538662496
DOIs	https://doi.org/10.1109/ICIP.2019.8804285
Publication status	Published - Sept 2019
Externally published	Yes
Event	26th IEEE International Conference on Image Processing, ICIP 2019 - Taipei, Taiwan, Province of China Duration: 22 Sept 2019 → 25 Sept 2019

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
Volume	2019-September
ISSN (Print)	1522-4880

Conference

Conference	26th IEEE International Conference on Image Processing, ICIP 2019
Country/Territory	Taiwan, Province of China
City	Taipei
Period	22/09/19 → 25/09/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

adversarial learning
generative model
image generation
semantic image manipulation

Access to Document

10.1109/ICIP.2019.8804285

Cite this

Yu, S., Dong, H., Liang, F., Mo, Y., Wu, C., & Guo, Y. (2019). SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks. In 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings (pp. 734-738). Article 8804285 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September). IEEE Computer Society. https://doi.org/10.1109/ICIP.2019.8804285

@inproceedings{4801f4b18dec47c89d55fa552cc30a63,

title = "SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks",

abstract = "Semantic image manipulation (SIM) aims to generate realistic images from an input source image and a target text description, such that the generated images not only match the content of the description, but also maintain text-irrelevant features of the source image. It requires to learn a good mapping between visual features and linguistic features. Previous works on SIM can only generate images of limited resolution that typically lack of fine and clear details. In this work, we aim to generate high-resolution photo-realistic images for SIM. Specifically, we propose SIMGAN, a generative adversarial networks (GAN) based architecture that is capable of generating images of size 256 × 256 for SIM. We demonstrate the effectiveness of SIMGAN and its superiority over existing methods via qualitative and quantitative evaluation on Caltech-200 and Oxford-102 datasets.",

keywords = "adversarial learning, generative model, image generation, semantic image manipulation",

author = "Simiao Yu and Hao Dong and Felix Liang and Yuanhan Mo and Chao Wu and Yike Guo",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 26th IEEE International Conference on Image Processing, ICIP 2019 ; Conference date: 22-09-2019 Through 25-09-2019",

year = "2019",

month = sep,

doi = "10.1109/ICIP.2019.8804285",

language = "English",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE Computer Society",

pages = "734--738",

booktitle = "2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings",

}

Yu, S, Dong, H, Liang, F, Mo, Y, Wu, C & Guo, Y 2019, SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks. in 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings., 8804285, Proceedings - International Conference on Image Processing, ICIP, vol. 2019-September, IEEE Computer Society, pp. 734-738, 26th IEEE International Conference on Image Processing, ICIP 2019, Taipei, Taiwan, Province of China, 22/09/19. https://doi.org/10.1109/ICIP.2019.8804285

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks. / Yu, Simiao; Dong, Hao; Liang, Felix et al.
2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings. IEEE Computer Society, 2019. p. 734-738 8804285 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September).

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

TY - GEN

T1 - SIMGAN

T2 - 26th IEEE International Conference on Image Processing, ICIP 2019

AU - Yu, Simiao

AU - Dong, Hao

AU - Liang, Felix

AU - Mo, Yuanhan

AU - Wu, Chao

AU - Guo, Yike

PY - 2019/9

Y1 - 2019/9

N2 - Semantic image manipulation (SIM) aims to generate realistic images from an input source image and a target text description, such that the generated images not only match the content of the description, but also maintain text-irrelevant features of the source image. It requires to learn a good mapping between visual features and linguistic features. Previous works on SIM can only generate images of limited resolution that typically lack of fine and clear details. In this work, we aim to generate high-resolution photo-realistic images for SIM. Specifically, we propose SIMGAN, a generative adversarial networks (GAN) based architecture that is capable of generating images of size 256 × 256 for SIM. We demonstrate the effectiveness of SIMGAN and its superiority over existing methods via qualitative and quantitative evaluation on Caltech-200 and Oxford-102 datasets.

AB - Semantic image manipulation (SIM) aims to generate realistic images from an input source image and a target text description, such that the generated images not only match the content of the description, but also maintain text-irrelevant features of the source image. It requires to learn a good mapping between visual features and linguistic features. Previous works on SIM can only generate images of limited resolution that typically lack of fine and clear details. In this work, we aim to generate high-resolution photo-realistic images for SIM. Specifically, we propose SIMGAN, a generative adversarial networks (GAN) based architecture that is capable of generating images of size 256 × 256 for SIM. We demonstrate the effectiveness of SIMGAN and its superiority over existing methods via qualitative and quantitative evaluation on Caltech-200 and Oxford-102 datasets.

KW - adversarial learning

KW - generative model

KW - image generation

KW - semantic image manipulation

UR - https://www.webofscience.com/wos/woscc/full-record/WOS:000521828600146

UR - https://openalex.org/W2970067499

UR - https://www.scopus.com/pages/publications/85076802313

U2 - 10.1109/ICIP.2019.8804285

DO - 10.1109/ICIP.2019.8804285

M3 - Conference Paper published in a book

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 734

EP - 738

BT - 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings

PB - IEEE Computer Society

Y2 - 22 September 2019 through 25 September 2019

ER -

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this