Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization

Rie Johnson; Tong Zhang

Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization

Rie Johnson^*
, Tong Zhang^*

^*Corresponding author for this work

Department of Computer Science and Engineering

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

2 Citations (Scopus)

Abstract

This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in a function space. We provide a theoretical analysis and empirical study of the training method derived from this framework. It is shown that the method leads to better performance than that of standard training techniques.

Original language	English
Title of host publication	37th International Conference on Machine Learning, ICML 2020
Editors	Hal Daume, Aarti Singh
Publisher	International Machine Learning Society (IMLS)
Pages	4870-4879
Number of pages	10
ISBN (Electronic)	9781713821120
Publication status	Published - 2020
Event	37th International Conference on Machine Learning, ICML 2020 - Virtual, Online Duration: 13 Jul 2020 → 18 Jul 2020

Publication series

Name	37th International Conference on Machine Learning, ICML 2020
Volume	PartF168147-7

Conference

Conference	37th International Conference on Machine Learning, ICML 2020
City	Virtual, Online
Period	13/07/20 → 18/07/20

Bibliographical note

Publisher Copyright:
© 2020 by the Authors.

Access to Document

https://hdl.handle.net/1783.1/111000

Cite this

Johnson, R., & Zhang, T. (2020). Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization. In H. Daume, & A. Singh (Eds.), 37th International Conference on Machine Learning, ICML 2020 (pp. 4870-4879). (37th International Conference on Machine Learning, ICML 2020; Vol. PartF168147-7). International Machine Learning Society (IMLS). https://hdl.handle.net/1783.1/111000

@inproceedings{65040a6b934b4fb9aff1e9d7152e97ab,

title = "Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization",

abstract = "This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in a function space. We provide a theoretical analysis and empirical study of the training method derived from this framework. It is shown that the method leads to better performance than that of standard training techniques.",

author = "Rie Johnson and Tong Zhang",

note = "Publisher Copyright: {\textcopyright} 2020 by the Authors.; 37th International Conference on Machine Learning, ICML 2020 ; Conference date: 13-07-2020 Through 18-07-2020",

year = "2020",

language = "English",

series = "37th International Conference on Machine Learning, ICML 2020",

publisher = "International Machine Learning Society (IMLS)",

pages = "4870--4879",

editor = "Hal Daume and Aarti Singh",

booktitle = "37th International Conference on Machine Learning, ICML 2020",

}

Johnson, R & Zhang, T 2020, Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization. in H Daume & A Singh (eds), 37th International Conference on Machine Learning, ICML 2020. 37th International Conference on Machine Learning, ICML 2020, vol. PartF168147-7, International Machine Learning Society (IMLS), pp. 4870-4879, 37th International Conference on Machine Learning, ICML 2020, Virtual, Online, 13/07/20. <https://hdl.handle.net/1783.1/111000>

Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization. / Johnson, Rie; Zhang, Tong.
37th International Conference on Machine Learning, ICML 2020. ed. / Hal Daume; Aarti Singh. International Machine Learning Society (IMLS), 2020. p. 4870-4879 (37th International Conference on Machine Learning, ICML 2020; Vol. PartF168147-7).

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

TY - GEN

T1 - Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization

AU - Johnson, Rie

AU - Zhang, Tong

PY - 2020

Y1 - 2020

N2 - This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in a function space. We provide a theoretical analysis and empirical study of the training method derived from this framework. It is shown that the method leads to better performance than that of standard training techniques.

AB - This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in a function space. We provide a theoretical analysis and empirical study of the training method derived from this framework. It is shown that the method leads to better performance than that of standard training techniques.

UR - https://www.scopus.com/pages/publications/85105121620

M3 - Conference Paper published in a book

AN - SCOPUS:85105121620

T3 - 37th International Conference on Machine Learning, ICML 2020

SP - 4870

EP - 4879

BT - 37th International Conference on Machine Learning, ICML 2020

A2 - Daume, Hal

A2 - Singh, Aarti

PB - International Machine Learning Society (IMLS)

T2 - 37th International Conference on Machine Learning, ICML 2020

Y2 - 13 July 2020 through 18 July 2020

ER -

Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this