Communication-efficient distributed optimization using an approximate Newton-type method

Ohad Shamir; Nathan Srebro; Tong Zhang

Communication-efficient distributed optimization using an approximate Newton-type method

Ohad Shamir, Nathan Srebro, Tong Zhang

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

Abstract

We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems. For quadratic objectives, the method enjoys a linear rate of convergence which prov- ably improves with the data size, requiring an essentially constant number of iterations under reasonable assumptions. We provide theoretical and empirical evidence of the advantages of our method compared to other approaches, such as one-shot parameter averaging and ADMM.

Original language	English
Title of host publication	31st International Conference on Machine Learning, ICML 2014
Publisher	International Machine Learning Society (IMLS)
Pages	2665-2681
Number of pages	17
ISBN (Electronic)	9781634393973
Publication status	Published - 2014
Externally published	Yes
Event	31st International Conference on Machine Learning, ICML 2014 - Beijing, China Duration: 21 Jun 2014 → 26 Jun 2014

Publication series

Name	31st International Conference on Machine Learning, ICML 2014
Volume	3

Conference

Conference	31st International Conference on Machine Learning, ICML 2014
Country/Territory	China
City	Beijing
Period	21/06/14 → 26/06/14

Bibliographical note

Publisher Copyright:
Copyright © (2014) by the International Machine Learning Society (IMLS) All rights reserved.

Access to Document

https://hdl.handle.net/1783.1/98233

Cite this

@inproceedings{6102a01af85e48f88b3ef9d7089272cc,

title = "Communication-efficient distributed optimization using an approximate Newton-type method",

abstract = "We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems. For quadratic objectives, the method enjoys a linear rate of convergence which prov- ably improves with the data size, requiring an essentially constant number of iterations under reasonable assumptions. We provide theoretical and empirical evidence of the advantages of our method compared to other approaches, such as one-shot parameter averaging and ADMM.",

author = "Ohad Shamir and Nathan Srebro and Tong Zhang",

note = "Publisher Copyright: Copyright {\textcopyright} (2014) by the International Machine Learning Society (IMLS) All rights reserved.; 31st International Conference on Machine Learning, ICML 2014 ; Conference date: 21-06-2014 Through 26-06-2014",

year = "2014",

language = "English",

series = "31st International Conference on Machine Learning, ICML 2014",

publisher = "International Machine Learning Society (IMLS)",

pages = "2665--2681",

booktitle = "31st International Conference on Machine Learning, ICML 2014",

}

Shamir, O, Srebro, N & Zhang, T 2014, Communication-efficient distributed optimization using an approximate Newton-type method. in 31st International Conference on Machine Learning, ICML 2014. 31st International Conference on Machine Learning, ICML 2014, vol. 3, International Machine Learning Society (IMLS), pp. 2665-2681, 31st International Conference on Machine Learning, ICML 2014, Beijing, China, 21/06/14. <https://hdl.handle.net/1783.1/98233>

Communication-efficient distributed optimization using an approximate Newton-type method. / Shamir, Ohad; Srebro, Nathan; Zhang, Tong.
31st International Conference on Machine Learning, ICML 2014. International Machine Learning Society (IMLS), 2014. p. 2665-2681 (31st International Conference on Machine Learning, ICML 2014; Vol. 3).

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

TY - GEN

T1 - Communication-efficient distributed optimization using an approximate Newton-type method

AU - Shamir, Ohad

AU - Srebro, Nathan

AU - Zhang, Tong

PY - 2014

Y1 - 2014

N2 - We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems. For quadratic objectives, the method enjoys a linear rate of convergence which prov- ably improves with the data size, requiring an essentially constant number of iterations under reasonable assumptions. We provide theoretical and empirical evidence of the advantages of our method compared to other approaches, such as one-shot parameter averaging and ADMM.

AB - We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems. For quadratic objectives, the method enjoys a linear rate of convergence which prov- ably improves with the data size, requiring an essentially constant number of iterations under reasonable assumptions. We provide theoretical and empirical evidence of the advantages of our method compared to other approaches, such as one-shot parameter averaging and ADMM.

UR - https://www.scopus.com/pages/publications/84919935839

M3 - Conference Paper published in a book

AN - SCOPUS:84919935839

T3 - 31st International Conference on Machine Learning, ICML 2014

SP - 2665

EP - 2681

BT - 31st International Conference on Machine Learning, ICML 2014

PB - International Machine Learning Society (IMLS)

T2 - 31st International Conference on Machine Learning, ICML 2014

Y2 - 21 June 2014 through 26 June 2014

ER -

Communication-efficient distributed optimization using an approximate Newton-type method

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this