High-throughput Generative Inference of Large Language Models with a Single GPU

Clark Barrett; Beidi Chen; Daniel Y. Fu; Joseph E. Gonzalez; Zhuohan Li; Percy Liang; Christopher Ré; Max Ryabinin; Ying Sheng; Ion Stoica; Zhiqiang Xie; Binhang Yuan; Ce Zhang; Lianmin Zheng

High-throughput Generative Inference of Large Language Models with a Single GPU

Clark Barrett, Beidi Chen, Daniel Y. Fu, Joseph E. Gonzalez, Zhuohan Li, Percy Liang, Christopher Ré, Max Ryabinin, Ying Sheng, Ion Stoica, Zhiqiang Xie, Binhang Yuan, Ce Zhang, Lianmin Zheng

The Hong Kong University of Science and Technology

Research output: Contribution to conference › Conference Paper

Original language	English
Publication status	Published - 2023
Event	40th International Conference on Machine Learning (ICML 2023) - Duration: 1 Jan 2023 → 1 Jan 2023

Conference

Conference	40th International Conference on Machine Learning (ICML 2023)
Period	1/01/23 → 1/01/23

Access to Document

https://hdl.handle.net/1783.1/128666

Cite this

@conference{87fe86517e114fe0a27f7f0e87bee9e7,

title = "High-throughput Generative Inference of Large Language Models with a Single GPU",

author = "Clark Barrett and Beidi Chen and Fu, \{Daniel Y.\} and Gonzalez, \{Joseph E.\} and Zhuohan Li and Percy Liang and Christopher R{\'e} and Max Ryabinin and Ying Sheng and Ion Stoica and Zhiqiang Xie and Binhang Yuan and Ce Zhang and Lianmin Zheng",

year = "2023",

language = "English",

note = "40th International Conference on Machine Learning (ICML 2023) ; Conference date: 01-01-2023 Through 01-01-2023",

}

TY - CONF

T1 - High-throughput Generative Inference of Large Language Models with a Single GPU

AU - Barrett, Clark

AU - Chen, Beidi

AU - Fu, Daniel Y.

AU - Gonzalez, Joseph E.

AU - Li, Zhuohan

AU - Liang, Percy

AU - Ré, Christopher

AU - Ryabinin, Max

AU - Sheng, Ying

AU - Stoica, Ion

AU - Xie, Zhiqiang

AU - Yuan, Binhang

AU - Zhang, Ce

AU - Zheng, Lianmin

PY - 2023

Y1 - 2023

M3 - Conference Paper

T2 - 40th International Conference on Machine Learning (ICML 2023)

Y2 - 1 January 2023 through 1 January 2023

ER -