Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing

Yang Katie Zhao; Shang Wu; Jingqun Zhang; Sixu Li; Chaojian Li; Yingyan Celine Lin

doi:10.1109/DAC56929.2023.10247710

Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing

Yang Katie Zhao^*, Shang Wu, Jingqun Zhang, Sixu Li, Chaojian Li, Yingyan Celine Lin

^*Corresponding author for this work

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

17 Citations (Scopus)

Abstract

Instant on-device Neural Radiance Fields (NeRFs) are in growing demand for unleashing the promise of immersive AR/VR experiences, but are still limited by their prohibitive training time. Our profiling analysis reveals a memory-bound inefficiency in NeRF training. To tackle this inefficiency, near-memory processing (NMP) promises to be an effective solution, but also faces challenges due to the unique workloads of NeRFs, including the random hash table lookup, random point processing sequence, and heterogeneous bottleneck steps. Therefore, we propose the first NMP framework, Instant-NeRF, dedicated to enabling instant on-device NeRF training. Experiments on eight datasets consistently validate the effectiveness of Instant-NeRF.

Original language	English
Title of host publication	2023 60th ACM/IEEE Design Automation Conference, DAC 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Number of pages	6
ISBN (Electronic)	9798350323481
ISBN (Print)	9798350323498
DOIs	https://doi.org/10.1109/DAC56929.2023.10247710
Publication status	Published - 15 Sept 2023
Externally published	Yes
Event	60th ACM/IEEE Design Automation Conference, DAC 2023 - San Francisco, United States Duration: 9 Jul 2023 → 13 Jul 2023

Publication series

Name	Proceedings - Design Automation Conference
Volume	2023-July
ISSN (Print)	0738-100X

Conference

Conference	60th ACM/IEEE Design Automation Conference, DAC 2023
Country/Territory	United States
City	San Francisco
Period	9/07/23 → 13/07/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

Neural Radiance Field
Algorithm-Accelerator Co-Design
Near-Memory Processing
On-Device Training

Access to Document

10.1109/DAC56929.2023.10247710

Cite this

Zhao, Y. K., Wu, S., Zhang, J., Li, S., Li, C., & Lin, Y. C. (2023). Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing. In 2023 60th ACM/IEEE Design Automation Conference, DAC 2023 Article 10247710 (Proceedings - Design Automation Conference; Vol. 2023-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DAC56929.2023.10247710

@inproceedings{d9aaf30c9e4844d2a821af79d354bbea,

title = "Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing",

abstract = "Instant on-device Neural Radiance Fields (NeRFs) are in growing demand for unleashing the promise of immersive AR/VR experiences, but are still limited by their prohibitive training time. Our profiling analysis reveals a memory-bound inefficiency in NeRF training. To tackle this inefficiency, near-memory processing (NMP) promises to be an effective solution, but also faces challenges due to the unique workloads of NeRFs, including the random hash table lookup, random point processing sequence, and heterogeneous bottleneck steps. Therefore, we propose the first NMP framework, Instant-NeRF, dedicated to enabling instant on-device NeRF training. Experiments on eight datasets consistently validate the effectiveness of Instant-NeRF.",

keywords = "Neural Radiance Field, Algorithm-Accelerator Co-Design, Near-Memory Processing, On-Device Training",

author = "Zhao, \{Yang Katie\} and Shang Wu and Jingqun Zhang and Sixu Li and Chaojian Li and Lin, \{Yingyan Celine\}",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 60th ACM/IEEE Design Automation Conference, DAC 2023 ; Conference date: 09-07-2023 Through 13-07-2023",

year = "2023",

month = sep,

day = "15",

doi = "10.1109/DAC56929.2023.10247710",

language = "English",

isbn = "9798350323498",

series = "Proceedings - Design Automation Conference",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2023 60th ACM/IEEE Design Automation Conference, DAC 2023",

}

Zhao, YK, Wu, S, Zhang, J, Li, S, Li, C & Lin, YC 2023, Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing. in 2023 60th ACM/IEEE Design Automation Conference, DAC 2023., 10247710, Proceedings - Design Automation Conference, vol. 2023-July, Institute of Electrical and Electronics Engineers Inc., 60th ACM/IEEE Design Automation Conference, DAC 2023, San Francisco, United States, 9/07/23. https://doi.org/10.1109/DAC56929.2023.10247710

Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing. / Zhao, Yang Katie; Wu, Shang; Zhang, Jingqun et al.
2023 60th ACM/IEEE Design Automation Conference, DAC 2023. Institute of Electrical and Electronics Engineers Inc., 2023. 10247710 (Proceedings - Design Automation Conference; Vol. 2023-July).

Research output: Chapter in Book/Conference Proceeding/Report › Conference Paper published in a book › peer-review

TY - GEN

T1 - Instant-NeRF

T2 - 60th ACM/IEEE Design Automation Conference, DAC 2023

AU - Zhao, Yang Katie

AU - Wu, Shang

AU - Zhang, Jingqun

AU - Li, Sixu

AU - Li, Chaojian

AU - Lin, Yingyan Celine

PY - 2023/9/15

Y1 - 2023/9/15

N2 - Instant on-device Neural Radiance Fields (NeRFs) are in growing demand for unleashing the promise of immersive AR/VR experiences, but are still limited by their prohibitive training time. Our profiling analysis reveals a memory-bound inefficiency in NeRF training. To tackle this inefficiency, near-memory processing (NMP) promises to be an effective solution, but also faces challenges due to the unique workloads of NeRFs, including the random hash table lookup, random point processing sequence, and heterogeneous bottleneck steps. Therefore, we propose the first NMP framework, Instant-NeRF, dedicated to enabling instant on-device NeRF training. Experiments on eight datasets consistently validate the effectiveness of Instant-NeRF.

AB - Instant on-device Neural Radiance Fields (NeRFs) are in growing demand for unleashing the promise of immersive AR/VR experiences, but are still limited by their prohibitive training time. Our profiling analysis reveals a memory-bound inefficiency in NeRF training. To tackle this inefficiency, near-memory processing (NMP) promises to be an effective solution, but also faces challenges due to the unique workloads of NeRFs, including the random hash table lookup, random point processing sequence, and heterogeneous bottleneck steps. Therefore, we propose the first NMP framework, Instant-NeRF, dedicated to enabling instant on-device NeRF training. Experiments on eight datasets consistently validate the effectiveness of Instant-NeRF.

KW - Neural Radiance Field

KW - Algorithm-Accelerator Co-Design

KW - Near-Memory Processing

KW - On-Device Training

UR - https://www.scopus.com/pages/publications/85173116393

U2 - 10.1109/DAC56929.2023.10247710

DO - 10.1109/DAC56929.2023.10247710

M3 - Conference Paper published in a book

AN - SCOPUS:85173116393

SN - 9798350323498

T3 - Proceedings - Design Automation Conference

BT - 2023 60th ACM/IEEE Design Automation Conference, DAC 2023

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 9 July 2023 through 13 July 2023

ER -

Zhao YK, Wu S, Zhang J, Li S, Li C, Lin YC. Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing. In 2023 60th ACM/IEEE Design Automation Conference, DAC 2023. Institute of Electrical and Electronics Engineers Inc. 2023. 10247710. (Proceedings - Design Automation Conference). doi: 10.1109/DAC56929.2023.10247710

Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this