This course is inspired by Stanford Stats 385, Theories of Deep Learning, taught by Prof. Dave Donoho, Dr. Hatef Monajemi and Mr. Vardan Papyan. The aim of this course is to be provide graduate students who are interested in deep learning a variety of mathematical and theoretical understanding of neural networks which are currently available in research, in addition to some preliminary tutorials. Students with mathematical maturity on approximation theory, optimization, and statistics will be helpful.