This course is an extension of Stanford Stats 385, Theories of Deep Learning, taught by Prof. Dave Donoho, Dr. Hatef Monajemi and Mr. Vardan Papyan. The aim of this course is to provide graduate students who are interested in deep learning some state-of-the-art mathematical and theoretical understanding of neural networks, in addition to some preliminary tutorials. It includes, but is not limited to, harmonic analysis perspectives on convolutional networks, statistical learning theory on generalization ability, and non-convex optimization methods. Familiarity on python programming with modern neural networks and mathematical maturity on approximation theory, optimization, and statistics will be helpful.