A current CMU dissertation defense: PhD Candidate: Shaojie Bai
*Title: **Equilibrium Approaches to Modern Deep Learning* Abstract: Deep learning (DL) has become one of the most successful and widely-adopted methods in modern artificial intelligence. Accompanying these successes are also increasingly complex and costly architectural designs, at the foundation of which has been a core concept: *layers*. This thesis challenges this fundamental role of layers, and provides an in-depth introduction to a new, layer-*less* paradigm of deep learning that computes the output as the fixed point of a dynamical system: deep equilibrium (DEQ) models. First, we introduce the general formulation of deep equilibrium models. We discuss how these models express “infinite-level” neural networks, decouple forward and backward passes, yet with the cost and design complexity of one traditional layer— even in some of the most competitive settings (e.g., language mode --- Frank C. Wimberly 140 Calle Ojo Feliz, Santa Fe, NM 87505 505 670-9918 Santa Fe, NM
.-- .- -. - / .- -.-. - .. --- -. ..--.. / -.-. --- -. .--- ..- --. .- - . FRIAM Applied Complexity Group listserv Zoom Fridays 9:30a-12p Mtn UTC-6 bit.ly/virtualfriam un/subscribe http://redfish.com/mailman/listinfo/friam_redfish.com FRIAM-COMIC http://friam-comic.blogspot.com/ archives: 5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/ 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/