The Misconception that Almost Stopped AI

Take your personal data back with Incogni! Use code WELCHLABS and get 60% off an annual plan: http://incogni.com/welchlabs

Loss Landscape Posters! 21:23
https://www.welchlabs.com/resources/loss-landscape-poster-17x19
https://www.welchlabs.com/resources/loss-landscape-poster-digital-download

Poster and Book Bundle
https://www.welchlabs.com/resources/loss-landscape-bundle-w-imaginary-numbers-book

Special Matte Black Edition Poster
https://www.welchlabs.com/resources/loss-landscape-poster-17x22-matte-black-special-edition

Welch Labs Book
https://www.welchlabs.com/resources/imaginary-numbers-book

Sections
0:00 - Intro
1:18 - How Incogni gets me more focus time
3:01 - What are we measuring again?
6:18 - How to make our loss go down?
7:32 - Tuning one parameter
9:11 - Tuning two parameters together
11:01 - Gradient descent
13:18 - Visualizing high dimensional surfaces
15:10 - Loss Landscapes
16:55 - Wormholes!
17:55 - Wikitext
18:55 - But where do the wormholes come from?
20:00 - Why local minima are not a problem
21:23 - Posters

Special Thanks to Patrons https://www.patreon.com/welchlabs

Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti, Brian Henry, Tim Palade, Petar Vecutin, Nicolas baumann, Jason Singh, Robert Riley, vornska, Barry Silverman, Jake Ehrlich, Mitch Jacobs

References
Li et al: Visualizing the Loss Landscape of Neural Nets. https://arxiv.org/abs/1712.09913
Talking Nets: An Oral History of Neural Networks. (2000). United Kingdom: MIT Press. Hinton quote is on p376.
Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. United Kingdom: MIT Press.
Prince, S. J. (2023). Understanding Deep Learning. United Kingdom: MIT Press.
Manim Animations: https://github.com/stephencwelch/manim_videos

Premium Beat IDs
MWROXNAY0SPXCMBS