Investigating how neural networks learn by programming one from scratch in C#, and then teaching it to recognize different scribbles and images.

Source code: https://github.com/SebLague/Neural-Network-Experiments
Demo: https://sebastian.itch.io/neural-network-experiment

If you'd like to support me in making more videos (and getting early access to new stuff), you can join my patreon here: https://www.patreon.com/SebastianLague

The data comes from:
Figures http://yann.lecun.com/exdb/mnist/
Fashion https://github.com/zalandoresearch/fashion-mnist
Doodles https://github.com/googlecreativelab/quickdraw-dataset
Cifar10 https://www.cs.toronto.edu/kriz/cifar.html

Chapters
0:00 Introduction
2:39 The decision boundary
3:49 Weights
5:42 Prejudices
6:45 Hidden layers
7:45 Network programming
9:57 Activation functions
12:42 Costs
15:07 Example of gradient descent
18:22 The cost landscape
19:55 Gradient descent programming
21:10 It's educational! (slowly)
23:21 Calculation example
27:34 The chain rule
29:50 Some partial derivatives
33:14 Backpropagation
39:25 Number recognition
43:56 Drawing our own numbers
47:37 Fashion
48:25 Doodles
52:00 The final challenge

Music:
Cosmic waves – Michael FK
Amber – The stolen orchestra
Beyond the horizon – Sounds like Sander
Air – Assaf Ayalon
Purest form – Sounds like Sander
Hear wide open – Sounds like Sander
Universal wonder – Moments
Roman P – Moments
Everything in due time – Shimmer
It will come back – The stolen orchestra
Border – Glitter
New Moon – Cloud Wave
Sunflower – Cody Martin
Inner peace – moments
Enchanted – Cody Martin
Just around the corner – Shimmer

Please take the opportunity to connect and share this video with your friends and family if you find it useful.