Backpropagation calculus | Appendix to deep learning chapter 3
This one is a bit more symbol heavy, and that's actually the point. The goal here is to represent in somewhat more formal terms the intuition for how ...
==========
The hard assumption here is that you’ve watched part 3, giving an intuitive walkthrough of the backpropagation algorithm. Here, we get a bit more formal and dive into the relevant calculus. It’s normal for this to be a little confusing, so the mantra to regularly pause and ponder certainly applies as much here as anywhere else. Our main goal is to show how people in machine learning commonly think about the chain rule from the calculus in the context of networks, which has a different feel for how much most introductory calculus courses approach the subject. For those of you uncomfortable with the relevant calculus, I do have a whole series on the topic. Let’s just start off with an extremely simple network, one where each layer has a single neuron in it. So this particular network is de...
==========
The hard assumption here is that you’ve watched part 3, giving an intuitive walkthrough of the backpropagation algorithm. Here, we get a bit more formal and dive into the relevant calculus. It’s normal for this to be a little confusing, so the mantra to regularly pause and ponder certainly applies as much here as anywhere else. Our main goal is to show how people in machine learning commonly think about the chain rule from the calculus in the context of networks, which has a different feel for how much most introductory calculus courses approach the subject. For those of you uncomfortable with the relevant calculus, I do have a whole series on the topic. Let’s just start off with an extremely simple network, one where each layer has a single neuron in it. So this particular network is de...
