12—Attentional Models

We asked in the last lesson whether “Memory Networks” lives up to the publicity it received when it came out, and came to the conclusion: almost certainly not! So why did be bother with it at all? Because it turns out that it provides much of the key foundations we need to understand something which have become one of the most important advances in the last year or two: Attentional Models. These models allow us to build systems that focus on the most important part of the input for the current task, and are critical, for instance, in creating translation systems (which we’ll cover in the next lesson).