Lesson 18 built up networks by composing simple functions into deeper ones. What it left unspecified is how a layer decides which pieces of its input to combine. Two very different answers to that question power most of modern deep learning, and each one is a bet about the data’s structure — not a universally better technique.
Convolution’s bet. A convolutional layer applies the same small filter (say, a 3×3 patch of weights) at every position in its input, sliding it across the whole thing. Two assumptions are baked into that design: locality — whatever matters for this position is found in a small neighborhood around it, not far away — and translation invariance — a pattern means the same thing no matter where it shows up, so it’s worth detecting with the same weights everywhere (an edge is an edge, a cat’s ear is a cat’s ear, whether it’s in the top-left or bottom-right of the photo). For images, both assumptions are excellent: pixels next to each other are almost always related, and a shape doesn’t change meaning when it moves across the frame.
Attention’s bet. An attention layer instead computes, for every position, a relevance score against every other position, then combines information weighted by those scores. There’s no built-in notion of “nearby” at all — position 1 and position 500 start out equally reachable; the model learns from data which positions actually matter to which, and that relationship is allowed to depend on content, not distance. (Chapter and verse on the mechanism is coming in Stage 6 — for now, the structural bet is the point.)
The task on the table: resolving what a pronoun refers to somewhere in a long paragraph, where the right answer could be one word back or forty words back, and which is right depends on the specific sentence, not on some fixed offset.
Your task: pick the option that correctly explains why convolution’s structural assumption is a poor fit here, while attention’s isn’t.