The transformer “explained”?

The closest thing to an intuitive explainer than I know of is “The Illustrated Transformer,” but IMO it’s too light on intuition and too heavy on near-pseudocode (including stuff like “now you divide by 8,” as the third of six enumerated “steps” which themselves only cover part of the whole computation!) A sentence can have a pronoun at one end whose antecedent is all the way at the other, for example. But instead of looking at the current position plus a small local window around it, they instead look at

When the input is textual, this feels a lot like “reading” – the RNN processes the the first word, writes down a summary of what it’s gleaned so far, then processes the second word in light of the summary, updates the summary, processes the third word in light of the summary, updates the summary, etc.

Source: nostalgebraist.tumblr.com