Attention

Each position projects into a query, a key and a value. Comparing a query against all keys yields a score per position; normalising those scores produces weights, and the weighted sum of values becomes that position’s new representation. Multi-head attention runs several of these in parallel with different projections, letting different heads specialise — some track syntax, others long-range dependencies — before the results are recombined. Causal masking prevents a position from attending to later ones during generation, which is what makes autoregressive decoding well-defined.

The all-pairs comparison is the source of both the capability and the cost. Because the model computes an explicit relationship between every pair of positions, it can connect a pronoun to a noun thousands of tokens earlier — but the work grows with the square of sequence length. Efficiency research targets exactly this: sparse and sliding-window patterns, grouped or multi-query key–value sharing, and memory-management schemes that keep the growing key-value cache from fragmenting a GPU.

The misconception worth flagging is that attention weights are explanations. They show where computation flowed, not why an answer was produced, and high attention on a token does not establish that the token caused the output. Attention maps are a debugging aid for researchers, not an interpretability guarantee for product decisions.