Attention Wasn't the Original Name

In the summer of 2014, a graduate student named Dzmitry Bahdanau sat down to fix what looked like an ordinary bug. He was interning at a machine learning lab in Montreal, working on software that translated one language into another, and the translations kept falling apart on long sentences. He was not trying to invent anything the field would still be talking about a decade later. He was trying to stop a specific model from losing the thread.

The video you just watched calls the fix "attention." Here's the part it didn't have time for: that isn't the name Bahdanau gave it. It isn't even the name his supervisor gave it, not at first.

Bahdanau was working under Kyunghyun Cho and Yoshua Bengio, at a Montreal lab that would later become one of the most influential machine learning research groups in the world. The translation software of the day had a structural flaw: it had to compress an entire sentence into one fixed summary before it could generate a single word of the translation. Short sentences survived that squeeze. Long ones didn't.

Bahdanau's fix sounds almost too modest for what it became. Instead of forcing one summary to carry everything, let the model glance back at the whole original sentence while producing each word of the translation, and decide, word by word, which parts of it actually mattered right now. He built it and gave it a working title only an engineer could love: RNNsearch. Technically accurate. Entirely forgettable.

Reading a late draft, Bengio reached for a different word than the one Bahdanau had been using. He had spent years around cognitive science research on how human attention narrows focus to one detail while the rest of a scene fades, and Bahdanau's mechanism did something similar. Bengio's word for that was "attention." It made it into the finished paper exactly once, in a single passing sentence explaining what the mechanism did. Everywhere else, on every page, the paper called itself something far less memorable: an "alignment model," running inside a system its own authors had named RNNsearch.

The paper that resulted, "Neural Machine Translation by Jointly Learning to Align and Translate," carries three names: Bahdanau, Cho, and Bengio. None of them were trying to write a landmark paper in artificial intelligence history. They were trying to fix a translation model. It would be another three years before a different team, building on exactly this idea, would title their own paper as if they already knew what they'd done.

The pattern is worth sitting with, because it shows up again and again in AI history: the field's biggest turns rarely announce themselves. They arrive disguised as a patch for a boring, specific problem, published by researchers who describe their own work modestly, in the plainest terms they have. Attention didn't enter the world billed as the idea that would end up inside every major language model built afterward. It entered as a fix for one translation model's bad habit on long sentences, with a name suggested almost in passing.

None of this is in the video: not the meeting-room comparison that makes the actual mechanism click, not the worked example of a sentence where "it" could mean two different things, not the moment Video Fourteen's ambiguous "crane" finally gets resolved. That's the mechanism itself, and it earns fifteen minutes doing it justice.

Bahdanau built a fix for one sentence at a time. It took his supervisor one word, borrowed from how human focus actually works, to name what he'd built.