Why would anyone let LLMs predict 4 tokens at once? Multi-Token Prediction Explained

Stable diffusion LLM

Your origin paper is incorrect, although you did show / mention it later in the video. The first paper to propose MTP was "Fast Inference from Transformers via Speculative Decoding" (2023), where the entire model predicts N future tokens (context + N-1 * mask -> N). MTP modified this by training a series of smaller models (heads) to predict the next N tokens from a single input (context -> N). DeepSeek then turned these heads into a transformer model with causal masking (still context -> N), which essentially treats the prediction like a RNN. In all cases though, the prediction is speculative, and might be discarded using the same method from the paper I mentioned. Furthermore, this technique has clearly been used by OpenAI since GPT 3.5, and is why the canvas editor in ChatGPT is so fast to make changes (they speculate on the previous canvas state - there's API documentation to support this). My guess is that this isn't being talked about much because it's orthogonal to other advancements, just like quantization is. Also, you never completed the initial motivation. Can DeepSeek's MTP method correctly guess the number of words in a sentence? Probably only slightly better than the next-token prediction objective. The diffusion methods should solve that, but are susceptible to block generation artifacts. Edit: I apologize if this comment came off as harsh, that wasn't my intention.

Bycloud and AI Explained are the GOATS.

Even better than diffusion over tokens. Would be diffusion in a latent space. Kind of like using a text autoencoder, and doing diffusion on that space

Can't wait for the Gemini diffusion video :D

0:51 Diffusion isn't an architecture. In fact, diffusion LLMs still use transformers.

Now I'm even more excited for V4 and R2

Check out LTX Video 13B now and experience the latest video gen breakthrough: https://bit.ly/ltxvbycloud

My favourite podcast channel (visuals are plain brainrot)

That moment when Will Smith eating spaghetti is a Video Gen AI benchmark.