AI isn't gonna keep improving

Gordon Moore was not a “Dev”. He was the co-founder of Fairchild Semiconductor and Intel (and former CEO of the Intel).

Calling Gordon Moore a "dev and hardware enthusiast" is hilarious. Dude literally founded Intel

The problem as always is when you have a 99% reliable system and you want a 99.9% reliable model. The .9% difference is 10x more than anything else

"Moore - a dev and hardware enthusiast" -- Theo That is... technically correct

As we reach higher benchmark scores, you have to flip your understanding of them. If accuracy goes from 80% to 90%, that feels like a 10% improvement, but in reality the error rate has gone down by half, which is basically a 100% improvement.

Calling Gordon Moore a dev/hardware enthusiast would’ve been funny, if it was intended as a joke

small factual correction: "One of the crazy things apple invented, was the idea of having different cores with different roles"... No they didn't, it was actually ARM. Hetereogeneous Computing strategies and Big/Little architectures were not invented by apple :)

It feels less like LLMs have plateaued and more like the benchmarks are all being gamed and optimized for. Claude 3.5 sonnet, for example, is a cut above all other models.

I no longer believe what I said in this video. Updated take here: https://youtu.be/Kzf-tL8zyfo?si=IU-0BajX1F6FbBHz

This happened with re-enforcement learning too. The models had so many nodes that back propagation had virtually no effect, meaning they became too big to train any more and even got worse with more training.