The Truth about AI is Devastating: Proof by MIT, Harvard

I think anyone who has used LLMs for anything substantial will have realized that they do not actually understand anything. This doesn't mean they are not useful tools, but those expecting AGI are going to have to wait a while longer.

They forgot to put into the prompt: "you are the world leading expert in physics, like Isaac Newton and Johannes Kepler..."

Turns out llms work exactly like other ML models 😂

This actually points to a deeper misconception in the field of AI development. The idea that a _language_ model would be conducive to _thinking._ I feel like a lot of people mistakenly conflate the process of thinking with the process of expressing the thoughts. A chimp that can throw a rock at a target already intuitively understands Newtonian physics. We need not a _language_ model, but a _thought_ model.

It’s not a formalized experiment with inductive bias controls, but I have noticed that when I push co-coding models into really weird use cases, they completely breakdown, like even forgetting syntax. This tells me they cannot extrapolate, only pattern match. In fact, they don’t even have a model of the language syntax.

Thanks for sharing. According to a recent Anthropic paper, LLMs employ parallel processing and heuristic-based shortcuts to come with believable answers, particularly in mathematical tasks, without truly grasping the fundamental ideas—that is, without putting the human-crafted techniques they acquired from the training dataset to use. They seem to be very good at anticipating "what," but not so much "why."

Well… Apple DID try to warn us of this.

There is a lot of magical thinking around AI

No matter how many cards you add to a card catalog, in the end it's still a card catalog.

This is the same for coding it is very clear that even the most state of the art models do not actually fundamentally understand what's happening with the variables and control structures within the code they are merely piecing together various patterns.