World Models Collapse into a Phase Transition - WHAT?

The phase-transition world-model collapse research is validating an intuition I have had about this for a while. Seeing the actual measurements is still startling. Excellent research.

Thank you for an excellent walkthrough on these papers. It really struck me that what we're really talking about here is the distinction between using language versus math as a proxy for representing the world. The "reasoning" in the JEPA models proceeds by the laws of physics, in LLMS it's using the logic of language. Is mathematics really a more granular somehow more real representation of the world than language itself? They're both going to be subject to this phase transition collapse, JEPA approaches will be based on lossy mathematical compression, language will more readily accommodate high dimensional capture. It seems obvious we're going to need a hybrid approach, and your discussion of these papers really made that clear to me. Appreciate all the hard work, take care

I think there should be a "Slapstick" benchmark. Slapstick isn't a linguistic phenomenon. It has everything to do with the violation, and near-violation, of the Causal World Model of the observer. Which is what makes it funny, in conjunction with the expectations derived from the semantic/cultural matrix from the LLM. A good World Model AGI should be able to do funny Slapstick.

Many experts say that world models are needed to surmount the shortcomings of LLMs, but these papers suggest that creating efficacious world models is no small task. Do you think all world models will be subject to collapse, or, could there be new mathematical paradigms which could help prevent that outcome?To what extent might this not just be a software problem but also a hardware problem?

Yes, they show phase transitions. I've been building these into my models for the past 6 months, but I push them towards the insight. I'll have to read these papers because I can already see exactly what they're doing and what they're missing out on. They still can't harness the attention right which is why they run into the hard limits. Their models are, quite literally, bored out of their minds.

Some days I feel like my phase transition is on the left, just depends on how much coffee I have that morning…

have thought about this for awhile and love your breakdown! iirc, the optimal state of brain function operates in a "near-critical" state, teetering on the boundary on the edge of chaos...

This goes way over my head...

Wait. I thought everyone was building their own offline harnesses that way

I have the same problem with my project on arc agi 3 and my human skills....the transition phases on my tool world model… And that’s exactly what I developed determinism, and micro NN for unknown situations