Context-CoT: Forcing LLMs to Actually Think (No ICL)

What I noticed quickly and kept looking for that seemed obvious was relationships from one modality of concept recognition tied to another. Kinda like translating from construction knowledge and experience to that of an engineer. This is fundamental in how we teach our children as well as communication with co-workers giving them a leg-up in comprehension of how, what, and why as well as the scope of the "new" realm they are to apply themselves.

i think if you use constants across math domains, like the planck constant and the plastic constant, as the ground floor math your model uses for reasoning, you can use the pisot numbers (the constants) to build bridges across contextual domains. it seems to work well, the constants become the container and the sieve for context.

true dat

Ok so we’ll eeeermnm. I’m an idiot, I get concepts easily but I have not a clue about the maths that are discussed on these video - go learn I hear you say. Well while I can learn my ADHD super power and my quickly accelerating to 50 renders that path likely longer than I have years left. My request is, can you make a version of your videos for those who aren’t stupid but just not as not stupid as others? Like AI lite. I enjoy your delivery, there are just some of us that need to be more gently lead. Keep up the good work.

Isn’t this just rehashing the conclusions drawn by Marcus about long tails in training data? Reasoning and knowledge are closely intertwined in both agents and humans.

No link to the study 😢

08:23-13:21 You are mistaken. Reasons: 1. Classical mechanics in its developed form is much closer to modern theories than you think, you will understand this if you make a sufficiently detailed description of the concept of a classical body and ask the AI to translate it into the language of Lie group generators and you will see that many modern concepts (stratification, , well, or you can try to build a theory of gravity based on Kepler's laws and the assumption that gravity is an elastic reaction of a material body to a change in the scale of space, these assumptions alone are enough to obtain part of the equations of modern physics, and adding the hypothesis of the finiteness of the speed of gravity will give most of the effects of GTR. Therefore, a classical mechanic in your hypothetical space with the observed effects of string theory will not be as helpless as you think. 2. The reason for the shortcomings you are talking about is that when training LLM, the fact that it is an interpreter of language is ontologically ignored and it is trained to directly generate text instead of learning to work with text as a linguist working on the basis of a meta-description of a dialogue. (for example, what I I'm giving a linguistic model of language and, as a result, I'm describing procedures for tracking shifts in the context of a discussion or the meaning of terms. I also recommend pausing the analysis and asking questions (this greatly improves the quality of the neural network's reasoning).

language is to messy for ai. i think even if we got to agi or rsi we would bottleneck on compute because the foundation is bogus, its bogus because our brains are very flexible and can do all the translation layers needed to understand langauge, we can literally understand any language, but thats the isue, we never had to make them logical. either ai has to become as flexible as us or we need a new more logical foundation imo

LLMs are not bad because they “do not care” about context. They are unreliable because next-token prediction plus attention over a long prompt does not automatically create stable, auditable, persistent rule-following cognition.

Finally! Somebody gets it!