MedAI without Reasoning - but right Answer. HOW?

In order for an llm to start by "no" as a first token, when answering a question, it must have already the answer before generating the whole paragraph. Which mean it reason beyond token generation. Also the network is a stateless database we query at each loop of the chat by reintroducing the whole context back. But we have known since the beginning of cots that replacing the cots in the context still leads to correct (or invalid result) irrelevant to the cots. But with experiment after deepseek grpio paper, we know that saddle specifically replacing saddle transition word shift the answer. To understand, it's like asking the model to reason in haiku. The surface constraint is irrelevant to the content, since the network itself is stateless, the encoding within the context is beyond surface appearance. Hence why the reasoning trace is similar, the code is elsewhere. An experiment on training a student by a teacher model, that likes owl, only on generating random number, has the student exhibiting the same behaviour, despite not being trained on anything but number generation. I always go against the fact that llm are next token predictor, they are not, they are concurrent concepts discriminator, which people naively call pattern matching, because the assumption I that reasoning is linear and sequential. But you can also do conceptual battle royal and auto regression shift the discriminator criteria to find the current next token. Ie it's closer to a parser tree. But eith sophisticated internal dynamics.

Whoa this is fascinating! There is a parallel with us: it's kind of like how humans start with emotions (based on potentially decades old triggers and long standing neural pathways) and then we look around and try to fill in the gap with why we feel this way (and we are often so wrong lol) "I feel lonely ... it must be because my husband didnt pay attention to me earlier when he was going to work. Yeah thats it! I can justifiably get mad at him!"

There is one big mystery: Where are all those ideal doctors to be found? Could it be that theoretical physicists are not too grounded in reality? I’d pour 10X more money into the learning of AIs than that of doctors, but evidence-based assessment of medical practise seems to be taboo. I’m not joking.

So, what needs to change in the way we train the models to make them learn "the more objective/grounded reasoning" (not just statistical relations of tokens/pixels which have no physical/emotional meaning to them)? Especially, considering that even though human brain is a powerful "prediction machine", we still train kids to reason about the world and life, and yet depending on the "fine-tuning" (eg. family, religious, scientific) that kids receive, their reasoning pathways wildly differ even within local groups. Also, about "expressed (!) reasoning": if humans were not bound by the risks of loss (reputational, economic or physical), don't your think their "reasoning traces" would also take very diverse routes? Think about the level of expressivity we allow in the spectrum of "med/weapons ... civil engineering ... arts", ie. Why do we expect the models trained in a general way (and lobotomized with specific guardrails) to be good at "reasoning" at all?

Por mi experiencia trabajando con modelos hospedados en diferentes tareas, casi todas agenticas, debo decir que no sirven de mucho, al final, todo se reduce a la potencia del Llm, es decir, los millones de parámetros en los que fue entrenador aún así, a pesar de rodearlos de arneses, con configuraciones de todo tipo, reglas generales, workflows definidos y skills al detalle para cada workflows, para 13 dominios de tareas que se le pueden pedir, como páginas web, software educativo, desarrollo de juegos, investigación, etc...y otros tantos arquetipos dentro de cada dominio como saas, moocs,k-12, etc..(para el dominio de software educativo) y así para otros 12 dominios, en mi afán de llevar una orquestación de varios agentes hasta la consecución de la tarea. Pero aun así, fallan estrepitosamente a pesar de añadir un agente scope que selecciona el dominio y el arquetipo y que te hace preguntan con el fin de clarificar de manera inequívoca que es lo que se le está pidiendo como tarea. En definitiva, no se sabe a ciencia cierta si el cuello de botella es el humano, sus prompts, los datos de su entrenamiento o las matemáticas de sus razonamientos internos. Cada vez veo menos plausible, al menos de momento que puedan sernos útiles a corto y medio plazo, a pesar de que cada vez que analizas y explicas un nuevo paper científico, parece que este un poco más cerca, que si graphrag,que si Turbo quant, que life-harness, que si H-Mem, que si esto nuevo, que si esto otro que salió ayer....Al final, nos damos de bruces con la realidad de que a pesar de que todos insistimos en lo inteligentes que son, no resuelven una tarea de principio a fin de forma determinista. Al menos esa es mi sensación y experiencia con modelos auto hospedados en un pc de gama media. Dios quiera que cuando podamos acceder a los nuevos pc de 128gb de VRAM integrados a nosecuantosmil petaflops por segundo, podamos incrementar la eficiencia/eficacia de los Llm en tareas end to end sin errores de atrasados mentales. Saludos. Sigue así, al menos descubrimos nuevas investigaciones interesantes que intentan indagar sobre el pozo oscuro y sin fondo que los los Llm actuales.

Isn't probability a kind of reasoning? It's not logical but it is statistical? So if an LLM is trained on human reasoned diagnoses, then the reward signal will point toward the most probabilistic conclusion which acts as a proxy for reasoning. Then the LLM finds the reasoning that was used and presents that as if it were its own cognitive process. So this isn't really anything new yes? We understand there is no actual logical reasoning going on in any of these models for any domain, medical or otherwise.

You highlight the criticism—what is your proposed plan for improvement?

I wonder if and by how much temperature settings changed the similarities if it did at all. Also curious how it compares to non human examples of thinking topologies

The invisible hand writ large.