Open 32B w/ AutoMemory beats Opus: HOW? (Stanford)

Interesting, useful, but we need methods that are not dependent on frontier models

It’s a bit uncanny how often your videos feel like they are crawling my recent agent logs to explain and expand the exact thing I’m working on at the time. 😂

So rather than just using Claude, I can now use Claude and Qwen and distill Claude knowledge into a LoRA for Qwen, and then I basically get near the same intelligence as Claude, provided that I keep interacting with the model about the same thing, and I only get this experience after some usage and training. Or ... I could have just used Claude. Sure, Claude is money, and Qwen is free, but even with these training loops, I still have to pay for Claude. If I have a Toyota car, and I want it to run better, it's like I'm being told to buy a Ferrari, take pieces out of the engine of the Ferrari, and put them into the Toyota, and then the Toyota will nearly run as good as the Ferrari. But if I bought a Ferrari, why wouldn't I just drive the Ferrari and ditch the Toyota all together? All this is is perpetual distillation, in which I pay for Claude to distill to Qwen to almost function at a Claude level. Interesting that it works, but I have better thing to waste my time with that provide a higher value of return. Not to mention, what about all the time I use Qwen pre-training, the wait time during traing, the post-trained usage where I'm confident in Qwen's results, yet might be engaging it on a topic it hadn't yet been trained and get a bad answer. None of this is useful.

What are the best Hermes agent memory addons? Is obsidian enough?

Can you check out the StateLM and CaveAgent papers?

It's LLMs all the way down.

More good stuff here. 👏

One of your future videos will be about abstracting  and three piece or more structures broken into math.... 🙂

Can't wait to see when 1B parameter model with well trained memory skill will outperform all of the beefy Terra-scale models, that require ton of memory and compute to run. This is really way forward - teach a model how to condense and distill the knowledge from the noise, instead of tuning billions of parameters by treating noise as 'useful data to learn' during training.

More epistemic foraging!