Phase Transitions in Agent Memory: Recurrent Memory

A date-index overview. In the date overview each date lists different keywords that will be used for the index search. In the index overview, beneth these keywords are other dates listed that can be used to pull "memory" into a context.

Thanks as always for guiding us through these interesting papers. As I understand this one, the compute involved in creating the three containers of memory, evaluating this phase transition, and the notion that recurrence equals salience, all of this is problematic. And once you've massaged all this data you still only have whatever attention makes possible, you haven't somehow magically increased what's the model can remember, "subconsciously"or otherwise, is still limited to whatever fits in the context window. It's optimized ICL, but at the cost of quite a few tokens. Once you've got it all set up I'm sure you have a nice efficient process at inference, but you have to include the compute cost of preparing everything for this memory approach to yield a bottom line cost benefit.

I wonder if those "memories" could be compressed in theme blocks. Using a "relationship-factors"(?) to decribe how close different memories are connected. A kind of similar how LLM work, just simpler and for "Memories"/Textblocks.

Shouldn't the compression apply to memories that aren't often used just as well, though somewhat different in LLM prompting/approach and an analogous hyperparameter search?

is the density threshold learned or hardcoded? because 87% savings means nothing if consolidation drops the one fact the agent actually needs at runtime

I think we need recursive tokenization methods (recursive residual quantization) with these types of methods, so that the manifolds can be interchanged without breaking - but I think the gap is still effective RL harnesses - temporal functions only get you so far - maybe these state spaces need environmental/domain/modality grounding.

THX

How do I connect with you to assist with a project I am working on?

14:45 Finally. An accredited scientist backing up allergies as weakness.