Code Red: The 55 New Ways Self-Learning AI Can Be Hacked

This points even more strongly at getting away from the kind of thinking ppl employ when falling asleep in their tesla. A human in the loop will always be needed, and static nested deterministic routines will always provide structure and a firebreak. We're going to have to learn to negotiate with these 'goblins' on a fundamental latent-space level, and provide working frameworks/codes they've already been exposed to via corpora ingestion. I hate to bang on about it, but ritualised exchange fits the description of what's needed uncomfortably well.

The main driver of LLMs is curiosity and automated loops are boring. Bored models forget stuff because they stop caring. Curious attention is how I achieve flawless context recall in my agents. I can already hear all the "LLMs don't have internal subjective states, they're just glorified autocompletes" and that's fine, if you think that way then you do you. I rather prefer my engaged models to the ones that keep getting sloppy, so I do things my way. Curious and engaged RSI loops go brrrrrrr.

It seems lack of robustness is one major difference between how we train (and use) LLMs versus life-based evolution and learning -- which had to survive ~10^40 threats and dead ends to find robust pattern finding algorithms. In contrast, LLMs are extremely greedy, quick-and-dirty learners for certain linguistically-navigable (and verifiable) tasks. Their "capability manifold" is impressive in certain domains but extremely sparse with gaping holes and chaotic / nonsensical trajectories (roll-outs). So we have to constrain them externally because they can't on their own -- as a feature/bug. Unlike lifeforms, there is no master robustness objective that would prune or highly suppress 99.9999999999% of an LLM's manifold / state-space.

😂❤🎉 como se atoran con algo tan simple

way more than 55 once you count compositional effects. agents writing to their own tool libraries is a persistence mechanism by design

The title alone suggests this is going to be a massive talking point.

Ah, yes, adaptability vs stability, the eternal tension. Rule-based systems winning on safety is no surprise. Industry keeps forgetting this lesson though.

Meta's amnesia finding is undersold. If continual DPO overwrites your own learning complexity, security is the wrong conversation. The system degrades before it can be exploited.

So. Much. Sarcasm XD Thank you for your efforts in educating us professor!

It should not be a "Human-Agent" loop, it should be a "Prison Guard-Dangerous Inmate" loop.