Orchestration Over Architecture: What Stanford Found

A wrapper now becomes an architecture 😂

The truth is: we software engineers love making simple things unnecessarily complicated, inventing new buzzwords for concepts that already exist just to sound more “cutting-edge,” and over-engineering stuff that should stay dead simple for one reason only: ego. And honestly? About 90% of the “new” AI engineering best practices that pop up every single week are straight-up bullshit. We’ve always done this as developers, but with the fear and anxiety AI created, this nonsense has probably tripled or quadrupled over the last year. What’s even crazier is seeing all these “AI gurus” selling AI courses and even full-blown graduate programs, while the entire space is still chaotic and nobody has — or will have — a definitive answer anytime soon. LLMs are getting major updates literally every week. And a lot of those updates are already shipping with built-in support for many of the “advanced practices” these gurus are trying to teach. Meaning people are paying thousands of dollars to learn workflows that will probably become obsolete in a matter of days because the models themselves will handle most of that orchestration natively by default. These guys are teaching things that could be outdated by next week. Nowadays, the first thing I check before consuming AI-related content is the upload date. If the video is two weeks old, there’s already a huge chance it’s outdated or full of nonsense. Now imagine an “AI graduate program” whose curriculum was planned six months ago…

My favorite engineer-tuber.

Haiku beating Opus with a better harness would be a big deal, but it doesn't appear to be shown here. (Re: 9:33) In the paper 76.4% was for Meta Harness on Opus not Haiku. Haiku was only 37.6%. "On Opus 4.6, Meta-Harness discovers a harness achieving 76.4% ... On the weaker Haiku 4.5 model, the improvement is larger: Meta-Harness achieves 37.6%"

Thanks for the very good content 🌹and also thanks for use dark background 🌹🌹🌹

Great presentation and thank you for your insights. What software do you use for this beautiful slides'

1. How do I know within Hermes with Deepseek v4 what is in my context that shouldn't be there? 2. The same for tool use.. is there a tool or skill that produces insights? 3. Same with verifiers and logic, where do I look for that within Hermes?

Audit harness before even thinking about model switch is the biggest takeaway from this video and hands down the best approach. Usually devs offset the agent issues by switching to a bigger and capable model which may hide the underlying issues but sooner or later they would come out in the form of edge cases. I also recently published a write-up on how we used Neo AI engineer to optimize a customer support RAG agent. It covers architectural changes like RAG similarity score optimization, context enhancement that were implemented before even considering a model switch. The changes alone led to 19% performance increase and model switch led to 79% cost reduction. So I guess both go hand in hand.

Awesome stuff keep it going with this harness engineering stuff you've convinced me to work on my own harness now, would you recommend starting with PI as a good minimalist base or is that still too much, best to start with even less?

its an interesting take. ive seen plenty of videos explaining the same thing, but with different words. seeing another perspective on the same ideas is good. give it a year and there will be a "standard" way to communicate these concepts, likely with several new words or abbreviations for abstracting specifics.