AI is Burning - 5 New Papers

As soon as the Attention paper was published, industry and academia forgot 70 years of principled AI, ML, math, CS, neuroscience research. Now LLM investments are too big to fail. 😢

LLMs are excellent tools for accessing and assessing huge data sets, the folk who believe they will become intelligent however are almost always motivated by their next round of funding.

I find it hilarious white space and the lack of using proper trim() functions even affects fancy AI. This is has been pet peeve of mine in programing for decades. You have to always assume spaces will be lingering about since humans can't see them.

Maybe Yann LeCun was much more correct than the hype machine gave him credit for in regards to LLM’s

Self-consistency... I once asked an LLM that claimed to be capable of writing stories to just... describe a character for a story. It gave them short sleeves and described the sleeves as being wrist-length in the next sentence.

I got a version of the inconsistency problem from multiple different models all asserting confidently that on a 38th floor balcony you could see over top of a 55 floor building - it was interesting that they consistently reached the same wrong conclusion

Feeling that only your channel can bring AI-Awareness content about research with value-for-human as a center focus, not the technique it-self. You are really concern about the ability to apply those technique into real useful application for developer or tech business people. You are a gem sir!

Autoformalization (translating natural language into formal language) has always seemed liked a useful use case for LLMs since you don't need them to reason but to get the format into something we already have tools to use. The unfortunate thing is that it doesn't seem that LLMs can even do that accurately at this point. Like there was this AutoEval paper published in the ICLR this year on this.

That representation reminds of university classes like expert systems where we had to learn ProLog. Quite a resemblance.

I suspect that in 10-20 years, we'll again find out that we have approached LLMs like every other aspect of AI research: ass-backwards. I wonder if we will have another AI Winter once the funding dries up.