The data black hole at the center of AI

My baby had seen three sinks before she knew what the completely different looking sink at the library was.

The objection to Karpathy's argument regarding evolution pre-training may be incomplete. The parameter encoding may not be restricted to the genome. There may be significant encoding in the geometry of the gamete, the distribution of its organelles, the enzymes, and other data-encoding molecules. Without the initial machinery, the genome is not sufficient to instantiate biological intelligence.

dwarkesh, its about time for this year's Trenton and Sholto podcast

Maybe the real ASI was the friends we made along the way...

4:47 You should remember the interview you JUST did with David Reich where he pointed out the epigentic coding the tells our genome the context in which any given gene should be expressed which amplifies the computational complexity of the genome MASSIVELY.

It seems like you will be getting Yann LeCun on your podcast. Your thesis here aligns with his vision on the future of AI

"We have models that are 5T parameters" >What did Dwarkesh saw?

Dwarkesh finally interviews himself and it's his best video yet

The next big leap for AI is to reach at least 0.000001 of the learning efficiency of the Human Brain.

I think we should probably be very concerned about this sample efficiency gap from a safety perspective. Supposing that the labs are correct that their automated R&D could realise algorithms with human level sample efficiency, then we might very suddenly move to a world where the relative quantity of available data jumps by orders of magnitude without much warning. What would it be like if a human could spend thousands of years learning? We have no idea.