Was "Machine Learning 2.0" All Hype? The Kolmogorov-Arnold Network Explained

Still waiting to wake up and realized all of this was just a dream.

What KAN is really cool for in my opinion is to find mathematical functions between data where there didn't exist any in the past. And since we know a lot about mathematical optimisation and things like the Taylor/Fourier series, we could theoretically calculate the input-output relationship much more cheaply (inference becomes commodity). Training would be more expensive however

Are you sure that a KAN will save VRAM? Yes, you need less parameters, but unless I misunderstood the video wouldn't a KAN need much 'bigger' parameters than a highly optimized MLP? A function should need a lot more bits to store than a 4-bit or 8-bit parameter.

Ok, but when Kan we use it? :^)

I really like that kan isnt a black box. Thags huge for alignment

Is it KAN-enough? We don't know but we'll find out eventually!

Streamline AI task delegation with HubSpot's Free Playbook: https://clickhubspot.com/9yu and check out my newsletter 😎 https://mail.bycloud.ai/

In mathematics we have "Generalized linear models". The simple explanation is that we know linear regression. What they forget to teach (not always) is that in order for that to work, all parameters and the result should have the should have the same distribution. For example Normal. What happens when they dont. We have to transform the output of the regression from one Distribution to another (or the other way around). This is easy for exponential distributions. Those S functions (or relu) are transformers from Normal to Categorical (we call that logistic regression). But that is not alway accurate ofcourse. It has been proven good enough though. In theory we could have different transform function that better map between those distributions. So the idea is pretty simple and I guess for many cases where logistic regression is obvious, it will fallback to obvious S-like functions. It would be interesting if that could be adaptive. Mean starting with simple Relus and by some criteria increase the Spline points etc

I've also been hearing a lot about "liquid networks". Been filling my YT feed lately. It'd be cool if you could make video on that.

Great analogy for curse of high dimensionality! I’ve never heard of the term, as I’m not in ML, but your analogy was easy to understand