Is Local AI Coding Actually Good?

Get started with MindsHub Cowork for free (it's open-source): https://mindshub.ai?utm_medium=influencers&utm_source=youtube&utm_campaign=26q3-tech-with-tim&utm_content=integration-1

Drop the middle man: Learn to use llama.cpp directly - Router mode or with llama.-switch..

The biggest thing on offer by local models is privacy. Using enterprise cloud models means sharing whatever data you are working on with a 3rd party which is some cases, isn't feasible due to laws, regulations, or company policies.

On two DGX Sparks running Deepseek V4 Flash 284b full native precision (MXFP4 + FP8 QAT model), 60 tok/s decode, 2k tok/s prefill, and ~2.2M tokens of total kv-cache right now. It's been shockingly stable in different 24/7 agent and is very good at non-vision coding tasks. Can't wait for the final checkpoint releasing soon with vision.

You are not limited to a model that fits in VRAM, llama.cpp + MoE model + MTP = sweet

Doesn't feel like you've answered the central question of the video. . You created a oneshot game and 1 of 2 didn't work an you say the reason might be the harness / model compability. So what should be the conclusion? My experience: Without promoting local AI, I make totally different experience 🙄 . That video is more a: Look, I have spent 10 minutes with a local model and this is what I have collected! Little bit of a timewaster...

nice video but let me correct you on something. Several points, but I'll go through one at least One of them being the CPU/GPU divide or in other words loading a part of your model on RAM. It's totally doable and you won't be facing 2-3t/s at all (unless you use your swap for this). I run my context window and my initial run of qwen3.6 (24gb) in size with 150K context window on my 8GB VRAM and the rest is being loaded on my 32GB RAM and I hit more often than not around 40-50 t/s. With the context window full it may drop to 20 t/s but it's very well above what I expected. The tricky part is to not use ollama or lm studio, but llama cpp (with llama swap) and on linux (dual boot laptop)

11:51 like that joke: - quick: add 1+1 - 3 - what?!?! - did you want speed or precision? LOL😂

Thank you, sir! I appreciate honest eyes-open takes on running open source LLMs on the hardware setup quality that most of us actually have. "Get the A.I. model for the rig you actually have, not the rig you want to have." :)

another important aspect is the license, just a few are really free to use for making profit