Ryzen AI Halo

The thing is - you ain't buying a system that runs you frontier grade models for $1200/$2400. For $1200 you have a choice between R9700 or Arc B70. So 32GB VRAM and around 600-640GB/s memory bandwidth. This will smoothly run you Qwen3.5 27B or 35B MoE, both roughly comparable to Anthropic's Haiku, a

Unified RAM, that's basically it. If Apple Silicon didn't have unified RAM then no one would be using it for AI, they'd instead use AMD Strix Halo or Nvidia DGX chips, both of which have some advantages over AS, like token prefill speeds and CUDA access.

Da, uzeo sam si AMD Strix Halo miniPC sa 128 GB unified RAM (GMKTec EVO-X2), prilična je zvjerka i možeš alocirati do cca 100 GB RAMa za GPU. Napravio sam za probu web-baziranu igru u jedan dan crunchanja sa OpenCode i Qwen-3.6-35B-A3B. Dobro radi i dovoljno je brzo. Reda veličine 50 tokena/sec na p

Intel already competes well with AMD in the high end of client with their current core counts. AVX-512, bLLC, sure though, that's catching up. The appeal behind unified core isn't that it's one architecture with different physical design layouts like what AMD does (though I'm sure Intel bean counter

If Nvidia thought RTX Spark would cause the bubble to pop, why on earth would they release it? Same for AMD. Nvidia, AMD, the hyperscalers, the neoclouds, and the labs all know that the frontier models are exclusive to datacenters and even if they weren't, these 1-2 year old chips wouldn't be able t

The AI environment right now is still very new. Local hosting LLM is still a very niche market. For 99% of the AI users, it's still better off going subscription route, because self hosting LLM requires hardware like DGS Spark, AMD Strix Halo or RTX spark now, which is very expensive and the model t

https://github.com/kyuz0/amd-strix-halo-vllm-toolboxesUse this. Its easy to use and well maintained but vllm is not as performant as llama.cpp with vulkan

And, critically, buy 'unified memory', we're talking about devices like a Macbook with lots of ram, or AMDs Strix Halo. None of which are cheap, but both run Qwen mixture-of-experts models pretty damn well.

Prompting is a pain in the ass. No one wants to do it. It’s truly a dark art, painfully tedious to experiment and find what works unless you build a testing harness that’s more or less automated and have the model run its own experiments and maybe use some standard benchmarks or invent new ones, gra

It might not have lived up to the hype but it gets the job done. People also underestimate how much noise, power and heat a system with duel full height graphic cards put out. The Mac Studio and DGX Sparks give you very capable systems in a small convenient package. AMD Strix Halo is also an option