Ryzen AI Halo

The thing is - you ain't buying a system that runs you frontier grade models for $1200/$2400. For $1200 you have a choice between R9700 or Arc B70. So 32GB VRAM and around 600-640GB/s memory bandwidth. This will smoothly run you Qwen3.5 27B or 35B MoE, both roughly comparable to Anthropic's Haiku, a

Intel already competes well with AMD in the high end of client with their current core counts. AVX-512, bLLC, sure though, that's catching up. The appeal behind unified core isn't that it's one architecture with different physical design layouts like what AMD does (though I'm sure Intel bean counter

If Nvidia thought RTX Spark would cause the bubble to pop, why on earth would they release it? Same for AMD. Nvidia, AMD, the hyperscalers, the neoclouds, and the labs all know that the frontier models are exclusive to datacenters and even if they weren't, these 1-2 year old chips wouldn't be able t

The AI environment right now is still very new. Local hosting LLM is still a very niche market. For 99% of the AI users, it's still better off going subscription route, because self hosting LLM requires hardware like DGS Spark, AMD Strix Halo or RTX spark now, which is very expensive and the model t

https://github.com/kyuz0/amd-strix-halo-vllm-toolboxesUse this. Its easy to use and well maintained but vllm is not as performant as llama.cpp with vulkan

Prompting is a pain in the ass. No one wants to do it. It’s truly a dark art, painfully tedious to experiment and find what works unless you build a testing harness that’s more or less automated and have the model run its own experiments and maybe use some standard benchmarks or invent new ones, gra

I don't think there's any doubt that TheDrummer's 123b models are great. At least for me, it just comes down to practicality. 70b models at q4 are a comfortable speed and q8 is right on the edge of being too slow. I've tried running behemoth v1.2 but the prefill and generation speed was painful even

not my research, but I did find this person who did load it into a strix, and gave alot of datapoints: Running DiffusionGemma on AMD Strix Halo and Decade-Old Tesla P40s | TinyComputers.io Seems it is not as much of a speed boost without a dedicated GPU.

[hal0] Installation on CachyOS (Arch-based) – detailed failure report I attempted to install hal0 v0.9.2 on a CachyOS system (Arch-based) and encountered several blockers. This post is to document the issues for the developers and for other Arch/CachyOS users considering this platform. Hardware: HP

Nice! I'm using 35B A3B with no thinking if I need fast. I'll give it a try, but I doubt that I'll get anywhere near 250t/s since I'm on an AMD Strix Halo.