Ryzen AI Halo

The thing is - you ain't buying a system that runs you frontier grade models for $1200/$2400. For $1200 you have a choice between R9700 or Arc B70. So 32GB VRAM and around 600-640GB/s memory bandwidth. This will smoothly run you Qwen3.5 27B or 35B MoE, both roughly comparable to Anthropic's Haiku, a

From what we know, it is physically impossible. The placement of the RAM chips themselves is an important part of how it achieves an extremely wide memory buss at low latency. According to Framework, when they first looked into making a computer with Strix Halo, they asked AMD to try and find a way

Unified memory can, and in Apple's case, does mean you can use the same data in CPU and GPU code without having to move the data back and forth. Apple Silicon has a memory bandwidth of 68 GB/s on the M1 chip (non-Pro/Max), the slowest processor package for macOS-operated computers, e.g. the MacBook

The Framework Desktop is an AMD Ryzen AI Max+ 395 board. It doesn't go higher than 128GB, so it's almost definitely just 128GB.

There are lots of options but The Steam Machine is typical pricing for sub 3 litre machines. A step up would be Atomman gaming options from Minisforum, high end would be the Rog NUC range and the new Ryzen AI Max+ 395 machines. I suspect there will be Panther lake options out there soon. For indie a

don't know how fast lpddr6x is gonna be, but lpddr6 should be at least 14 Gbit/s. random article talking about lpddr6 samples: https://videocardz.com/newz/sk-hynix-details-16gb-lpddr6-at-14-4gbps-samsung-sends-lpddr6x-samples-to-qualcomm and again crucial to remember, that lpddr6 camm style modules

GPU is not really the bottleneck for large models on a M4 Max or M3 Ultra. It's RAM/VRAM bandwidth: the same bogeyman that haunts the DGX Spark, AMD Strix Halo, and other platforms finally catching up to Apple in the "Unified RAM" game. No matter what you choose, you're going to encounter limits. Wh

Ive found the atrributing slowdown in Vulkan for AMD, im pretty sure it effects Intel GPUs too, Id be keen for someone to test an experimental branch I have, The slowdown ive found is caused by Memory channel camping, not all memory channels engage causing the prefill slowdown. https://github.com/Na

Right off the bat you will have less capability on a local system. You just won't get something equal to the large frontier models running on crazy hardware. You might be able to get something that is good enough for your use case. You will probably have a bad time with only 6gb of vram but you can

With that hardware you are very limited. Realistically, 8B models on q4, q5 like qwen are your safe limit (I am guessing it is a 12GB vram 3060), above that the model would spill into the system ram and therefore inference speed will drastically go down. One model I encourage you to try is Gemma 4 1