Ryzen AI Halo

Unified RAM, that's basically it. If Apple Silicon didn't have unified RAM then no one would be using it for AI, they'd instead use AMD Strix Halo or Nvidia DGX chips, both of which have some advantages over AS, like token prefill speeds and CUDA access.

Da, uzeo sam si AMD Strix Halo miniPC sa 128 GB unified RAM (GMKTec EVO-X2), prilična je zvjerka i možeš alocirati do cca 100 GB RAMa za GPU. Napravio sam za probu web-baziranu igru u jedan dan crunchanja sa OpenCode i Qwen-3.6-35B-A3B. Dobro radi i dovoljno je brzo. Reda veličine 50 tokena/sec na p

And, critically, buy 'unified memory', we're talking about devices like a Macbook with lots of ram, or AMDs Strix Halo. None of which are cheap, but both run Qwen mixture-of-experts models pretty damn well.

It might not have lived up to the hype but it gets the job done. People also underestimate how much noise, power and heat a system with duel full height graphic cards put out. The Mac Studio and DGX Sparks give you very capable systems in a small convenient package. AMD Strix Halo is also an option

APU even have the compute to do AI in a reasonabe amount of time It's not the worst. The thing is with bigger models, you ideally want the entire thing in memory. That's why you'll find articles about how this runs the bigger Deepseek models faster than a 5090 or whatever. I'm kinda tempted to get o

Benchmarks speak louder than words: Dgx Spark: https://spark-arena.com/leaderboard Framework: https://kyuz0.github.io/amd-strix-halo-toolboxes/ (llamacpp) and https://kyuz0.github.io/amd-strix-halo-vllm-toolboxes/ (vllm) Apple Chips: https://omlx.ai/benchmarks The advantage of VLLM, which is NVIDIA-

Two AMD Ryzen AI max+ 395 wow just wow, to bad no fsr4. I’m thinking of honestly just waiting for the Medusa Halo to come out since it’ll have the knew RDNA 5 or whatever they’re calling it.

the APU with up to 128GB of shared memory would allow you to load some decent medium-sized models with usable tok/sec speeds. according to here: For compute and memory bandwidth, you can think of the Strix Halo GPU like a Radeon RX 7600 XT, but with up to 124GiB+ of VRAM I googled a bit for some LLM

Owned or benchmarked most of these in the last 6 months. Practical take: 2x AMD R9700 (32 GB each, 64 GB total VRAM) - this is the best price/performance right now if you're okay tinkering. ROCm 6.4 + vLLM 0.7 with tensor parallelism gets you ~55 tok/s on Qwen3 32B FP8 with 64k context. Cross-GPU wo

I disagree. Between MoE and the new VibeThinker-style LLMs, we're seeing better outputs at dramatically reduced training and inference times from even very small 1.5B models. Admittedly, they do better when the problem they are solving has a deterministic outcome like coding, but nevertheless "small