DGX Spark

NVIDIA did - they call it the DGX Spark. I have one. 128 GB of LPDDR5X memory to the Grace Blackwell chip. $3999 retail. Apple could smash this thing to bits with an M6 with a bajillion neural cores and the same RAM. They could price it at $1999 and take the whole market. They’re already doing it wi

Firstly you should understand that the Ryzen AI Max+ 395 chip is not intended to be an all-purpose AI chip. It comes with a lot of memory, but does not support CUDA, does not have a lot of compute, and most importantly does not have a lot of memory bandwidth. Not supporting CUDA means the options fo

RAM prices have skyrocketed after OpenAI bought 40% of the memory wafers, and NVIDIA is producing less consumer GPUs. They want the compute concentrated in cloud datacenters under their control. Self-hosted local AI is the way to go – Mac Studio makes a 512GB model and there is DGX Spark and AMD Str

Believe me, I have been waiting for ROCm to be in a state where it is not painful for researchers to use, much less ready for consumers expecting the using the libraries off the shelf, but I have waited more than 10 years and ROCm is still about as matured as it was 10 years ago.The "good" news thou

As someone who runs local models on a Mac Studio and Nvidia DGX Spark you aren’t running anything close to foundation models from a year ago on a Mac Mini; especially one with 32GB of memory. You need something with a minimum of 64GB to fit a large enough context window and run something like Qwen 3

Best a Mac mini can do is 64GB. They can only use a Pro M SOC. To get a 128GB M4 Max SOC you need at least a Mac Studio (or a MacBook Pro). At that point it is hard to say no to the Nvidia DGX Spark @ $5k. That said wait until March 4th to see what new goodies Apple drops. I read rumors there could

I'd love to show the entire desk, speakers, etc. but I'm afraid everything surrounding is a bit of a mess 'PC' is an Nvidia DGX Spark. Needed something with plenty of memory, strong CUDA support, & something small I can take with me for travel. It is a hair better than a 9950x in CPU performance & a

Honestly it's fairly niche. It's meant for local LLM development/testing when working with large datasets (that's why it comes with 200Gb/s networking). But it's actually pretty damn bad at actually running LLMs (as in, you can load a large model but it's really slow), you want a beefier workstation

If you want long context processing, Mac is not a good choice. Actually, the worst among other choices, because of extremely slow reading speed. Note that this is different from the TG speed you mentioned. But for your use case, it is far more important, as others have mentioned. So Prefill speed (P

It competes with things like Nvidia DGX Spark, which has 273 GBs (and a lot less cores). It's not a bad product, the worst thing is that AMD's ecosystem is not as large as Nvidia's.