DGX Spark

You wont buy it https://www.microcenter.com/product/699008/nvidia-dgx-spark

Welcome to the club. The harder thing to resist is buying a second one. forums.developer.nvidia.com/c/accelerated-computing/dgx-spark-gb10/dgx-spark-gb10/721 awesome resource Current top-pick for single spark: Qwen 3.5 122B-A10B, Qwen 3.6 27B (with MTP) or even the 35B-A3B if speed matters more. htt

Post on over to r/localllama or check out the wiki there for setups. DGX sparks are not an inference powerhouse for the $$$$ you’d spend. It’s not an enterprise grade system for 1500 users either. It’s for local, personal prototyping on the same environment as you’d deploy when you deploy to a large

Believe me, I have been waiting for ROCm to be in a state where it is not painful for researchers to use, much less ready for consumers expecting the using the libraries off the shelf, but I have waited more than 10 years and ROCm is still about as matured as it was 10 years ago.The "good" news thou

RAM prices have skyrocketed after OpenAI bought 40% of the memory wafers, and NVIDIA is producing less consumer GPUs. They want the compute concentrated in cloud datacenters under their control. Self-hosted local AI is the way to go – Mac Studio makes a 512GB model and there is DGX Spark and AMD Str

Best a Mac mini can do is 64GB. They can only use a Pro M SOC. To get a 128GB M4 Max SOC you need at least a Mac Studio (or a MacBook Pro). At that point it is hard to say no to the Nvidia DGX Spark @ $5k. That said wait until March 4th to see what new goodies Apple drops. I read rumors there could

If you want long context processing, Mac is not a good choice. Actually, the worst among other choices, because of extremely slow reading speed. Note that this is different from the TG speed you mentioned. But for your use case, it is far more important, as others have mentioned. So Prefill speed (P

Training using the full Nvidia professional stack absolutely justifies it for a certain demographic. You're not it. You aren't going to train anything on a 5070ti with 256GB/s memory bandwidth. Also training is done with Fp16 so 128GB RAM limits you to about 60B parameters. People doing training use

Yeah, forgot to write that in. For sure, to me it seems like a no-brainer decision to make, more so to Nvidia than AMD. The die space contributing to the memory controllers IMO wouldn't really dent Nvidia's bank, although I suspect is a move they've done on purpose to keep customers on their DC prod

At that price why buy it over the DGX Spark or one of the partner systems? You don't get access to the Nvidia software stack and no ConnectX-7 for clustering. The only benefit is the RYZEN AI Halo box can run Windows and seeing it's x86_64 it has uses outside of AI; but at $4k there are plenty of be