M5

It depends if you’re talking about dense models or MOE models. On 70B+ MOE models, you have benchmarks above that, and you won’t get better performance with a single GPU with 24 or 32GB of VRAM and the rest in RAM. The only time you’ll do better with GPUs is if your entire model fits in VRAM, includ

Ci sono dei MiniPc col Ryzen su base Strix Halo + 128GB di RAM che grazie alla possibilita' di assegnare uno spazio Vram fino a 96GB permettono di "smanettare" benone con l'IA (e non solo) senza spendere diverse migliaia di euro...tipo questo: https://www.notebookcheck.it/Mini-PC-con-AMD-Strix-Halo-

my suggestion is a minipc using Strix Halo. the 8060S gpu built in to the system runs about as well as a 4060 (so 1080p medium-to-high in most AAA games) GMKtec EVO-X2 AI Mini PC AMD Ryzen™ AI Max+ 395, $1699 (64gb) to 2699 (128GB)https://www.gmktec.com/products/amd-ryzen%E2%84%A2-ai-max-395-evo-x2-