M5

also bought Bosgame M5 with 128G. its enough for testing and also running some "production" for example in n8n. There are another few important points: 1) electricity - i meter up to 80-100W when ollama running (had 3090 before and whole pc was 550+w), 2) compact size, 3) when you find you dont need

I thought about building a suitcase pc for my son before his next deployment. But there's sweet mini-pcs out there now with the AMD AI 395 Strix Halo that can game better than most gaming laptops. They use integrated graphics the 8060s, with similar performance to a 4070. Bosgame M5 AI Mini DesktopG

It depends if you’re talking about dense models or MOE models. On 70B+ MOE models, you have benchmarks above that, and you won’t get better performance with a single GPU with 24 or 32GB of VRAM and the rest in RAM. The only time you’ll do better with GPUs is if your entire model fits in VRAM, includ