Two 128GB boxes, one bandwidth problem

Two machines arrived in the same window with the same headline number: 128GB of unified memory, addressable by the accelerator, on a desk, for a few thousand dollars. NVIDIA’s DGX Spark and AMD’s Ryzen AI Max+ 395 (“Strix Halo”).

Clients keep asking me which one to buy. The honest answer starts with something neither vendor’s marketing leads on: for token generation, these two machines are much closer to each other than either is to a real discrete GPU — and the thing that makes them interesting is capacity, not speed.

(Hardware figures below come from NVIDIA’s and AMD’s own documentation, linked where it matters. Prices are deliberately vague: the 2026 memory shortage is moving them fast enough that any figure I print here will be wrong by the time you read it. More on that under the table.)

The specs, side by side

Look at the bandwidth row. 273 versus 256 GB/s. That is a rounding error, and it is the number that decides how fast either machine writes tokens.

Then look at the three rows below it — native 4-bit, interconnect, clustering. That’s where the machines actually diverge, and none of it shows up in a tokens-per-second benchmark.

The price gap has closed

At launch, a 128GB Strix Halo box cost roughly half what a DGX Spark did, and that was a central plank of the argument for it. That is no longer true.

As I write this, 128GB Strix Halo mini-PCs are selling in the same neighbourhood as a Spark — the Minisforum MS-S1 Max 128GB at about $3,639 against a $4,549 list, and the GMKtec EVO-X2 at about $3,650 against a $3,999 list, are representative. At list price, some of them now cost more than a DGX Spark.

The cause is the 2026 DRAM shortage, which has pushed memory prices up hard across the whole industry — Apple has raised Mac and iPad pricing, and contract memory has in places roughly doubled. A machine whose bill of materials is dominated by 128GB of LPDDR5X absorbs that increase almost directly, while a fixed-price appliance absorbs it more slowly. The gap didn’t close because the Spark got cheaper.

Two consequences. First, any specific price in this post is probably stale by the time you read it — check before you plan around it. Second, and more important: the “AMD is simply the cheaper option” argument is much weaker than it was, and that changes the recommendations at the end of this post.

The technical comparison is unaffected. Bandwidth, FP4, concurrency and clustering behave the same regardless of what the machines cost.

Why bandwidth is the number that matters

Generating a token is memory-bound, not compute-bound. For every single token, the machine reads the model’s weights out of memory. So there’s a hard ceiling:

tokens/sec  ≈  memory bandwidth ÷ bytes read per token

Run it for a dense 70B model at 4-bit — roughly 40GB of weights:

273 GB/s ÷ 40 GB ≈ 6.8 tokens/sec, before any real-world inefficiency. Expect meaningfully less in practice. That is slower than you read.

Now compare against a current discrete gaming GPU at roughly 1.8 TB/s. For a model that fits in its VRAM, that card is around 7× faster than either 128GB box. Not 7% — 7×.

This is the whole story. These machines are not fast. They are large. You buy one because the model does not fit anywhere else at this price, and you accept the speed that comes with it.

Which is why mixture-of-experts changes the verdict

I wrote previously that MoE models are fast on modest hardware, not small on it. On a 128GB unified-memory box, that property stops being a footnote and becomes the entire reason to own one.

A dense 70B reads ~40GB per token. A large MoE with a small active-parameter count — say 100B+ total but only a few billion active — reads only the active experts, perhaps 3–5GB per token. Same bandwidth, same box, an order of magnitude more tokens per second, and a far more capable model.

128GB of relatively slow memory is close to the perfect shape for that class of model: enough capacity to hold every expert, and low enough per-token reads that the bandwidth ceiling stops binding. If you’re buying either machine, buy it for MoE inference. Dense 70B-class models will technically load and will disappoint you on both.

Where they genuinely differ: prefill

Generation is memory-bound and the two are near-identical. Prompt processing is compute-bound, and here they are not close at all.

Prefill is the pass over your input before the first token comes out. For a chat message it’s irrelevant. For RAG with 8k–32k tokens of retrieved context on every single query — which is the actual workload most of my clients are building — it can dominate the response time the user feels.

DGX Spark’s compute budget is in a different league. If your workload pushes long contexts through the machine constantly, that gap is the most important practical difference between these two boxes, and it does not show up in a tokens-per-second benchmark run on a short prompt.

The next section explains why the gap is as wide as it is — and why it isn’t something AMD can close with a driver update.

FP4: the difference that’s architectural, not incremental

The “~1 PFLOP FP4” row looks like spec-sheet garnish. It isn’t. It’s the one capability gap between these machines that you cannot close with better software.

Blackwell’s fifth-generation tensor cores implement NVFP4 in hardware. The format is a 4-bit float (1 sign, 2 exponent, 1 mantissa) carrying two levels of scale: an FP8 scale for every 16-value micro-block, plus an FP32 scale per tensor. The hardware handles the grouping, the dynamic scaling and the 4-bit matrix operation itself. NVIDIA’s published figures put the cost at about 4.5 bits per weight — roughly 3.5× smaller than FP16 and 1.8× smaller than FP8 — for under 1% accuracy loss against FP8 across seven benchmarks on DeepSeek-R1.

That buys you two separate things, and it’s worth keeping them apart:

The memory half is available to everyone. Fewer bytes per weight means fewer bytes read per token, which lifts the bandwidth ceiling from the formula above. Any machine that can store 4-bit weights gets this, AMD included.

The compute half only exists where the tensor cores speak FP4. And on AMD’s RDNA line they don’t. The WMMA instruction menu covers FP16, BF16, INT8 and INT4 — there is no FP4 form. A 4-bit model therefore gets dequantised back to FP16 in a prelude shader before the matmul runs, landing at exactly the FP16 throughput ceiling. The weights are small; the arithmetic is not fast. Strix Halo is RDNA 3.5, a generation older still — it predates even FP8 in that instruction set.

To be fair to AMD: this is an RDNA-branch gap, not a company-wide one. Their CDNA datacenter parts do carry native microscaled-FP4 matrix support. It just isn’t in the silicon you’d put on a desk.

So the honest summary is: FP4 narrows the generation-speed gap for both machines and widens the prefill gap dramatically. It lands squarely on the compute-bound half of the workload — the half I just said was the biggest practical difference between them.

The catch is that it isn’t free. You need the model to exist in NVFP4 (increasingly it does — prequantised checkpoints are published, and llama.cpp gained a Blackwell-native FP4 path) and a serving stack that dispatches to it. That’s maturing quickly, but “the hardware supports it” and “your stack uses it” are still two different claims. Check the second one before you pay for the first.

Serving more than one person: what vLLM does and doesn’t fix

Everything above assumes one request at a time. Most people buying one of these boxes intend to point a small team at it, and there’s a widely-held intuition that this is nearly free: decode is memory-bound, batching reads the weights once per step and computes for the whole batch, so ten users should cost roughly what one user costs. That intuition is correct on a datacenter GPU. It is where vLLM’s continuous batching and paged KV cache earn their reputation.

On these machines the crossover arrives much sooner than people expect.

vLLM’s own DGX Spark guidance is unusually blunt about it. The recommended recipe sets --max-num-seqs 4, and the reasoning is explicit: above roughly four concurrent decode streams “the per-token bandwidth tax can outweigh continuous-batching gains, and time-to-first-token spikes.” Their summary is that Spark is “better suited to small-batch inference than high-concurrency serving.”

Three things drive that ceiling:

Batching trades bandwidth for compute. That trade is a bargain when you have compute to spare. A datacenter part has headroom in both dimensions, so you can push batch sizes into the dozens before arithmetic becomes the limit. With 273 GB/s and a desktop power budget, the batch size at which you go compute-bound is small.

MoE quietly works against you. MoE is the model class these boxes are best at — but the advantage is that a single token activates only a few experts. Add concurrent users and their tokens route to different experts, so the union of experts touched per step grows, and bytes-read-per-step grows with it. The memory-bandwidth win erodes exactly as you add people. This is not a niche effect: NVIDIA’s large-scale expert-parallelism work exists specifically to solve it, by spreading experts across many GPUs to pool aggregate bandwidth. On one box, there is nothing to pool.

Prefill contention is what users actually feel. Concurrent requests each need a prompt processed, prefill is compute-bound, and TTFT is the number people perceive as “slow.” For RAG, where every query drags 8k–32k tokens of context, this is the first thing to degrade under load.

Usefully, vLLM’s measurements show KV-cache utilisation staying under 5% in single-user tests and under 30% under small-batch demo traffic on a 120B NVFP4 MoE. That’s worth sitting with: on a 128GB box you will not run out of KV cache. You will run out of memory bandwidth first. Capacity is the thing you bought, and it is not the thing that limits you.

The AMD side of this is a software story, not a hardware one. vLLM is a first-class, documented, officially-imaged path on Spark. On Strix Halo, gfx1151 is not fully supported by ROCm — AMD’s own vLLM containers have failed on Ryzen AI Max+ with invalid device function, and the working route is community-patched toolboxes and nightly ROCm builds. People do get it running. They also write things like “8+ days of debugging” and “unstable and brittle.” Meanwhile llama.cpp, which does work well there, has weaker batching than vLLM.

So if multi-user serving is the actual goal, that gap matters far more than the 17 GB/s of memory bandwidth between the two machines.

One lever genuinely worth pulling: prefix caching. If your users share a system prompt or hit the same document set, vLLM will reuse the cached prefix instead of re-prefilling it per request. For a team pointed at one RAG corpus this can matter more than any other setting on this list.

Stacking boxes: when one isn’t enough

Both platforms let you gang machines together to run models that don’t fit in 128GB. They go about it very differently.

DGX Spark pairs. Each unit ships with a ConnectX-7 SmartNIC and QSFP ports. Link two with a single 200GbE direct-attach cable and you get a point-to-point RoCE (RDMA over Ethernet) connection — NCCL runs its collectives straight over it, and vLLM or TensorRT-LLM shard the model with tensor parallelism. 256GB combined, and NVIDIA’s headline claim for the pair is Llama 3.1 405B in FP4. Two is the supported configuration; going further is a recurring request on NVIDIA’s own forums rather than a shipping feature.

Strix Halo clusters over plain Ethernet. No special NIC, no RDMA — llama.cpp’s RPC mode shards the model across nodes at load time and each node exposes its memory and compute to a controller. AMD’s own walkthrough runs a one-trillion-parameter model across four Ryzen AI Max+ machines, with 96GB of dedicated VRAM per node — 384GB in aggregate.

Now that per-unit prices have converged, the interesting difference here is no longer cost per gigabyte — four nodes costs about twice what two nodes costs on either platform. It’s the ceiling. Spark’s supported cluster stops at two units and 256GB; Strix Halo will keep going as long as you keep buying boxes and tolerating an Ethernet interconnect. What NVIDIA gives you in exchange is a far better interconnect and a production-grade distributed stack.

But here is the part that matters more than either. Stacking does not pool memory bandwidth. Each node still reads weights at its own ~256–273 GB/s. Adding machines raises the ceiling on model size, concurrency and KV-cache headroom — it does not make a single response stream faster. A 405B model across two Sparks runs; it runs slowly.

And the prefill problem compounds. In that same AMD walkthrough, time-to-first-token on the four-node cluster was 90.5 seconds at an 8k prompt and 239 seconds at 16k — with Flash Attention enabled; it ran out of memory without it. That’s an extreme configuration and not what you’d see from a normal model on one box, but it shows the shape of the thing: on hardware without native low-precision matmul, prompt processing is where the time goes, and clustering doesn’t fix it.

If you’re stacking to serve a bigger model to a few people, both approaches work. If you’re stacking hoping to make it fast, neither will.

Where they genuinely differ: the software

DGX Spark runs CUDA. Everything works: vLLM, TensorRT-LLM, the NIM containers, the profilers, every quantization tool, every tutorial written in the last decade.

Strix Halo runs ROCm and Vulkan. llama.cpp support is good and improving; the broader ecosystem is rougher. You will hit things that need a workaround, and you should budget time for that rather than being surprised by it.

If you’re a business evaluating these, the sharpest way to frame it is: DGX Spark isn’t really a server, it’s a development box with production parity. The pitch is that what you build there runs unchanged on NVIDIA hardware in a datacenter or a cloud, same containers, same CUDA. If that migration path is your plan, the premium buys something real.

If it isn’t your plan, you’re paying for a moat you’ll never cross.

Where Strix Halo quietly wins

It’s a normal computer. It runs Windows, it runs your IDE, it plays games, it comes in laptops and mini-PCs and a small desktop. Around half the power draw, and when it isn’t serving a model it’s still a genuinely fast 16-core workstation.

DGX Spark is an appliance. It does one job well and it is not your daily driver.

That versatility used to come with a large discount attached. Now that the price gap has mostly closed, the case for Strix Halo rests on the things money can’t buy back: it’s a general-purpose machine you’ll use for other work, it runs Windows, and it’s the only one of the two you can scale past two nodes. Those are real arguments. They’re just no longer cheaper arguments.

What I actually tell clients

Buy neither if your model fits in 24–32GB. A discrete GPU will be several times faster for less money. Capacity you don’t need is the most common way to waste a hardware budget — and this is by far the most frequent mistake I see.

Plan around four concurrent streams, not forty. vLLM’s continuous batching is real and it’s the difference between serving one person and serving a small team — but vLLM’s own Spark recipe caps --max-num-seqs at 4, and that’s a reasonable planning number for either machine. A handful of colleagues sharing a box is fine. A department is not. If you need to serve dozens of concurrent users with long contexts, this class of hardware is the wrong tool and a rented GPU is the right one.

Buy DGX Spark if you’re developing against NVIDIA production targets, or your workload is prefill-heavy, or you intend to serve several people through vLLM. The native FP4 path is the strongest technical argument for it, and it’s the one thing here that software cannot equalise. With the price gap largely gone, this is the default recommendation for most serving workloads in a way it wasn’t a year ago.

Buy Strix Halo if you want a machine that’s also a genuinely useful workstation, you’re happy running MoE models through llama.cpp, or you expect to scale past two nodes — which Spark’s supported cluster cannot do. Note that “it’s much cheaper” has mostly stopped being one of the reasons.

Don’t buy a cluster to buy speed. Whichever side you pick, adding nodes buys capacity and concurrency, never faster single-response generation — the bandwidth ceiling is per-node on both platforms. If a stakeholder is expecting a stacked rig to feel snappier, correct that expectation before the purchase order, not after.

And consider buying neither for now. Rent the equivalent for a month, learn what your workflow actually demands, and then buy hardware against a measured number instead of a spec sheet. That’s the same advice I gave about cloud versus local cost, and 128GB of unified memory on your desk doesn’t change it.

The exciting part of both machines isn’t the benchmark. It’s that a model class which needed a datacenter two years ago now sits under a desk for the price of a laptop. Just don’t confuse fits with fast.