The Intel Arc Pro B60 for Self-Hosted LLMs: 24 GB, SR-IOV, and a Software Stack That's Finally Catching Up

The economics of self-hosted inference come down to one number: dollars per gigabyte of fast VRAM. Weights and KV cache have to fit on the card, and the moment a model spills to system RAM your tokens-per-second falls off a cliff. That's why Intel's Arc Pro B60 is interesting — it puts 24 GB of GDDR6 on a single-slot professional card at a price that undercuts every 24 GB NVIDIA option, and it brings features the consumer cards don't: SR-IOV hardware partitioning and a reference design for stacking many cards into one box.

We've been running production LLM workloads on a pair of B60s in our Kubernetes cluster for a couple of months. This post covers what the card is, how the software stack behaves, and — with real numbers from our benchmark harness — where it wins and where it still hurts.

The card

The Arc Pro B60 is built on Intel's Xe2 "Battlemage" architecture (the BMG-G21 die — the same silicon as the consumer Arc B580, but wired for 24 GB instead of 12 GB and tuned for a lower, workstation-friendly power envelope). It's a single-slot, blower-cooled professional card aimed squarely at AI inference and content-creation workstations.

SR-IOV and partitioning

One genuinely differentiating feature of the Arc Pro B-series over consumer Arc is SR-IOV — single-root I/O virtualization. The card exposes multiple Virtual Functions, so a hypervisor can hand a slice of the GPU to each of several VMs or containers with hardware-level isolation. It's worth being precise about the status, though: the B60 is hardware-capable but SR-IOV isn't the factory default — Intel enabled it via a February 2026 firmware update (flashed with the igsc utility), and on Linux it wants a recent kernel (6.17+, Intel xe driver) plus SR-IOV / Above-4G decoding in BIOS. Community reports put it at up to 7 Virtual Functions per GPU (we haven't found this in a primary Intel doc, so treat the exact count as provisional). Intel pairs all this with a reference multi-GPU inference platform ("Project Battlematrix") that stacks up to eight B60s into a single 192 GB node for models past 70B. Consumer Arc B580 has no SR-IOV path at all — this is the "Pro" tax paying for itself in a virtualized fleet.

That said, we don't use SR-IOV in our cluster, and it's worth being honest about why. Our nodes run the Intel GPU device plugin, which time-shares each physical card as several schedulable gpu.intel.com/xe units (we run sharedDevNum: 4, so two B60s present as eight units to Kubernetes). That's oversubscription, not isolation: it's simpler to operate, lets several small pods share a card, and needs no VF plumbing — but it gives you no hard memory or fault boundary between tenants. SR-IOV is the right tool when you need strict isolation (multi-tenant, untrusted workloads); device-plugin time-sharing is the right tool when you control every workload and just want to pack the card. Pick per your threat model.

The MaxSun dual-card: 48 GB in one slot

Because the B60 is a lean, low-power design, board partners can put two GPUs on a single card. The best-known is the MaxSun Arc Pro B60 Dual 48G — two full B60 GPUs and 48 GB total (2×24) on one dual-slot board (~400 W TBP, a single 12V-2×6 connector, ~$1,200 street). There's no bridge chip: the ×16 slot is bifurcated to ×8/×8, one PCIe 5.0 ×8 link per GPU (128 GB/s each — equivalent to PCIe 4.0 ×16, so no bandwidth regression). The important caveat: the OS sees two independent GPUs, not one 48 GB device — you need motherboard bifurcation support, and software has to explicitly span both GPUs (tensor parallelism, or layer-split in llama.cpp) to use the full 48 GB for one model.

That's exactly the topology of our test rig: two B60s on node5, enumerated by llama.cpp as SYCL0 and SYCL1. So everything below about single-card vs multi-card behaviour maps directly onto what a MaxSun Dual owner would see.

The software stack

We benchmarked two runtimes, both in-cluster on Kubernetes:

• llama.cpp — the SYCL backend (Intel's oneAPI path) and the Vulkan backend, built from source into Harbor-hosted images. GGUF weights at Unsloth's UD-Q4_K_XL dynamic 4-bit quant.
• vLLM — the official v0.22.0 XPU build (compiled from source for Intel GPUs), FP8 weights.

Models: Gemma 4 26B-A4B (a mixture-of-experts model), Gemma 4 31B (dense), Gemma 4 12B, and Qwen3.6-35B-A3B. Every benchmark cell was gated by a coherence canary ("capital of France" → "Paris") — numbers from incoherent runs were thrown out.

Interactive vs batch

The single most important finding is that the two runtimes are good at opposite things. llama.cpp gives you the best single-stream latency; vLLM's continuous batching wins decisively once you have real concurrency.

At one stream, llama.cpp decodes ~20 t/s while vLLM manages only ~5 (vLLM runs --enforce-eager on XPU, which disables CUDA-graph-style capture and leaves single-stream decode slow). But vLLM scales ~16× from 1→64 concurrent streams to 96.5 t/s aggregate, while llama.cpp tops out around 8 streams. Use llama.cpp for a chat assistant; use vLLM for a batch/API backend.

Within llama.cpp, SYCL and Vulkan trade blows depending on concurrency:

oneAPI 2026: JIT fixed correctness and quadrupled decode

The most dramatic result came from a toolchain upgrade. On the ahead-of-time-compiled (AOT) SYCL build, Gemma 4 decode topped out around 14–19 t/s — and worse, the oneAPI 2026 AOT build started producing non-deterministic gibberish on greedy decode. We root-caused it to the offline IGC optimizer miscompiling the attention kernel (-cl-opt-disable restores determinism but tanks throughput). The fix turned out to be switching from AOT to JIT kernel compilation, which is both correct and far faster at steady state:

That's a ~4× jump in single-stream decode versus the original FP32 AOT build — from a compiler flag, not new hardware.

Context and memory

The 26B-A4B MoE model runs the full 256k context on a single card — its sparse activation and Gemma's sliding-window attention keep the KV cache small. The dense 31B and Qwen3.6 hit the memory wall much sooner:

And here's the VRAM budget that drives all of it — note how close f16 KV at 256k runs to the 24 GB ceiling, and how q4_0 KV buys back headroom:

Decode being bandwidth-bound is also why concurrency barely helps a single model's throughput — the card is already saturated moving weights and KV each token:

Two cards, 48 GB

On the older builds, llama.cpp's multi-GPU split garbled output across two B60s — a real blocker for anyone hoping to pool a MaxSun Dual's 48 GB. On the current b9820 / oneAPI-2026 JIT build, --split-mode layer is coherent across both cards:

Throughput is essentially the same as a single card (decode is bandwidth-bound, and layer-split doesn't add bandwidth) — so the payoff isn't speed, it's capacity: 48 GB lets you hold models or contexts that don't fit 24 GB. One caveat remains: --split-mode row still segfaults on load, so stick to layer-split.

B60 vs NVIDIA

To place the B60 in context we ran the same small Gemma 4 models — E2B and E4B — through an identical llama.cpp harness on the three NVIDIA cards in the spec table above: the modern RTX 4080 (CUDA build, run locally) and the cluster's Turing pair, the 2070 SUPER and 1660 SUPER.

On raw single-stream decode the ranking is unambiguous. The RTX 4080 is fastest by a wide margin (~178 t/s on E2B against the B60's 65), the eight-year-old 2070S still beats the B60 on these tiny models, and the B60 lands between the two Turing cards — ahead of the 1660S on E4B, a touch behind it on E2B. Ada's tensor cores and a decade of CUDA kernel tuning simply move small models faster than first-generation SYCL does.

The nuance the chart can't show is the VRAM wall. E4B at full 128k context already needs ~6.3 GB of f16 KV, and neither Turing card can even load the 12B/26B/31B models the B60 runs all day — nor can the 16 GB 4080 hold them at long context. Fast NVIDIA silicon is quick right up until the model doesn't fit; the B60's 24 GB is the entire point.

Small models: caveats

Running the full grid on both the B60 and the 4080 surfaced four things worth knowing before you trust any single headline number.

Decode fades as context grows. Neither card holds its 8k decode rate out to 128k — the KV cache grows and decode is bandwidth-bound. The B60 drops from ~81 → ~35 t/s (E2B) across 8k → 128k; the 4080 from ~195 → ~136. The gap between them stays roughly constant.

q8_0 KV cache is a SYCL trap. Quantizing the KV cache to q8_0 is meant to trade a little speed for memory. On CUDA it does — the 4080 keeps ~80% of its f16 decode. On Battlemage/SYCL it craters: the B60 retains only ~38%. Same models, same llama.cpp, wildly different penalty — a runtime bug, not a model property. Stay on f16 KV on the B60; if you must quantize, q4_0 behaves where q8_0 collapses.

flash-attn off doesn't crash the small models — but it rebalances them. We expected --flash-attn off to take down the SYCL backend. On E2B/E4B it ran cleanly with coherent output, but the throughput shape flipped: decode fell ~37% while prefill roughly doubled. On the 4080, turning flash-attn off just made everything slower. Keep it on for interactive (decode-bound) work; the only reason to turn it off is a prefill-heavy, low-generation workload.

mmap barely matters; the JIT warmup does. Toggling memory-mapped weights (--no-mmap) moved decode by under 1% on both cards — the models are small and storage is fast. What does cost is the SYCL JIT: the first server start on a fresh B60 pod spent ~118 s compiling kernels, while every later load was ~8 s. Budget one cold start per pod, not per request. (The CUDA 4080 loads in ~3–4 s throughout — no JIT step.)

The 256k point failed identically on both cards. Our harness's 256k-context fill returned nothing on the B60 and on the 16 GB 4080 — the same failure on cards with very different VRAM, so it's a harness/long-prompt artifact, not a hardware ceiling. We report the clean 8k–128k ramp and flag 256k as untested here rather than as a limit.

Recommendations

• Interactive assistant, one model: llama.cpp SYCL on the b9820 oneAPI-2026 JIT build. Best single-stream latency, and now coherent and fast.
• Batch / multi-user API: vLLM v0.22.0 XPU with -tp 2. Slow at one stream, excellent at 64.
• Long context (256k): the 26B-A4B MoE on a single card. Use q4_0 KV if you want headroom.
• Need 48 GB: the MaxSun Dual (or 2× B60) with --split-mode layer. Capacity, not speed.
• Multi-tenant isolation: this is where SR-IOV VFs earn their keep over device-plugin time-sharing.

Bottom line

The Arc Pro B60 is not the fastest card you can buy, and its software stack will still bite you. But it delivers the thing that actually gates self-hosted inference: a lot of fast memory, cheaply, with Pro-tier features — SR-IOV, multi-card reference designs, a 48 GB dual-GPU board — that the consumer cards don't offer, and the oneAPI-2026 JIT toolchain clears the last correctness-and-speed blocker. The economics are lopsided: at ~$599–799 street the B60 lands around $25–33 per GB of VRAM, versus ~$62/GB for NVIDIA's 24 GB-class pro cards and ~$67–83/GB for an RTX 4090 — 2–3× the fast memory per dollar. For a self-hosted 24–30B assistant on a budget, it's the most VRAM you can rack per dollar today.

Card specifications and pricing from Intel's product page and hands-on coverage at StorageReview, ServeTheHome, and EmbeddedLLM. All throughput, VRAM, and context numbers are from our own benchmark runs on the cluster described above.