Two things hurt most when a large model runs on your own machine. One is that the card has too little memory to hold a long context. The other is that it is slow enough to make work unpleasant. Strata 0.2.10 attacks both at once: the default speculative decoder is replaced by a small helper of a little over six hundred megabytes, and an 8 GB card can now open a 128K context.

What this release changes

Strata is a large-model client that runs on your own computer, written in C++, MIT licensed, hosted on GitHub and currently sitting near forty thousand stars. The positioning is narrow and clear: consumer hardware, two entry points in a desktop app and a terminal, pick a model after install and start chatting, writing code or reading images, with no account and no server in the path.

The version numbers need care. The app and the terminal are both 0.2.10, while the control component and the engine packages are 0.1.56. The release notes list the points plainly: DFlash2 becomes the default drafter, the multi-card full sharding engine switches on automatically on Linux, a new ISTA-DAS Coder model arrives, the Mac can read images, and 128K context works starting from an 8 GB card.

Six hundred megabytes replace a speculative layer of five gigabytes

Speculative decoding lets a small helper model guess what comes next, then has the main model check those guesses in a batch; the more of them land, the less time goes into generating token by token. The previous release used a speculative layer called MTP that took about 5.2 GB of video memory; this one uses DFlash2 at roughly 625 MB, so the difference frees a bit over four gigabytes.

The freed memory turns directly into usable context. The published comparison, on the same engine and the same files: on a Ryzen AI Max machine under Windows short answers gain 12 percent and long answers 8.8; on an RTX PRO 5000 with greedy decoding 7.9; on an Apple M5 Max 7.4; on a DGX Spark 5.8. There is a counterexample as well, an Intel Arc A750 runs 2.6 slower than MTP, and the notes say so plainly.

The price is concurrency. DFlash2 handles one request at a time, so running several at once means going back to MTP, and the same holds when the model family is not Qwen3.8. The automatic choice is built into the program: on automatic it picks by hardware and scenario, a wrong pick can be overridden by hand, and only one drafter loads per run rather than both at once.

How 128K fits into 8 GB of video memory

Long context eats cache. On an 8 GB card, opening 64K or more used to fail at launch as soon as the expert cache filled up, and users fell back to around 48K. This release changes how resources are placed: when video memory gets tight, data shifts between it and system memory dynamically, the context length the user asked for is preserved as far as possible, and when necessary part of the cache spills into system memory or even onto the SSD.

The measured numbers look like this: with the test machine constrained to the conditions of an 8 GB card, Qwen3.8 at the IQ3_S quantization with 128K enabled generated 63 to 73 tokens per second in DFlash2 mode and read a 32K prompt in 19.8 seconds, with 390 MB of video memory still free. On a real Intel Arc A750 with 8 GB, all four 128K launch combinations succeeded at 12.7 to 14.8 tokens per second. A 12 GB card is unaffected and keeps its original speed.

There is a threshold on this path. Once the pressure is shared with system memory and the disk, the demand on the machine's RAM rises, and the community rule of thumb is 32 GB to start, 48 GB to be comfortable enough, 64 GB to stop thinking about it. The disk needs room too, since the model files themselves are not small.

Two cards together, and code editing nearly doubles

The old approach with several cards is layer splitting: each card owns part of the layers and passes the result to the next. This release enables a different split by default in the Linux terminal, called full sharding: every card holds a slice of every layer, and all cards read weights and compute at the same moment. The gain is that memory spreads more evenly; the cost is that the cards exchange intermediate results constantly, so interconnect bandwidth starts to matter.

The project measured two RTX A6000 cards under Linux: code editing went from 195 tokens per second to 377 to 394, writing new code from 139 to 144 up to 189 to 232, plain text from 93 to 139 to 146, and an agent loop from 112 to 119 up to 165. Reading a long prompt is slightly slower, a 32K prompt going from 7.65 seconds to 10.1, which is throughput bought with first-token latency.

Auto-enabling has strict conditions: it has to be Linux, it has to be two NVIDIA cards at compute capability 8.6 or 12.0, or four Radeon AI PRO R9700 with specific quantization files; the model has to be Qwen3.8, which the engine can handle; all routed experts have to fit in video memory; and about 43 GB of free system memory has to remain. When the conditions are not met it falls back to layer splitting, explains why, and the automatic behaviour can also be switched off by hand.

Two things fixed along the way

The first is startup on Windows. In some background launch scenarios the disk read priority was pushed down so far that model loading slowed noticeably, and this release raises it again. The second is image input on the Mac: on an M5 Max, eight image requests were read correctly eight times, generation with an image ran 19 to 22 tokens per second against 21 to 22 for plain text, the image tower took an extra 319 MB of video memory, and the engine's peak memory stayed at 74 GB, level with the text-only case.

The new ISTA-DAS Coder takes 192 of the 384 experts in DeepSeek-V4.1-Flash at every layer; on four R9700 cards it decodes at 59.2 tokens per second, on a single card 22.8, and on an Apple M5 Max 19.1 to 22.5. All of these come from the release notes and were taken on different machines, so a comparison across them means little; the order of magnitude is the point.

Who should update

Anyone with an 8 GB to 12 GB consumer card who wants a long context should switch, because this turns a scenario that simply would not start into one that runs. Dual-card users running Qwen3.8 on Linux get a fairly direct gain from full sharding. For someone with a single large card who mostly chats, the difference after upgrading is mainly that the first start downloads about 1.2 GB of drafter files.

One warning: projects like this iterate quickly, and a version number a day is normal. The documentation is quite detailed about hardware, quantization levels and concurrency settings, and it is worth reading it against your own configuration before acting rather than copying someone else's parameters.