Qwen3.8-Flash-Next Review - Running It on an RTX 5090 + R9700 Pair

Three quantization swaps and sixteen rounds of advice later, the story of a local model swap that shipped to production

This is the English version of a post originally written in Korean for my algorithmic trading system devlog(new tab).

The AI recommendation pipeline in my trading bot has a research model(new tab). Its job is to read news and disclosures the way a person would, then write a report backed by reasoning.

I recently put a new open-source model up as a candidate for that role: Qwen3.8-Flash-Next, an MoE model with roughly 125 billion total parameters but only about 6 billion actually active per token, positioned as a preview of Qwen's next generation. This post is far more specific than usual. What I'm describing here is pure infrastructure — which GPUs, how they were configured, exactly which quantization I picked — not the decision logic itself, so there's no real reason to hold it back. It should also be useful reference for anyone running local LLMs themselves, so I'm leaving the process in unfiltered.

This post runs long, so here's the short version up front: the quantization I ultimately shipped to production was Q3_K_XL. This morning (09-08, 07:18) the pipeline finished all 100 tickers without a skip for the first time, and the server log from that actual run works out to roughly 610 tok/s prefill and 124 tok/s decode (summed across 4 concurrent slots, about 31 tok/s per slot).

The rest of the post walks through the other quantization candidates I tried before landing on that, and a measured table of how prefill/decode speed shifted every time I changed the concurrency count (-np) or batch size (-b). If you're curious about that process, read on.

Goal and hardware

The goal was simple: get the research pipeline through its usual sweep of 100 tickers in under 10 hours.

Serving ran on two cards: an NVIDIA RTX 5090 (32GB) and an AMD Radeon AI PRO R9700 (32GB), two different vendors' cards split across layers on a single llama.cpp server. Combined VRAM was about 63.6GiB, and the model backbone alone took up most of that — there wasn't much room to spare.

Mixing vendors brought its own quirks. The AMD card had a bug where dropping into GTT (system memory the GPU borrows) power-saving mode slowed responses down, so I had to disable runtime power management entirely via a udev rule (99-r9700-no-runtime-pm.rules).

The layer split I eventually settled on was -sm layer -ts 23,25 (23 layers on the 5090, 25 on the R9700), with -np 4 concurrent requests, batch size -b 512, and context -c 131072.

Actual input/output tokens per ticker

Before getting into speed, it's worth pinning down what actually goes in and out of this model. Sweeping one ticker takes several stages (news analysis, disclosure analysis, debate, and so on), each triggering its own model call — these are the measured figures per individual call.

The 6.6x gap matters because it means the real bottleneck in this workload leans more toward "how fast can it read a long input" (prefill) than "how fast can it generate an answer" (decode). That prefill weight keeps showing up as the limiting factor in the concurrency experiments below.

First choice - starting aggressive on quantization

Fitting the whole model into VRAM meant GGUF quantization. I started with Unsloth's UD-IQ3_XXS (about 77-82GB, with half the expert parameters down at IQ2_S-grade 2.5-bit precision).

Measured throughput came out to 521 seconds per ticker — 14.5 hours for 100 tickers, 1.45x the 10-hour target.

Before the raw numbers, what actually bothered me was "how much judgment quality did this precision cost." I wasn't confident IQ3_XXS was safe to go with, so I asked for advice from another AI model several times, using Unsloth's published quantization quality comparisons (perplexity retention, KL divergence) as the basis.

That settled the picture across three candidates. Q4_K_XL (92.3% quality retention, KLD 0.047, the highest precision) needed 77GiB for the backbone alone — over the combined 64GiB VRAM budget, so it was disqualified outright. Q3_K_XL (88.3%, KLD 0.107) gave up less quality than IQ3_XXS (85.4%, KLD 0.165) while its 57GiB backbone nearly fit inside VRAM. I dropped the original IQ3_XXS and pulled down Q3_K_XL instead (about 84GB).

Raising concurrency crashed the GPU outright

Chasing more speed, I tried raising the number of concurrent requests. At a certain combination, the GPU itself simply stopped responding.

I misdiagnosed the cause twice. First I blamed a software setting, then ruled that out too. The real cause turned out to be sitting in the kernel log the whole time: NVRM: Xid 8 — the RC (Row Chase) watchdog forcibly resetting the GPU after 7 seconds. That's a safety mechanism that force-resets the GPU when it goes unresponsive too long, and the np=8 concurrency combination pushed past that window.

Shrinking the prompt bought 14%

If the model itself couldn't change, the next lever was shrinking what got fed into it. I cleaned up the boilerplate at the front of the instructions repeated on every research call, and applied a second patch asking the model for more concise output.

An A/B comparison under identical conditions came out to 457.7 seconds before the patches, 391.8 seconds after — 14.4% faster. Projected across 100 tickers, that's a drop from 12.7 hours to 10.9 hours.

I also checked whether turning the patches on shook judgment quality. A handful of tickers did change grade, but this model already flips grades on roughly half its calls even on repeated identical runs (self-reproducibility of 51.5%), so I ruled this variation as pre-existing noise rather than something the patches caused.

Moving layers to the other card backfired

I also tried routing expert-layer parameters preferentially onto whichever card had more headroom. Single-request benchmarks looked promising — 8-17% faster.

But retested under real server conditions (multiple concurrent requests), it came out 6.7% slower instead. Digging further, that batching approach turned out to inflate time-to-first-token (prefill) by 20%, and once several concurrent requests piled up, that side effect ate the entire gain the single-request benchmark had shown. I dropped this approach.

The moment the benchmark numbers lied

After receiving the mid-precision version, a standard benchmarking tool measured prefill speed as 22-25% worse than the previous version — a number bad enough to force reconsidering the whole choice.

Asking for advice again to dig into the cause, it turned out the benchmark tool lazy-loads part of its data on the model's first run, and that cold-cache first pass was what artificially tanked the number. Corrected for real conditions (a warmed cache), the version was actually 2-7% faster.

That reinforced a lesson: never trust a single benchmark tool's number at face value. From then on, every judgment was made only from values reproduced with an actual running server.

Re-verified on the real server, the mid-precision version came out to 395.7 seconds per ticker — only a 1% loss versus the first version's 391.8 seconds. Given how much less it had been compressed, that near-zero speed cost made it the version I ultimately adopted.

What the low-temperature experiment revealed - a bug that wasn't there

I also tried lowering output diversity (temperature) to make model output more consistent. But while asking for advice on this, something strange surfaced.

The setting was actually getting dropped somewhere in the plumbing that passes configuration to the server, so every run I thought had lowered the temperature had actually silently fallen back to default the whole time. The patch I'd suspected earlier turned out to be innocent, so I redirected the work toward fixing this plumbing bug instead.

Closing the loop with a 20-ticker paired comparison

To settle every judgment made up to this point in one pass, I ran the same 20 tickers end-to-end with the patches off and with them on, head to head.

Patches off came out to 423.5 seconds per ticker, patches on to 390.5 seconds — 8% faster. Grades matched exactly on 11 of the 20 tickers, and a sign test on the remaining differences found no significant directionality (p=1.0). That confirmed the patches raise speed without shaking judgment.

The final configuration — mid-precision quantization with both patches on — landed at 390.5 seconds per ticker, 10.85 hours for 100 tickers. That's 1.085x the 10-hour target, as close as I got it.

Cutting over to production, and two incidents right away

With everything validated up to this point, I decided to cut over to production — took down the previous model and restarted five resident services onto this one.

Two problems surfaced the moment it went live. First, the logic that resumes an interrupted run wasn't checking which model had produced the existing results, so leftover data from the just-retired model was about to get mixed in under the new model's label. I added a parameter to block resuming whenever the model doesn't match.

Second, I only discovered at that point that there was no automatic fallback to the previous model if something went wrong, and no automatic backup either. I manually backed things up first, then hastily built a safety-net script to cover the gap.

Night one after the cutover, skipped for a completely unrelated reason

The first night after cutover, only 77 of 100 tickers got processed; the rest were skipped. I suspected the new model and dug through logs, but the actual cause was somewhere else entirely.

A test script written for code review purposes had, during cleanup, run a command to kill every llama-server process on port 8090 — which happened to be the exact port the freshly-deployed production server was using. The test had killed the live service.

With the server unresponsive for 34 minutes, the system misread it as an external network failure, and that misdiagnosis cascaded into skipping the remaining tickers. I added a safeguard to the test script (a guard that blocks it from touching a real port unless it's substituting a mock response).

The next day - sweeping concurrency (np) and batch size (-b) all day

The day after cutover, I spent the whole day sweeping combinations of concurrency (-np), batch size (-b), and the layer split between the two cards (-ts). Here are the combinations actually re-verified on the real server.

A few things worth revisiting from this sweep.

I first suspected batch size (-b 1024) as the cause of the OOMs, since they kept showing up on np=3/np=5 runs. Digging back in, though, -b only changes how many tokens the GPU batches per pass — the memory reservation itself is governed by a separate setting (-ub, left unset here), independent of -b. I'd been suspecting the wrong variable.

Odder still: np=5 succeeded on the exact same settings (-c 133,120, ts23/25) where np=3 failed. Fewer concurrent requests running short on memory is the opposite of what you'd expect. I couldn't pin down the cause from that day's experiments alone, and left it unresolved with two competing hypotheses — "the GPU happened to have less free memory available at that particular moment" versus "fewer slots means each slot's buffer grows larger" — since chasing it further (worth maybe 3%) wasn't worth the priority over other things.

One observation stood out along the way: the AMD card was almost always pinned at 100% utilization and was the bottleneck, while the NVIDIA card sat idle 17-26% of the time. The performance gap between the two cards showed up directly as the bottleneck.

In the end, I locked in the combination settled on day one — -np 4 -b 512 -c 131072 ts23/25. Nothing else beat it, and with the first full completion after cutover as the goal, there was no reason to switch to a less stable combination (np=5's zeroed-out context margin).

The result - today's actual completion data, and a question still unanswered

On the second morning after cutover (2026-09-08, 07:18), the run that started the previous evening at 21:02 finished all 100 tickers without a single skip for the first time — 10 hours 17 minutes end to end. These numbers come from directly parsing that actual run's server log, not a test.

369.9s/ticker is even faster than the 390-396s range seen during validation. Validation ran on samples of a few to a few dozen tickers, so some difference from a genuine full 100-ticker run is expected.

The speed-tuning goal itself was settled there.

One thing I still haven't checked, though: whether this model actually makes good calls — quality measured against real trading performance — hasn't been measured even once up to this point. Everything covered here was about "how fast" and "whether the patches shake judgment," never "how good is the judgment itself." That's still homework left for later.

Looking back

The biggest lesson to come out of this was the habit of never trusting a single benchmark tool's number outright. Both the fact that the tool was measuring a cold cache and the fact that the temperature setting was leaking somewhere in the plumbing only surfaced because a number "felt off" enough to go ask for advice again.

I've written before(new tab) about a time a hardware swap and a misread benchmark flipped a verdict twice, and the same lesson repeated here. A real value reproduced under the exact conditions you'll actually use was always more trustworthy than catalog numbers or a one-off benchmark.