Testing Qwen3.6-35B-A3B

It's amazing to see how much progress models and llama.cpp have made in just a few months. Before this, I tested the Qwen3.6-27B model. It wouldn't fit in my 16 GB of VRAM, ran quite slowly, and generated at 12–18 tokens per second.

It took about 20 hours to complete my test task: building a Markdown editor in Rust with GTK on Linux.

It was an interesting experiment, but practically useless. Way too slow.

And in the meantime, the Qwen3.6-35B-A3B model was released. It uses a MoE architecture: 35B total parameters, but only ~3B are activated per token (8 routed experts + 1 shared expert out of 256). This makes the model compute much more efficiently while retaining quality on par with much larger models in the same class. llama.cpp has added support for this architecture, along with numerous performance improvements and MTP (Multi-Token Prediction) — an additional "head" trained to predict multiple tokens ahead, alongside the main model. MTP delivers a ~1.4–2.2x generation speedup without sacrificing accuracy, though for MoE models the boost is usually more modest (~1.15–1.25x).

So today I decided to feed my test task to this model. First, I had Claude Code run several tests to find the optimal settings for it.

Saying I'm surprised is an understatement. First, the entire model fits comfortably in VRAM. Second, I'm getting a 256K context window, which I've never been able to use with other models (even smaller ones struggled with this).

Prompt processing is running at roughly 260–400 tokens per second. Generation speed on an empty context was nearly 100 tok/s, dropping to 62–65 tok/s as the context filled up. Still, it's damn fast.

With the 27B model, it took about 10 hours just to reach a state where the app would run, even with a buggy and broken UI. Today, the 35B model got there in 50 minutes!

Energized by the speed, I started telling the model what to fix. In just 4 hours, I got a fully working application with all the flaws and even minor details corrected.

Looking forward to running this model on my real projects now. Who knows, maybe the dream of a capable local coding model isn't so far out of reach after all.