u/frontsideair

How they did MCP with code mode is huge. I slept on it and am amazed how well they solved it. 

The problem with most harnesses is that there are two environments, where the client runs and where the sandbox runs. MCP servers are authenticated so they have to run on the client, but code has to run on the sandbox for containment. But MCP tools generate huge responses that pollute the context. You can delegate summarization to a subagent with mixed to poor results. 

Their solution is to ship a minimal sandbox inside the client, so the model can write code to tie MCP responses together without shell access, and the response can be trimmed down before it gets fed back into the context. 

They expose tools into the sandbox so the generated code can call them, and there’s some persistence as far as I can tell. Pretty amazing. 

Just tried, also gone, likely archived as well. *tinfoil hat* Maybe Nvidia is cracking down on popular llama.cpp forks? (I don't actually believe that, just having fun.)

Even with pi's high cache hit rates, server restarts or garbage collection due to running other threads destroy performance, especially with Qwen3.8-flash-next low prompt processing speed on llama.cpp mainline. But if we had the ~1400 t/s speeds of forks, near-full context would still cost 3 whole minutes, which isn't great.

Luckily there are already a few forks but only time will tell if they'll invest in the same way as the original author. I was also interested in giving it a try, and curious if the persistent cache survives upgrades, which could potentially be brittle.

I personally don't like leaving agents unattended, so I'd still expect responsive token generation (and prefill) speeds. 27b can work ~30t/s in smallish contexts, which is borderline usable. Flash Next is around the same speed but without optimizations or MTP at all, so I'm hopeful for it and MoE models in general.

Without 27b dense they are still plenty smart (benchmarks corroborate that) and have better world knowledge.

My personal recommendation is to go with the highest memory you can afford. The prices are highest they've ever been, but there's no hope of them ever going down and models tend to grow in side, that seems to be the trend. The last few months has seen some smaller models sizes to be dropped entirely and it's reasonable to expect the same going forward.

I have Minisforum MS-S1 Max at 128GB and happy about my purchase, I've been able to try out models like DSv4 Flash and Hy3 at very low quants and slow speeds, but they were surprisingly usable. It's also nice to keep a few models with different capabilities and sizes in memory at the same time.

Should run faster than 122B since A6B is less to read than A10B, and PLE should definitely help for the same reasons it helped Gemma4 E2B and E4B.

Yeah, I agree that it looks more like a bandaid than an actual solution. Please update when you have given it a try!

I researched this a bit and found out while there are manual APIs for KV cache persistence, it's not currently planned in llama.cpp, but if you're using pi, there's a pi-llama fork that uses those manual APIs. I haven't tried this but it's an option.

I wonder if what they mean by “early access” is that there won’t be an Unsloth quant on release day. 

There is no variable/adaptive reasoning for this model.

I honestly have no idea how frontier models are doing it, but Claude the model seems to have adaptive reasoning that's also available in the API, which isn't a guarantee, but a strong indicator that it's possible.

I can imagine that their training data for fine tuning has a lot of examples with these specific reasoning level variations that are rewarded for their conciseness or thoroughness, and thus models are likely highly attuned to these settings. While I can't know for sure, that sounds pretty reasonable to me.

I did lower the reasoning for myself, and I'm perfectly fine with the fact they decided to make xhigh the default, I'm glad they did show their best.

Sorry if I wasn’t clearer, I meant that Artificial Analysis performs benchmarks for top AI providers like Anthropic and OpenAI at multiple reasoning levels, but they usually do only one reasoning level for smaller labs. 

I also didn’t edit my post, it’s the same as before. 

I’m under the impression that the model is trained with certain versions of the prompt template so it carries more weight than a random modification. But I also agree that the model should be versatile enough to react to changes well. 

Sure, it just looks like an outlier, and shows the intelligence density of 27B. 

I didn't know this! Do you have a source?

For coding and agentic tasks in general, it makes more sense to increase reasoning level, you don't want to babysit the model at each turn, you want it to run to completion.

For more general tasks it's important to get a good enough answer quickly, and it would be useful if it had adaptive reasoning depending on the task. Maybe it can be done with a jinja update, I don't know how feasible it would be.

I've given up hope that Laguna will be benchmarked after waiting for weeks now. Not that important, but gives perspective.

Yeah, one-shots are the worst, but for sure they are flashy. In real world tasks I almost never care if a model one-shots something or not.

According to the jinja template, both low and xhigh inject extra instructions while medium is the default. That doesn't tell us anything about the training of course, just my observation.

It's really task dependent, for long horizon agentic tasks long reasoning is a feature so it doesn't give up before the task is done. I think this is basically why it scored so high.