u/Digger412

The model was trained in MXFP4, there isn't a BF16 version that exists like you're envisioning. Their technical report goes over some of the MXFP4 training details.

I'll have to do a bit of investigation / research, pretty sure that's a mixed FP8 / FP4 model so it'd end up being an mxfp4-native conversion for the experts; it might benefit more from Ed's target-bpw quantization then like I do for Kimi. Will see what it looks like on the charts.

Not really, gemma-4 doesn't really refuse with even a modest JB-like prompt for many topics. I'd think if you're really running into refusals, you're either doing something pretty extreme or your system prompt isn't set up to "tell" the model it's allowed to do X, Y, Z things.

The full weights and quants are available though, if you really did want to abliterate it you could. It would almost certainly damage the model's intelligence though.

26B-A4B is doable, it's just extra work to do the initial abliteration / J-Space lens construction and also the training because of the experts. Doing it on a dense model was simpler. I'm not planning on doing the 26B-A4B as of right now though.

I do this as a hobby, so I appreciate the offer but I'm not looking for funding help at this time :)

I glossed over it pretty quickly, but if you read the bullet point after the one about picking the lowest refusal (which I omitted saying was on the pareto frontier), the projection of the abliteration LoRA through the J-Space removes about 80% of the abliteration edit weight.

I picked a trial that had 4/100 refusals and 0.2 KLD IIRC, and after the projection it was up to 91/100 refusals and 0.008 KLD or similar. So, to gain the most effect since the projection preserves intelligence and discards most of the edit that isn't verbalozable you want to select a very aggressive trial. 

I'm not trying to balance refusals and KLD because I know that projection step will revert the majority of the changes and restore the intelligence.

I haven't used the QAT version, and as I mentioned in another comment:

As far as I'm aware, the QAT is a Q4_0 already so you wouldn't train a finetune on that. For accuracy and stability you want to base finetunes on BF16 models.

It should be possible to extract the LoRA stack and apply it to the Q4_0, but given the nature of how I did successive finetunes I think that would lead to compounding inaccuracies and that's pretty firmly in experimental territory. And I definitely don't have the time or compute to redo the QAT from scratch.

Thank you, glad you're enjoying the model.

As far as I'm aware, the QAT is a Q4_0 already so you wouldn't train a finetune on that. For accuracy and stability you want to base finetunes on BF16 models.

It should be possible to extract the LoRA stack and apply it to the Q4_0, but given the nature of how I did successive finetunes I think that would lead to compounding inaccuracies and that's pretty firmly in experimental territory. And I definitely don't have the time or compute to redo the QAT from scratch.

I should be able to make an nvfp4 quant, but I'm not sure if that would be more accurate that the Q/IQ quants. I didn't PPL/KLD test them, I'll see about doing that tomorrow evening. 

Glad you're enjoying it; it should be a totally capable model for assistant work too. The DPO finetuning is distribution sharpening, I tried an SFT on top to add variety but it traded intelligence and adherence for creativity and didn't feel right when tested.

I'm pretty sure this is a just-as-capable-but-nicer-to-talk-to model compared to baseline gemma-4-31b-it and should be able to replace it for assistant work entirely adequately.

Hi, this is a really quick and dirty writeup of the steps I went through, more or less:

• Heretic abliterate gemma-4-31b-it, pick the most aggressive trial that results in the least refusals
• Train a J-Space lens on gemma-4-31b-it (the lens is published in the first scotoma repo; it gives you a local linear map from weight-edit directions to what changes in the model's output)
• Do some funky math to project the abliteration LoRA through the J-Space lens (basically, an orthogonal projection onto "what is verbalizable through J-space", and leave the "computational" bits alone; this is where the intelligence is preserved but it also undoes most of the refusal work in the abliteration but leaves the model in a sort of verbally de-aligned state? which I think helps break some of the burned-in assistant voice.)
• Apply that lens-pojected LoRA to the model
• Start with ~500 RP prompts
• Run completions with those prompts with the lens-ablated model
• Use some metric identification for the patterns to identify the structural tics (paired adjectives, negation, etc)
• Have the lens-ablated model rewrite the identified spans to not exhibit the tics (it's important that the only change is in the identified areas, you want to leave the rest of the response as-is)
• Do DPO with the (rejected from lens-ablated, chosen from edited) to push the model away from producing those tics (there were 3 DPO steps, and at each step I redid the completion pairs so that each trained step was on-policy)

All of the training dataset was English, and I haven't tested it in other languages. If you test it out and it feels better/different, I'd be interested to know.

This was something I noticed as well, and I think it might be something that has surfaced due to the removal of some of the other tics. I.e., longer, comma-connected clause sentences are showing up more now that they aren't being overridden by other prominent constructions.

I'm afraid that chasing that and fixing it might end up being a whack-a-mole exercise in the end though.

Hi, scotoma author here - it should work fine with instructions and thinking. I've tried to maintain the capabilities of gemma, since finetuning carries risks of hurting the model's intelligence etc.

Yes, I'm the author :)

I haven't seen what Gryphe has done, but this finetune was mostly DPO-based preference tuning to push the model away from those specific prose constructions. I know that SFT is a common route for changing a model's style, but that can also affect intelligence and prompt adherence.

Gemma can already RP fine, it just needs to stop tripping over its lack of prose diversity IMO.

Yeah, there are too many llm-isms to really hit them all. In this tune I was aiming for the ones that irritated me the most which were the structural and sentence-style tics that Gemma4 likes to do.

So things like ozone, predatory, etc., are probably not shifted that much. It's something we as humans notice from our experience using different models and pattern matching, but single words in isolation are better served with logit biases to reduce their occurrence.

Hi, AesSedai here - yep, can confirm, am a guy :)

(and, being suuuuuuper lore-specific, Aes Sedai during the Age of Legends referred to both men and women channelers!)

I don't have the HF storage for all of those, the Laguna's I'll try to do but the rest I'm not sure about. I've already got a QuantPasture org that I'll have to move the older minimaxes to I think.

AesSedai here - tested out the conversion and it works (gj pwilkin!)

Working on a small imatrix with the full quality MXFP4 gguf, but it's at the absolute limit of what my system will load. I'm unsure if I'll make / upload MoE-quants to HF because frankly this one is a beast and a half.

17.40.359.649 I compute_imatrix: computing over 16 chunks, n_ctx=8192, batch_size=8192, n_seq=1
27.00.443.409 I compute_imatrix: 560.08 seconds per pass - ETA 2 hours 29.35 minutes
[1]3.7688,[2]2.6482,

I've gotten feedback that the Q4_K_M is good, the IQ4_XS is about 1BPW smaller which is ~64GB lighter but in the KLD graph the IQ4_XS is nearly 3x the absolute KLD of Q4_K_M. If you've got 304GB, you should be able to fit the Q4_K_M but I'm not sure if you'd fit it at 100k ctx without dropping to Q8_0 for cache quantization perhaps.

So I'd say try the Q4_K_M, if it doesn't fit at your ctx needs then the IQ4_XS.

<3

I've uploaded the rest to my repo on HF now :)

Hi, I've just finished uploading my MoE-optimized quants: https://huggingface.co/AesSedai/MiniMax-M3-GGUF

Nope, posted a picture in another comment. I have three PCIe x16 -> MCIO x8x8 splitters and two MCIO headers on the board. The cards are attached via MCIO -> PCIe risers.

Interesting, there'll probably be some improvements in the next couple of weeks as the b6k discord folks get it dialed in too.