u/PcChip

i wondered why those plastic barriers were gone lately

i can fit all of DS4.1F in RAM but it's so much slower for me than QFN, like way slower

it was listed on the project maya github page, I just told qwen3.8 flash next to download and install project maya with the exl3 3bit quant version of deepseek 4.1 1017 and it did it - just tell your agent to figure it out

for exl3 I run tabbyapi with the 5.05bpw weights, and then i told qwen 3.8 flash next to download the full 16-bit ngram table and put that in RAM instead of the quantized version

tabbyapi is great

tell the ai to speed itself up
and yes, I'm serious - give it ssh credentials and let it go to town tweaking and tuning all the parameters :D

after it was done speeding itself up, I told it to try power limits for all GPUs in 5% increments between 65% and 85% and produce a graph of the results

when you decide to create a model from scratch, how do you decide how many parameters total and active to use?

any chance you could support the exl3 3bit quant? that's what I use in project maya for better quality than 2bit quant, but i only get 35 tok/s on my four 24GB GPUs

i'm running maya with the exl3bit quant right now, i can only get up to about 35 tok/s decode and prefill is terribly slow

let me know if you try dsx and it's faster, i might try switching

are your map chunks completely flat?
from the video it looks like completely flat land, so i'm wondering why you even need VBA's per chunk and not something simple like one giant quad

how are you handling simulations for chunks that are offscreen/unloaded?
I'm debating on the way to handle it myself and wondering what you did

would it be hard to make it support exl3 quants instead of unsloth gguf?
I currently run the exl3 quant of 5.05bpw QFN with tabbyapi but would love to try this for even more speed; I have four mixed nvidia 24GB cards and 512GB RAM

400t/s doesn't mean much unless you specify the quant

are we talking 1bit or 8bit here?

interesting!

https://preview.redd.it/2p2zg6yg9wxh1.png?width=2637&format=png&auto=webp&s=4ab116b8ae0b7881c79bb8303c0890f46a8dbb71

love this, thank you for what you're doing for the community

try tabbyapi with qwen 3.8 flash next, maybe 5.05bpw exl3 quant (or 4.05 if the 5.05 is too slow)
then have the ai figure out how to load the full 16bit n-gram table instead of the quantized one, to sit in your RAM

i tried it at home with project maya and the 3 bit exl3 quant and was only getting 35 tok/s decode, and prefill was very slow. I figured openrouter providers would have much higher speeds

why didn't you list the quants of each that you used?

why no warmup?

also why not test tabbyapi with a similar sized exl3 quant?

would love to see actual benchmark results besides just speeds - results are more important than decode speeds I think

tl;dr - you made your own quant of deepseek and it doomloops?

i'd be afraid to let that lobotomized thing touch my computer...

>How do I overcome this?

by starting to do it yourself too

find out what AI the company has accepted (for data privacy reasons), and start testing it out. Tell it something like "can you write me a powershell script that will pull all hyper-v hosts from this cluster and show me important details about them?"

after getting comfortable with that, you can move onto installing a harness and having it start developing code, using the lessons that you learned by starting gradually. Remember to not just let it do whatever it wants, if you see something that makes you scratch your head, call it out and steer it in the direction you want

love the comments on HN

https://preview.redd.it/kwxntnhgp2xh1.png?width=1071&format=png&auto=webp&s=e534b0b6b91c77c47c48fcd1f184b4571ec1ac98

a 35B beats GLM-5.3? 🙄

any way to actually test it against agentic tasks with toolcalls to see how they compare?

ideally they should be exactly the same right?

tl;dr you doubled your speeds and intels are good now?