u/Hedede

What? A100 was released in 2020.

That's true. There are a lot of misconceptions about this.

I guess the hype is coming from the fact that some software ships (or at least used to) with FP32 path enabled by default. For example, I get 10-20% faster generation with --fast fp16_accumulation in ComfyUI and probably not everyone knows that.

It does use slightly more power, but it doesn't go over the power limit. My results with 4070 Ti Super:

We're talking about FP16 with FP32 accumulation.

Not applicable. P100 doesn't have tensor cores.

No, 50-series have a lock that's harder to bypass.

No, V100 is already running at full speed.

Both use the same tensor cores, the difference is that fp32 accumulate uses more registers to store the result at higher precision.

They're buying 5090s in bulk to convert them for datacenter use.

Likely just a measurement error. Also FP32 uses slightly more power so the GPU runs at slightly lower clocks.

Here are my results on 4070 Ti (FP16 matrix multiplication):

Because this limitation is mostly aimed at training.

Serving models isn't the problem. API inference runs at 70-80% margins (per SemiAnalysis), and subscriptions are only a small part of the revenue, everyone else is already paying per token.

What is unprofitable, and what is actually funded by debt, is releasing a new model every few months, because training them costs billions.

This isn't a "real" AI filter. This is just a tag. It doesn't cover all AI slop. And there are games themed about AI but not made with AI which this filters out.

Source? Anyone can swap the memory chips, but getting them to initialise properly is the hard part. Does 5090 VBIOS contain straps for 48 GB?

It uses more power. Doesn't change the TDP.

But it's very likely that you don't need it if you aren't fine-tuning models. If you were using FP16 accumulate you were already running at full speed and using the same power as hacked FP32 accumulate.

No, not enough, WSL doesn't have direct access to hardware.

Which part isn't true? It does increase the power draw but:

(1) It doesn't make GPU go beyond the power cap you set, period.

(2) The GPU was already hitting the same load with FP16 accumulate, so the GPU is already built for it.

If the chinese could figure out how to turn 4090 24GB into 48GB, why would it suddenly be an issue with rtx50 series?

Because there's a leaked 48GB VBIOS for RTX 4090. Chinese didn't figure anything out, they just took an existing VBIOS.

Compare to RTX 3090 that has no leaked VBIOS: people tired modding it by swapping memory chips for bigger capacity ones, but the card still "sees" only 24GB.

Do we know if disabling the cap has ZERO stability/heat/longevity issues.

I'd say there's a lot of evidence that this won't cause any issues.

Basically, there are two instructions: * FP16 matmul with FP16 accumulate -- runs at FULL speed; * FP16 matmul with FP32 accumulate -- runs at HALF speed;

Both are doing the exact same FP16 matmul on the same tensor cores; FP32 accumulate just keeps the running sum at higher precision. So uncapped, the load is essentially the same as FP16-accumulate workloads, which every GeForce card already runs at full speed.

And the exact same dies run FP32 accumulate at full rate in pro cards, e.g. the same full GA102 die is used in A6000/A40 and 3090 Ti, yet 3090 Ti has halved FP32 accumulate.

Plus there's evidence that FP16 accumulate internally computes at higher precision and then rounds down to FP16.

For comparison: when a chip genuinely can’t sustain an instruction at full power, vendors handle it dynamically, e.g. Skylake-SP Xeons drop to lower clocks when running heavy AVX-512 loads, rather than running a fixed 50% cap in microcode.

There's no guarantee that a 4090 that has 2,000 cuda cores disabled will be able turn those cores on and have them work reliably.

But this thread started with discussion unlocking instructions, not cores.

FP32 accumulate is an instruction replicated across every SM, a defect disables the whole SM, it doesn't occur uniformly in every single SM.

It might not fry the card but it might make it highly unstable.

It's not silicon lottery. FP32 accumulate is an instruction implemented in every SM. A defect doesn't occur uniformly across every SM. And FP16 accumulate already runs at full rate on those exact tensor cores.

If you didn't understand:

• FP16 matmul with FP16 accumulate -- FULL rate;
• FP16 matmul with FP32 accumulate -- HALF rate;

In a matmul, almost all the work is the multiplies, i.e. FP16 x FP16, accumulate is only a small step, and these cards would already highly unstalbe if they couldn't do full-rate tensor work.

They're literally discontinuing 5090 production so they can make more enterprise/datacenter cards. Is there a chance you will be fine applying this to your 4090?

There’s far lower demand for RTX 6000 than for 4090s, so plenty of fully-functioning dies end up in 4090s. They’re reportedly diverting dies from the 5090 now because RTX PRO demand went up. That only works if those dies were capable to begin with.

Higher power draw and potentially going beyond what the card may've been designed for, can kill your card. Even if it doesn't die instantly, it may still kill it faster.

I'm tired of people repeating this.

First off, this doesn't make the GPU go beyond the power limit. This just removes an artificial performance cap for certain instructions.

Second, yes it does raise the power draw. But you could get the exact same draw by running FP16 matmuls with FP16 accumulate, which have always run at full rate on GeForce.

It's not even a locked off area. It's just artificially capping the rate of certain instructions.

Unlocking does raise power consumption obviously because the GPU does more computations, but it's the same as switchting to full-rate FP16-with-FP16 accumulate which was never locked in the first place.

Are you sure? Official datasheet lists full BF16 rate for L40. Since BF16 doesn't support FP16 accumulate, it means that L40 has full rate FP32 accumulate.

Zero chance. It's the same chip as A40 which has it unlocked.