u/Timotheeee1

in about half of these cases the person actually did the test and told me, but fair enough I might be wrong about some of the rest

from a large stockpile built up over ages

it's just the kerafactor cap but cheaper

this is an LED cap, which is useless

https://i.imgur.com/U6y9ymV.png

usually yes, because the web versions tend to include enormous system prompts with hundreds of instructions, while the API has none

Anthropic shows the first 1k tokens or so, then a summary

go to meetup events

what happens if you instead use a specialized calibration dataset that contains only code or only english writing? you could probably prune the 235B down quite a lot more and make several specialist models.

what will the speed and cost be with a reasonable batch size once available on openrouter?

only the boss drops the schematic

actually I did build beds after building all of the farms

closed-source frontier models can be used to generate high quality data for fine-tuning local models that are specialized in specific tasks. (especially this one as it shows the reasoning traces)

they also provide a preview of the capabilities that open models will likely have in the future.

try asking in the unsloth discord

no, gradient checkpointing only offloads activations, but my idea would be to additionally offload model weights too

It would be cool if when doing QLoRA you could offload some layers of the model to the CPU and have them streamed to the GPU over PCIe as they are needed. in theory this shouldn't make the speed that much worse since PCIe has 63GB/s, enough bandwidth to stream the weights of a 32B 4 times per second, and it usually takes much longer than that do process one batch. this could allow for fine-tuning of larger models on colab and local hardware

It was solved a few months ago: https://arxiv.org/pdf/2409.12517v1

did you ever manage to fix this?

I think your project could still be useful for applying changes in a big file with 20k tokens, as you can't really expect the model to just re-output everything