u/Rusty_devl

Now, for extra fun or confusion, look at the project goal for Mir move elimination, if you want to see cases where LLVM isn't able to do so. :D

reddit is the wrong place for rustc discussions, you should join https://rust-lang.zulipchat.com for more answers. Probably ask in the t-compiler channel.

Congrats on finding a research topic involving Rust!

https://hyperboleandahalf.blogspot.com/2010/04/alot-is-better-than-you-at-everything.html

I'd love to chat, are you around during the next hour?

Neovim

Thanks for writing it up! I wanted to write a chess engine as one of my first rust projects, but fell into a rabbit hole along the way, so I'm even more happy to see others working on it. Are you also looking into improving your own engine?

They .. make a noise?

Very happy we now have a sensible fast-math in Rust 

yes-ish. omp (offload) is like three modes in a trench coat, some of which have a measurable overhead. Since I don't know which one people have in mind or experienced in the past, I try to avoid the comparison. My co-authors are luckily quite involved/knowledgable there, so we could avoid some past mistakes. On the Rust side we used some sufficiently low-level APIs from the omp/offload toolbox that we should at least for now avoid all overhead, I think the benchmarks show that quite well.

And indeed, we previously discussed instead developing a gpu version of something like rayon, but we felt that the current design is more in line with general Rust requirements.

I worked for half a year in the Julia Lab and I'm pretty sure I also referenced KA somewhere in my Project Goal proposals, it indeed gave me some inspirations. I like their idea of having vendor-agnostic features live there, and delegating people to the vendor-specific projects if you want features that are either too new to have made it into the general lib, or if you want features that are too vendor-unique to provide shared abstractions. In talks I mentioned I aim to have 80% of the Kernels being safe, idiomatic Rust and 20% using either unsafe or vendor specific features. My feeling is that Rust will have a harder time to become as composable as Julia, but in exchange we have less perf. pitfalls like Type Unstable Code or GPU memory overhead, so I'm ok with these trade-offs. Time will tell if we find equally nice abstractions, I think we have a fair chance. (Obviously not talking on behalf of any actual Julia devs, just my observations.)

On the autodiff side, I'd also like to see something like DI for a friendly competition, but I unfortunately don't see enough competing compiler-based autodiff tools to merit it yet.

I'm doing my PhD in a Quantum-Chemistry group (https://www.matter.toronto.edu/), so all of my applied work also ends up being wrapped in PyO3, otherwise 90% of my colleagues wouldn't touch it ^{^} I gave a tutorial at the scientific computing in Rust conference where for the same reason I showed that we can easily wrap std::autodiff in PyO3, the same should work for std::offload.

https://rustc-dev-guide.rust-lang.org/offload/usage.html#compile-instructions has instructions on this.

You have both a target (amd or nvidia) and a target-cpu (e.g. -Ctarget-cpu=gfx90a in the docs). If you use a newer target cpu you'll probably get better performance on newer GPUs, but it won't run on older ones. If you target an older one, you'll get relatively worse performance, but have more GPUs on which it will work.

Offload supports fat binaries, so you can have one host code with various Device codes embedded, you'd just need to run step 2 (the device compilation) a few times, once per target. At runtime offload could the pick the suitable one.

That is btw. very similar to CPUs, you generally can't compile code for x86 and then just run it on aarm64. You also have the opportunity to give a target-cpu to rustc when compiling for the CPU, but the perf differences are much smaller compared to GPUs, so it's usually just not that relevant.

For the offload side, so far we've tested Rust on a few AMD APUs (MI300A and some laptop ones), as well as desktop and server GPUs. Intel support is not exposed in Rust yet, but it was recently added on the LLVM side. I also know of experiments running Offload on Apple GPUs, but that's even more experimental than the Intel support.

The Pi specifically consumes Vulkan/SPIR-V. LLVM's Offload recently gained support from Intel for their GPUs, which also require SPIR-V, but those are different SPIR-V environments, so it doesn't necessarily translate: https://xkcd.com/927/

In general though, LLVM developers are very welcoming, so feel free to open a thread on https://discourse.llvm.org/ if you want to ask about future support. Otherwise, Rust-GPU might already work, since it targets the matching Vulkan/SPIR-V environment.

Glad to hear! After having worked for 6 years on autodiff, I'm quite happy with how quickly we got offload up and running. But there are also two more posts for autodiff coming up in the near term, once I finally stop distracting myself from writing.

Thanks!

We have a rustc-dev-guide chapter about the design, for contributors: https://rustc-dev-guide.rust-lang.org/offload/internals.html. Just yesterday we also merged a slightly higher-level offload macro for users, documented here: https://doc.rust-lang.org/nightly/std/offload/macro.offload.html#examples. We will add more user-facing docs as we get closer to nightly.

Wrt. Vectorware, Rust-Cuda was originally announced in 2021, so they had more years to implement features we don't have yet. In exchange, it often uses raw pointers, whereas offload and cuda-oxide provide safe interfaces. Rust-GPU is cross-vendor just like offload, but it started from a gaming/graphics background, which uses a quite different software stack compared to Offload (LLVM). They gave some very interesting talks at RustWeek about the trade-offs, e.g. how to represent pointers in SPIR-V. std::offload is based on LLVM, so we reuse the existing AMD/NVIDIA targets of rustc and (once available in rustc) can also add additional vendor support based on LLVM support.

Based on their latest blogpost it also sounds like Vectorware is exploring the same solution space libc-for-gpu offers on the LLVM side, so both should in the future support std in a similar fashion, in the case of Offload via libc-for-gpu. It's probably best to read their blog posts for the details of their longer-term goals.

My lab is pretty large (>10 postdocs, >20 PhDs), I had ~1 meeting with my PI per year and if I wanted an answer on one of my update emails I have to ask his staff team to remind him. I am at the end of my PhD and absolutely loved the liberties, but like every applicant I was also told early to expect this situation. It also doesn't mean a lack of mentorship, most people just collect feedback from PostDocs, and in my case I just gained (and also came with) a lot of external collaborators since my open source work gave me a little visibility.

Yes they allow FMA. As part of my work in a HPC lab I've used those, and they give good perf improvements both when using autodiff and GPU offload.  It's just a better version of -ffast-math since it skips two of it's foot guns which usually provide little benefits

Nice progress, and glad to hear that autodiff is good enough for you to be used here even without custom derivatives. Please ping me on any issues you encounter. (Fwiw I should really finish the blogpost to announce autodiff, lol)

Given that Clarabel and autodiff are also available in Julia, do you plan to do some benchmarking against Julia libraries? They obviously have been optimized for way longer, but my hope and theory is that it's easier to optimize Rust code to the same level, so I'd be interested in numbers.

Most posts here are just pure LLM outputs these days, I was positively surprised to see something that appears to be hand written for once. 

Isn't it also handwritten? https://github.com/minerscale/diffable/commits/main/

Medium Tracks how many article ppl have read on their side, only a very low amount is free, then it forces registration.