u/dbaupp

I think the point is more that one needs to do the bookkeeping right, or the compiler will accidentally affirm the consequent… which is, as you say, invalid.

It’s not as simple as “ignore an impl exists when checking it”, because one can have (mutual) recursion: given trait Magic { type T: Magic; } one might have impl Magic for A { type T = A; } that needs A: Magic for T or impl Magic for B { type T = C; } impl Magic for C { type T = B; } that need each other’s impl.

Validating that requires having some concept of “pending” impls, or similar. Not at all impossible, and not a violation of logic, just finicky bookkeeping that can lead to unsoundness and/or infinite recursion if done wrong. 

(The recursion can get far more complicated and indirect, via super-traits or generics and/or mutual recursion of traits/associated types.) 

Peculiar! The function itself only does anything if there’s 2 or more worktrees (source code), so maybe something else is going on.

I’ve used the snippet in the post in repos both with many worktrees and just the “default” one, without apparent issue.

That’s a very nice win! The single threaded nature of XProtect is such a drag.

Thank you for writing this. The pants build system has been struggling with the same problem on macOS, but we hadn’t yet discovered the secret setting! https://github.com/pantsbuild/pants/issues/22617

Huh, that’s an interesting idea! I’m not quite sure exactly how it’d fit together, but I can see that it might.

It’s not a bad idea, but it often seems to lead to worse code. The optimiser ends up being unable to decipher the dynamic mutations and so each iteration does the dynamic match to work out what to execute next, rather than just directly executing the next step.

That’s fine if the code being run is “heavyweight” (the dynamic checks are pretty quick), but not so good if it’s a tight bit of numeric code, where those extra checks end up being a large percentage of the overall time.

I am not aware! That seems like exactly the same as this hypothetical fallthrough, but expressed far better. Thanks for linking.

Yep. There’s two that I know of, although I’ve used neither and thus don’t know how well they work in practice:

• https://github.com/pythonesque/fallthrough
• https://github.com/Jules-Bertholet/fallthrough

Yes, you're right. That just happened to a real-world example of the perfect size for a blog post.

A hypothetical fallthrough keyword could also take a value that binds to the pattern, e.g. fallthrough Some(1).

match opt {
    None => fallthrough Some(1),
    Some(x) => {
        // `x` == 1 in the `None` case
    }
}

One could even allow "falling through" to an arbitrary other arm, by specifying a matching value, turning match into a state-machine executor (maybe with some restrictions like "the relevant branch to jump to should be statically known", and "match arms with if aren't supported"):

match state {
    State::A => if foo() { fallthrough State::B(1) } else { fallthrough State::C }
    State::B(x) => { ...; fallthrough State::D("foo") }
    State::C => { ...; fallthrough State::D("bar") }
    State::D(y) => { ....; fallthrough State::A }
}

Which would have two benefits:

• efficient implementations of state machines become easy (and they're "automatically" resumable, in some ways)
• match becomes Rust's 4th looping construct (and, I think, all others can be desugared to it)!

For the explicit enum  in Rust vs the goto of C, one can get much better code using labelled breaks, and a whole bunch of nesting.

The principle is a layer of nesting for each label (in C), from last to first (’done: loop { ’S9: loop { ‘S8: loop { … } … } …} … }). The actual code for each state is placed directly after the corresponding labelled loop (within the parent), so break ‘S9 starts running that code.  This thus behaves as goto but only for collections of jumps that make a DAG, so that one can nest the labels appropriately (in reverse topological ordering).

Example for the tight inner loop of a prime sieve (very opaque, though):

• generator: https://github.com/huonw/primal/blob/140650b0ebd0a571898a8834e3a6912daea62aa6/generators/src/bin/wheel-generator.rs#L214
• generated code example: https://github.com/huonw/primal/blob/140650b0ebd0a571898a8834e3a6912daea62aa6/primal-sieve/src/wheel/wheel30.rs#L201

(NB I suspect one might not need the loops any more. This is some old Rust code.)

Ah interesting. I was hoping the page fault could be handled asynchronously somehow; that is, still triggered and flip to kernel, but it returns quickly like any other non-blocking syscall, with the work happening in the background.

But, based on what you say, maybe that’s too much to hope for! I don’t know and haven’t investigated.

Yep! I think there’s already a fair amount of awareness that calling std::fs::File::read within async code is bad, but less awareness that memory mapping has the same problems (mmap is sometimes treated as magic go-fast juice, as someone else in this thread mentions).

Thanks for the kind words.

Using spawn_blocking would be one way to do this properly. However, the  blog post is intentionally exploring the consequence of incorrect code, answering “how bad is using mmap naively?” given the syntax makes it so easy. It isn’t trying to explore “how to use mmap properly with async/await”.

Yeah, that’d be one way to do this properly.

The blog post is intentionally exploring the consequence of incorrect code, answering “how bad is using mmap naively?” given the syntax makes it so easy. It isn’t trying to explore “how to use mmap properly with async/await”.

Thanks for the input! Are you using “blocking” in the specific technical sense of O_NONBLOCK / SOCK_NONBLOCK etc?

Is there a better word for operations like reading a file (or waiting for a page fault, in this case) that involve a syscall or other kernel operations that cause the thread to block/be descheduled for a moderate amount of time? (That is, not potentially-unbounded time like for network IO, but still in the many-microsecond to millisecond (or more) range.)

Hello, I appreciate the sentiment! I’m definitely only on the periphery of Rust now, just reading the TWiRs and generally following along. All of my open source energy is now going into https://github.com/pantsbuild/pants, which is a Rust-core/Python-outer application.

it wasn't obvious to me until the end of the article that this was benchmarking performance of mmap.. on XNU, macos's kernel

Ah, sorry for being misleading. I've added reference to macOS earlier in the article now.

Yeah, nice one. I've added them to the questions section.

It’s not magic run-real-fast sauce

Yeah, definitely agreed. I think it is sometimes talked of/used in these terms, though, hence I thought it worth diving into the details and confirm my understanding in reality.

Of course a synchronous call that could fetch a file into memory is blocking I/O.

Yeah, of course a synchronous call that might block the thread is blocking IO, I agree... but, if I didn't have the context of "we're in a comment thread about a blog post (I wrote) about mmap", I'm pretty sure I wouldn't flag `x[i]` on a `&[u8]` (or any other access) as a "synchronous call" that I might need to worry about.

Hence the discussion of subtlety in https://huonw.github.io/blog/2024/08/async-hazard-mmap/#at-a-distance

I was involved in some of the work, but only around the edges. I don’t recall who lead the fundamental design, but it was not me!

For that problem, you can work with smaller numbers by working in “log space”: taking the logarithm of all the formula and expanding. This turns division into subtraction, multiplication into addition, and exponentiation into multiplication. All of these will be much smaller (floating point) values. The final result can be computed by exponentiating at the end…

Of course, working with bigints might be more fun!

The 16 bytes of a UUID can be encoded in 22 characters in base64, which just fits under the 24 byte limit. Although that’s an unconventional representation and is more likely to result in false positives if attempting to parse unknown strings: the word electroencephalographs is a base64 representation of the UUID 5417da29-239d-453d-8cfc-6f8676cbce6f.

(As others point out though, HashMap<Uuid, T> would be better if possible.)