lz4_flex 0.7.2 reaches parity with cpp reference implementation on speed and ratio

I tend to rg unsafe and go from there, or even type unsafe in github search box.

In this case I looked up the commit message because I was curious how they sped that up, and realized they don't actually validate the value before adding it to the pointer.

https://doc.rust-lang.org/nomicon/ is a good resource for understanding what kind of unsafe code is OK and what isn't.

Fuzzing is also helpful for larger or more complex codebases.

As I understand it this is an api convention to signal the programmer unsafe code and does not cause UB itself, by missing it.

For private APIs, practically yes. However, it is still considered good style to mark even those functions as unsafe if they rely on a safety invariant that has to be upheld by the caller.

I'm not saying you should but you could have a macro like:

#[cfg(not(feature = "safe-encode"))]
macro_rules! maybe_unsafe {
    ($inner:expr) => {
        unsafe { $inner }
    };
}
#[cfg(feature = "safe-encode")]
macro_rules! maybe_unsafe {
    ($inner:expr) => {
        { $inner }
    };
}

that would allow you to write it as:

let duplicate_length = maybe_unsafe!(count_same_bytes(input, candidate as usize + MINMATCH, &mut cur));

Note that currently rustfmt will format code inside macros that are invoked with parenthesis ( (not brackets [ or {), so formatting should still work.

I think it boils down to “should private unsafe functions be marked as such even if they can’t be used unsafy by the public API” - which I don’t think there is a consensus on...

I would err on the side of marking unsafe just because a future dev might work on the project and use the function in an unsound way.

Fixed formatting.

Hello, Pascalius: code blocks using triple backticks (```) don't work on all versions of Reddit!

Some users see this / this instead.

To fix this, indent every line with 4 spaces instead.

FAQ

^{You can opt out by replying with backtickopt6 to this comment.}

if you don't trust your input

You should never trust your input.

If you aren't confident that the unsafe version is correct, please consider adding a disclaimer to the README.

You might already know this, but you can use cargo-fuzz, valgrind and miri to detect some problems.

Without checked-decode an out of bounds write could occur

Does that mean that your public API is marked as unsafe when not using checked-decode?