tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

rust: helpers: #define __rust_helper

Because of LLVM inling checks, it's generally not possible to inline a C
helper into Rust code, even with LTO:

* LLVM doesn't want to inline functions compiled with
`-fno-delete-null-pointer-checks` with code compiled without. The C
CGUs all have this enabled and Rust CGUs don't. Inlining is okay since
this is one of the hardening features that does not change the ABI,
and we shouldn't have null pointer dereferences in these helpers.

* LLVM doesn't want to inline functions with different list of builtins. C
side has `-fno-builtin-wcslen`; `wcslen` is not a Rust builtin, so
they should be compatible, but LLVM does not perform inlining due to
attributes mismatch.

* clang and Rust doesn't have the exact target string. Clang generates
`+cmov,+cx8,+fxsr` but Rust doesn't enable them (in fact, Rust will
complain if `-Ctarget-feature=+cmov,+cx8,+fxsr` is used). x86-64
always enable these features, so they are in fact the same target
string, but LLVM doesn't understand this and so inlining is inhibited.
This can be bypassed with `--ignore-tti-inline-compatible`, but this
is a hidden option.

To fix this, we can add __always_inline on every helper, which skips
these LLVM inlining checks. For this purpose, introduce a new
__rust_helper macro that needs to be added to every helper.

Most helpers already have __rust_helper specified, but there are a few
missing. The only consequence of this is that those specific helpers do
not get inlined.

Signed-off-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Andreas Hindborg <a.hindborg@kernel.org>
Reviewed-by: Andreas Hindborg <a.hindborg@kernel.org>
Link: https://patch.msgid.link/20260203-inline-helpers-v2-2-beb8547a03c9@google.com
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>

authored by

Gary Guo and committed by
Miguel Ojeda db702816 e90f97ce

+29
+29
rust/helpers/helpers.c
··· 7 7 * Sorted alphabetically. 8 8 */ 9 9 10 + #include <linux/compiler_types.h> 11 + 12 + #ifdef __BINDGEN__ 13 + // Omit `inline` for bindgen as it ignores inline functions. 10 14 #define __rust_helper 15 + #else 16 + // The helper functions are all inline functions. 17 + // 18 + // We use `__always_inline` here to bypass LLVM inlining checks, in case the 19 + // helpers are inlined directly into Rust CGUs. 20 + // 21 + // The LLVM inlining checks are false positives: 22 + // * LLVM doesn't want to inline functions compiled with 23 + // `-fno-delete-null-pointer-checks` with code compiled without. 24 + // The C CGUs all have this enabled and Rust CGUs don't. Inlining is okay 25 + // since this is one of the hardening features that does not change the ABI, 26 + // and we shouldn't have null pointer dereferences in these helpers. 27 + // * LLVM doesn't want to inline functions with different list of builtins. C 28 + // side has `-fno-builtin-wcslen`; `wcslen` is not a Rust builtin, so they 29 + // should be compatible, but LLVM does not perform inlining due to attributes 30 + // mismatch. 31 + // * clang and Rust doesn't have the exact target string. Clang generates 32 + // `+cmov,+cx8,+fxsr` but Rust doesn't enable them (in fact, Rust will 33 + // complain if `-Ctarget-feature=+cmov,+cx8,+fxsr` is used). x86-64 always 34 + // enable these features, so they are in fact the same target string, but 35 + // LLVM doesn't understand this and so inlining is inhibited. This can be 36 + // bypassed with `--ignore-tti-inline-compatible`, but this is a hidden 37 + // option. 38 + #define __rust_helper __always_inline 39 + #endif 11 40 12 41 #include "atomic.c" 13 42 #include "atomic_ext.c"