Currently the only way for Rust code to call a static inline function is to go through a helper in rust/helpers/. This introduces performance costs due to additional function calls and also clutters backtraces and flame graphs with helper symbols.
To get rid of these helper symbols, provide functionality to inline helpers into Rust using llvm-link. This option complements full LTO, by being much cheaper and avoiding incompatibility with BTF. I ran a microbenchmark showing the benefit of this. All the benchmark does is call refcount_inc() in a loop. This was chosen since refcounting is quite hot in Binder. The results are that Rust spends 6.35 ns per call vs 5.73 ns per call in C. When enabling this option, the two languages become equally fast, and disassembly confirms the exact same machine code is used (in particular there is no call to rust_helper_refcount_inc). Benchmarking Binder also results in an improvement from this change. This patch is complementary to: https://lore.kernel.org/all/[email protected]/ Signed-off-by: Alice Ryhl <[email protected]> --- Alice Ryhl (1): vmalloc: export vrealloc_node_align_noprof Gary Guo (3): rust: helpers: #define __rust_helper kbuild: rust: add `CONFIG_RUSTC_CLANG_LLVM_COMPATIBLE` build: rust: provide an option to inline C helpers into Rust Makefile | 4 +++- init/Kconfig | 15 +++++++++++++++ lib/Kconfig.debug | 15 +++++++++++++++ mm/vmalloc.c | 1 + rust/Makefile | 26 ++++++++++++++++++++++---- rust/exports.c | 5 ++++- rust/helpers/atomic.c | 5 ----- rust/helpers/helpers.c | 31 +++++++++++++++++++++++++++++++ scripts/Makefile.build | 5 ++++- scripts/atomic/gen-rust-atomic-helpers.sh | 5 ----- 10 files changed, 95 insertions(+), 17 deletions(-) --- base-commit: 54e3eae855629702c566bd2e130d9f40e7f35bde change-id: 20251202-inline-helpers-996f4db65e18 Best regards, -- Alice Ryhl <[email protected]>
