module: Begin to move from RCU-sched to RCU.

The RCU usage in module was introduced in commit d72b37513cdfb ("Remove
stop_machine during module load v2") and it claimed not to be RCU but
similar. Then there was another improvement in commit e91defa26c527
("module: don't use stop_machine on module load"). It become a mix of
RCU and RCU-sched and was eventually fixed 0be964be0d450 ("module:
Sanitize RCU usage and locking"). Later RCU & RCU-sched was merged in
commit cb2f55369d3a9 ("modules: Replace synchronize_sched() and
call_rcu_sched()") so that was aligned.

Looking at it today, there is still leftovers. The preempt_disable() was
used instead rcu_read_lock_sched(). The RCU & RCU-sched merge was not
complete as there is still rcu_dereference_sched() for module::kallsyms.

The RCU-list modules and unloaded_tainted_modules are always accessed
under RCU protection or the module_mutex. The modules list iteration can
always happen safely because the module will not disappear.
Once the module is removed (free_module()) then after removing the
module from the list, there is a synchronize_rcu() which waits until
every RCU reader left the section. That means iterating over the list
within a RCU-read section is enough, there is no need to disable
preemption. module::kallsyms is first assigned in add_kallsyms() before
the module is added to the list. At this point, it points to init data.
This pointer is later updated and before the init code is removed there
is also synchronize_rcu() in do_free_init(). That means A RCU read lock
is enough for protection and rcu_dereference() can be safely used.

Convert module code and its users step by step. Update comments and
convert print_modules() to use RCU.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20250108090457.512198-3-bigeasy@linutronix.de
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>

authored by

Sebastian Andrzej Siewior and committed by

Petr Pavlu 1 year ago 838e6dd8 aa0fdccd

+8 -9

2 changed files

expand all

kernel

module

main.c

tree_lookup.c

+4 -5

kernel/module/main.c

··· 67 67 68 68 /* 69 69 * Mutex protects: 70 - * 1) List of modules (also safely readable with preempt_disable), 70 + * 1) List of modules (also safely readable within RCU read section), 71 71 * 2) module_use links, 72 72 * 3) mod_tree.addr_min/mod_tree.addr_max. 73 73 * (delete and add uses RCU list operations). ··· 1348 1348 mod_tree_remove(mod); 1349 1349 /* Remove this module from bug list, this uses list_del_rcu */ 1350 1350 module_bug_cleanup(mod); 1351 - /* Wait for RCU-sched synchronizing before releasing mod->list and buglist. */ 1351 + /* Wait for RCU synchronizing before releasing mod->list and buglist. */ 1352 1352 synchronize_rcu(); 1353 1353 if (try_add_tainted_module(mod)) 1354 1354 pr_err("%s: adding tainted module to the unloaded tainted modules list failed.\n", ··· 3049 3049 #endif 3050 3050 /* 3051 3051 * We want to free module_init, but be aware that kallsyms may be 3052 - * walking this with preempt disabled. In all the failure paths, we 3052 + * walking this within an RCU read section. In all the failure paths, we 3053 3053 * call synchronize_rcu(), but we don't want to slow down the success 3054 3054 * path. execmem_free() cannot be called in an interrupt, so do the 3055 3055 * work and call synchronize_rcu() in a work queue. ··· 3836 3836 3837 3837 printk(KERN_DEFAULT "Modules linked in:"); 3838 3838 /* Most callers should already have preempt disabled, but make sure */ 3839 - preempt_disable(); 3839 + guard(rcu)(); 3840 3840 list_for_each_entry_rcu(mod, &modules, list) { 3841 3841 if (mod->state == MODULE_STATE_UNFORMED) 3842 3842 continue; ··· 3844 3844 } 3845 3845 3846 3846 print_unloaded_tainted_modules(); 3847 - preempt_enable(); 3848 3847 if (last_unloaded_module.name[0]) 3849 3848 pr_cont(" [last unloaded: %s%s]", last_unloaded_module.name, 3850 3849 last_unloaded_module.taints);

+4 -4

kernel/module/tree_lookup.c

··· 12 12 13 13 /* 14 14 * Use a latched RB-tree for __module_address(); this allows us to use 15 - * RCU-sched lookups of the address from any context. 15 + * RCU lookups of the address from any context. 16 16 * 17 - * This is conditional on PERF_EVENTS || TRACING because those can really hit 18 - * __module_address() hard by doing a lot of stack unwinding; potentially from 19 - * NMI context. 17 + * This is conditional on PERF_EVENTS || TRACING || CFI_CLANG because those can 18 + * really hit __module_address() hard by doing a lot of stack unwinding; 19 + * potentially from NMI context. 20 20 */ 21 21 22 22 static __always_inline unsigned long __mod_tree_val(struct latch_tree_node *n)

Configure Feed

Configure Feed