Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

slub: avoid list_lock contention from __refill_objects_any()

Kernel test robot has reported a regression in the patch "slab: refill
sheaves from all nodes". When taken in isolation like this, there is
indeed a tradeoff - we prefer to use remote objects prior to allocating
new local slabs. It is replicating a behavior that existed before
sheaves for replenishing cpu (partial) slabs - now called
get_from_any_partial() to allocate a single object.

So the possibility of allocating remote objects is intended even if
remote accesses are then slower. But the profiles in the report also
suggested a contention on the list_lock spinlock. And that's something
we can try to avoid without much tradeoff - if someone else has the
spin_lock, it's more likely they are allocating from the node than
freeing to it, so we can skip it even if it means allocating a new local
slab - contributing to that lock's contention isn't worth it. It should
not result in partial slabs accumulating on the remote node.

Thus add an allow_spin parameter to __refill_objects_node() and
get_partial_node_bulk() to make the attempts from __refill_objects_any()
use only a trylock.

Reported-by: kernel test robot <oliver.sang@intel.com>
Link: https://lore.kernel.org/oe-lkp/202601132136.77efd6d7-lkp@intel.com
Link: https://patch.msgid.link/20260129-b4-refill_any_trylock-v1-1-de7420b25840@suse.cz
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>

+13 -6
+13 -6
mm/slub.c
··· 3378 3378 3379 3379 static bool get_partial_node_bulk(struct kmem_cache *s, 3380 3380 struct kmem_cache_node *n, 3381 - struct partial_bulk_context *pc) 3381 + struct partial_bulk_context *pc, 3382 + bool allow_spin) 3382 3383 { 3383 3384 struct slab *slab, *slab2; 3384 3385 unsigned int total_free = 0; ··· 3391 3390 3392 3391 INIT_LIST_HEAD(&pc->slabs); 3393 3392 3394 - spin_lock_irqsave(&n->list_lock, flags); 3393 + if (allow_spin) 3394 + spin_lock_irqsave(&n->list_lock, flags); 3395 + else if (!spin_trylock_irqsave(&n->list_lock, flags)) 3396 + return false; 3395 3397 3396 3398 list_for_each_entry_safe(slab, slab2, &n->partial, slab_list) { 3397 3399 struct freelist_counters flc; ··· 6548 6544 6549 6545 static unsigned int 6550 6546 __refill_objects_node(struct kmem_cache *s, void **p, gfp_t gfp, unsigned int min, 6551 - unsigned int max, struct kmem_cache_node *n) 6547 + unsigned int max, struct kmem_cache_node *n, 6548 + bool allow_spin) 6552 6549 { 6553 6550 struct partial_bulk_context pc; 6554 6551 struct slab *slab, *slab2; ··· 6561 6556 pc.min_objects = min; 6562 6557 pc.max_objects = max; 6563 6558 6564 - if (!get_partial_node_bulk(s, n, &pc)) 6559 + if (!get_partial_node_bulk(s, n, &pc, allow_spin)) 6565 6560 return 0; 6566 6561 6567 6562 list_for_each_entry_safe(slab, slab2, &pc.slabs, slab_list) { ··· 6655 6650 n->nr_partial <= s->min_partial) 6656 6651 continue; 6657 6652 6658 - r = __refill_objects_node(s, p, gfp, min, max, n); 6653 + r = __refill_objects_node(s, p, gfp, min, max, n, 6654 + /* allow_spin = */ false); 6659 6655 refilled += r; 6660 6656 6661 6657 if (r >= min) { ··· 6697 6691 return 0; 6698 6692 6699 6693 refilled = __refill_objects_node(s, p, gfp, min, max, 6700 - get_node(s, local_node)); 6694 + get_node(s, local_node), 6695 + /* allow_spin = */ true); 6701 6696 if (refilled >= min) 6702 6697 return refilled; 6703 6698