Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm: page_alloc: avoid kswapd thrashing due to NUMA restrictions

On NUMA systems without bindings, allocations check all nodes for free
space, then wake up the kswapds on all nodes and retry. This ensures
all available space is evenly used before reclaim begins. However,
when one process or certain allocations have node restrictions, they
can cause kswapds on only a subset of nodes to be woken up.

Since kswapd hysteresis targets watermarks that are *higher* than
needed for allocation, even *unrestricted* allocations can now get
suckered onto such nodes that are already pressured. This ends up
concentrating all allocations on them, even when there are idle nodes
available for the unrestricted requests.

This was observed with two numa nodes, where node0 is normal and node1
is ZONE_MOVABLE to facilitate hotplugging: a kernel allocation wakes
kswapd on node0 only (since node1 is not eligible); once kswapd0 is
active, the watermarks hover between low and high, and then even the
movable allocations end up on node0, only to be kicked out again;
meanwhile node1 is empty and idle.

Similar behavior is possible when a process with NUMA bindings is
causing selective kswapd wakeups.

To fix this, on NUMA systems augment the (misleading) watermark test
with a check for whether kswapd is already active during the first
iteration through the zonelist. If this fails to place the request,
kswapd must be running everywhere already, and the watermark test is
good enough to decide placement.

With this patch, unrestricted requests successfully make use of node1,
even while kswapd is reclaiming node0 for restricted allocations.

[gourry@gourry.net: don't retry if no kswapds were active]
Link: https://lkml.kernel.org/r/20250919162134.1098208-1-hannes@cmpxchg.org
Signed-off-by: Gregory Price <gourry@gourry.net>
Tested-by: Joshua Hahn <joshua.hahnjy@gmail.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Zi Yan <ziy@nvidia.com>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Johannes Weiner and committed by
Andrew Morton
19c5fb83 fde591da

+24
+24
mm/page_alloc.c
··· 3735 3735 struct pglist_data *last_pgdat = NULL; 3736 3736 bool last_pgdat_dirty_ok = false; 3737 3737 bool no_fallback; 3738 + bool skip_kswapd_nodes = nr_online_nodes > 1; 3739 + bool skipped_kswapd_nodes = false; 3738 3740 3739 3741 retry: 3740 3742 /* ··· 3797 3795 alloc_flags &= ~ALLOC_NOFRAGMENT; 3798 3796 goto retry; 3799 3797 } 3798 + } 3799 + 3800 + /* 3801 + * If kswapd is already active on a node, keep looking 3802 + * for other nodes that might be idle. This can happen 3803 + * if another process has NUMA bindings and is causing 3804 + * kswapd wakeups on only some nodes. Avoid accidental 3805 + * "node_reclaim_mode"-like behavior in this case. 3806 + */ 3807 + if (skip_kswapd_nodes && 3808 + !waitqueue_active(&zone->zone_pgdat->kswapd_wait)) { 3809 + skipped_kswapd_nodes = true; 3810 + continue; 3800 3811 } 3801 3812 3802 3813 cond_accept_memory(zone, order, alloc_flags); ··· 3901 3886 goto try_this_zone; 3902 3887 } 3903 3888 } 3889 + } 3890 + 3891 + /* 3892 + * If we skipped over nodes with active kswapds and found no 3893 + * idle nodes, retry and place anywhere the watermarks permit. 3894 + */ 3895 + if (skip_kswapd_nodes && skipped_kswapd_nodes) { 3896 + skip_kswapd_nodes = false; 3897 + goto retry; 3904 3898 } 3905 3899 3906 3900 /*