tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

memcg: make memcg_rstat_updated nmi safe

Currently kernel maintains memory related stats updates per-cgroup to
optimize stats flushing. The stats_updates is defined as atomic64_t which
is not nmi-safe on some archs. Actually we don't really need 64bit atomic
as the max value stats_updates can get should be less than nr_cpus *
MEMCG_CHARGE_BATCH. A normal atomic_t should suffice.

Also the function cgroup_rstat_updated() is still not nmi-safe but there
is parallel effort to make it nmi-safe, so until then let's ignore it in
the nmi context.

Link: https://lkml.kernel.org/r/20250519063142.111219-6-shakeel.butt@linux.dev
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tejun Heo <tj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Shakeel Butt and committed by
Andrew Morton 3ac4638a 15ca4fa9

+9 -7
+9 -7
mm/memcontrol.c
··· 531 531 unsigned long events_pending[NR_MEMCG_EVENTS]; 532 532 533 533 /* Stats updates since the last flush */ 534 - atomic64_t stats_updates; 534 + atomic_t stats_updates; 535 535 }; 536 536 537 537 /* ··· 557 557 558 558 static bool memcg_vmstats_needs_flush(struct memcg_vmstats *vmstats) 559 559 { 560 - return atomic64_read(&vmstats->stats_updates) > 560 + return atomic_read(&vmstats->stats_updates) > 561 561 MEMCG_CHARGE_BATCH * num_online_cpus(); 562 562 } 563 563 ··· 571 571 if (!val) 572 572 return; 573 573 574 - cgroup_rstat_updated(memcg->css.cgroup, cpu); 574 + /* TODO: add to cgroup update tree once it is nmi-safe. */ 575 + if (!in_nmi()) 576 + cgroup_rstat_updated(memcg->css.cgroup, cpu); 575 577 statc_pcpu = memcg->vmstats_percpu; 576 578 for (; statc_pcpu; statc_pcpu = statc->parent_pcpu) { 577 579 statc = this_cpu_ptr(statc_pcpu); ··· 591 589 continue; 592 590 593 591 stats_updates = this_cpu_xchg(statc_pcpu->stats_updates, 0); 594 - atomic64_add(stats_updates, &statc->vmstats->stats_updates); 592 + atomic_add(stats_updates, &statc->vmstats->stats_updates); 595 593 } 596 594 } 597 595 ··· 599 597 { 600 598 bool needs_flush = memcg_vmstats_needs_flush(memcg->vmstats); 601 599 602 - trace_memcg_flush_stats(memcg, atomic64_read(&memcg->vmstats->stats_updates), 600 + trace_memcg_flush_stats(memcg, atomic_read(&memcg->vmstats->stats_updates), 603 601 force, needs_flush); 604 602 605 603 if (!force && !needs_flush) ··· 4121 4119 } 4122 4120 WRITE_ONCE(statc->stats_updates, 0); 4123 4121 /* We are in a per-cpu loop here, only do the atomic write once */ 4124 - if (atomic64_read(&memcg->vmstats->stats_updates)) 4125 - atomic64_set(&memcg->vmstats->stats_updates, 0); 4122 + if (atomic_read(&memcg->vmstats->stats_updates)) 4123 + atomic_set(&memcg->vmstats->stats_updates, 0); 4126 4124 } 4127 4125 4128 4126 static void mem_cgroup_fork(struct task_struct *task)