tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

memcg: mm_update_next_owner: kill the "retry" logic

Add the new helper, try_to_set_owner(), which tries to update mm->owner
once we see c->mm == mm. This way mm_update_next_owner() doesn't need to
restart the list_for_each_entry/for_each_process loops from the very
beginning if it races with exit/exec, it can just continue.

Unlike the current code, try_to_set_owner() re-checks tsk->mm == mm before
it drops tasklist_lock, so it doesn't need get/put_task_struct().

Link: https://lkml.kernel.org/r/20240626152924.GA17933@redhat.com
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Eric W. Biederman <ebiederm@xmission.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Jinliang Zheng <alexjlzheng@tencent.com>
Cc: Mateusz Guzik <mjguzik@gmail.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Tycho Andersen <tandersen@netflix.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Oleg Nesterov and committed by
Andrew Morton 2a22b773 1419ff98

+27 -30
+27 -30
kernel/exit.c
··· 439 439 } 440 440 441 441 #ifdef CONFIG_MEMCG 442 + /* drops tasklist_lock if succeeds */ 443 + static bool try_to_set_owner(struct task_struct *tsk, struct mm_struct *mm) 444 + { 445 + bool ret = false; 446 + 447 + task_lock(tsk); 448 + if (likely(tsk->mm == mm)) { 449 + /* tsk can't pass exit_mm/exec_mmap and exit */ 450 + read_unlock(&tasklist_lock); 451 + WRITE_ONCE(mm->owner, tsk); 452 + lru_gen_migrate_mm(mm); 453 + ret = true; 454 + } 455 + task_unlock(tsk); 456 + return ret; 457 + } 458 + 442 459 /* 443 460 * A task is exiting. If it owned this mm, find a new owner for the mm. 444 461 */ ··· 463 446 { 464 447 struct task_struct *c, *g, *p = current; 465 448 466 - retry: 467 449 /* 468 450 * If the exiting or execing task is not the owner, it's 469 451 * someone else's problem. ··· 484 468 * Search in the children 485 469 */ 486 470 list_for_each_entry(c, &p->children, sibling) { 487 - if (c->mm == mm) 488 - goto assign_new_owner; 471 + if (c->mm == mm && try_to_set_owner(c, mm)) 472 + goto ret; 489 473 } 490 474 491 475 /* 492 476 * Search in the siblings 493 477 */ 494 478 list_for_each_entry(c, &p->real_parent->children, sibling) { 495 - if (c->mm == mm) 496 - goto assign_new_owner; 479 + if (c->mm == mm && try_to_set_owner(c, mm)) 480 + goto ret; 497 481 } 498 482 499 483 /* ··· 505 489 if (g->flags & PF_KTHREAD) 506 490 continue; 507 491 for_each_thread(g, c) { 508 - if (c->mm == mm) 509 - goto assign_new_owner; 510 - if (c->mm) 492 + struct mm_struct *c_mm = READ_ONCE(c->mm); 493 + if (c_mm == mm) { 494 + if (try_to_set_owner(c, mm)) 495 + goto ret; 496 + } else if (c_mm) 511 497 break; 512 498 } 513 499 } ··· 520 502 * ptrace or page migration (get_task_mm()). Mark owner as NULL. 521 503 */ 522 504 WRITE_ONCE(mm->owner, NULL); 505 + ret: 523 506 return; 524 507 525 - assign_new_owner: 526 - BUG_ON(c == p); 527 - get_task_struct(c); 528 - /* 529 - * The task_lock protects c->mm from changing. 530 - * We always want mm->owner->mm == mm 531 - */ 532 - task_lock(c); 533 - /* 534 - * Delay read_unlock() till we have the task_lock() 535 - * to ensure that c does not slip away underneath us 536 - */ 537 - read_unlock(&tasklist_lock); 538 - if (c->mm != mm) { 539 - task_unlock(c); 540 - put_task_struct(c); 541 - goto retry; 542 - } 543 - WRITE_ONCE(mm->owner, c); 544 - lru_gen_migrate_mm(mm); 545 - task_unlock(c); 546 - put_task_struct(c); 547 508 } 548 509 #endif /* CONFIG_MEMCG */ 549 510