tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm/balloon_compaction: remove dependency on page lock

Let's stop using the page lock in balloon code and instead use only the
balloon_device_lock.

As soon as we set the PG_movable_ops flag, we might now get isolation
callbacks for that page as we are no longer holding the page lock. In
there, we'll simply synchronize using the balloon_device_lock.

So in balloon_page_isolate() lookup the balloon_dev_info through
page->private under balloon_device_lock.

It's crucial that we update page->private under the balloon_device_lock,
so the isolation callback can properly deal with concurrent deflation.

Consequently, make sure that balloon_page_finalize() is called under
balloon_device_lock as we remove a page from the list and clear
page->private. balloon_page_insert() is already called with the
balloon_device_lock held.

Note that the core will still lock the pages, for example in
isolate_movable_ops_page(). The lock is there still relevant for handling
the PageMovableOpsIsolated flag, but that can be later changed to use an
atomic test-and-set instead, or moved into the movable_ops backends.

Link: https://lkml.kernel.org/r/20260119230133.3551867-10-david@kernel.org
Signed-off-by: David Hildenbrand (Red Hat) <david@kernel.org>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Eugenio Pérez <eperezma@redhat.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jason Wang <jasowang@redhat.com>
Cc: Jerrin Shaji George <jerrin.shaji-george@broadcom.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: SeongJae Park <sj@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

David Hildenbrand (Red Hat) and committed by
Andrew Morton a3fafdd3 8202313e

+25 -38
+13 -12
include/linux/balloon_compaction.h
··· 12 12 * is derived from the page type (PageOffline()) combined with the 13 13 * PG_movable_ops flag (PageMovableOps()). 14 14 * 15 + * Once the page type and the PG_movable_ops are set, migration code 16 + * can initiate page isolation by invoking the 17 + * movable_operations()->isolate_page() callback 18 + * 19 + * As long as page->private is set, the page is either on the balloon list 20 + * or isolated for migration. If page->private is not set, the page is 21 + * either still getting inflated, or was deflated to be freed by the balloon 22 + * driver soon. Isolation is impossible in both cases. 23 + * 15 24 * As the page isolation scanning step a compaction thread does is a lockless 16 25 * procedure (from a page standpoint), it might bring some racy situations while 17 26 * performing balloon page compaction. In order to sort out these racy scenarios 18 27 * and safely perform balloon's page compaction and migration we must, always, 19 28 * ensure following these simple rules: 20 29 * 21 - * i. Setting the PG_movable_ops flag and page->private with the following 22 - * lock order 23 - * +-page_lock(page); 24 - * +--spin_lock_irq(&balloon_pages_lock); 30 + * i. Inflation/deflation must set/clear page->private under the 31 + * balloon_pages_lock 25 32 * 26 33 * ii. isolation or dequeueing procedure must remove the page from balloon 27 34 * device page list under balloon_pages_lock 28 - * 29 - * The functions provided by this interface are placed to help on coping with 30 - * the aforementioned balloon page corner case, as well as to ensure the simple 31 - * set of exposed rules are satisfied while we are dealing with balloon pages 32 - * compaction / migration. 33 35 * 34 36 * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini <aquini@redhat.com> 35 37 */ ··· 95 93 * @balloon : pointer to balloon device 96 94 * @page : page to be assigned as a 'balloon page' 97 95 * 98 - * Caller must ensure the page is locked and the spin_lock protecting balloon 99 - * pages list is held before inserting a page into the balloon device. 96 + * Caller must ensure the balloon_pages_lock is held. 100 97 */ 101 98 static inline void balloon_page_insert(struct balloon_dev_info *balloon, 102 99 struct page *page) ··· 120 119 * balloon list for release to the page allocator 121 120 * @page: page to be released to the page allocator 122 121 * 123 - * Caller must ensure that the page is locked. 122 + * Caller must ensure the balloon_pages_lock is held. 124 123 */ 125 124 static inline void balloon_page_finalize(struct page *page) 126 125 {
+12 -26
mm/balloon_compaction.c
··· 20 20 static void balloon_page_enqueue_one(struct balloon_dev_info *b_dev_info, 21 21 struct page *page) 22 22 { 23 - /* 24 - * Block others from accessing the 'page' when we get around to 25 - * establishing additional references. We should be the only one 26 - * holding a reference to the 'page' at this point. If we are not, then 27 - * memory corruption is possible and we should stop execution. 28 - */ 29 - BUG_ON(!trylock_page(page)); 30 23 balloon_page_insert(b_dev_info, page); 31 - unlock_page(page); 32 24 if (b_dev_info->adjust_managed_page_count) 33 25 adjust_managed_page_count(page, -1); 34 26 __count_vm_event(BALLOON_INFLATE); ··· 85 93 list_for_each_entry_safe(page, tmp, &b_dev_info->pages, lru) { 86 94 if (n_pages == n_req_pages) 87 95 break; 88 - 89 - /* 90 - * Block others from accessing the 'page' while we get around to 91 - * establishing additional references and preparing the 'page' 92 - * to be released by the balloon driver. 93 - */ 94 - if (!trylock_page(page)) 95 - continue; 96 - 97 96 list_del(&page->lru); 98 97 if (b_dev_info->adjust_managed_page_count) 99 98 adjust_managed_page_count(page, 1); 100 99 balloon_page_finalize(page); 101 100 __count_vm_event(BALLOON_DEFLATE); 102 101 list_add(&page->lru, pages); 103 - unlock_page(page); 104 102 dec_node_page_state(page, NR_BALLOON_PAGES); 105 103 n_pages++; 106 104 } ··· 195 213 static bool balloon_page_isolate(struct page *page, isolate_mode_t mode) 196 214 197 215 { 198 - struct balloon_dev_info *b_dev_info = balloon_page_device(page); 216 + struct balloon_dev_info *b_dev_info; 199 217 unsigned long flags; 200 218 201 - if (!b_dev_info) 202 - return false; 203 - 204 219 spin_lock_irqsave(&balloon_pages_lock, flags); 220 + b_dev_info = balloon_page_device(page); 221 + if (!b_dev_info) { 222 + /* 223 + * The page already got deflated and removed from the 224 + * balloon list. 225 + */ 226 + spin_unlock_irqrestore(&balloon_pages_lock, flags); 227 + return false; 228 + } 205 229 list_del(&page->lru); 206 230 b_dev_info->isolated_pages++; 207 231 spin_unlock_irqrestore(&balloon_pages_lock, flags); ··· 240 252 struct balloon_dev_info *b_dev_info = balloon_page_device(page); 241 253 unsigned long flags; 242 254 int rc; 243 - 244 - VM_BUG_ON_PAGE(!PageLocked(page), page); 245 - VM_BUG_ON_PAGE(!PageLocked(newpage), newpage); 246 255 247 256 /* 248 257 * When we isolated the page, the page was still inflated in a balloon ··· 278 293 } 279 294 280 295 b_dev_info->isolated_pages--; 281 - spin_unlock_irqrestore(&balloon_pages_lock, flags); 282 296 283 297 /* Free the now-deflated page we isolated in balloon_page_isolate(). */ 284 298 balloon_page_finalize(page); 299 + spin_unlock_irqrestore(&balloon_pages_lock, flags); 300 + 285 301 put_page(page); 286 302 287 303 return 0;