tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm/vmalloc: request large order pages from buddy allocator

Sometimes, vm_area_alloc_pages() will want many pages from the buddy
allocator. Rather than making requests to the buddy allocator for at most
100 pages at a time, we can eagerly request large order pages a smaller
number of times.

We still split the large order pages down to order-0 as the rest of the
vmalloc code (and some callers) depend on it. We still defer to the bulk
allocator and fallback path in case of order-0 pages or failure.

Running 1000 iterations of allocations on a small 4GB system finds:

1000 2mb allocations:
[Baseline] [This patch]
real 46.310s real 0m34.582
user 0.001s user 0.006s
sys 46.058s sys 0m34.365s

10000 200kb allocations:
[Baseline] [This patch]
real 56.104s real 0m43.696
user 0.001s user 0.003s
sys 55.375s sys 0m42.995s

Link: https://lkml.kernel.org/r/20251021194455.33351-2-vishal.moola@gmail.com
Signed-off-by: Vishal Moola (Oracle) <vishal.moola@gmail.com>
Reviewed-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Vishal Moola (Oracle) and committed by
Andrew Morton a0615780 645a3c42

+36
+36
mm/vmalloc.c
··· 3619 3619 unsigned int order, unsigned int nr_pages, struct page **pages) 3620 3620 { 3621 3621 unsigned int nr_allocated = 0; 3622 + unsigned int nr_remaining = nr_pages; 3623 + unsigned int max_attempt_order = MAX_PAGE_ORDER; 3622 3624 struct page *page; 3623 3625 int i; 3626 + gfp_t large_gfp = (gfp & 3627 + ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL | __GFP_COMP)) 3628 + | __GFP_NOWARN; 3629 + unsigned int large_order = ilog2(nr_remaining); 3630 + 3631 + large_order = min(max_attempt_order, large_order); 3632 + 3633 + /* 3634 + * Initially, attempt to have the page allocator give us large order 3635 + * pages. Do not attempt allocating smaller than order chunks since 3636 + * __vmap_pages_range() expects physically contigous pages of exactly 3637 + * order long chunks. 3638 + */ 3639 + while (large_order > order && nr_remaining) { 3640 + if (nid == NUMA_NO_NODE) 3641 + page = alloc_pages_noprof(large_gfp, large_order); 3642 + else 3643 + page = alloc_pages_node_noprof(nid, large_gfp, large_order); 3644 + 3645 + if (unlikely(!page)) { 3646 + max_attempt_order = --large_order; 3647 + continue; 3648 + } 3649 + 3650 + split_page(page, large_order); 3651 + for (i = 0; i < (1U << large_order); i++) 3652 + pages[nr_allocated + i] = page + i; 3653 + 3654 + nr_allocated += 1U << large_order; 3655 + nr_remaining = nr_pages - nr_allocated; 3656 + 3657 + large_order = ilog2(nr_remaining); 3658 + large_order = min(max_attempt_order, large_order); 3659 + } 3624 3660 3625 3661 /* 3626 3662 * For order-0 pages we make use of bulk allocator, if