tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm/slab: avoid allocating slabobj_ext array from its own slab

When allocating slabobj_ext array in alloc_slab_obj_exts(), the array
can be allocated from the same slab we're allocating the array for.
This led to obj_exts_in_slab() incorrectly returning true [1],
although the array is not allocated from wasted space of the slab.

Vlastimil Babka observed that this problem should be fixed even when
ignoring its incompatibility with obj_exts_in_slab(), because it creates
slabs that are never freed as there is always at least one allocated
object.

To avoid this, use the next kmalloc size or large kmalloc when
the array can be allocated from the same cache we're allocating
the array for.

In case of random kmalloc caches, there are multiple kmalloc caches
for the same size and the cache is selected based on the caller address.
Because it is fragile to ensure the same caller address is passed to
kmalloc_slab(), kmalloc_noprof(), and kmalloc_node_noprof(), bump the
size to (s->object_size + 1) when the sizes are equal, instead of
directly comparing the kmem_cache pointers.

Note that this doesn't happen when memory allocation profiling is
disabled, as when the allocation of the array is triggered by memory
cgroup (KMALLOC_CGROUP), the array is allocated from KMALLOC_NORMAL.

Reported-by: kernel test robot <oliver.sang@intel.com>
Closes: https://lore.kernel.org/oe-lkp/202601231457.f7b31e09-lkp@intel.com [1]
Cc: stable@vger.kernel.org
Fixes: 4b8736964640 ("mm/slab: add allocation accounting into slab allocation and free paths")
Signed-off-by: Harry Yoo <harry.yoo@oracle.com>
Link: https://patch.msgid.link/20260126125714.88008-1-harry.yoo@oracle.com
Reviewed-by: Hao Li <hao.li@linux.dev>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>

authored by

Harry Yoo and committed by
Vlastimil Babka 280ea9c3 d907bf43

+53 -7
+53 -7
mm/slub.c
··· 2092 2092 slab->obj_exts = 0; 2093 2093 } 2094 2094 2095 + /* 2096 + * Calculate the allocation size for slabobj_ext array. 2097 + * 2098 + * When memory allocation profiling is enabled, the obj_exts array 2099 + * could be allocated from the same slab cache it's being allocated for. 2100 + * This would prevent the slab from ever being freed because it would 2101 + * always contain at least one allocated object (its own obj_exts array). 2102 + * 2103 + * To avoid this, increase the allocation size when we detect the array 2104 + * may come from the same cache, forcing it to use a different cache. 2105 + */ 2106 + static inline size_t obj_exts_alloc_size(struct kmem_cache *s, 2107 + struct slab *slab, gfp_t gfp) 2108 + { 2109 + size_t sz = sizeof(struct slabobj_ext) * slab->objects; 2110 + struct kmem_cache *obj_exts_cache; 2111 + 2112 + /* 2113 + * slabobj_ext array for KMALLOC_CGROUP allocations 2114 + * are served from KMALLOC_NORMAL caches. 2115 + */ 2116 + if (!mem_alloc_profiling_enabled()) 2117 + return sz; 2118 + 2119 + if (sz > KMALLOC_MAX_CACHE_SIZE) 2120 + return sz; 2121 + 2122 + if (!is_kmalloc_normal(s)) 2123 + return sz; 2124 + 2125 + obj_exts_cache = kmalloc_slab(sz, NULL, gfp, 0); 2126 + /* 2127 + * We can't simply compare s with obj_exts_cache, because random kmalloc 2128 + * caches have multiple caches per size, selected by caller address. 2129 + * Since caller address may differ between kmalloc_slab() and actual 2130 + * allocation, bump size when sizes are equal. 2131 + */ 2132 + if (s->object_size == obj_exts_cache->object_size) 2133 + return obj_exts_cache->object_size + 1; 2134 + 2135 + return sz; 2136 + } 2137 + 2095 2138 int alloc_slab_obj_exts(struct slab *slab, struct kmem_cache *s, 2096 2139 gfp_t gfp, bool new_slab) 2097 2140 { ··· 2143 2100 unsigned long new_exts; 2144 2101 unsigned long old_exts; 2145 2102 struct slabobj_ext *vec; 2103 + size_t sz; 2146 2104 2147 2105 gfp &= ~OBJCGS_CLEAR_MASK; 2148 2106 /* Prevent recursive extension vector allocation */ 2149 2107 gfp |= __GFP_NO_OBJ_EXT; 2108 + 2109 + sz = obj_exts_alloc_size(s, slab, gfp); 2150 2110 2151 2111 /* 2152 2112 * Note that allow_spin may be false during early boot and its ··· 2157 2111 * architectures with cmpxchg16b, early obj_exts will be missing for 2158 2112 * very early allocations on those. 2159 2113 */ 2160 - if (unlikely(!allow_spin)) { 2161 - size_t sz = objects * sizeof(struct slabobj_ext); 2162 - 2114 + if (unlikely(!allow_spin)) 2163 2115 vec = kmalloc_nolock(sz, __GFP_ZERO | __GFP_NO_OBJ_EXT, 2164 2116 slab_nid(slab)); 2165 - } else { 2166 - vec = kcalloc_node(objects, sizeof(struct slabobj_ext), gfp, 2167 - slab_nid(slab)); 2168 - } 2117 + else 2118 + vec = kmalloc_node(sz, gfp | __GFP_ZERO, slab_nid(slab)); 2119 + 2169 2120 if (!vec) { 2170 2121 /* 2171 2122 * Try to mark vectors which failed to allocate. ··· 2175 2132 2176 2133 return -ENOMEM; 2177 2134 } 2135 + 2136 + VM_WARN_ON_ONCE(virt_to_slab(vec) != NULL && 2137 + virt_to_slab(vec)->slab_cache == s); 2178 2138 2179 2139 new_exts = (unsigned long)vec; 2180 2140 if (unlikely(!allow_spin))