tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Merge tag 'slab-for-6.9' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab

Pull slab updates from Vlastimil Babka:

- Freelist loading optimization (Chengming Zhou)

When the per-cpu slab is depleted and a new one loaded from the cpu
partial list, optimize the loading to avoid an irq enable/disable
cycle. This results in a 3.5% performance improvement on the "perf
bench sched messaging" test.

- Kernel boot parameters cleanup after SLAB removal (Xiongwei Song)

Due to two different main slab implementations we've had boot
parameters prefixed either slab_ and slub_ with some later becoming
an alias as both implementations gained the same functionality (i.e.
slab_nomerge vs slub_nomerge). In order to eventually get rid of the
implementation-specific names, the canonical and documented
parameters are now all prefixed slab_ and the slub_ variants become
deprecated but still working aliases.

- SLAB_ kmem_cache creation flags cleanup (Vlastimil Babka)

The flags had hardcoded #define values which became tedious and
error-prone when adding new ones. Assign the values via an enum that
takes care of providing unique bit numbers. Also deprecate
SLAB_MEM_SPREAD which was only used by SLAB, so it's a no-op since
SLAB removal. Assign it an explicit zero value. The removals of the
flag usage are handled independently in the respective subsystems,
with a final removal of any leftover usage planned for the next
release.

- Misc cleanups and fixes (Chengming Zhou, Xiaolei Wang, Zheng Yejian)

Includes removal of unused code or function parameters and a fix of a
memleak.

* tag 'slab-for-6.9' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab:
slab: remove PARTIAL_NODE slab_state
mm, slab: remove memcg_from_slab_obj()
mm, slab: remove the corner case of inc_slabs_node()
mm/slab: Fix a kmemleak in kmem_cache_destroy()
mm, slab, kasan: replace kasan_never_merge() with SLAB_NO_MERGE
mm, slab: use an enum to define SLAB_ cache creation flags
mm, slab: deprecate SLAB_MEM_SPREAD flag
mm, slab: fix the comment of cpu partial list
mm, slab: remove unused object_size parameter in kmem_cache_flags()
mm/slub: remove parameter 'flags' in create_kmalloc_caches()
mm/slub: remove unused parameter in next_freelist_entry()
mm/slub: remove full list manipulation for non-debug slab
mm/slub: directly load freelist from cpu partial slab in the likely case
mm/slub: make the description of slab_min_objects helpful in doc
mm/slub: replace slub_$params with slab_$params in slub.rst
mm/slub: unify all sl[au]b parameters with "slab_$param"
Documentation: kernel-parameters: remove noaliencache

Linus Torvalds 0ea680ed cc4a875c

+212 -217
+34 -45
Documentation/admin-guide/kernel-parameters.txt
··· 3771 3771 no5lvl [X86-64,RISCV,EARLY] Disable 5-level paging mode. Forces 3772 3772 kernel to use 4-level paging instead. 3773 3773 3774 - noaliencache [MM, NUMA, SLAB] Disables the allocation of alien 3775 - caches in the slab allocator. Saves per-node memory, 3776 - but will impact performance. 3777 - 3778 3774 noalign [KNL,ARM] 3779 3775 3780 3776 noaltinstr [S390,EARLY] Disables alternative instructions ··· 5926 5930 simeth= [IA-64] 5927 5931 simscsi= 5928 5932 5929 - slram= [HW,MTD] 5933 + slab_debug[=options[,slabs][;[options[,slabs]]...] [MM] 5934 + Enabling slab_debug allows one to determine the 5935 + culprit if slab objects become corrupted. Enabling 5936 + slab_debug can create guard zones around objects and 5937 + may poison objects when not in use. Also tracks the 5938 + last alloc / free. For more information see 5939 + Documentation/mm/slub.rst. 5940 + (slub_debug legacy name also accepted for now) 5941 + 5942 + slab_max_order= [MM] 5943 + Determines the maximum allowed order for slabs. 5944 + A high setting may cause OOMs due to memory 5945 + fragmentation. For more information see 5946 + Documentation/mm/slub.rst. 5947 + (slub_max_order legacy name also accepted for now) 5930 5948 5931 5949 slab_merge [MM] 5932 5950 Enable merging of slabs with similar size when the 5933 5951 kernel is built without CONFIG_SLAB_MERGE_DEFAULT. 5952 + (slub_merge legacy name also accepted for now) 5953 + 5954 + slab_min_objects= [MM] 5955 + The minimum number of objects per slab. SLUB will 5956 + increase the slab order up to slab_max_order to 5957 + generate a sufficiently large slab able to contain 5958 + the number of objects indicated. The higher the number 5959 + of objects the smaller the overhead of tracking slabs 5960 + and the less frequently locks need to be acquired. 5961 + For more information see Documentation/mm/slub.rst. 5962 + (slub_min_objects legacy name also accepted for now) 5963 + 5964 + slab_min_order= [MM] 5965 + Determines the minimum page order for slabs. Must be 5966 + lower or equal to slab_max_order. For more information see 5967 + Documentation/mm/slub.rst. 5968 + (slub_min_order legacy name also accepted for now) 5934 5969 5935 5970 slab_nomerge [MM] 5936 5971 Disable merging of slabs with similar size. May be ··· 5975 5948 unchanged). Debug options disable merging on their 5976 5949 own. 5977 5950 For more information see Documentation/mm/slub.rst. 5951 + (slub_nomerge legacy name also accepted for now) 5978 5952 5979 - slab_max_order= [MM, SLAB] 5980 - Determines the maximum allowed order for slabs. 5981 - A high setting may cause OOMs due to memory 5982 - fragmentation. Defaults to 1 for systems with 5983 - more than 32MB of RAM, 0 otherwise. 5984 - 5985 - slub_debug[=options[,slabs][;[options[,slabs]]...] [MM, SLUB] 5986 - Enabling slub_debug allows one to determine the 5987 - culprit if slab objects become corrupted. Enabling 5988 - slub_debug can create guard zones around objects and 5989 - may poison objects when not in use. Also tracks the 5990 - last alloc / free. For more information see 5991 - Documentation/mm/slub.rst. 5992 - 5993 - slub_max_order= [MM, SLUB] 5994 - Determines the maximum allowed order for slabs. 5995 - A high setting may cause OOMs due to memory 5996 - fragmentation. For more information see 5997 - Documentation/mm/slub.rst. 5998 - 5999 - slub_min_objects= [MM, SLUB] 6000 - The minimum number of objects per slab. SLUB will 6001 - increase the slab order up to slub_max_order to 6002 - generate a sufficiently large slab able to contain 6003 - the number of objects indicated. The higher the number 6004 - of objects the smaller the overhead of tracking slabs 6005 - and the less frequently locks need to be acquired. 6006 - For more information see Documentation/mm/slub.rst. 6007 - 6008 - slub_min_order= [MM, SLUB] 6009 - Determines the minimum page order for slabs. Must be 6010 - lower than slub_max_order. 6011 - For more information see Documentation/mm/slub.rst. 6012 - 6013 - slub_merge [MM, SLUB] 6014 - Same with slab_merge. 6015 - 6016 - slub_nomerge [MM, SLUB] 6017 - Same with slab_nomerge. This is supported for legacy. 6018 - See slab_nomerge for more information. 5953 + slram= [HW,MTD] 6019 5954 6020 5955 smart2= [HW] 6021 5956 Format: <io1>[,<io2>[,...,<io8>]]
+30 -30
Documentation/mm/slub.rst
··· 9 9 an impact on overall system performance which may make a bug more 10 10 difficult to find. 11 11 12 - In order to switch debugging on one can add an option ``slub_debug`` 12 + In order to switch debugging on one can add an option ``slab_debug`` 13 13 to the kernel command line. That will enable full debugging for 14 14 all slabs. 15 15 ··· 26 26 available without debugging on and validation can only partially 27 27 be performed if debugging was not switched on. 28 28 29 - Some more sophisticated uses of slub_debug: 29 + Some more sophisticated uses of slab_debug: 30 30 ------------------------------------------- 31 31 32 - Parameters may be given to ``slub_debug``. If none is specified then full 32 + Parameters may be given to ``slab_debug``. If none is specified then full 33 33 debugging is enabled. Format: 34 34 35 - slub_debug=<Debug-Options> 35 + slab_debug=<Debug-Options> 36 36 Enable options for all slabs 37 37 38 - slub_debug=<Debug-Options>,<slab name1>,<slab name2>,... 38 + slab_debug=<Debug-Options>,<slab name1>,<slab name2>,... 39 39 Enable options only for select slabs (no spaces 40 40 after a comma) 41 41 ··· 60 60 61 61 F.e. in order to boot just with sanity checks and red zoning one would specify:: 62 62 63 - slub_debug=FZ 63 + slab_debug=FZ 64 64 65 65 Trying to find an issue in the dentry cache? Try:: 66 66 67 - slub_debug=,dentry 67 + slab_debug=,dentry 68 68 69 69 to only enable debugging on the dentry cache. You may use an asterisk at the 70 70 end of the slab name, in order to cover all slabs with the same prefix. For 71 71 example, here's how you can poison the dentry cache as well as all kmalloc 72 72 slabs:: 73 73 74 - slub_debug=P,kmalloc-*,dentry 74 + slab_debug=P,kmalloc-*,dentry 75 75 76 76 Red zoning and tracking may realign the slab. We can just apply sanity checks 77 77 to the dentry cache with:: 78 78 79 - slub_debug=F,dentry 79 + slab_debug=F,dentry 80 80 81 81 Debugging options may require the minimum possible slab order to increase as 82 82 a result of storing the metadata (for example, caches with PAGE_SIZE object ··· 84 84 in low memory situations or if there's high fragmentation of memory. To 85 85 switch off debugging for such caches by default, use:: 86 86 87 - slub_debug=O 87 + slab_debug=O 88 88 89 89 You can apply different options to different list of slab names, using blocks 90 90 of options. This will enable red zoning for dentry and user tracking for 91 91 kmalloc. All other slabs will not get any debugging enabled:: 92 92 93 - slub_debug=Z,dentry;U,kmalloc-* 93 + slab_debug=Z,dentry;U,kmalloc-* 94 94 95 95 You can also enable options (e.g. sanity checks and poisoning) for all caches 96 96 except some that are deemed too performance critical and don't need to be 97 97 debugged by specifying global debug options followed by a list of slab names 98 98 with "-" as options:: 99 99 100 - slub_debug=FZ;-,zs_handle,zspage 100 + slab_debug=FZ;-,zs_handle,zspage 101 101 102 102 The state of each debug option for a slab can be found in the respective files 103 103 under:: ··· 105 105 /sys/kernel/slab/<slab name>/ 106 106 107 107 If the file contains 1, the option is enabled, 0 means disabled. The debug 108 - options from the ``slub_debug`` parameter translate to the following files:: 108 + options from the ``slab_debug`` parameter translate to the following files:: 109 109 110 110 F sanity_checks 111 111 Z red_zone ··· 129 129 Slab validation 130 130 =============== 131 131 132 - SLUB can validate all object if the kernel was booted with slub_debug. In 132 + SLUB can validate all object if the kernel was booted with slab_debug. In 133 133 order to do so you must have the ``slabinfo`` tool. Then you can do 134 134 :: 135 135 ··· 150 150 governed by the order of the allocation for each slab. The allocations 151 151 can be influenced by kernel parameters: 152 152 153 - .. slub_min_objects=x (default 4) 154 - .. slub_min_order=x (default 0) 155 - .. slub_max_order=x (default 3 (PAGE_ALLOC_COSTLY_ORDER)) 153 + .. slab_min_objects=x (default: automatically scaled by number of cpus) 154 + .. slab_min_order=x (default 0) 155 + .. slab_max_order=x (default 3 (PAGE_ALLOC_COSTLY_ORDER)) 156 156 157 - ``slub_min_objects`` 157 + ``slab_min_objects`` 158 158 allows to specify how many objects must at least fit into one 159 159 slab in order for the allocation order to be acceptable. In 160 160 general slub will be able to perform this number of 161 161 allocations on a slab without consulting centralized resources 162 162 (list_lock) where contention may occur. 163 163 164 - ``slub_min_order`` 164 + ``slab_min_order`` 165 165 specifies a minimum order of slabs. A similar effect like 166 - ``slub_min_objects``. 166 + ``slab_min_objects``. 167 167 168 - ``slub_max_order`` 169 - specified the order at which ``slub_min_objects`` should no 168 + ``slab_max_order`` 169 + specified the order at which ``slab_min_objects`` should no 170 170 longer be checked. This is useful to avoid SLUB trying to 171 - generate super large order pages to fit ``slub_min_objects`` 171 + generate super large order pages to fit ``slab_min_objects`` 172 172 of a slab cache with large object sizes into one high order 173 173 page. Setting command line parameter 174 174 ``debug_guardpage_minorder=N`` (N > 0), forces setting 175 - ``slub_max_order`` to 0, what cause minimum possible order of 175 + ``slab_max_order`` to 0, what cause minimum possible order of 176 176 slabs allocation. 177 177 178 178 SLUB Debug output ··· 219 219 FIX kmalloc-8: Restoring Redzone 0xc90f6d28-0xc90f6d2b=0xcc 220 220 221 221 If SLUB encounters a corrupted object (full detection requires the kernel 222 - to be booted with slub_debug) then the following output will be dumped 222 + to be booted with slab_debug) then the following output will be dumped 223 223 into the syslog: 224 224 225 225 1. Description of the problem encountered ··· 239 239 pid=<pid of the process> 240 240 241 241 (Object allocation / free information is only available if SLAB_STORE_USER is 242 - set for the slab. slub_debug sets that option) 242 + set for the slab. slab_debug sets that option) 243 243 244 244 2. The object contents if an object was involved. 245 245 ··· 262 262 the object boundary. 263 263 264 264 (Redzone information is only available if SLAB_RED_ZONE is set. 265 - slub_debug sets that option) 265 + slab_debug sets that option) 266 266 267 267 Padding <address> : <bytes> 268 268 Unused data to fill up the space in order to get the next object ··· 296 296 297 297 Minimal debugging (sanity checks alone) can be enabled by booting with:: 298 298 299 - slub_debug=F 299 + slab_debug=F 300 300 301 301 This will be generally be enough to enable the resiliency features of slub 302 302 which will keep the system running even if a bad kernel component will ··· 311 311 312 312 I.e.:: 313 313 314 - slub_debug=F,dentry 314 + slab_debug=F,dentry 315 315 316 316 If the corruption occurs by writing after the end of the object then it 317 317 may be advisable to enable a Redzone to avoid corrupting the beginning 318 318 of other objects:: 319 319 320 - slub_debug=FZ,dentry 320 + slab_debug=FZ,dentry 321 321 322 322 Extended slabinfo mode and plotting 323 323 ===================================
+1 -1
drivers/misc/lkdtm/heap.c
··· 48 48 * correctly. 49 49 * 50 50 * This should get caught by either memory tagging, KASan, or by using 51 - * CONFIG_SLUB_DEBUG=y and slub_debug=ZF (or CONFIG_SLUB_DEBUG_ON=y). 51 + * CONFIG_SLUB_DEBUG=y and slab_debug=ZF (or CONFIG_SLUB_DEBUG_ON=y). 52 52 */ 53 53 static void lkdtm_SLAB_LINEAR_OVERFLOW(void) 54 54 {
-6
include/linux/kasan.h
··· 429 429 }; 430 430 431 431 size_t kasan_metadata_size(struct kmem_cache *cache, bool in_object); 432 - slab_flags_t kasan_never_merge(void); 433 432 void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, 434 433 slab_flags_t *flags); 435 434 ··· 442 443 /* Tag-based KASAN modes do not use per-object metadata. */ 443 444 static inline size_t kasan_metadata_size(struct kmem_cache *cache, 444 445 bool in_object) 445 - { 446 - return 0; 447 - } 448 - /* And thus nothing prevents cache merging. */ 449 - static inline slab_flags_t kasan_never_merge(void) 450 446 { 451 447 return 0; 452 448 }
+69 -28
include/linux/slab.h
··· 21 21 #include <linux/cleanup.h> 22 22 #include <linux/hash.h> 23 23 24 + enum _slab_flag_bits { 25 + _SLAB_CONSISTENCY_CHECKS, 26 + _SLAB_RED_ZONE, 27 + _SLAB_POISON, 28 + _SLAB_KMALLOC, 29 + _SLAB_HWCACHE_ALIGN, 30 + _SLAB_CACHE_DMA, 31 + _SLAB_CACHE_DMA32, 32 + _SLAB_STORE_USER, 33 + _SLAB_PANIC, 34 + _SLAB_TYPESAFE_BY_RCU, 35 + _SLAB_TRACE, 36 + #ifdef CONFIG_DEBUG_OBJECTS 37 + _SLAB_DEBUG_OBJECTS, 38 + #endif 39 + _SLAB_NOLEAKTRACE, 40 + _SLAB_NO_MERGE, 41 + #ifdef CONFIG_FAILSLAB 42 + _SLAB_FAILSLAB, 43 + #endif 44 + #ifdef CONFIG_MEMCG_KMEM 45 + _SLAB_ACCOUNT, 46 + #endif 47 + #ifdef CONFIG_KASAN_GENERIC 48 + _SLAB_KASAN, 49 + #endif 50 + _SLAB_NO_USER_FLAGS, 51 + #ifdef CONFIG_KFENCE 52 + _SLAB_SKIP_KFENCE, 53 + #endif 54 + #ifndef CONFIG_SLUB_TINY 55 + _SLAB_RECLAIM_ACCOUNT, 56 + #endif 57 + _SLAB_OBJECT_POISON, 58 + _SLAB_CMPXCHG_DOUBLE, 59 + _SLAB_FLAGS_LAST_BIT 60 + }; 61 + 62 + #define __SLAB_FLAG_BIT(nr) ((slab_flags_t __force)(1U << (nr))) 63 + #define __SLAB_FLAG_UNUSED ((slab_flags_t __force)(0U)) 24 64 25 65 /* 26 66 * Flags to pass to kmem_cache_create(). 27 67 * The ones marked DEBUG need CONFIG_SLUB_DEBUG enabled, otherwise are no-op 28 68 */ 29 69 /* DEBUG: Perform (expensive) checks on alloc/free */ 30 - #define SLAB_CONSISTENCY_CHECKS ((slab_flags_t __force)0x00000100U) 70 + #define SLAB_CONSISTENCY_CHECKS __SLAB_FLAG_BIT(_SLAB_CONSISTENCY_CHECKS) 31 71 /* DEBUG: Red zone objs in a cache */ 32 - #define SLAB_RED_ZONE ((slab_flags_t __force)0x00000400U) 72 + #define SLAB_RED_ZONE __SLAB_FLAG_BIT(_SLAB_RED_ZONE) 33 73 /* DEBUG: Poison objects */ 34 - #define SLAB_POISON ((slab_flags_t __force)0x00000800U) 74 + #define SLAB_POISON __SLAB_FLAG_BIT(_SLAB_POISON) 35 75 /* Indicate a kmalloc slab */ 36 - #define SLAB_KMALLOC ((slab_flags_t __force)0x00001000U) 76 + #define SLAB_KMALLOC __SLAB_FLAG_BIT(_SLAB_KMALLOC) 37 77 /* Align objs on cache lines */ 38 - #define SLAB_HWCACHE_ALIGN ((slab_flags_t __force)0x00002000U) 78 + #define SLAB_HWCACHE_ALIGN __SLAB_FLAG_BIT(_SLAB_HWCACHE_ALIGN) 39 79 /* Use GFP_DMA memory */ 40 - #define SLAB_CACHE_DMA ((slab_flags_t __force)0x00004000U) 80 + #define SLAB_CACHE_DMA __SLAB_FLAG_BIT(_SLAB_CACHE_DMA) 41 81 /* Use GFP_DMA32 memory */ 42 - #define SLAB_CACHE_DMA32 ((slab_flags_t __force)0x00008000U) 82 + #define SLAB_CACHE_DMA32 __SLAB_FLAG_BIT(_SLAB_CACHE_DMA32) 43 83 /* DEBUG: Store the last owner for bug hunting */ 44 - #define SLAB_STORE_USER ((slab_flags_t __force)0x00010000U) 84 + #define SLAB_STORE_USER __SLAB_FLAG_BIT(_SLAB_STORE_USER) 45 85 /* Panic if kmem_cache_create() fails */ 46 - #define SLAB_PANIC ((slab_flags_t __force)0x00040000U) 86 + #define SLAB_PANIC __SLAB_FLAG_BIT(_SLAB_PANIC) 47 87 /* 48 88 * SLAB_TYPESAFE_BY_RCU - **WARNING** READ THIS! 49 89 * ··· 135 95 * Note that SLAB_TYPESAFE_BY_RCU was originally named SLAB_DESTROY_BY_RCU. 136 96 */ 137 97 /* Defer freeing slabs to RCU */ 138 - #define SLAB_TYPESAFE_BY_RCU ((slab_flags_t __force)0x00080000U) 139 - /* Spread some memory over cpuset */ 140 - #define SLAB_MEM_SPREAD ((slab_flags_t __force)0x00100000U) 98 + #define SLAB_TYPESAFE_BY_RCU __SLAB_FLAG_BIT(_SLAB_TYPESAFE_BY_RCU) 141 99 /* Trace allocations and frees */ 142 - #define SLAB_TRACE ((slab_flags_t __force)0x00200000U) 100 + #define SLAB_TRACE __SLAB_FLAG_BIT(_SLAB_TRACE) 143 101 144 102 /* Flag to prevent checks on free */ 145 103 #ifdef CONFIG_DEBUG_OBJECTS 146 - # define SLAB_DEBUG_OBJECTS ((slab_flags_t __force)0x00400000U) 104 + # define SLAB_DEBUG_OBJECTS __SLAB_FLAG_BIT(_SLAB_DEBUG_OBJECTS) 147 105 #else 148 - # define SLAB_DEBUG_OBJECTS 0 106 + # define SLAB_DEBUG_OBJECTS __SLAB_FLAG_UNUSED 149 107 #endif 150 108 151 109 /* Avoid kmemleak tracing */ 152 - #define SLAB_NOLEAKTRACE ((slab_flags_t __force)0x00800000U) 110 + #define SLAB_NOLEAKTRACE __SLAB_FLAG_BIT(_SLAB_NOLEAKTRACE) 153 111 154 112 /* 155 113 * Prevent merging with compatible kmem caches. This flag should be used ··· 159 121 * - performance critical caches, should be very rare and consulted with slab 160 122 * maintainers, and not used together with CONFIG_SLUB_TINY 161 123 */ 162 - #define SLAB_NO_MERGE ((slab_flags_t __force)0x01000000U) 124 + #define SLAB_NO_MERGE __SLAB_FLAG_BIT(_SLAB_NO_MERGE) 163 125 164 126 /* Fault injection mark */ 165 127 #ifdef CONFIG_FAILSLAB 166 - # define SLAB_FAILSLAB ((slab_flags_t __force)0x02000000U) 128 + # define SLAB_FAILSLAB __SLAB_FLAG_BIT(_SLAB_FAILSLAB) 167 129 #else 168 - # define SLAB_FAILSLAB 0 130 + # define SLAB_FAILSLAB __SLAB_FLAG_UNUSED 169 131 #endif 170 132 /* Account to memcg */ 171 133 #ifdef CONFIG_MEMCG_KMEM 172 - # define SLAB_ACCOUNT ((slab_flags_t __force)0x04000000U) 134 + # define SLAB_ACCOUNT __SLAB_FLAG_BIT(_SLAB_ACCOUNT) 173 135 #else 174 - # define SLAB_ACCOUNT 0 136 + # define SLAB_ACCOUNT __SLAB_FLAG_UNUSED 175 137 #endif 176 138 177 139 #ifdef CONFIG_KASAN_GENERIC 178 - #define SLAB_KASAN ((slab_flags_t __force)0x08000000U) 140 + #define SLAB_KASAN __SLAB_FLAG_BIT(_SLAB_KASAN) 179 141 #else 180 - #define SLAB_KASAN 0 142 + #define SLAB_KASAN __SLAB_FLAG_UNUSED 181 143 #endif 182 144 183 145 /* ··· 185 147 * Intended for caches created for self-tests so they have only flags 186 148 * specified in the code and other flags are ignored. 187 149 */ 188 - #define SLAB_NO_USER_FLAGS ((slab_flags_t __force)0x10000000U) 150 + #define SLAB_NO_USER_FLAGS __SLAB_FLAG_BIT(_SLAB_NO_USER_FLAGS) 189 151 190 152 #ifdef CONFIG_KFENCE 191 - #define SLAB_SKIP_KFENCE ((slab_flags_t __force)0x20000000U) 153 + #define SLAB_SKIP_KFENCE __SLAB_FLAG_BIT(_SLAB_SKIP_KFENCE) 192 154 #else 193 - #define SLAB_SKIP_KFENCE 0 155 + #define SLAB_SKIP_KFENCE __SLAB_FLAG_UNUSED 194 156 #endif 195 157 196 158 /* The following flags affect the page allocator grouping pages by mobility */ 197 159 /* Objects are reclaimable */ 198 160 #ifndef CONFIG_SLUB_TINY 199 - #define SLAB_RECLAIM_ACCOUNT ((slab_flags_t __force)0x00020000U) 161 + #define SLAB_RECLAIM_ACCOUNT __SLAB_FLAG_BIT(_SLAB_RECLAIM_ACCOUNT) 200 162 #else 201 - #define SLAB_RECLAIM_ACCOUNT ((slab_flags_t __force)0) 163 + #define SLAB_RECLAIM_ACCOUNT __SLAB_FLAG_UNUSED 202 164 #endif 203 165 #define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */ 166 + 167 + /* Obsolete unused flag, to be removed */ 168 + #define SLAB_MEM_SPREAD __SLAB_FLAG_UNUSED 204 169 205 170 /* 206 171 * ZERO_SIZE_PTR will be returned for zero sized kmalloc requests.
+3 -3
mm/Kconfig.debug
··· 64 64 help 65 65 Boot with debugging on by default. SLUB boots by default with 66 66 the runtime debug capabilities switched off. Enabling this is 67 - equivalent to specifying the "slub_debug" parameter on boot. 67 + equivalent to specifying the "slab_debug" parameter on boot. 68 68 There is no support for more fine grained debug control like 69 - possible with slub_debug=xxx. SLUB debugging may be switched 69 + possible with slab_debug=xxx. SLUB debugging may be switched 70 70 off in a kernel built with CONFIG_SLUB_DEBUG_ON by specifying 71 - "slub_debug=-". 71 + "slab_debug=-". 72 72 73 73 config PAGE_OWNER 74 74 bool "Track page owner"
+6 -16
mm/kasan/generic.c
··· 334 334 DEFINE_ASAN_SET_SHADOW(f5); 335 335 DEFINE_ASAN_SET_SHADOW(f8); 336 336 337 - /* Only allow cache merging when no per-object metadata is present. */ 338 - slab_flags_t kasan_never_merge(void) 339 - { 340 - if (!kasan_requires_meta()) 341 - return 0; 342 - return SLAB_KASAN; 343 - } 344 - 345 337 /* 346 338 * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. 347 339 * For larger allocations larger redzones are used. ··· 362 370 return; 363 371 364 372 /* 365 - * SLAB_KASAN is used to mark caches that are sanitized by KASAN 366 - * and that thus have per-object metadata. 367 - * Currently this flag is used in two places: 368 - * 1. In slab_ksize() to account for per-object metadata when 369 - * calculating the size of the accessible memory within the object. 370 - * 2. In slab_common.c via kasan_never_merge() to prevent merging of 371 - * caches with per-object metadata. 373 + * SLAB_KASAN is used to mark caches that are sanitized by KASAN and 374 + * that thus have per-object metadata. Currently, this flag is used in 375 + * slab_ksize() to account for per-object metadata when calculating the 376 + * size of the accessible memory within the object. Additionally, we use 377 + * SLAB_NO_MERGE to prevent merging of caches with per-object metadata. 372 378 */ 373 - *flags |= SLAB_KASAN; 379 + *flags |= SLAB_KASAN | SLAB_NO_MERGE; 374 380 375 381 ok_size = *size; 376 382
+3 -8
mm/slab.h
··· 363 363 enum slab_state { 364 364 DOWN, /* No slab functionality yet */ 365 365 PARTIAL, /* SLUB: kmem_cache_node available */ 366 - PARTIAL_NODE, /* SLAB: kmalloc size for node struct available */ 367 366 UP, /* Slab caches usable but not all extras yet */ 368 367 FULL /* Everything is working */ 369 368 }; ··· 386 387 387 388 /* Kmalloc array related functions */ 388 389 void setup_kmalloc_cache_index_table(void); 389 - void create_kmalloc_caches(slab_flags_t); 390 + void create_kmalloc_caches(void); 390 391 391 392 extern u8 kmalloc_size_index[24]; 392 393 ··· 421 422 int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); 422 423 423 424 void __init kmem_cache_init(void); 424 - void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, 425 - slab_flags_t flags); 426 425 extern void create_boot_cache(struct kmem_cache *, const char *name, 427 426 unsigned int size, slab_flags_t flags, 428 427 unsigned int useroffset, unsigned int usersize); ··· 432 435 __kmem_cache_alias(const char *name, unsigned int size, unsigned int align, 433 436 slab_flags_t flags, void (*ctor)(void *)); 434 437 435 - slab_flags_t kmem_cache_flags(unsigned int object_size, 436 - slab_flags_t flags, const char *name); 438 + slab_flags_t kmem_cache_flags(slab_flags_t flags, const char *name); 437 439 438 440 static inline bool is_kmalloc_cache(struct kmem_cache *s) 439 441 { ··· 465 469 SLAB_STORE_USER | \ 466 470 SLAB_TRACE | \ 467 471 SLAB_CONSISTENCY_CHECKS | \ 468 - SLAB_MEM_SPREAD | \ 469 472 SLAB_NOLEAKTRACE | \ 470 473 SLAB_RECLAIM_ACCOUNT | \ 471 474 SLAB_TEMPORARY | \ ··· 523 528 #endif 524 529 525 530 /* 526 - * Returns true if any of the specified slub_debug flags is enabled for the 531 + * Returns true if any of the specified slab_debug flags is enabled for the 527 532 * cache. Use only for flags parsed by setup_slub_debug() as it also enables 528 533 * the static key. 529 534 */
+17 -12
mm/slab_common.c
··· 50 50 */ 51 51 #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ 52 52 SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ 53 - SLAB_FAILSLAB | SLAB_NO_MERGE | kasan_never_merge()) 53 + SLAB_FAILSLAB | SLAB_NO_MERGE) 54 54 55 55 #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ 56 56 SLAB_CACHE_DMA32 | SLAB_ACCOUNT) ··· 172 172 size = ALIGN(size, sizeof(void *)); 173 173 align = calculate_alignment(flags, align, size); 174 174 size = ALIGN(size, align); 175 - flags = kmem_cache_flags(size, flags, name); 175 + flags = kmem_cache_flags(flags, name); 176 176 177 177 if (flags & SLAB_NEVER_MERGE) 178 178 return NULL; ··· 282 282 283 283 #ifdef CONFIG_SLUB_DEBUG 284 284 /* 285 - * If no slub_debug was enabled globally, the static key is not yet 285 + * If no slab_debug was enabled globally, the static key is not yet 286 286 * enabled by setup_slub_debug(). Enable it if the cache is being 287 287 * created with any of the debugging flags passed explicitly. 288 288 * It's also possible that this is the first cache created with ··· 404 404 */ 405 405 static void kmem_cache_release(struct kmem_cache *s) 406 406 { 407 - sysfs_slab_unlink(s); 408 - sysfs_slab_release(s); 407 + if (slab_state >= FULL) { 408 + sysfs_slab_unlink(s); 409 + sysfs_slab_release(s); 410 + } else { 411 + slab_kmem_cache_release(s); 412 + } 409 413 } 410 414 #else 411 415 static void kmem_cache_release(struct kmem_cache *s) ··· 770 766 } 771 767 772 768 /* 773 - * kmalloc_info[] is to make slub_debug=,kmalloc-xx option work at boot time. 769 + * kmalloc_info[] is to make slab_debug=,kmalloc-xx option work at boot time. 774 770 * kmalloc_index() supports up to 2^21=2MB, so the final entry of the table is 775 771 * kmalloc-2M. 776 772 */ ··· 857 853 return max(minalign, arch_slab_minalign()); 858 854 } 859 855 860 - void __init 861 - new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) 856 + static void __init 857 + new_kmalloc_cache(int idx, enum kmalloc_cache_type type) 862 858 { 859 + slab_flags_t flags = 0; 863 860 unsigned int minalign = __kmalloc_minalign(); 864 861 unsigned int aligned_size = kmalloc_info[idx].size; 865 862 int aligned_idx = idx; ··· 907 902 * may already have been created because they were needed to 908 903 * enable allocations for slab creation. 909 904 */ 910 - void __init create_kmalloc_caches(slab_flags_t flags) 905 + void __init create_kmalloc_caches(void) 911 906 { 912 907 int i; 913 908 enum kmalloc_cache_type type; ··· 918 913 for (type = KMALLOC_NORMAL; type < NR_KMALLOC_TYPES; type++) { 919 914 for (i = KMALLOC_SHIFT_LOW; i <= KMALLOC_SHIFT_HIGH; i++) { 920 915 if (!kmalloc_caches[type][i]) 921 - new_kmalloc_cache(i, type, flags); 916 + new_kmalloc_cache(i, type); 922 917 923 918 /* 924 919 * Caches that are not of the two-to-the-power-of size. ··· 927 922 */ 928 923 if (KMALLOC_MIN_SIZE <= 32 && i == 6 && 929 924 !kmalloc_caches[type][1]) 930 - new_kmalloc_cache(1, type, flags); 925 + new_kmalloc_cache(1, type); 931 926 if (KMALLOC_MIN_SIZE <= 64 && i == 7 && 932 927 !kmalloc_caches[type][2]) 933 - new_kmalloc_cache(2, type, flags); 928 + new_kmalloc_cache(2, type); 934 929 } 935 930 } 936 931 #ifdef CONFIG_RANDOM_KMALLOC_CACHES
+49 -67
mm/slub.c
··· 295 295 296 296 /* 297 297 * Debugging flags that require metadata to be stored in the slab. These get 298 - * disabled when slub_debug=O is used and a cache's min order increases with 298 + * disabled when slab_debug=O is used and a cache's min order increases with 299 299 * metadata. 300 300 */ 301 301 #define DEBUG_METADATA_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER) ··· 306 306 307 307 /* Internal SLUB flags */ 308 308 /* Poison object */ 309 - #define __OBJECT_POISON ((slab_flags_t __force)0x80000000U) 309 + #define __OBJECT_POISON __SLAB_FLAG_BIT(_SLAB_OBJECT_POISON) 310 310 /* Use cmpxchg_double */ 311 311 312 312 #ifdef system_has_freelist_aba 313 - #define __CMPXCHG_DOUBLE ((slab_flags_t __force)0x40000000U) 313 + #define __CMPXCHG_DOUBLE __SLAB_FLAG_BIT(_SLAB_CMPXCHG_DOUBLE) 314 314 #else 315 - #define __CMPXCHG_DOUBLE ((slab_flags_t __force)0U) 315 + #define __CMPXCHG_DOUBLE __SLAB_FLAG_UNUSED 316 316 #endif 317 317 318 318 /* ··· 391 391 }; 392 392 struct slab *slab; /* The slab from which we are allocating */ 393 393 #ifdef CONFIG_SLUB_CPU_PARTIAL 394 - struct slab *partial; /* Partially allocated frozen slabs */ 394 + struct slab *partial; /* Partially allocated slabs */ 395 395 #endif 396 396 local_lock_t lock; /* Protects the fields above */ 397 397 #ifdef CONFIG_SLUB_STATS ··· 1498 1498 { 1499 1499 struct kmem_cache_node *n = get_node(s, node); 1500 1500 1501 - /* 1502 - * May be called early in order to allocate a slab for the 1503 - * kmem_cache_node structure. Solve the chicken-egg 1504 - * dilemma by deferring the increment of the count during 1505 - * bootstrap (see early_kmem_cache_node_alloc). 1506 - */ 1507 - if (likely(n)) { 1508 - atomic_long_inc(&n->nr_slabs); 1509 - atomic_long_add(objects, &n->total_objects); 1510 - } 1501 + atomic_long_inc(&n->nr_slabs); 1502 + atomic_long_add(objects, &n->total_objects); 1511 1503 } 1512 1504 static inline void dec_slabs_node(struct kmem_cache *s, int node, int objects) 1513 1505 { ··· 1608 1616 } 1609 1617 1610 1618 /* 1611 - * Parse a block of slub_debug options. Blocks are delimited by ';' 1619 + * Parse a block of slab_debug options. Blocks are delimited by ';' 1612 1620 * 1613 1621 * @str: start of block 1614 1622 * @flags: returns parsed flags, or DEBUG_DEFAULT_FLAGS if none specified ··· 1669 1677 break; 1670 1678 default: 1671 1679 if (init) 1672 - pr_err("slub_debug option '%c' unknown. skipped\n", *str); 1680 + pr_err("slab_debug option '%c' unknown. skipped\n", *str); 1673 1681 } 1674 1682 } 1675 1683 check_slabs: ··· 1728 1736 /* 1729 1737 * For backwards compatibility, a single list of flags with list of 1730 1738 * slabs means debugging is only changed for those slabs, so the global 1731 - * slub_debug should be unchanged (0 or DEBUG_DEFAULT_FLAGS, depending 1739 + * slab_debug should be unchanged (0 or DEBUG_DEFAULT_FLAGS, depending 1732 1740 * on CONFIG_SLUB_DEBUG_ON). We can extended that to multiple lists as 1733 1741 * long as there is no option specifying flags without a slab list. 1734 1742 */ ··· 1752 1760 return 1; 1753 1761 } 1754 1762 1755 - __setup("slub_debug", setup_slub_debug); 1763 + __setup("slab_debug", setup_slub_debug); 1764 + __setup_param("slub_debug", slub_debug, setup_slub_debug, 0); 1756 1765 1757 1766 /* 1758 1767 * kmem_cache_flags - apply debugging options to the cache 1759 - * @object_size: the size of an object without meta data 1760 1768 * @flags: flags to set 1761 1769 * @name: name of the cache 1762 1770 * 1763 1771 * Debug option(s) are applied to @flags. In addition to the debug 1764 1772 * option(s), if a slab name (or multiple) is specified i.e. 1765 - * slub_debug=<Debug-Options>,<slab name1>,<slab name2> ... 1773 + * slab_debug=<Debug-Options>,<slab name1>,<slab name2> ... 1766 1774 * then only the select slabs will receive the debug option(s). 1767 1775 */ 1768 - slab_flags_t kmem_cache_flags(unsigned int object_size, 1769 - slab_flags_t flags, const char *name) 1776 + slab_flags_t kmem_cache_flags(slab_flags_t flags, const char *name) 1770 1777 { 1771 1778 char *iter; 1772 1779 size_t len; ··· 1841 1850 struct slab *slab) {} 1842 1851 static inline void remove_full(struct kmem_cache *s, struct kmem_cache_node *n, 1843 1852 struct slab *slab) {} 1844 - slab_flags_t kmem_cache_flags(unsigned int object_size, 1845 - slab_flags_t flags, const char *name) 1853 + slab_flags_t kmem_cache_flags(slab_flags_t flags, const char *name) 1846 1854 { 1847 1855 return flags; 1848 1856 } ··· 2028 2038 obj_cgroup_uncharge(objcg, objects * obj_full_size(s)); 2029 2039 } 2030 2040 #else /* CONFIG_MEMCG_KMEM */ 2031 - static inline struct mem_cgroup *memcg_from_slab_obj(void *ptr) 2032 - { 2033 - return NULL; 2034 - } 2035 - 2036 2041 static inline void memcg_free_slab_cgroups(struct slab *slab) 2037 2042 { 2038 2043 } ··· 2228 2243 } 2229 2244 2230 2245 /* Get the next entry on the pre-computed freelist randomized */ 2231 - static void *next_freelist_entry(struct kmem_cache *s, struct slab *slab, 2246 + static void *next_freelist_entry(struct kmem_cache *s, 2232 2247 unsigned long *pos, void *start, 2233 2248 unsigned long page_limit, 2234 2249 unsigned long freelist_count) ··· 2267 2282 start = fixup_red_left(s, slab_address(slab)); 2268 2283 2269 2284 /* First entry is used as the base of the freelist */ 2270 - cur = next_freelist_entry(s, slab, &pos, start, page_limit, 2271 - freelist_count); 2285 + cur = next_freelist_entry(s, &pos, start, page_limit, freelist_count); 2272 2286 cur = setup_object(s, cur); 2273 2287 slab->freelist = cur; 2274 2288 2275 2289 for (idx = 1; idx < slab->objects; idx++) { 2276 - next = next_freelist_entry(s, slab, &pos, start, page_limit, 2290 + next = next_freelist_entry(s, &pos, start, page_limit, 2277 2291 freelist_count); 2278 2292 next = setup_object(s, next); 2279 2293 set_freepointer(s, cur, next); ··· 3247 3263 oo_order(s->min)); 3248 3264 3249 3265 if (oo_order(s->min) > get_order(s->object_size)) 3250 - pr_warn(" %s debugging increased min order, use slub_debug=O to disable.\n", 3266 + pr_warn(" %s debugging increased min order, use slab_debug=O to disable.\n", 3251 3267 s->name); 3252 3268 3253 3269 for_each_kmem_cache_node(s, node, n) { ··· 3310 3326 counters = slab->counters; 3311 3327 3312 3328 new.counters = counters; 3313 - VM_BUG_ON(!new.frozen); 3314 3329 3315 3330 new.inuse = slab->objects; 3316 3331 new.frozen = freelist != NULL; ··· 3481 3498 3482 3499 slab = slub_percpu_partial(c); 3483 3500 slub_set_percpu_partial(c, slab); 3484 - local_unlock_irqrestore(&s->cpu_slab->lock, flags); 3485 - stat(s, CPU_PARTIAL_ALLOC); 3486 3501 3487 - if (unlikely(!node_match(slab, node) || 3488 - !pfmemalloc_match(slab, gfpflags))) { 3489 - slab->next = NULL; 3490 - __put_partials(s, slab); 3491 - continue; 3502 + if (likely(node_match(slab, node) && 3503 + pfmemalloc_match(slab, gfpflags))) { 3504 + c->slab = slab; 3505 + freelist = get_freelist(s, slab); 3506 + VM_BUG_ON(!freelist); 3507 + stat(s, CPU_PARTIAL_ALLOC); 3508 + goto load_freelist; 3492 3509 } 3493 3510 3494 - freelist = freeze_slab(s, slab); 3495 - goto retry_load_slab; 3511 + local_unlock_irqrestore(&s->cpu_slab->lock, flags); 3512 + 3513 + slab->next = NULL; 3514 + __put_partials(s, slab); 3496 3515 } 3497 3516 #endif 3498 3517 ··· 3777 3792 zero_size = orig_size; 3778 3793 3779 3794 /* 3780 - * When slub_debug is enabled, avoid memory initialization integrated 3795 + * When slab_debug is enabled, avoid memory initialization integrated 3781 3796 * into KASAN and instead zero out the memory via the memset below with 3782 3797 * the proper size. Otherwise, KASAN might overwrite SLUB redzones and 3783 3798 * cause false-positive reports. This does not lead to a performance 3784 - * penalty on production builds, as slub_debug is not intended to be 3799 + * penalty on production builds, as slab_debug is not intended to be 3785 3800 * enabled there. 3786 3801 */ 3787 3802 if (__slub_debug_enabled()) ··· 4172 4187 * then add it. 4173 4188 */ 4174 4189 if (!kmem_cache_has_cpu_partial(s) && unlikely(!prior)) { 4175 - remove_full(s, n, slab); 4176 4190 add_partial(n, slab, DEACTIVATE_TO_TAIL); 4177 4191 stat(s, FREE_ADD_PARTIAL); 4178 4192 } ··· 4185 4201 */ 4186 4202 remove_partial(n, slab); 4187 4203 stat(s, FREE_REMOVE_PARTIAL); 4188 - } else { 4189 - /* Slab must be on the full list */ 4190 - remove_full(s, n, slab); 4191 4204 } 4192 4205 4193 4206 spin_unlock_irqrestore(&n->list_lock, flags); ··· 4683 4702 * activity on the partial lists which requires taking the list_lock. This is 4684 4703 * less a concern for large slabs though which are rarely used. 4685 4704 * 4686 - * slub_max_order specifies the order where we begin to stop considering the 4687 - * number of objects in a slab as critical. If we reach slub_max_order then 4705 + * slab_max_order specifies the order where we begin to stop considering the 4706 + * number of objects in a slab as critical. If we reach slab_max_order then 4688 4707 * we try to keep the page order as low as possible. So we accept more waste 4689 4708 * of space in favor of a small page order. 4690 4709 * ··· 4751 4770 * and backing off gradually. 4752 4771 * 4753 4772 * We start with accepting at most 1/16 waste and try to find the 4754 - * smallest order from min_objects-derived/slub_min_order up to 4755 - * slub_max_order that will satisfy the constraint. Note that increasing 4773 + * smallest order from min_objects-derived/slab_min_order up to 4774 + * slab_max_order that will satisfy the constraint. Note that increasing 4756 4775 * the order can only result in same or less fractional waste, not more. 4757 4776 * 4758 4777 * If that fails, we increase the acceptable fraction of waste and try 4759 4778 * again. The last iteration with fraction of 1/2 would effectively 4760 4779 * accept any waste and give us the order determined by min_objects, as 4761 - * long as at least single object fits within slub_max_order. 4780 + * long as at least single object fits within slab_max_order. 4762 4781 */ 4763 4782 for (unsigned int fraction = 16; fraction > 1; fraction /= 2) { 4764 4783 order = calc_slab_order(size, min_order, slub_max_order, ··· 4768 4787 } 4769 4788 4770 4789 /* 4771 - * Doh this slab cannot be placed using slub_max_order. 4790 + * Doh this slab cannot be placed using slab_max_order. 4772 4791 */ 4773 4792 order = get_order(size); 4774 4793 if (order <= MAX_PAGE_ORDER) ··· 4838 4857 slab = new_slab(kmem_cache_node, GFP_NOWAIT, node); 4839 4858 4840 4859 BUG_ON(!slab); 4841 - inc_slabs_node(kmem_cache_node, slab_nid(slab), slab->objects); 4842 4860 if (slab_nid(slab) != node) { 4843 4861 pr_err("SLUB: Unable to allocate memory from node %d\n", node); 4844 4862 pr_err("SLUB: Allocating a useless per node structure in order to be able to continue\n"); ··· 5084 5104 5085 5105 static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) 5086 5106 { 5087 - s->flags = kmem_cache_flags(s->size, flags, s->name); 5107 + s->flags = kmem_cache_flags(flags, s->name); 5088 5108 #ifdef CONFIG_SLAB_FREELIST_HARDENED 5089 5109 s->random = get_random_long(); 5090 5110 #endif ··· 5293 5313 return 1; 5294 5314 } 5295 5315 5296 - __setup("slub_min_order=", setup_slub_min_order); 5316 + __setup("slab_min_order=", setup_slub_min_order); 5317 + __setup_param("slub_min_order=", slub_min_order, setup_slub_min_order, 0); 5318 + 5297 5319 5298 5320 static int __init setup_slub_max_order(char *str) 5299 5321 { ··· 5308 5326 return 1; 5309 5327 } 5310 5328 5311 - __setup("slub_max_order=", setup_slub_max_order); 5329 + __setup("slab_max_order=", setup_slub_max_order); 5330 + __setup_param("slub_max_order=", slub_max_order, setup_slub_max_order, 0); 5312 5331 5313 5332 static int __init setup_slub_min_objects(char *str) 5314 5333 { ··· 5318 5335 return 1; 5319 5336 } 5320 5337 5321 - __setup("slub_min_objects=", setup_slub_min_objects); 5338 + __setup("slab_min_objects=", setup_slub_min_objects); 5339 + __setup_param("slub_min_objects=", slub_min_objects, setup_slub_min_objects, 0); 5322 5340 5323 5341 #ifdef CONFIG_HARDENED_USERCOPY 5324 5342 /* ··· 5647 5663 5648 5664 /* Now we can use the kmem_cache to allocate kmalloc slabs */ 5649 5665 setup_kmalloc_cache_index_table(); 5650 - create_kmalloc_caches(0); 5666 + create_kmalloc_caches(); 5651 5667 5652 5668 /* Setup random freelists for each cache */ 5653 5669 init_freelist_randomization(); ··· 6776 6792 6777 6793 void sysfs_slab_unlink(struct kmem_cache *s) 6778 6794 { 6779 - if (slab_state >= FULL) 6780 - kobject_del(&s->kobj); 6795 + kobject_del(&s->kobj); 6781 6796 } 6782 6797 6783 6798 void sysfs_slab_release(struct kmem_cache *s) 6784 6799 { 6785 - if (slab_state >= FULL) 6786 - kobject_put(&s->kobj); 6800 + kobject_put(&s->kobj); 6787 6801 } 6788 6802 6789 6803 /*
-1
tools/include/linux/slab.h
··· 18 18 enum slab_state { 19 19 DOWN, 20 20 PARTIAL, 21 - PARTIAL_NODE, 22 21 UP, 23 22 FULL 24 23 };