tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

zsmalloc: introduce SG-list based object read API

Currently, zsmalloc performs address linearization on read (which
sometimes requires memcpy() to a local buffer). Not all zsmalloc users
need a linear address. For example, Crypto API supports SG-list,
performing linearization under the hood, if needed. In addition, some
compressors can have native SG-list support, completely avoiding the
linearization step.

Provide an SG-list based zsmalloc read API:
- zs_obj_read_sg_begin()
- zs_obj_read_sg_end()

This API allows callers to obtain an SG representation of the object (one
entry for objects that are contained in a single page and two entries for
spanning objects), avoiding the need for a bounce buffer and memcpy.

[senozhatsky@chromium.org: make zs_obj_read_sg_begin() return void, per Yosry]
Link: https://lkml.kernel.org/r/20260117024900.792237-1-senozhatsky@chromium.org
Link: https://lkml.kernel.org/r/20260113034645.2729998-1-senozhatsky@chromium.org
Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Tested-by: Yosry Ahmed <yosry.ahmed@linux.dev>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Brian Geffon <bgeffon@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Sergey Senozhatsky and committed by
Andrew Morton dc2e4982 737dfe7d

+67
+4
include/linux/zsmalloc.h
··· 22 22 }; 23 23 24 24 struct zs_pool; 25 + struct scatterlist; 25 26 26 27 struct zs_pool *zs_create_pool(const char *name); 27 28 void zs_destroy_pool(struct zs_pool *pool); ··· 44 43 size_t mem_len, void *local_copy); 45 44 void zs_obj_read_end(struct zs_pool *pool, unsigned long handle, 46 45 size_t mem_len, void *handle_mem); 46 + void zs_obj_read_sg_begin(struct zs_pool *pool, unsigned long handle, 47 + struct scatterlist *sg, size_t mem_len); 48 + void zs_obj_read_sg_end(struct zs_pool *pool, unsigned long handle); 47 49 void zs_obj_write(struct zs_pool *pool, unsigned long handle, 48 50 void *handle_mem, size_t mem_len); 49 51
+63
mm/zsmalloc.c
··· 30 30 #include <linux/highmem.h> 31 31 #include <linux/string.h> 32 32 #include <linux/slab.h> 33 + #include <linux/scatterlist.h> 33 34 #include <linux/spinlock.h> 34 35 #include <linux/sprintf.h> 35 36 #include <linux/shrinker.h> ··· 1141 1140 zspage_read_unlock(zspage); 1142 1141 } 1143 1142 EXPORT_SYMBOL_GPL(zs_obj_read_end); 1143 + 1144 + void zs_obj_read_sg_begin(struct zs_pool *pool, unsigned long handle, 1145 + struct scatterlist *sg, size_t mem_len) 1146 + { 1147 + struct zspage *zspage; 1148 + struct zpdesc *zpdesc; 1149 + unsigned long obj, off; 1150 + unsigned int obj_idx; 1151 + struct size_class *class; 1152 + 1153 + /* Guarantee we can get zspage from handle safely */ 1154 + read_lock(&pool->lock); 1155 + obj = handle_to_obj(handle); 1156 + obj_to_location(obj, &zpdesc, &obj_idx); 1157 + zspage = get_zspage(zpdesc); 1158 + 1159 + /* Make sure migration doesn't move any pages in this zspage */ 1160 + zspage_read_lock(zspage); 1161 + read_unlock(&pool->lock); 1162 + 1163 + class = zspage_class(pool, zspage); 1164 + off = offset_in_page(class->size * obj_idx); 1165 + 1166 + if (!ZsHugePage(zspage)) 1167 + off += ZS_HANDLE_SIZE; 1168 + 1169 + if (off + mem_len <= PAGE_SIZE) { 1170 + /* this object is contained entirely within a page */ 1171 + sg_init_table(sg, 1); 1172 + sg_set_page(sg, zpdesc_page(zpdesc), mem_len, off); 1173 + } else { 1174 + size_t sizes[2]; 1175 + 1176 + /* this object spans two pages */ 1177 + sizes[0] = PAGE_SIZE - off; 1178 + sizes[1] = mem_len - sizes[0]; 1179 + 1180 + sg_init_table(sg, 2); 1181 + sg_set_page(sg, zpdesc_page(zpdesc), sizes[0], off); 1182 + 1183 + zpdesc = get_next_zpdesc(zpdesc); 1184 + sg = sg_next(sg); 1185 + 1186 + sg_set_page(sg, zpdesc_page(zpdesc), sizes[1], 0); 1187 + } 1188 + } 1189 + EXPORT_SYMBOL_GPL(zs_obj_read_sg_begin); 1190 + 1191 + void zs_obj_read_sg_end(struct zs_pool *pool, unsigned long handle) 1192 + { 1193 + struct zspage *zspage; 1194 + struct zpdesc *zpdesc; 1195 + unsigned long obj; 1196 + unsigned int obj_idx; 1197 + 1198 + obj = handle_to_obj(handle); 1199 + obj_to_location(obj, &zpdesc, &obj_idx); 1200 + zspage = get_zspage(zpdesc); 1201 + 1202 + zspage_read_unlock(zspage); 1203 + } 1204 + EXPORT_SYMBOL_GPL(zs_obj_read_sg_end); 1144 1205 1145 1206 void zs_obj_write(struct zs_pool *pool, unsigned long handle, 1146 1207 void *handle_mem, size_t mem_len)