Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
SUNRPC: Optimize rq_respages allocation in svc_alloc_arg
svc_alloc_arg() invokes alloc_pages_bulk() with the full rq_maxpages
count (~259 for 1MB messages) for the rq_respages array, causing a
full-array scan despite most slots holding valid pages.
svc_rqst_release_pages() NULLs only the range
[rq_respages, rq_next_page)
after each RPC, so only that range contains NULL entries. Limit the
rq_respages fill in svc_alloc_arg() to that range instead of
scanning the full array.
svc_init_buffer() initializes rq_next_page to span the entire
rq_respages array, so the first svc_alloc_arg() call fills all
slots.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
+12
-1
+4
include/linux/sunrpc/svc.h
+4
include/linux/sunrpc/svc.h
···
152
152
* still in transport use, and set rq_pages_nfree to the count.
153
153
* svc_alloc_arg() refills only that many rq_pages entries.
154
154
*
155
+
* For rq_respages, svc_rqst_release_pages() NULLs entries in
156
+
* [rq_respages, rq_next_page) after each RPC. svc_alloc_arg()
157
+
* refills only that range.
158
+
*
155
159
* xdr_buf holds responses; the structure fits NFS read responses
156
160
* (header, data pages, optional tail) and enables sharing of
157
161
* client-side routines.
+1
net/sunrpc/svc.c
+1
net/sunrpc/svc.c
+7
-1
net/sunrpc/svc_xprt.c
+7
-1
net/sunrpc/svc_xprt.c
···
686
686
rqstp->rq_pages_nfree = 0;
687
687
}
688
688
689
-
if (!svc_fill_pages(rqstp, rqstp->rq_respages, pages))
689
+
if (WARN_ON_ONCE(rqstp->rq_next_page < rqstp->rq_respages))
690
690
return false;
691
+
nfree = rqstp->rq_next_page - rqstp->rq_respages;
692
+
if (nfree) {
693
+
if (!svc_fill_pages(rqstp, rqstp->rq_respages, nfree))
694
+
return false;
695
+
}
696
+
691
697
rqstp->rq_next_page = rqstp->rq_respages;
692
698
rqstp->rq_page_end = &rqstp->rq_respages[pages];
693
699
/* svc_rqst_replace_page() dereferences *rq_next_page even