tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

nfsd: don't use sv_nrthreads in connection limiting calculations.

The heuristic for limiting the number of incoming connections to nfsd
currently uses sv_nrthreads - allowing more connections if more threads
were configured.

A future patch will allow number of threads to grow dynamically so that
there will be no need to configure sv_nrthreads. So we need a different
solution for limiting connections.

It isn't clear what problem is solved by limiting connections (as
mentioned in a code comment) but the most likely problem is a connection
storm - many connections that are not doing productive work. These will
be closed after about 6 minutes already but it might help to slow down a
storm.

This patch adds a per-connection flag XPT_PEER_VALID which indicates
that the peer has presented a filehandle for which it has some sort of
access. i.e the peer is known to be trusted in some way. We now only
count connections which have NOT been determined to be valid. There
should be relative few of these at any given time.

If the number of non-validated peer exceed a limit - currently 64 - we
close the oldest non-validated peer to avoid having too many of these
useless connections.

Note that this patch significantly changes the meaning of the various
configuration parameters for "max connections". The next patch will
remove all of these.

Signed-off-by: NeilBrown <neilb@suse.de>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

authored by

NeilBrown and committed by
Chuck Lever eccbbc7c de71d4e2

+38 -23
-4
fs/nfs/callback.c
··· 211 211 return ERR_PTR(-ENOMEM); 212 212 } 213 213 cb_info->serv = serv; 214 - /* As there is only one thread we need to over-ride the 215 - * default maximum of 80 connections 216 - */ 217 - serv->sv_maxconn = 1024; 218 214 dprintk("nfs_callback_create_svc: service created\n"); 219 215 return serv; 220 216 }
+1
fs/nfs/callback_xdr.c
··· 984 984 nfs_put_client(cps.clp); 985 985 goto out_invalidcred; 986 986 } 987 + svc_xprt_set_valid(rqstp->rq_xprt); 987 988 } 988 989 989 990 cps.minorversion = hdr_arg.minorversion;
+2 -2
fs/nfsd/netns.h
··· 129 129 unsigned char writeverf[8]; 130 130 131 131 /* 132 - * Max number of connections this nfsd container will allow. Defaults 133 - * to '0' which is means that it bases this on the number of threads. 132 + * Max number of non-validated connections this nfsd container 133 + * will allow. Defaults to '0' gets mapped to 64. 134 134 */ 135 135 unsigned int max_connections; 136 136
+2
fs/nfsd/nfsfh.c
··· 382 382 if (error) 383 383 goto out; 384 384 385 + svc_xprt_set_valid(rqstp->rq_xprt); 386 + 385 387 /* Finally, check access permissions. */ 386 388 error = nfsd_permission(cred, exp, dentry, access); 387 389 out:
+1 -1
include/linux/sunrpc/svc.h
··· 81 81 unsigned int sv_xdrsize; /* XDR buffer size */ 82 82 struct list_head sv_permsocks; /* all permanent sockets */ 83 83 struct list_head sv_tempsocks; /* all temporary sockets */ 84 - int sv_tmpcnt; /* count of temporary sockets */ 84 + int sv_tmpcnt; /* count of temporary "valid" sockets */ 85 85 struct timer_list sv_temptimer; /* timer for aging temporary sockets */ 86 86 87 87 char * sv_name; /* service name */
+16
include/linux/sunrpc/svc_xprt.h
··· 99 99 XPT_HANDSHAKE, /* xprt requests a handshake */ 100 100 XPT_TLS_SESSION, /* transport-layer security established */ 101 101 XPT_PEER_AUTH, /* peer has been authenticated */ 102 + XPT_PEER_VALID, /* peer has presented a filehandle that 103 + * it has access to. It is NOT counted 104 + * in ->sv_tmpcnt. 105 + */ 102 106 }; 107 + 108 + static inline void svc_xprt_set_valid(struct svc_xprt *xpt) 109 + { 110 + if (test_bit(XPT_TEMP, &xpt->xpt_flags) && 111 + !test_and_set_bit(XPT_PEER_VALID, &xpt->xpt_flags)) { 112 + struct svc_serv *serv = xpt->xpt_server; 113 + 114 + spin_lock(&serv->sv_lock); 115 + serv->sv_tmpcnt -= 1; 116 + spin_unlock(&serv->sv_lock); 117 + } 118 + } 103 119 104 120 static inline void unregister_xpt_user(struct svc_xprt *xpt, struct svc_xpt_user *u) 105 121 {
+16 -16
net/sunrpc/svc_xprt.c
··· 606 606 } 607 607 608 608 /* 609 - * Make sure that we don't have too many active connections. If we have, 609 + * Make sure that we don't have too many connections that have not yet 610 + * demonstrated that they have access to the NFS server. If we have, 610 611 * something must be dropped. It's not clear what will happen if we allow 611 612 * "too many" connections, but when dealing with network-facing software, 612 613 * we have to code defensively. Here we do that by imposing hard limits. ··· 626 625 */ 627 626 static void svc_check_conn_limits(struct svc_serv *serv) 628 627 { 629 - unsigned int limit = serv->sv_maxconn ? serv->sv_maxconn : 630 - (serv->sv_nrthreads+3) * 20; 628 + unsigned int limit = serv->sv_maxconn ? serv->sv_maxconn : 64; 631 629 632 630 if (serv->sv_tmpcnt > limit) { 633 - struct svc_xprt *xprt = NULL; 631 + struct svc_xprt *xprt = NULL, *xprti; 634 632 spin_lock_bh(&serv->sv_lock); 635 633 if (!list_empty(&serv->sv_tempsocks)) { 636 - /* Try to help the admin */ 637 - net_notice_ratelimited("%s: too many open connections, consider increasing the %s\n", 638 - serv->sv_name, serv->sv_maxconn ? 639 - "max number of connections" : 640 - "number of threads"); 641 634 /* 642 635 * Always select the oldest connection. It's not fair, 643 - * but so is life 636 + * but nor is life. 644 637 */ 645 - xprt = list_entry(serv->sv_tempsocks.prev, 646 - struct svc_xprt, 647 - xpt_list); 648 - set_bit(XPT_CLOSE, &xprt->xpt_flags); 649 - svc_xprt_get(xprt); 638 + list_for_each_entry_reverse(xprti, &serv->sv_tempsocks, 639 + xpt_list) { 640 + if (!test_bit(XPT_PEER_VALID, &xprti->xpt_flags)) { 641 + xprt = xprti; 642 + set_bit(XPT_CLOSE, &xprt->xpt_flags); 643 + svc_xprt_get(xprt); 644 + break; 645 + } 646 + } 650 647 } 651 648 spin_unlock_bh(&serv->sv_lock); 652 649 ··· 1038 1039 1039 1040 spin_lock_bh(&serv->sv_lock); 1040 1041 list_del_init(&xprt->xpt_list); 1041 - if (test_bit(XPT_TEMP, &xprt->xpt_flags)) 1042 + if (test_bit(XPT_TEMP, &xprt->xpt_flags) && 1043 + !test_bit(XPT_PEER_VALID, &xprt->xpt_flags)) 1042 1044 serv->sv_tmpcnt--; 1043 1045 spin_unlock_bh(&serv->sv_lock); 1044 1046