tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Merge branch 'net-smc-parallelism'

D. Wythe says:

====================
net/smc: optimize the parallelism of SMC-R connections

This patch set attempts to optimize the parallelism of SMC-R connections,
mainly to reduce unnecessary blocking on locks, and to fix exceptions that
occur after thoses optimization.

According to Off-CPU graph, SMC worker's off-CPU as that:

smc_close_passive_work (1.09%)
smcr_buf_unuse (1.08%)
smc_llc_flow_initiate (1.02%)

smc_listen_work (48.17%)
__mutex_lock.isra.11 (47.96%)

An ideal SMC-R connection process should only block on the IO events
of the network, but it's quite clear that the SMC-R connection now is
queued on the lock most of the time.

The goal of this patchset is to achieve our ideal situation where
network IO events are blocked for the majority of the connection lifetime.

There are three big locks here:

1. smc_client_lgr_pending & smc_server_lgr_pending

2. llc_conf_mutex

3. rmbs_lock & sndbufs_lock

And an implementation issue:

1. confirm/delete rkey msg can't be sent concurrently while
protocol allows indeed.

Unfortunately,The above problems together affect the parallelism of
SMC-R connection. If any of them are not solved. our goal cannot
be achieved.

After this patch set, we can get a quite ideal off-CPU graph as
following:

smc_close_passive_work (41.58%)
smcr_buf_unuse (41.57%)
smc_llc_do_delete_rkey (41.57%)

smc_listen_work (39.10%)
smc_clc_wait_msg (13.18%)
tcp_recvmsg_locked (13.18)
smc_listen_find_device (25.87%)
smcr_lgr_reg_rmbs (25.87%)
smc_llc_do_confirm_rkey (25.87%)

We can see that most of the waiting times are waiting for network IO
events. This also has a certain performance improvement on our
short-lived conenction wrk/nginx benchmark test:

+--------------+------+------+-------+--------+------+--------+
|conns/qps |c4 | c8 | c16 | c32 | c64 | c200 |
+--------------+------+------+-------+--------+------+--------+
|SMC-R before |9.7k | 10k | 10k | 9.9k | 9.1k | 8.9k |
+--------------+------+------+-------+--------+------+--------+
|SMC-R now |13k | 19k | 18k | 16k | 15k | 12k |
+--------------+------+------+-------+--------+------+--------+
|TCP |15k | 35k | 51k | 80k | 100k | 162k |
+--------------+------+------+-------+--------+------+--------+

The reason why the benefit is not obvious after the number of connections
has increased dues to workqueue. If we try to change workqueue to UNBOUND,
we can obtain at least 4-5 times performance improvement, reach up to half
of TCP. However, this is not an elegant solution, the optimization of it
will be much more complicated. But in any case, we will submit relevant
optimization patches as soon as possible.

Please note that the premise here is that the lock related problem
must be solved first, otherwise, no matter how we optimize the workqueue,
there won't be much improvement.

Because there are a lot of related changes to the code, if you have
any questions or suggestions, please let me know.

Thanks
D. Wythe

v1 -> v2:

1. Fix panic in SMC-D scenario
2. Fix lnkc related hashfn calculation exception, caused by operator
priority
3. Only wake up one connection if the lnk is not active
4. Delete obsolete unlock logic in smc_listen_work()
5. PATCH format, do Reverse Christmas tree
6. PATCH format, change all xxx_lnk_xxx function to xxx_link_xxx
7. PATCH format, add correct fix tag for the patches for fixes.
8. PATCH format, fix some spelling error
9. PATCH format, rename slow to do_slow

v2 -> v3:

1. add SMC-D support, remove the concept of link cluster since SMC-D has
no link at all. Replace it by lgr decision maker, who provides suggestions
to SMC-D and SMC-R on whether to create new link group.

2. Fix the corruption problem described by PATCH 'fix application
data exception' on SMC-D.

v3 -> v4:

1. Fix panic caused by uninitialization map.

v4 -> v5:

1. Make SMC-D buf creation be serial to avoid Potential error
2. Add a flag to synchronize the success of the first contact
with the ready of the link group, including SMC-D and SMC-R.
3. Fixed possible reference count leak in smc_llc_flow_start().
4. reorder the patch, make bugfix PATCH be ahead.

v5 -> v6:

1. Separate the bugfix patches to make it independent.
2. Merge patch 'fix SMC_CLC_DECL_ERR_REGRMB without smc_server_lgr_pending'
with patch 'remove locks smc_client_lgr_pending and smc_server_lgr_pending'
3. Format code styles, including alignment and reverse christmas tree
style.
4. Fix a possible memory leak in smc_llc_rmt_delete_rkey()
and smc_llc_rmt_conf_rkey().

v6 -> v7:

1. Discard patch attempting to remove global locks
2. Discard patch attempting make confirm/delete rkey process concurrently
====================

Signed-off-by: David S. Miller <davem@davemloft.net>

David S. Miller 042b7858 88c940cc

+78 -62
+20 -5
net/smc/af_smc.c
··· 502 502 return -EINVAL; 503 503 504 504 /* protect against parallel smcr_link_reg_buf() */ 505 - mutex_lock(&lgr->llc_conf_mutex); 505 + down_write(&lgr->llc_conf_mutex); 506 506 for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) { 507 507 if (!smc_link_active(&lgr->lnk[i])) 508 508 continue; ··· 510 510 if (rc) 511 511 break; 512 512 } 513 - mutex_unlock(&lgr->llc_conf_mutex); 513 + up_write(&lgr->llc_conf_mutex); 514 514 return rc; 515 515 } 516 516 ··· 519 519 struct smc_buf_desc *rmb_desc) 520 520 { 521 521 struct smc_link_group *lgr = link->lgr; 522 + bool do_slow = false; 522 523 int i, rc = 0; 523 524 524 525 rc = smc_llc_flow_initiate(lgr, SMC_LLC_FLOW_RKEY); 525 526 if (rc) 526 527 return rc; 528 + 529 + down_read(&lgr->llc_conf_mutex); 530 + for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) { 531 + if (!smc_link_active(&lgr->lnk[i])) 532 + continue; 533 + if (!rmb_desc->is_reg_mr[link->link_idx]) { 534 + up_read(&lgr->llc_conf_mutex); 535 + goto slow_path; 536 + } 537 + } 538 + /* mr register already */ 539 + goto fast_path; 540 + slow_path: 541 + do_slow = true; 527 542 /* protect against parallel smc_llc_cli_rkey_exchange() and 528 543 * parallel smcr_link_reg_buf() 529 544 */ 530 - mutex_lock(&lgr->llc_conf_mutex); 545 + down_write(&lgr->llc_conf_mutex); 531 546 for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) { 532 547 if (!smc_link_active(&lgr->lnk[i])) 533 548 continue; ··· 550 535 if (rc) 551 536 goto out; 552 537 } 553 - 538 + fast_path: 554 539 /* exchange confirm_rkey msg with peer */ 555 540 rc = smc_llc_do_confirm_rkey(link, rmb_desc); 556 541 if (rc) { ··· 559 544 } 560 545 rmb_desc->is_conf_rkey = true; 561 546 out: 562 - mutex_unlock(&lgr->llc_conf_mutex); 547 + do_slow ? up_write(&lgr->llc_conf_mutex) : up_read(&lgr->llc_conf_mutex); 563 548 smc_llc_flow_stop(lgr, &lgr->llc_flow_lcl); 564 549 return rc; 565 550 }
+38 -37
net/smc/smc_core.c
··· 854 854 lgr->freeing = 0; 855 855 lgr->vlan_id = ini->vlan_id; 856 856 refcount_set(&lgr->refcnt, 1); /* set lgr refcnt to 1 */ 857 - mutex_init(&lgr->sndbufs_lock); 858 - mutex_init(&lgr->rmbs_lock); 857 + init_rwsem(&lgr->sndbufs_lock); 858 + init_rwsem(&lgr->rmbs_lock); 859 859 rwlock_init(&lgr->conns_lock); 860 860 for (i = 0; i < SMC_RMBE_SIZES; i++) { 861 861 INIT_LIST_HEAD(&lgr->sndbufs[i]); ··· 1098 1098 static void smcr_buf_unuse(struct smc_buf_desc *buf_desc, bool is_rmb, 1099 1099 struct smc_link_group *lgr) 1100 1100 { 1101 - struct mutex *lock; /* lock buffer list */ 1101 + struct rw_semaphore *lock; /* lock buffer list */ 1102 1102 int rc; 1103 1103 1104 1104 if (is_rmb && buf_desc->is_conf_rkey && !list_empty(&lgr->list)) { ··· 1106 1106 rc = smc_llc_flow_initiate(lgr, SMC_LLC_FLOW_RKEY); 1107 1107 if (!rc) { 1108 1108 /* protect against smc_llc_cli_rkey_exchange() */ 1109 - mutex_lock(&lgr->llc_conf_mutex); 1109 + down_read(&lgr->llc_conf_mutex); 1110 1110 smc_llc_do_delete_rkey(lgr, buf_desc); 1111 1111 buf_desc->is_conf_rkey = false; 1112 - mutex_unlock(&lgr->llc_conf_mutex); 1112 + up_read(&lgr->llc_conf_mutex); 1113 1113 smc_llc_flow_stop(lgr, &lgr->llc_flow_lcl); 1114 1114 } 1115 1115 } ··· 1118 1118 /* buf registration failed, reuse not possible */ 1119 1119 lock = is_rmb ? &lgr->rmbs_lock : 1120 1120 &lgr->sndbufs_lock; 1121 - mutex_lock(lock); 1121 + down_write(lock); 1122 1122 list_del(&buf_desc->list); 1123 - mutex_unlock(lock); 1123 + up_write(lock); 1124 1124 1125 1125 smc_buf_free(lgr, is_rmb, buf_desc); 1126 1126 } else { ··· 1224 1224 int i; 1225 1225 1226 1226 for (i = 0; i < SMC_RMBE_SIZES; i++) { 1227 - mutex_lock(&lgr->rmbs_lock); 1227 + down_write(&lgr->rmbs_lock); 1228 1228 list_for_each_entry_safe(buf_desc, bf, &lgr->rmbs[i], list) 1229 1229 smcr_buf_unmap_link(buf_desc, true, lnk); 1230 - mutex_unlock(&lgr->rmbs_lock); 1231 - mutex_lock(&lgr->sndbufs_lock); 1230 + up_write(&lgr->rmbs_lock); 1231 + 1232 + down_write(&lgr->sndbufs_lock); 1232 1233 list_for_each_entry_safe(buf_desc, bf, &lgr->sndbufs[i], 1233 1234 list) 1234 1235 smcr_buf_unmap_link(buf_desc, false, lnk); 1235 - mutex_unlock(&lgr->sndbufs_lock); 1236 + up_write(&lgr->sndbufs_lock); 1236 1237 } 1237 1238 } 1238 1239 ··· 1378 1377 int i; 1379 1378 1380 1379 if (!lgr->is_smcd) { 1381 - mutex_lock(&lgr->llc_conf_mutex); 1380 + down_write(&lgr->llc_conf_mutex); 1382 1381 for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) { 1383 1382 if (lgr->lnk[i].state != SMC_LNK_UNUSED) 1384 1383 smcr_link_clear(&lgr->lnk[i], false); 1385 1384 } 1386 - mutex_unlock(&lgr->llc_conf_mutex); 1385 + up_write(&lgr->llc_conf_mutex); 1387 1386 smc_llc_lgr_clear(lgr); 1388 1387 } 1389 1388 ··· 1697 1696 } else { 1698 1697 if (lgr->llc_flow_lcl.type != SMC_LLC_FLOW_NONE) { 1699 1698 /* another llc task is ongoing */ 1700 - mutex_unlock(&lgr->llc_conf_mutex); 1699 + up_write(&lgr->llc_conf_mutex); 1701 1700 wait_event_timeout(lgr->llc_flow_waiter, 1702 1701 (list_empty(&lgr->list) || 1703 1702 lgr->llc_flow_lcl.type == SMC_LLC_FLOW_NONE), 1704 1703 SMC_LLC_WAIT_TIME); 1705 - mutex_lock(&lgr->llc_conf_mutex); 1704 + down_write(&lgr->llc_conf_mutex); 1706 1705 } 1707 1706 if (!list_empty(&lgr->list)) { 1708 1707 smc_llc_send_delete_link(to_lnk, del_link_id, ··· 1762 1761 if (list_empty(&lgr->list)) 1763 1762 return; 1764 1763 wake_up_all(&lgr->llc_msg_waiter); 1765 - mutex_lock(&lgr->llc_conf_mutex); 1764 + down_write(&lgr->llc_conf_mutex); 1766 1765 smcr_link_down(link); 1767 - mutex_unlock(&lgr->llc_conf_mutex); 1766 + up_write(&lgr->llc_conf_mutex); 1768 1767 } 1769 1768 1770 1769 static int smc_vlan_by_tcpsk_walk(struct net_device *lower_dev, ··· 1991 1990 * buffer size; if not available, return NULL 1992 1991 */ 1993 1992 static struct smc_buf_desc *smc_buf_get_slot(int compressed_bufsize, 1994 - struct mutex *lock, 1993 + struct rw_semaphore *lock, 1995 1994 struct list_head *buf_list) 1996 1995 { 1997 1996 struct smc_buf_desc *buf_slot; 1998 1997 1999 - mutex_lock(lock); 1998 + down_read(lock); 2000 1999 list_for_each_entry(buf_slot, buf_list, list) { 2001 2000 if (cmpxchg(&buf_slot->used, 0, 1) == 0) { 2002 - mutex_unlock(lock); 2001 + up_read(lock); 2003 2002 return buf_slot; 2004 2003 } 2005 2004 } 2006 - mutex_unlock(lock); 2005 + up_read(lock); 2007 2006 return NULL; 2008 2007 } 2009 2008 ··· 2112 2111 return 0; 2113 2112 } 2114 2113 2115 - static int _smcr_buf_map_lgr(struct smc_link *lnk, struct mutex *lock, 2114 + static int _smcr_buf_map_lgr(struct smc_link *lnk, struct rw_semaphore *lock, 2116 2115 struct list_head *lst, bool is_rmb) 2117 2116 { 2118 2117 struct smc_buf_desc *buf_desc, *bf; 2119 2118 int rc = 0; 2120 2119 2121 - mutex_lock(lock); 2120 + down_write(lock); 2122 2121 list_for_each_entry_safe(buf_desc, bf, lst, list) { 2123 2122 if (!buf_desc->used) 2124 2123 continue; ··· 2127 2126 goto out; 2128 2127 } 2129 2128 out: 2130 - mutex_unlock(lock); 2129 + up_write(lock); 2131 2130 return rc; 2132 2131 } 2133 2132 ··· 2160 2159 int i, rc = 0; 2161 2160 2162 2161 /* reg all RMBs for a new link */ 2163 - mutex_lock(&lgr->rmbs_lock); 2162 + down_write(&lgr->rmbs_lock); 2164 2163 for (i = 0; i < SMC_RMBE_SIZES; i++) { 2165 2164 list_for_each_entry_safe(buf_desc, bf, &lgr->rmbs[i], list) { 2166 2165 if (!buf_desc->used) 2167 2166 continue; 2168 2167 rc = smcr_link_reg_buf(lnk, buf_desc); 2169 2168 if (rc) { 2170 - mutex_unlock(&lgr->rmbs_lock); 2169 + up_write(&lgr->rmbs_lock); 2171 2170 return rc; 2172 2171 } 2173 2172 } 2174 2173 } 2175 - mutex_unlock(&lgr->rmbs_lock); 2174 + up_write(&lgr->rmbs_lock); 2176 2175 2177 2176 if (lgr->buf_type == SMCR_PHYS_CONT_BUFS) 2178 2177 return rc; 2179 2178 2180 2179 /* reg all vzalloced sndbufs for a new link */ 2181 - mutex_lock(&lgr->sndbufs_lock); 2180 + down_write(&lgr->sndbufs_lock); 2182 2181 for (i = 0; i < SMC_RMBE_SIZES; i++) { 2183 2182 list_for_each_entry_safe(buf_desc, bf, &lgr->sndbufs[i], list) { 2184 2183 if (!buf_desc->used || !buf_desc->is_vm) 2185 2184 continue; 2186 2185 rc = smcr_link_reg_buf(lnk, buf_desc); 2187 2186 if (rc) { 2188 - mutex_unlock(&lgr->sndbufs_lock); 2187 + up_write(&lgr->sndbufs_lock); 2189 2188 return rc; 2190 2189 } 2191 2190 } 2192 2191 } 2193 - mutex_unlock(&lgr->sndbufs_lock); 2192 + up_write(&lgr->sndbufs_lock); 2194 2193 return rc; 2195 2194 } 2196 2195 ··· 2248 2247 int i, rc = 0, cnt = 0; 2249 2248 2250 2249 /* protect against parallel link reconfiguration */ 2251 - mutex_lock(&lgr->llc_conf_mutex); 2250 + down_read(&lgr->llc_conf_mutex); 2252 2251 for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) { 2253 2252 struct smc_link *lnk = &lgr->lnk[i]; 2254 2253 ··· 2261 2260 cnt++; 2262 2261 } 2263 2262 out: 2264 - mutex_unlock(&lgr->llc_conf_mutex); 2263 + up_read(&lgr->llc_conf_mutex); 2265 2264 if (!rc && !cnt) 2266 2265 rc = -EINVAL; 2267 2266 return rc; ··· 2310 2309 struct smc_link_group *lgr = conn->lgr; 2311 2310 struct list_head *buf_list; 2312 2311 int bufsize, bufsize_short; 2312 + struct rw_semaphore *lock; /* lock buffer list */ 2313 2313 bool is_dgraded = false; 2314 - struct mutex *lock; /* lock buffer list */ 2315 2314 int sk_buf_size; 2316 2315 2317 2316 if (is_rmb) ··· 2359 2358 SMC_STAT_RMB_ALLOC(smc, is_smcd, is_rmb); 2360 2359 SMC_STAT_RMB_SIZE(smc, is_smcd, is_rmb, bufsize); 2361 2360 buf_desc->used = 1; 2362 - mutex_lock(lock); 2361 + down_write(lock); 2363 2362 list_add(&buf_desc->list, buf_list); 2364 - mutex_unlock(lock); 2363 + up_write(lock); 2365 2364 break; /* found */ 2366 2365 } 2367 2366 ··· 2435 2434 /* create rmb */ 2436 2435 rc = __smc_buf_create(smc, is_smcd, true); 2437 2436 if (rc) { 2438 - mutex_lock(&smc->conn.lgr->sndbufs_lock); 2437 + down_write(&smc->conn.lgr->sndbufs_lock); 2439 2438 list_del(&smc->conn.sndbuf_desc->list); 2440 - mutex_unlock(&smc->conn.lgr->sndbufs_lock); 2439 + up_write(&smc->conn.lgr->sndbufs_lock); 2441 2440 smc_buf_free(smc->conn.lgr, false, smc->conn.sndbuf_desc); 2442 2441 smc->conn.sndbuf_desc = NULL; 2443 2442 }
+3 -3
net/smc/smc_core.h
··· 252 252 unsigned short vlan_id; /* vlan id of link group */ 253 253 254 254 struct list_head sndbufs[SMC_RMBE_SIZES];/* tx buffers */ 255 - struct mutex sndbufs_lock; /* protects tx buffers */ 255 + struct rw_semaphore sndbufs_lock; /* protects tx buffers */ 256 256 struct list_head rmbs[SMC_RMBE_SIZES]; /* rx buffers */ 257 - struct mutex rmbs_lock; /* protects rx buffers */ 257 + struct rw_semaphore rmbs_lock; /* protects rx buffers */ 258 258 259 259 u8 id[SMC_LGR_ID_SIZE]; /* unique lgr id */ 260 260 struct delayed_work free_work; /* delayed freeing of an lgr */ ··· 298 298 /* queue for llc events */ 299 299 spinlock_t llc_event_q_lock; 300 300 /* protects llc_event_q */ 301 - struct mutex llc_conf_mutex; 301 + struct rw_semaphore llc_conf_mutex; 302 302 /* protects lgr reconfig. */ 303 303 struct work_struct llc_add_link_work; 304 304 struct work_struct llc_del_link_work;
+17 -17
net/smc/smc_llc.c
··· 608 608 609 609 prim_lnk_idx = link->link_idx; 610 610 lnk_idx = link_new->link_idx; 611 - mutex_lock(&lgr->rmbs_lock); 611 + down_write(&lgr->rmbs_lock); 612 612 ext->num_rkeys = lgr->conns_num; 613 613 if (!ext->num_rkeys) 614 614 goto out; ··· 628 628 } 629 629 len += i * sizeof(ext->rt[0]); 630 630 out: 631 - mutex_unlock(&lgr->rmbs_lock); 631 + up_write(&lgr->rmbs_lock); 632 632 return len; 633 633 } 634 634 ··· 889 889 int rc = 0; 890 890 int i; 891 891 892 - mutex_lock(&lgr->rmbs_lock); 892 + down_write(&lgr->rmbs_lock); 893 893 num_rkeys_send = lgr->conns_num; 894 894 buf_pos = smc_llc_get_first_rmb(lgr, &buf_lst); 895 895 do { ··· 916 916 break; 917 917 } while (num_rkeys_send || num_rkeys_recv); 918 918 919 - mutex_unlock(&lgr->rmbs_lock); 919 + up_write(&lgr->rmbs_lock); 920 920 return rc; 921 921 } 922 922 ··· 999 999 ext = (struct smc_llc_msg_add_link_v2_ext *)((u8 *)lgr->wr_rx_buf_v2 + 1000 1000 SMC_WR_TX_SIZE); 1001 1001 max = min_t(u8, ext->num_rkeys, SMC_LLC_RKEYS_PER_MSG_V2); 1002 - mutex_lock(&lgr->rmbs_lock); 1002 + down_write(&lgr->rmbs_lock); 1003 1003 for (i = 0; i < max; i++) { 1004 1004 smc_rtoken_set(lgr, link->link_idx, link_new->link_idx, 1005 1005 ext->rt[i].rmb_key, 1006 1006 ext->rt[i].rmb_vaddr_new, 1007 1007 ext->rt[i].rmb_key_new); 1008 1008 } 1009 - mutex_unlock(&lgr->rmbs_lock); 1009 + up_write(&lgr->rmbs_lock); 1010 1010 } 1011 1011 1012 1012 static void smc_llc_save_add_link_info(struct smc_link *link, ··· 1202 1202 1203 1203 qentry = smc_llc_flow_qentry_clr(&lgr->llc_flow_lcl); 1204 1204 1205 - mutex_lock(&lgr->llc_conf_mutex); 1205 + down_write(&lgr->llc_conf_mutex); 1206 1206 if (smc_llc_is_local_add_link(&qentry->msg)) 1207 1207 smc_llc_cli_add_link_invite(qentry->link, qentry); 1208 1208 else 1209 1209 smc_llc_cli_add_link(qentry->link, qentry); 1210 - mutex_unlock(&lgr->llc_conf_mutex); 1210 + up_write(&lgr->llc_conf_mutex); 1211 1211 } 1212 1212 1213 1213 static int smc_llc_active_link_count(struct smc_link_group *lgr) ··· 1313 1313 int rc = 0; 1314 1314 int i; 1315 1315 1316 - mutex_lock(&lgr->rmbs_lock); 1316 + down_write(&lgr->rmbs_lock); 1317 1317 num_rkeys_send = lgr->conns_num; 1318 1318 buf_pos = smc_llc_get_first_rmb(lgr, &buf_lst); 1319 1319 do { ··· 1338 1338 smc_llc_flow_qentry_del(&lgr->llc_flow_lcl); 1339 1339 } while (num_rkeys_send || num_rkeys_recv); 1340 1340 out: 1341 - mutex_unlock(&lgr->rmbs_lock); 1341 + up_write(&lgr->rmbs_lock); 1342 1342 return rc; 1343 1343 } 1344 1344 ··· 1509 1509 1510 1510 qentry = smc_llc_flow_qentry_clr(&lgr->llc_flow_lcl); 1511 1511 1512 - mutex_lock(&lgr->llc_conf_mutex); 1512 + down_write(&lgr->llc_conf_mutex); 1513 1513 rc = smc_llc_srv_add_link(link, qentry); 1514 1514 if (!rc && lgr->type == SMC_LGR_SYMMETRIC) { 1515 1515 /* delete any asymmetric link */ 1516 1516 smc_llc_delete_asym_link(lgr); 1517 1517 } 1518 - mutex_unlock(&lgr->llc_conf_mutex); 1518 + up_write(&lgr->llc_conf_mutex); 1519 1519 kfree(qentry); 1520 1520 } 1521 1521 ··· 1582 1582 smc_lgr_terminate_sched(lgr); 1583 1583 goto out; 1584 1584 } 1585 - mutex_lock(&lgr->llc_conf_mutex); 1585 + down_write(&lgr->llc_conf_mutex); 1586 1586 /* delete single link */ 1587 1587 for (lnk_idx = 0; lnk_idx < SMC_LINKS_PER_LGR_MAX; lnk_idx++) { 1588 1588 if (lgr->lnk[lnk_idx].link_id != del_llc->link_num) ··· 1616 1616 smc_lgr_terminate_sched(lgr); 1617 1617 } 1618 1618 out_unlock: 1619 - mutex_unlock(&lgr->llc_conf_mutex); 1619 + up_write(&lgr->llc_conf_mutex); 1620 1620 out: 1621 1621 kfree(qentry); 1622 1622 } ··· 1652 1652 int active_links; 1653 1653 int i; 1654 1654 1655 - mutex_lock(&lgr->llc_conf_mutex); 1655 + down_write(&lgr->llc_conf_mutex); 1656 1656 qentry = smc_llc_flow_qentry_clr(&lgr->llc_flow_lcl); 1657 1657 lnk = qentry->link; 1658 1658 del_llc = &qentry->msg.delete_link; ··· 1708 1708 smc_llc_add_link_local(lnk); 1709 1709 } 1710 1710 out: 1711 - mutex_unlock(&lgr->llc_conf_mutex); 1711 + up_write(&lgr->llc_conf_mutex); 1712 1712 kfree(qentry); 1713 1713 } 1714 1714 ··· 2126 2126 spin_lock_init(&lgr->llc_flow_lock); 2127 2127 init_waitqueue_head(&lgr->llc_flow_waiter); 2128 2128 init_waitqueue_head(&lgr->llc_msg_waiter); 2129 - mutex_init(&lgr->llc_conf_mutex); 2129 + init_rwsem(&lgr->llc_conf_mutex); 2130 2130 lgr->llc_testlink_time = READ_ONCE(net->smc.sysctl_smcr_testlink_time); 2131 2131 } 2132 2132