tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

page_pool: fix inconsistency for page_pool_ring_[un]lock()

page_pool_ring_[un]lock() use in_softirq() to decide which
spin lock variant to use, and when they are called in the
context with in_softirq() being false, spin_lock_bh() is
called in page_pool_ring_lock() while spin_unlock() is
called in page_pool_ring_unlock(), because spin_lock_bh()
has disabled the softirq in page_pool_ring_lock(), which
causes inconsistency for spin lock pair calling.

This patch fixes it by returning in_softirq state from
page_pool_producer_lock(), and use it to decide which
spin lock variant to use in page_pool_producer_unlock().

As pool->ring has both producer and consumer lock, so
rename it to page_pool_producer_[un]lock() to reflect
the actual usage. Also move them to page_pool.c as they
are only used there, and remove the 'inline' as the
compiler may have better idea to do inlining or not.

Fixes: 7886244736a4 ("net: page_pool: Add bulk support for ptr_ring")
Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Acked-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Link: https://lore.kernel.org/r/20230522031714.5089-1-linyunsheng@huawei.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

authored by

Yunsheng Lin and committed by
Jakub Kicinski 368d3cb4 3632679d

+26 -20
-18
include/net/page_pool.h
··· 399 399 page_pool_update_nid(pool, new_nid); 400 400 } 401 401 402 - static inline void page_pool_ring_lock(struct page_pool *pool) 403 - __acquires(&pool->ring.producer_lock) 404 - { 405 - if (in_softirq()) 406 - spin_lock(&pool->ring.producer_lock); 407 - else 408 - spin_lock_bh(&pool->ring.producer_lock); 409 - } 410 - 411 - static inline void page_pool_ring_unlock(struct page_pool *pool) 412 - __releases(&pool->ring.producer_lock) 413 - { 414 - if (in_softirq()) 415 - spin_unlock(&pool->ring.producer_lock); 416 - else 417 - spin_unlock_bh(&pool->ring.producer_lock); 418 - } 419 - 420 402 #endif /* _NET_PAGE_POOL_H */
+26 -2
net/core/page_pool.c
··· 134 134 #define recycle_stat_add(pool, __stat, val) 135 135 #endif 136 136 137 + static bool page_pool_producer_lock(struct page_pool *pool) 138 + __acquires(&pool->ring.producer_lock) 139 + { 140 + bool in_softirq = in_softirq(); 141 + 142 + if (in_softirq) 143 + spin_lock(&pool->ring.producer_lock); 144 + else 145 + spin_lock_bh(&pool->ring.producer_lock); 146 + 147 + return in_softirq; 148 + } 149 + 150 + static void page_pool_producer_unlock(struct page_pool *pool, 151 + bool in_softirq) 152 + __releases(&pool->ring.producer_lock) 153 + { 154 + if (in_softirq) 155 + spin_unlock(&pool->ring.producer_lock); 156 + else 157 + spin_unlock_bh(&pool->ring.producer_lock); 158 + } 159 + 137 160 static int page_pool_init(struct page_pool *pool, 138 161 const struct page_pool_params *params) 139 162 { ··· 640 617 int count) 641 618 { 642 619 int i, bulk_len = 0; 620 + bool in_softirq; 643 621 644 622 for (i = 0; i < count; i++) { 645 623 struct page *page = virt_to_head_page(data[i]); ··· 659 635 return; 660 636 661 637 /* Bulk producer into ptr_ring page_pool cache */ 662 - page_pool_ring_lock(pool); 638 + in_softirq = page_pool_producer_lock(pool); 663 639 for (i = 0; i < bulk_len; i++) { 664 640 if (__ptr_ring_produce(&pool->ring, data[i])) { 665 641 /* ring full */ ··· 668 644 } 669 645 } 670 646 recycle_stat_add(pool, ring, i); 671 - page_pool_ring_unlock(pool); 647 + page_pool_producer_unlock(pool, in_softirq); 672 648 673 649 /* Hopefully all pages was return into ptr_ring */ 674 650 if (likely(i == bulk_len))