tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

page_pool: unlink from napi during destroy

Jesper points out that we must prevent recycling into cache
after page_pool_destroy() is called, because page_pool_destroy()
is not synchronized with recycling (some pages may still be
outstanding when destroy() gets called).

I assumed this will not happen because NAPI can't be scheduled
if its page pool is being destroyed. But I missed the fact that
NAPI may get reused. For instance when user changes ring configuration
driver may allocate a new page pool, stop NAPI, swap, start NAPI,
and then destroy the old pool. The NAPI is running so old page
pool will think it can recycle to the cache, but the consumer
at that point is the destroy() path, not NAPI.

To avoid extra synchronization let the drivers do "unlinking"
during the "swap" stage while NAPI is indeed disabled.

Fixes: 8c48eea3adf3 ("page_pool: allow caching from safely localized NAPI")
Reported-by: Jesper Dangaard Brouer <jbrouer@redhat.com>
Link: https://lore.kernel.org/all/e8df2654-6a5b-3c92-489d-2fe5e444135f@redhat.com/
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Link: https://lore.kernel.org/r/20230419182006.719923-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Jakub Kicinski dd64b232 4bb7aac7

+22 -1
+5
include/net/page_pool.h
··· 247 247 struct xdp_mem_info; 248 248 249 249 #ifdef CONFIG_PAGE_POOL 250 + void page_pool_unlink_napi(struct page_pool *pool); 250 251 void page_pool_destroy(struct page_pool *pool); 251 252 void page_pool_use_xdp_mem(struct page_pool *pool, void (*disconnect)(void *), 252 253 struct xdp_mem_info *mem); ··· 255 254 void page_pool_put_page_bulk(struct page_pool *pool, void **data, 256 255 int count); 257 256 #else 257 + static inline void page_pool_unlink_napi(struct page_pool *pool) 258 + { 259 + } 260 + 258 261 static inline void page_pool_destroy(struct page_pool *pool) 259 262 { 260 263 }
+17 -1
net/core/page_pool.c
··· 839 839 pool->xdp_mem_id = mem->id; 840 840 } 841 841 842 + void page_pool_unlink_napi(struct page_pool *pool) 843 + { 844 + if (!pool->p.napi) 845 + return; 846 + 847 + /* To avoid races with recycling and additional barriers make sure 848 + * pool and NAPI are unlinked when NAPI is disabled. 849 + */ 850 + WARN_ON(!test_bit(NAPI_STATE_SCHED, &pool->p.napi->state) || 851 + READ_ONCE(pool->p.napi->list_owner) != -1); 852 + 853 + WRITE_ONCE(pool->p.napi, NULL); 854 + } 855 + EXPORT_SYMBOL(page_pool_unlink_napi); 856 + 842 857 void page_pool_destroy(struct page_pool *pool) 843 858 { 844 859 if (!pool) ··· 862 847 if (!page_pool_put(pool)) 863 848 return; 864 849 850 + page_pool_unlink_napi(pool); 865 851 page_pool_free_frag(pool); 866 852 867 853 if (!page_pool_release(pool)) ··· 916 900 * in the same context as the consumer would run, so there's 917 901 * no possible race. 918 902 */ 919 - napi = pp->p.napi; 903 + napi = READ_ONCE(pp->p.napi); 920 904 allow_direct = napi_safe && napi && 921 905 READ_ONCE(napi->list_owner) == smp_processor_id(); 922 906