Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

powerpc/powernv: Fix concurrency issue with npu->mmio_atsd_usage

We've encountered a performance issue when multiple processors stress
{get,put}_mmio_atsd_reg(). These functions contend for
mmio_atsd_usage, an unsigned long used as a bitmask.

The accesses to mmio_atsd_usage are done using test_and_set_bit_lock()
and clear_bit_unlock(). As implemented, both of these will require
a (successful) stwcx to that same cache line.

What we end up with is thread A, attempting to unlock, being slowed by
other threads repeatedly attempting to lock. A's stwcx instructions
fail and retry because the memory reservation is lost every time a
different thread beats it to the punch.

There may be a long-term way to fix this at a larger scale, but for
now resolve the immediate problem by gating our call to
test_and_set_bit_lock() with one to test_bit(), which is obviously
implemented without using a store.

Fixes: 1ab66d1fbada ("powerpc/powernv: Introduce address translation services for Nvlink2")
Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
Acked-by: Alistair Popple <alistair@popple.id.au>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>

authored by

Reza Arbab and committed by
Michael Ellerman
9eab9901 06832fc0

+3 -2
+3 -2
arch/powerpc/platforms/powernv/npu-dma.c
··· 440 440 int i; 441 441 442 442 for (i = 0; i < npu->mmio_atsd_count; i++) { 443 - if (!test_and_set_bit_lock(i, &npu->mmio_atsd_usage)) 444 - return i; 443 + if (!test_bit(i, &npu->mmio_atsd_usage)) 444 + if (!test_and_set_bit_lock(i, &npu->mmio_atsd_usage)) 445 + return i; 445 446 } 446 447 447 448 return -ENOSPC;