Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

bitops: let the compiler optimize {__,}assign_bit()

Since commit b03fc1173c0c ("bitops: let optimize out non-atomic bitops
on compile-time constants"), the compilers are able to expand inline
bitmap operations to compile-time initializers when possible.
However, during the round of replacement if-__set-else-__clear with
__assign_bit() as per Andy's advice, bloat-o-meter showed +1024 bytes
difference in object code size for one module (even one function),
where the pattern:

DECLARE_BITMAP(foo) = { }; // on the stack, zeroed

if (a)
__set_bit(const_bit_num, foo);
if (b)
__set_bit(another_const_bit_num, foo);
...

is heavily used, although there should be no difference: the bitmap is
zeroed, so the second half of __assign_bit() should be compiled-out as
a no-op.
I either missed the fact that __assign_bit() has bitmap pointer marked
as `volatile` (as we usually do for bitops) or was hoping that the
compilers would at least try to look past the `volatile` for
__always_inline functions. Anyhow, due to that attribute, the compilers
were always compiling the whole expression and no mentioned compile-time
optimizations were working.

Convert __assign_bit() to a macro since it's a very simple if-else and
all of the checks are performed inside __set_bit() and __clear_bit(),
thus that wrapper has to be as transparent as possible. After that
change, despite it showing only -20 bytes change for vmlinux (due to
that it's still relatively unpopular), no drastic code size changes
happen when replacing if-set-else-clear for onstack bitmaps with
__assign_bit(), meaning the compiler now expands them to the actual
operations will all the expected optimizations.

Atomic assign_bit() is less affected due to its nature, but let's
convert it to a macro as well to keep the code consistent and not
leave a place for possible suboptimal codegen. Moreover, with certain
kernel configuration it actually gives some saves (x86):

do_ip_setsockopt 4154 4099 -55

Suggested-by: Yury Norov <yury.norov@gmail.com> # assign_bit(), too
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Acked-by: Yury Norov <yury.norov@gmail.com>
Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

authored by

Alexander Lobakin and committed by
David S. Miller
5259401e 7d8296b2

+4 -16
+4 -16
include/linux/bitops.h
··· 275 275 * @addr: the address to start counting from 276 276 * @value: the value to assign 277 277 */ 278 - static __always_inline void assign_bit(long nr, volatile unsigned long *addr, 279 - bool value) 280 - { 281 - if (value) 282 - set_bit(nr, addr); 283 - else 284 - clear_bit(nr, addr); 285 - } 278 + #define assign_bit(nr, addr, value) \ 279 + ((value) ? set_bit((nr), (addr)) : clear_bit((nr), (addr))) 286 280 287 - static __always_inline void __assign_bit(long nr, volatile unsigned long *addr, 288 - bool value) 289 - { 290 - if (value) 291 - __set_bit(nr, addr); 292 - else 293 - __clear_bit(nr, addr); 294 - } 281 + #define __assign_bit(nr, addr, value) \ 282 + ((value) ? __set_bit((nr), (addr)) : __clear_bit((nr), (addr))) 295 283 296 284 /** 297 285 * __ptr_set_bit - Set bit in a pointer's value