Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

vmlinux.lds.h: Adjust symbol ordering in text output section

When the -ffunction-sections compiler option is enabled, each function
is placed in a separate section named .text.function_name rather than
putting all functions in a single .text section.

However, using -function-sections can cause problems with the
linker script. The comments included in include/asm-generic/vmlinux.lds.h
note these issues.:
“TEXT_MAIN here will match .text.fixup and .text.unlikely if dead
code elimination is enabled, so these sections should be converted
to use ".." first.”

It is unclear whether there is a straightforward method for converting
a suffix to "..".

This patch modifies the order of subsections within the text output
section. Specifically, it changes current order:
.text.hot, .text, .text_unlikely, .text.unknown, .text.asan
to the new order:
.text.asan, .text.unknown, .text_unlikely, .text.hot, .text

Here is the rationale behind the new layout:

The majority of the code resides in three sections: .text.hot, .text,
and .text.unlikely, with .text.unknown containing a negligible amount.
.text.asan is only generated in ASAN builds.

The primary goal is to group code segments based on their execution
frequency (hotness).

First, we want to place .text.hot adjacent to .text. Since we cannot put
.text.hot after .text (Due to constraints with -ffunction-sections,
placing .text.hot after .text is problematic), we need to put
.text.hot before .text.

Then it comes to .text.unlikely, we cannot put it after .text (same
-ffunction-sections issue) . Therefore, we position .text.unlikely
before .text.hot.

.text.unknown and .tex.asan follow the same logic.

This revised ordering effectively reverses the original arrangement (for
.text.unlikely, .text.unknown, and .tex.asan), maintaining a similar level
of affinity between sections.

It also places .text.hot section at the beginning of a page to better
utilize the TLB entry.

Note that the limitation arises because the linker script employs glob
patterns instead of regular expressions for string matching. While there
is a method to maintain the current order using complex patterns, this
significantly complicates the pattern and increases the likelihood of
errors.

This patch also changes vmlinux.lds.S for the sparc64 architecture to
accommodate specific symbol placement requirements.

Co-developed-by: Han Shen <shenhan@google.com>
Signed-off-by: Han Shen <shenhan@google.com>
Signed-off-by: Rong Xu <xur@google.com>
Suggested-by: Sriraman Tallam <tmsriram@google.com>
Suggested-by: Krzysztof Pszeniczny <kpszeniczny@google.com>
Tested-by: Yonghong Song <yonghong.song@linux.dev>
Tested-by: Yabin Cui <yabinc@google.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

authored by

Rong Xu and committed by
Masahiro Yamada
0043ecea 52892ed6

+17 -7
+5
arch/sparc/kernel/vmlinux.lds.S
··· 48 48 { 49 49 _text = .; 50 50 HEAD_TEXT 51 + ALIGN_FUNCTION(); 52 + #ifdef CONFIG_SPARC64 53 + /* Match text section symbols in head_64.S first */ 54 + *head_64.o(.text) 55 + #endif 51 56 TEXT_TEXT 52 57 SCHED_TEXT 53 58 LOCK_TEXT
+12 -7
include/asm-generic/vmlinux.lds.h
··· 553 553 * .text section. Map to function alignment to avoid address changes 554 554 * during second ld run in second ld pass when generating System.map 555 555 * 556 - * TEXT_MAIN here will match .text.fixup and .text.unlikely if dead 557 - * code elimination is enabled, so these sections should be converted 558 - * to use ".." first. 556 + * TEXT_MAIN here will match symbols with a fixed pattern (for example, 557 + * .text.hot or .text.unlikely) if dead code elimination or 558 + * function-section is enabled. Match these symbols first before 559 + * TEXT_MAIN to ensure they are grouped together. 560 + * 561 + * Also placing .text.hot section at the beginning of a page, this 562 + * would help the TLB performance. 559 563 */ 560 564 #define TEXT_TEXT \ 561 565 ALIGN_FUNCTION(); \ 566 + *(.text.asan.* .text.tsan.*) \ 567 + *(.text.unknown .text.unknown.*) \ 568 + *(.text.unlikely .text.unlikely.*) \ 569 + . = ALIGN(PAGE_SIZE); \ 562 570 *(.text.hot .text.hot.*) \ 563 571 *(TEXT_MAIN .text.fixup) \ 564 - *(.text.unlikely .text.unlikely.*) \ 565 - *(.text.unknown .text.unknown.*) \ 566 572 NOINSTR_TEXT \ 567 - *(.ref.text) \ 568 - *(.text.asan.* .text.tsan.*) 573 + *(.ref.text) 569 574 570 575 571 576 /* sched.text is aling to function alignment to secure we have same