Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

fgraph: Do not call handlers direct when not using ftrace_ops

The function graph tracer was modified to us the ftrace_ops of the
function tracer. This simplified the code as well as allowed more features
of the function graph tracer.

Not all architectures were converted over as it required the
implementation of HAVE_DYNAMIC_FTRACE_WITH_ARGS to implement. For those
architectures, it still did it the old way where the function graph tracer
handle was called by the function tracer trampoline. The handler then had
to check the hash to see if the registered handlers wanted to be called by
that function or not.

In order to speed up the function graph tracer that used ftrace_ops, if
only one callback was registered with function graph, it would call its
function directly via a static call.

Now, if the architecture does not support the use of using ftrace_ops and
still has the ftrace function trampoline calling the function graph
handler, then by doing a direct call it removes the check against the
handler's hash (list of functions it wants callbacks to), and it may call
that handler for functions that the handler did not request calls for.

On 32bit x86, which does not support the ftrace_ops use with function
graph tracer, it shows the issue:

~# trace-cmd start -p function -l schedule
~# trace-cmd show
# tracer: function_graph
#
# CPU DURATION FUNCTION CALLS
# | | | | | | |
2) * 11898.94 us | schedule();
3) # 1783.041 us | schedule();
1) | schedule() {
------------------------------------------
1) bash-8369 => kworker-7669
------------------------------------------
1) | schedule() {
------------------------------------------
1) kworker-7669 => bash-8369
------------------------------------------
1) + 97.004 us | }
1) | schedule() {
[..]

Now by starting the function tracer is another instance:

~# trace-cmd start -B foo -p function

This causes the function graph tracer to trace all functions (because the
function trace calls the function graph tracer for each on, and the
function graph trace is doing a direct call):

~# trace-cmd show
# tracer: function_graph
#
# CPU DURATION FUNCTION CALLS
# | | | | | | |
1) 1.669 us | } /* preempt_count_sub */
1) + 10.443 us | } /* _raw_spin_unlock_irqrestore */
1) | tick_program_event() {
1) | clockevents_program_event() {
1) 1.044 us | ktime_get();
1) 6.481 us | lapic_next_event();
1) + 10.114 us | }
1) + 11.790 us | }
1) ! 181.223 us | } /* hrtimer_interrupt */
1) ! 184.624 us | } /* __sysvec_apic_timer_interrupt */
1) | irq_exit_rcu() {
1) 0.678 us | preempt_count_sub();

When it should still only be tracing the schedule() function.

To fix this, add a macro FGRAPH_NO_DIRECT to be set to 0 when the
architecture does not support function graph use of ftrace_ops, and set to
1 otherwise. Then use this macro to know to allow function graph tracer to
call the handlers directly or not.

Cc: stable@vger.kernel.org
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Link: https://patch.msgid.link/20260218104244.5f14dade@gandalf.local.home
Fixes: cc60ee813b503 ("function_graph: Use static_call and branch to optimize entry function")
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>

+21 -4
+10 -3
include/linux/ftrace.h
··· 1092 1092 1093 1093 #ifdef CONFIG_FUNCTION_GRAPH_TRACER 1094 1094 #ifndef ftrace_graph_func 1095 - #define ftrace_graph_func ftrace_stub 1096 - #define FTRACE_OPS_GRAPH_STUB FTRACE_OPS_FL_STUB 1095 + # define ftrace_graph_func ftrace_stub 1096 + # define FTRACE_OPS_GRAPH_STUB FTRACE_OPS_FL_STUB 1097 + /* 1098 + * The function graph is called every time the function tracer is called. 1099 + * It must always test the ops hash and cannot just directly call 1100 + * the handler. 1101 + */ 1102 + # define FGRAPH_NO_DIRECT 1 1097 1103 #else 1098 - #define FTRACE_OPS_GRAPH_STUB 0 1104 + # define FTRACE_OPS_GRAPH_STUB 0 1105 + # define FGRAPH_NO_DIRECT 0 1099 1106 #endif 1100 1107 #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ 1101 1108
+11 -1
kernel/trace/fgraph.c
··· 539 539 static struct fgraph_ops *fgraph_direct_gops = &fgraph_stub; 540 540 DEFINE_STATIC_CALL(fgraph_func, ftrace_graph_entry_stub); 541 541 DEFINE_STATIC_CALL(fgraph_retfunc, ftrace_graph_ret_stub); 542 + #if FGRAPH_NO_DIRECT 543 + static DEFINE_STATIC_KEY_FALSE(fgraph_do_direct); 544 + #else 542 545 static DEFINE_STATIC_KEY_TRUE(fgraph_do_direct); 546 + #endif 543 547 544 548 /** 545 549 * ftrace_graph_stop - set to permanently disable function graph tracing ··· 847 843 bitmap = get_bitmap_bits(current, offset); 848 844 849 845 #ifdef CONFIG_HAVE_STATIC_CALL 850 - if (static_branch_likely(&fgraph_do_direct)) { 846 + if (!FGRAPH_NO_DIRECT && static_branch_likely(&fgraph_do_direct)) { 851 847 if (test_bit(fgraph_direct_gops->idx, &bitmap)) 852 848 static_call(fgraph_retfunc)(&trace, fgraph_direct_gops, fregs); 853 849 } else ··· 1289 1285 trace_func_graph_ret_t retfunc = NULL; 1290 1286 int i; 1291 1287 1288 + if (FGRAPH_NO_DIRECT) 1289 + return; 1290 + 1292 1291 if (gops) { 1293 1292 func = gops->entryfunc; 1294 1293 retfunc = gops->retfunc; ··· 1315 1308 1316 1309 static void ftrace_graph_disable_direct(bool disable_branch) 1317 1310 { 1311 + if (FGRAPH_NO_DIRECT) 1312 + return; 1313 + 1318 1314 if (disable_branch) 1319 1315 static_branch_disable(&fgraph_do_direct); 1320 1316 static_call_update(fgraph_func, ftrace_graph_entry_stub);