Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Merge tag 'trace-tools-v6.17' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing tools updates from Steven Rostedt:
- Introduce enum timerlat_tracing_mode
Now that BPF based sampling has been added to timerlat, add an enum
to represent which mode timerlat is running in
- Add action on timelat threshold feature
A new option, --on-threshold, is added, taking an argument that
further specifies the action. Actions added in this patch are:
- trace[,file=<filename>]: Saves tracefs buffer, optionally taking a
filename
- signal,num=<sig>,pid=<pid>: Sends signal to process. "parent" might
be specified instead of number to send signal to parent process
- shell,command=<command>: Execute shell command
- Allow resuming tracing in timerlat bpf
rtla-timerlat BPF program uses a global variable stored in a .bss
section to store whether tracing has been stopped. Map it to allow it
to resume tracing after it has been stopped
- Add continue action to timerlat
Introduce option to resume tracing after a latency threshold
overflow. The option is implemented as an action named "continue"
- Add action on end feature to timerlat
Implement actions on end next to actions on threshold. A new option,
--on-end is added, parallel to --on-threshold. Instead of being
executed whenever a latency threshold is reached, it is executed at
the end of the measurement
- Have rtla tests check output with grep
Add argument to the check command in the test suite that takes a
regular expression that the output of rtla command is checked
against. This allows testing for specific information in rtla output
in addition to checking the return value
- Add tests for timerlat actions
- Update the documentation for the new features
* tag 'trace-tools-v6.17' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
rtla/tests: Test timerlat -P option using actions
rtla/tests: Add grep checks for base test cases
Documentation/rtla: Add actions feature
rtla/tests: Limit duration to maximum of 10s
rtla/tests: Add tests for actions
rtla/tests: Check rtla output with grep
rtla/timerlat: Add action on end feature
rtla/timerlat: Add continue action
rtla/timerlat_bpf: Allow resuming tracing
rtla/timerlat: Add action on threshold feature
rtla/timerlat: Introduce enum timerlat_tracing_mode
+739
-131
+64
Documentation/tools/rtla/common_timerlat_options.rst
+64
Documentation/tools/rtla/common_timerlat_options.rst
···
55
55
Set timerlat to run without workload, waiting for the user to dispatch a per-cpu
56
56
task that waits for a new period on the tracing/osnoise/per_cpu/cpu$ID/timerlat_fd.
57
57
See linux/tools/rtla/sample/timerlat_load.py for an example of user-load code.
58
+
59
+
**--on-threshold** *action*
60
+
61
+
Defines an action to be executed when tracing is stopped on a latency threshold
62
+
specified by **-i/--irq** or **-T/--thread**.
63
+
64
+
Multiple --on-threshold actions may be specified, and they will be executed in
65
+
the order they are provided. If any action fails, subsequent actions in the list
66
+
will not be executed.
67
+
68
+
Supported actions are:
69
+
70
+
- *trace[,file=<filename>]*
71
+
72
+
Saves trace output, optionally taking a filename. Alternative to -t/--trace.
73
+
Note that nlike -t/--trace, specifying this multiple times will result in
74
+
the trace being saved multiple times.
75
+
76
+
- *signal,num=<sig>,pid=<pid>*
77
+
78
+
Sends signal to process. "parent" might be specified in place of pid to target
79
+
the parent process of rtla.
80
+
81
+
- *shell,command=<command>*
82
+
83
+
Execute shell command.
84
+
85
+
- *continue*
86
+
87
+
Continue tracing after actions are executed instead of stopping.
88
+
89
+
Example:
90
+
91
+
$ rtla timerlat -T 20 --on-threshold trace
92
+
--on-threshold shell,command="grep ipi_send timerlat_trace.txt"
93
+
--on-threshold signal,num=2,pid=parent
94
+
95
+
This will save a trace with the default filename "timerlat_trace.txt", print its
96
+
lines that contain the text "ipi_send" on standard output, and send signal 2
97
+
(SIGINT) to the parent process.
98
+
99
+
Performance Considerations:
100
+
101
+
For time-sensitive actions, it is recommended to run **rtla timerlat** with BPF
102
+
support and RT priority. Note that due to implementational limitations, actions
103
+
might be delayed up to one second after tracing is stopped if BPF mode is not
104
+
available or disabled.
105
+
106
+
**--on-end** *action*
107
+
108
+
Defines an action to be executed at the end of **rtla timerlat** tracing.
109
+
110
+
Multiple --on-end actions can be specified, and they will be executed in the order
111
+
they are provided. If any action fails, subsequent actions in the list will not be
112
+
executed.
113
+
114
+
See the documentation for **--on-threshold** for the list of supported actions, with
115
+
the exception that *continue* has no effect.
116
+
117
+
Example:
118
+
119
+
$ rtla timerlat -d 5s --on-end trace
120
+
121
+
This runs rtla timerlat with default options and save trace output at the end.
+1
tools/tracing/rtla/src/Build
+1
tools/tracing/rtla/src/Build
+260
tools/tracing/rtla/src/actions.c
+260
tools/tracing/rtla/src/actions.c
···
1
+
// SPDX-License-Identifier: GPL-2.0
2
+
#include <stdlib.h>
3
+
#include <string.h>
4
+
#include <signal.h>
5
+
#include <unistd.h>
6
+
7
+
#include "actions.h"
8
+
#include "trace.h"
9
+
#include "utils.h"
10
+
11
+
/*
12
+
* actions_init - initialize struct actions
13
+
*/
14
+
void
15
+
actions_init(struct actions *self)
16
+
{
17
+
self->size = action_default_size;
18
+
self->list = calloc(self->size, sizeof(struct action));
19
+
self->len = 0;
20
+
self->continue_flag = false;
21
+
22
+
memset(&self->present, 0, sizeof(self->present));
23
+
24
+
/* This has to be set by the user */
25
+
self->trace_output_inst = NULL;
26
+
}
27
+
28
+
/*
29
+
* actions_destroy - destroy struct actions
30
+
*/
31
+
void
32
+
actions_destroy(struct actions *self)
33
+
{
34
+
/* Free any action-specific data */
35
+
for (struct action *action = self->list; action < self->list + self->len; action++) {
36
+
if (action->type == ACTION_SHELL)
37
+
free(action->command);
38
+
if (action->type == ACTION_TRACE_OUTPUT)
39
+
free(action->trace_output);
40
+
}
41
+
42
+
/* Free action list */
43
+
free(self->list);
44
+
}
45
+
46
+
/*
47
+
* actions_new - Get pointer to new action
48
+
*/
49
+
static struct action *
50
+
actions_new(struct actions *self)
51
+
{
52
+
if (self->size >= self->len) {
53
+
self->size *= 2;
54
+
self->list = realloc(self->list, self->size * sizeof(struct action));
55
+
}
56
+
57
+
return &self->list[self->len++];
58
+
}
59
+
60
+
/*
61
+
* actions_add_trace_output - add an action to output trace
62
+
*/
63
+
int
64
+
actions_add_trace_output(struct actions *self, const char *trace_output)
65
+
{
66
+
struct action *action = actions_new(self);
67
+
68
+
self->present[ACTION_TRACE_OUTPUT] = true;
69
+
action->type = ACTION_TRACE_OUTPUT;
70
+
action->trace_output = calloc(strlen(trace_output) + 1, sizeof(char));
71
+
if (!action->trace_output)
72
+
return -1;
73
+
strcpy(action->trace_output, trace_output);
74
+
75
+
return 0;
76
+
}
77
+
78
+
/*
79
+
* actions_add_trace_output - add an action to send signal to a process
80
+
*/
81
+
int
82
+
actions_add_signal(struct actions *self, int signal, int pid)
83
+
{
84
+
struct action *action = actions_new(self);
85
+
86
+
self->present[ACTION_SIGNAL] = true;
87
+
action->type = ACTION_SIGNAL;
88
+
action->signal = signal;
89
+
action->pid = pid;
90
+
91
+
return 0;
92
+
}
93
+
94
+
/*
95
+
* actions_add_shell - add an action to execute a shell command
96
+
*/
97
+
int
98
+
actions_add_shell(struct actions *self, const char *command)
99
+
{
100
+
struct action *action = actions_new(self);
101
+
102
+
self->present[ACTION_SHELL] = true;
103
+
action->type = ACTION_SHELL;
104
+
action->command = calloc(strlen(command) + 1, sizeof(char));
105
+
if (!action->command)
106
+
return -1;
107
+
strcpy(action->command, command);
108
+
109
+
return 0;
110
+
}
111
+
112
+
/*
113
+
* actions_add_continue - add an action to resume measurement
114
+
*/
115
+
int
116
+
actions_add_continue(struct actions *self)
117
+
{
118
+
struct action *action = actions_new(self);
119
+
120
+
self->present[ACTION_CONTINUE] = true;
121
+
action->type = ACTION_CONTINUE;
122
+
123
+
return 0;
124
+
}
125
+
126
+
/*
127
+
* actions_parse - add an action based on text specification
128
+
*/
129
+
int
130
+
actions_parse(struct actions *self, const char *trigger)
131
+
{
132
+
enum action_type type = ACTION_NONE;
133
+
char *token;
134
+
char trigger_c[strlen(trigger)];
135
+
136
+
/* For ACTION_SIGNAL */
137
+
int signal = 0, pid = 0;
138
+
139
+
/* For ACTION_TRACE_OUTPUT */
140
+
char *trace_output;
141
+
142
+
strcpy(trigger_c, trigger);
143
+
token = strtok(trigger_c, ",");
144
+
145
+
if (strcmp(token, "trace") == 0)
146
+
type = ACTION_TRACE_OUTPUT;
147
+
else if (strcmp(token, "signal") == 0)
148
+
type = ACTION_SIGNAL;
149
+
else if (strcmp(token, "shell") == 0)
150
+
type = ACTION_SHELL;
151
+
else if (strcmp(token, "continue") == 0)
152
+
type = ACTION_CONTINUE;
153
+
else
154
+
/* Invalid trigger type */
155
+
return -1;
156
+
157
+
token = strtok(NULL, ",");
158
+
159
+
switch (type) {
160
+
case ACTION_TRACE_OUTPUT:
161
+
/* Takes no argument */
162
+
if (token == NULL)
163
+
trace_output = "timerlat_trace.txt";
164
+
else {
165
+
if (strlen(token) > 5 && strncmp(token, "file=", 5) == 0) {
166
+
trace_output = token + 5;
167
+
} else {
168
+
/* Invalid argument */
169
+
return -1;
170
+
}
171
+
172
+
token = strtok(NULL, ",");
173
+
if (token != NULL)
174
+
/* Only one argument allowed */
175
+
return -1;
176
+
}
177
+
return actions_add_trace_output(self, trace_output);
178
+
case ACTION_SIGNAL:
179
+
/* Takes two arguments, num (signal) and pid */
180
+
while (token != NULL) {
181
+
if (strlen(token) > 4 && strncmp(token, "num=", 4) == 0) {
182
+
signal = atoi(token + 4);
183
+
} else if (strlen(token) > 4 && strncmp(token, "pid=", 4) == 0) {
184
+
if (strncmp(token + 4, "parent", 7) == 0)
185
+
pid = -1;
186
+
else
187
+
pid = atoi(token + 4);
188
+
} else {
189
+
/* Invalid argument */
190
+
return -1;
191
+
}
192
+
193
+
token = strtok(NULL, ",");
194
+
}
195
+
196
+
if (!signal || !pid)
197
+
/* Missing argument */
198
+
return -1;
199
+
200
+
return actions_add_signal(self, signal, pid);
201
+
case ACTION_SHELL:
202
+
if (token == NULL)
203
+
return -1;
204
+
if (strlen(token) > 8 && strncmp(token, "command=", 8) == 0)
205
+
return actions_add_shell(self, token + 8);
206
+
return -1;
207
+
case ACTION_CONTINUE:
208
+
/* Takes no argument */
209
+
if (token != NULL)
210
+
return -1;
211
+
return actions_add_continue(self);
212
+
default:
213
+
return -1;
214
+
}
215
+
}
216
+
217
+
/*
218
+
* actions_perform - perform all actions
219
+
*/
220
+
int
221
+
actions_perform(struct actions *self)
222
+
{
223
+
int pid, retval;
224
+
const struct action *action;
225
+
226
+
for (action = self->list; action < self->list + self->len; action++) {
227
+
switch (action->type) {
228
+
case ACTION_TRACE_OUTPUT:
229
+
retval = save_trace_to_file(self->trace_output_inst, action->trace_output);
230
+
if (retval) {
231
+
err_msg("Error saving trace\n");
232
+
return retval;
233
+
}
234
+
break;
235
+
case ACTION_SIGNAL:
236
+
if (action->pid == -1)
237
+
pid = getppid();
238
+
else
239
+
pid = action->pid;
240
+
retval = kill(pid, action->signal);
241
+
if (retval) {
242
+
err_msg("Error sending signal\n");
243
+
return retval;
244
+
}
245
+
break;
246
+
case ACTION_SHELL:
247
+
retval = system(action->command);
248
+
if (retval)
249
+
return retval;
250
+
break;
251
+
case ACTION_CONTINUE:
252
+
self->continue_flag = true;
253
+
return 0;
254
+
default:
255
+
break;
256
+
}
257
+
}
258
+
259
+
return 0;
260
+
}
+52
tools/tracing/rtla/src/actions.h
+52
tools/tracing/rtla/src/actions.h
···
1
+
/* SPDX-License-Identifier: GPL-2.0 */
2
+
#include <tracefs.h>
3
+
#include <stdbool.h>
4
+
5
+
enum action_type {
6
+
ACTION_NONE = 0,
7
+
ACTION_TRACE_OUTPUT,
8
+
ACTION_SIGNAL,
9
+
ACTION_SHELL,
10
+
ACTION_CONTINUE,
11
+
ACTION_FIELD_N
12
+
};
13
+
14
+
struct action {
15
+
enum action_type type;
16
+
union {
17
+
struct {
18
+
/* For ACTION_TRACE_OUTPUT */
19
+
char *trace_output;
20
+
};
21
+
struct {
22
+
/* For ACTION_SIGNAL */
23
+
int signal;
24
+
int pid;
25
+
};
26
+
struct {
27
+
/* For ACTION_SHELL */
28
+
char *command;
29
+
};
30
+
};
31
+
};
32
+
33
+
static const int action_default_size = 8;
34
+
35
+
struct actions {
36
+
struct action *list;
37
+
int len, size;
38
+
bool present[ACTION_FIELD_N];
39
+
bool continue_flag;
40
+
41
+
/* External dependencies */
42
+
struct tracefs_instance *trace_output_inst;
43
+
};
44
+
45
+
void actions_init(struct actions *self);
46
+
void actions_destroy(struct actions *self);
47
+
int actions_add_trace_output(struct actions *self, const char *trace_output);
48
+
int actions_add_signal(struct actions *self, int signal, int pid);
49
+
int actions_add_shell(struct actions *self, const char *command);
50
+
int actions_add_continue(struct actions *self);
51
+
int actions_parse(struct actions *self, const char *trigger);
52
+
int actions_perform(struct actions *self);
+9
-4
tools/tracing/rtla/src/timerlat.bpf.c
+9
-4
tools/tracing/rtla/src/timerlat.bpf.c
···
29
29
} summary_irq SEC(".maps"), summary_thread SEC(".maps"), summary_user SEC(".maps");
30
30
31
31
struct {
32
+
__uint(type, BPF_MAP_TYPE_ARRAY);
33
+
__uint(max_entries, 1);
34
+
__type(key, unsigned int);
35
+
__type(value, unsigned long long);
36
+
} stop_tracing SEC(".maps");
37
+
38
+
struct {
32
39
__uint(type, BPF_MAP_TYPE_RINGBUF);
33
40
__uint(max_entries, 1);
34
41
} signal_stop_tracing SEC(".maps");
···
47
40
const volatile int irq_threshold;
48
41
const volatile int thread_threshold;
49
42
const volatile bool aa_only;
50
-
51
-
int stop_tracing;
52
43
53
44
nosubprog unsigned long long map_get(void *map,
54
45
unsigned int key)
···
114
109
int value = 0;
115
110
116
111
/* Suppress further sample processing */
117
-
stop_tracing = 1;
112
+
map_set(&stop_tracing, 0, 1);
118
113
119
114
/* Signal to userspace */
120
115
bpf_ringbuf_output(&signal_stop_tracing, &value, sizeof(value), 0);
···
126
121
unsigned long long latency, latency_us;
127
122
int bucket;
128
123
129
-
if (stop_tracing)
124
+
if (map_get(&stop_tracing, 0))
130
125
return 0;
131
126
132
127
latency = tp_args->timer_latency / output_divisor;
+15
-9
tools/tracing/rtla/src/timerlat.c
+15
-9
tools/tracing/rtla/src/timerlat.c
···
40
40
CPU_SET(i, ¶ms->monitored_cpus);
41
41
}
42
42
43
-
retval = osnoise_set_stop_us(tool->context, params->stop_us);
44
-
if (retval) {
45
-
err_msg("Failed to set stop us\n");
46
-
goto out_err;
47
-
}
43
+
if (params->mode != TRACING_MODE_BPF) {
44
+
/*
45
+
* In tracefs and mixed mode, timerlat tracer handles stopping
46
+
* on threshold
47
+
*/
48
+
retval = osnoise_set_stop_us(tool->context, params->stop_us);
49
+
if (retval) {
50
+
err_msg("Failed to set stop us\n");
51
+
goto out_err;
52
+
}
48
53
49
-
retval = osnoise_set_stop_total_us(tool->context, params->stop_total_us);
50
-
if (retval) {
51
-
err_msg("Failed to set stop total us\n");
52
-
goto out_err;
54
+
retval = osnoise_set_stop_total_us(tool->context, params->stop_total_us);
55
+
if (retval) {
56
+
err_msg("Failed to set stop total us\n");
57
+
goto out_err;
58
+
}
53
59
}
54
60
55
61
+23
-1
tools/tracing/rtla/src/timerlat.h
+23
-1
tools/tracing/rtla/src/timerlat.h
···
1
1
// SPDX-License-Identifier: GPL-2.0
2
+
#include "actions.h"
2
3
#include "osnoise.h"
4
+
5
+
/*
6
+
* Define timerlat tracing mode.
7
+
*
8
+
* There are three tracing modes:
9
+
* - tracefs-only, used when BPF is unavailable.
10
+
* - BPF-only, used when BPF is available and neither trace saving nor
11
+
* auto-analysis are enabled.
12
+
* - mixed mode, used when BPF is available and either trace saving or
13
+
* auto-analysis is enabled (which rely on sample collection through
14
+
* tracefs).
15
+
*/
16
+
enum timerlat_tracing_mode {
17
+
TRACING_MODE_BPF,
18
+
TRACING_MODE_TRACEFS,
19
+
TRACING_MODE_MIXED,
20
+
};
3
21
4
22
struct timerlat_params {
5
23
/* Common params */
6
24
char *cpus;
7
25
cpu_set_t monitored_cpus;
8
-
char *trace_output;
9
26
char *cgroup_name;
10
27
unsigned long long runtime;
11
28
long long stop_us;
···
47
30
cpu_set_t hk_cpu_set;
48
31
struct sched_attr sched_param;
49
32
struct trace_events *events;
33
+
enum timerlat_tracing_mode mode;
34
+
35
+
struct actions threshold_actions;
36
+
struct actions end_actions;
37
+
50
38
union {
51
39
struct {
52
40
/* top only */
+13
tools/tracing/rtla/src/timerlat_bpf.c
+13
tools/tracing/rtla/src/timerlat_bpf.c
···
106
106
return retval;
107
107
}
108
108
109
+
/*
110
+
* timerlat_bpf_restart_tracing - restart stopped tracing
111
+
*/
112
+
int timerlat_bpf_restart_tracing(void)
113
+
{
114
+
unsigned int key = 0;
115
+
unsigned long long value = 0;
116
+
117
+
return bpf_map__update_elem(bpf->maps.stop_tracing,
118
+
&key, sizeof(key),
119
+
&value, sizeof(value), BPF_ANY);
120
+
}
121
+
109
122
static int get_value(struct bpf_map *map_irq,
110
123
struct bpf_map *map_thread,
111
124
struct bpf_map *map_user,
+3
tools/tracing/rtla/src/timerlat_bpf.h
+3
tools/tracing/rtla/src/timerlat_bpf.h
···
18
18
void timerlat_bpf_detach(void);
19
19
void timerlat_bpf_destroy(void);
20
20
int timerlat_bpf_wait(int timeout);
21
+
int timerlat_bpf_restart_tracing(void);
21
22
int timerlat_bpf_get_hist_value(int key,
22
23
long long *value_irq,
23
24
long long *value_thread,
···
29
28
long long *value_thread,
30
29
long long *value_user,
31
30
int cpus);
31
+
32
32
static inline int have_libbpf_support(void) { return 1; }
33
33
#else
34
34
static inline int timerlat_bpf_init(struct timerlat_params *params)
···
40
38
static inline void timerlat_bpf_detach(void) { };
41
39
static inline void timerlat_bpf_destroy(void) { };
42
40
static inline int timerlat_bpf_wait(int timeout) { return -1; }
41
+
static inline int timerlat_bpf_restart_tracing(void) { return -1; };
43
42
static inline int timerlat_bpf_get_hist_value(int key,
44
43
long long *value_irq,
45
44
long long *value_thread,
+106
-34
tools/tracing/rtla/src/timerlat_hist.c
+106
-34
tools/tracing/rtla/src/timerlat_hist.c
···
757
757
" --warm-up s: let the workload run for s seconds before collecting data",
758
758
" --trace-buffer-size kB: set the per-cpu trace buffer size in kB",
759
759
" --deepest-idle-state n: only go down to idle state n on cpus used by timerlat to reduce exit from idle latency",
760
+
" --on-threshold <action>: define action to be executed at latency threshold, multiple are allowed",
761
+
" --on-end <action>: define action to be executed at measurement end, multiple are allowed",
760
762
NULL,
761
763
};
762
764
···
788
786
int auto_thresh;
789
787
int retval;
790
788
int c;
789
+
char *trace_output = NULL;
791
790
792
791
params = calloc(1, sizeof(*params));
793
792
if (!params)
794
793
exit(1);
794
+
795
+
actions_init(¶ms->threshold_actions);
796
+
actions_init(¶ms->end_actions);
795
797
796
798
/* disabled by default */
797
799
params->dma_latency = -1;
···
807
801
params->output_divisor = 1000;
808
802
params->bucket_size = 1;
809
803
params->entries = 256;
804
+
805
+
/* default to BPF mode */
806
+
params->mode = TRACING_MODE_BPF;
810
807
811
808
while (1) {
812
809
static struct option long_options[] = {
···
847
838
{"warm-up", required_argument, 0, '\2'},
848
839
{"trace-buffer-size", required_argument, 0, '\3'},
849
840
{"deepest-idle-state", required_argument, 0, '\4'},
841
+
{"on-threshold", required_argument, 0, '\5'},
842
+
{"on-end", required_argument, 0, '\6'},
850
843
{0, 0, 0, 0}
851
844
};
852
845
···
874
863
params->print_stack = auto_thresh;
875
864
876
865
/* set trace */
877
-
params->trace_output = "timerlat_trace.txt";
866
+
trace_output = "timerlat_trace.txt";
878
867
879
868
break;
880
869
case 'c':
···
964
953
case 't':
965
954
if (optarg) {
966
955
if (optarg[0] == '=')
967
-
params->trace_output = &optarg[1];
956
+
trace_output = &optarg[1];
968
957
else
969
-
params->trace_output = &optarg[0];
958
+
trace_output = &optarg[0];
970
959
} else if (optind < argc && argv[optind][0] != '-')
971
-
params->trace_output = argv[optind];
960
+
trace_output = argv[optind];
972
961
else
973
-
params->trace_output = "timerlat_trace.txt";
962
+
trace_output = "timerlat_trace.txt";
974
963
break;
975
964
case 'u':
976
965
params->user_workload = 1;
···
1040
1029
case '\4':
1041
1030
params->deepest_idle_state = get_llong_from_str(optarg);
1042
1031
break;
1032
+
case '\5':
1033
+
retval = actions_parse(¶ms->threshold_actions, optarg);
1034
+
if (retval) {
1035
+
err_msg("Invalid action %s\n", optarg);
1036
+
exit(EXIT_FAILURE);
1037
+
}
1038
+
break;
1039
+
case '\6':
1040
+
retval = actions_parse(¶ms->end_actions, optarg);
1041
+
if (retval) {
1042
+
err_msg("Invalid action %s\n", optarg);
1043
+
exit(EXIT_FAILURE);
1044
+
}
1045
+
break;
1043
1046
default:
1044
1047
timerlat_hist_usage("Invalid option");
1045
1048
}
1046
1049
}
1050
+
1051
+
if (trace_output)
1052
+
actions_add_trace_output(¶ms->threshold_actions, trace_output);
1047
1053
1048
1054
if (geteuid()) {
1049
1055
err_msg("rtla needs root permission\n");
···
1081
1053
1082
1054
if (params->kernel_workload && params->user_workload)
1083
1055
timerlat_hist_usage("--kernel-threads and --user-threads are mutually exclusive!");
1056
+
1057
+
/*
1058
+
* If auto-analysis or trace output is enabled, switch from BPF mode to
1059
+
* mixed mode
1060
+
*/
1061
+
if (params->mode == TRACING_MODE_BPF &&
1062
+
(params->threshold_actions.present[ACTION_TRACE_OUTPUT] ||
1063
+
params->end_actions.present[ACTION_TRACE_OUTPUT] || !params->no_aa))
1064
+
params->mode = TRACING_MODE_MIXED;
1084
1065
1085
1066
return params;
1086
1067
}
···
1186
1149
pthread_t timerlat_u;
1187
1150
int retval;
1188
1151
int nr_cpus, i;
1189
-
bool no_bpf = false;
1190
1152
1191
1153
params = timerlat_hist_parse_args(argc, argv);
1192
1154
if (!params)
···
1197
1161
goto out_exit;
1198
1162
}
1199
1163
1200
-
retval = timerlat_hist_apply_config(tool, params);
1201
-
if (retval) {
1202
-
err_msg("Could not apply config\n");
1203
-
goto out_free;
1204
-
}
1205
-
1206
1164
trace = &tool->trace;
1207
1165
/*
1208
1166
* Save trace instance into global variable so that SIGINT can stop
···
1205
1175
*/
1206
1176
hist_inst = trace;
1207
1177
1178
+
/*
1179
+
* Try to enable BPF, unless disabled explicitly.
1180
+
* If BPF enablement fails, fall back to tracefs mode.
1181
+
*/
1208
1182
if (getenv("RTLA_NO_BPF") && strncmp(getenv("RTLA_NO_BPF"), "1", 2) == 0) {
1209
1183
debug_msg("RTLA_NO_BPF set, disabling BPF\n");
1210
-
no_bpf = true;
1211
-
}
1212
-
1213
-
if (!no_bpf && !tep_find_event_by_name(trace->tep, "osnoise", "timerlat_sample")) {
1184
+
params->mode = TRACING_MODE_TRACEFS;
1185
+
} else if (!tep_find_event_by_name(trace->tep, "osnoise", "timerlat_sample")) {
1214
1186
debug_msg("osnoise:timerlat_sample missing, disabling BPF\n");
1215
-
no_bpf = true;
1216
-
}
1217
-
1218
-
if (!no_bpf) {
1187
+
params->mode = TRACING_MODE_TRACEFS;
1188
+
} else {
1219
1189
retval = timerlat_bpf_init(params);
1220
1190
if (retval) {
1221
1191
debug_msg("Could not enable BPF\n");
1222
-
no_bpf = true;
1192
+
params->mode = TRACING_MODE_TRACEFS;
1223
1193
}
1194
+
}
1195
+
1196
+
retval = timerlat_hist_apply_config(tool, params);
1197
+
if (retval) {
1198
+
err_msg("Could not apply config\n");
1199
+
goto out_free;
1224
1200
}
1225
1201
1226
1202
retval = enable_timerlat(trace);
···
1281
1245
}
1282
1246
}
1283
1247
1284
-
if (params->trace_output) {
1248
+
if (params->threshold_actions.present[ACTION_TRACE_OUTPUT] ||
1249
+
params->end_actions.present[ACTION_TRACE_OUTPUT]) {
1285
1250
record = osnoise_init_trace_tool("timerlat");
1286
1251
if (!record) {
1287
1252
err_msg("Failed to enable the trace instance\n");
1288
1253
goto out_free;
1289
1254
}
1255
+
params->threshold_actions.trace_output_inst = record->trace.inst;
1256
+
params->end_actions.trace_output_inst = record->trace.inst;
1290
1257
1291
1258
if (params->events) {
1292
1259
retval = trace_events_enable(&record->trace, params->events);
···
1355
1316
* tracing while enabling other instances. The trace instance is the
1356
1317
* one with most valuable information.
1357
1318
*/
1358
-
if (params->trace_output)
1319
+
if (record)
1359
1320
trace_instance_start(&record->trace);
1360
1321
if (!params->no_aa)
1361
1322
trace_instance_start(&aa->trace);
1362
-
if (no_bpf) {
1323
+
if (params->mode == TRACING_MODE_TRACEFS) {
1363
1324
trace_instance_start(trace);
1364
1325
} else {
1365
1326
retval = timerlat_bpf_attach();
···
1372
1333
tool->start_time = time(NULL);
1373
1334
timerlat_hist_set_signals(params);
1374
1335
1375
-
if (no_bpf) {
1336
+
if (params->mode == TRACING_MODE_TRACEFS) {
1376
1337
while (!stop_tracing) {
1377
1338
sleep(params->sleep_time);
1378
1339
···
1387
1348
goto out_hist;
1388
1349
}
1389
1350
1390
-
if (osnoise_trace_is_off(tool, record))
1391
-
break;
1351
+
if (osnoise_trace_is_off(tool, record)) {
1352
+
actions_perform(¶ms->threshold_actions);
1353
+
1354
+
if (!params->threshold_actions.continue_flag)
1355
+
/* continue flag not set, break */
1356
+
break;
1357
+
1358
+
/* continue action reached, re-enable tracing */
1359
+
if (record)
1360
+
trace_instance_start(&record->trace);
1361
+
if (!params->no_aa)
1362
+
trace_instance_start(&aa->trace);
1363
+
trace_instance_start(trace);
1364
+
}
1392
1365
1393
1366
/* is there still any user-threads ? */
1394
1367
if (params->user_workload) {
···
1410
1359
}
1411
1360
}
1412
1361
}
1413
-
} else
1414
-
timerlat_bpf_wait(-1);
1362
+
} else {
1363
+
while (!stop_tracing) {
1364
+
timerlat_bpf_wait(-1);
1415
1365
1416
-
if (!no_bpf) {
1366
+
if (!stop_tracing) {
1367
+
/* Threshold overflow, perform actions on threshold */
1368
+
actions_perform(¶ms->threshold_actions);
1369
+
1370
+
if (!params->threshold_actions.continue_flag)
1371
+
/* continue flag not set, break */
1372
+
break;
1373
+
1374
+
/* continue action reached, re-enable tracing */
1375
+
if (record)
1376
+
trace_instance_start(&record->trace);
1377
+
if (!params->no_aa)
1378
+
trace_instance_start(&aa->trace);
1379
+
timerlat_bpf_restart_tracing();
1380
+
}
1381
+
}
1382
+
}
1383
+
1384
+
if (params->mode != TRACING_MODE_TRACEFS) {
1417
1385
timerlat_bpf_detach();
1418
1386
retval = timerlat_hist_bpf_pull_data(tool);
1419
1387
if (retval) {
···
1448
1378
1449
1379
timerlat_print_stats(params, tool);
1450
1380
1381
+
actions_perform(¶ms->end_actions);
1382
+
1451
1383
return_value = PASSED;
1452
1384
1453
1385
if (osnoise_trace_is_off(tool, record) && !stop_tracing) {
···
1458
1386
if (!params->no_aa)
1459
1387
timerlat_auto_analysis(params->stop_us, params->stop_total_us);
1460
1388
1461
-
save_trace_to_file(record ? record->trace.inst : NULL,
1462
-
params->trace_output);
1463
1389
return_value = FAILED;
1464
1390
}
1465
1391
···
1479
1409
osnoise_destroy_tool(aa);
1480
1410
osnoise_destroy_tool(record);
1481
1411
osnoise_destroy_tool(tool);
1412
+
actions_destroy(¶ms->threshold_actions);
1413
+
actions_destroy(¶ms->end_actions);
1414
+
if (params->mode != TRACING_MODE_TRACEFS)
1415
+
timerlat_bpf_destroy();
1482
1416
free(params);
1483
1417
free_cpu_idle_disable_states();
1484
-
if (!no_bpf)
1485
-
timerlat_bpf_destroy();
1486
1418
out_exit:
1487
1419
exit(return_value);
1488
1420
}
+118
-61
tools/tracing/rtla/src/timerlat_top.c
+118
-61
tools/tracing/rtla/src/timerlat_top.c
···
516
516
" --warm-up s: let the workload run for s seconds before collecting data",
517
517
" --trace-buffer-size kB: set the per-cpu trace buffer size in kB",
518
518
" --deepest-idle-state n: only go down to idle state n on cpus used by timerlat to reduce exit from idle latency",
519
+
" --on-threshold <action>: define action to be executed at latency threshold, multiple are allowed",
520
+
" --on-end: define action to be executed at measurement end, multiple are allowed",
519
521
NULL,
520
522
};
521
523
···
547
545
long long auto_thresh;
548
546
int retval;
549
547
int c;
548
+
char *trace_output = NULL;
550
549
551
550
params = calloc(1, sizeof(*params));
552
551
if (!params)
553
552
exit(1);
553
+
554
+
actions_init(¶ms->threshold_actions);
555
+
actions_init(¶ms->end_actions);
554
556
555
557
/* disabled by default */
556
558
params->dma_latency = -1;
···
564
558
565
559
/* display data in microseconds */
566
560
params->output_divisor = 1000;
561
+
562
+
/* default to BPF mode */
563
+
params->mode = TRACING_MODE_BPF;
567
564
568
565
while (1) {
569
566
static struct option long_options[] = {
···
598
589
{"warm-up", required_argument, 0, '6'},
599
590
{"trace-buffer-size", required_argument, 0, '7'},
600
591
{"deepest-idle-state", required_argument, 0, '8'},
592
+
{"on-threshold", required_argument, 0, '9'},
593
+
{"on-end", required_argument, 0, '\1'},
601
594
{0, 0, 0, 0}
602
595
};
603
596
···
625
614
params->print_stack = auto_thresh;
626
615
627
616
/* set trace */
628
-
params->trace_output = "timerlat_trace.txt";
617
+
trace_output = "timerlat_trace.txt";
618
+
629
619
break;
630
620
case '5':
631
621
/* it is here because it is similar to -a */
···
721
709
case 't':
722
710
if (optarg) {
723
711
if (optarg[0] == '=')
724
-
params->trace_output = &optarg[1];
712
+
trace_output = &optarg[1];
725
713
else
726
-
params->trace_output = &optarg[0];
714
+
trace_output = &optarg[0];
727
715
} else if (optind < argc && argv[optind][0] != '-')
728
-
params->trace_output = argv[optind];
716
+
trace_output = argv[optind];
729
717
else
730
-
params->trace_output = "timerlat_trace.txt";
731
-
718
+
trace_output = "timerlat_trace.txt";
732
719
break;
733
720
case 'u':
734
721
params->user_workload = true;
···
779
768
case '8':
780
769
params->deepest_idle_state = get_llong_from_str(optarg);
781
770
break;
771
+
case '9':
772
+
retval = actions_parse(¶ms->threshold_actions, optarg);
773
+
if (retval) {
774
+
err_msg("Invalid action %s\n", optarg);
775
+
exit(EXIT_FAILURE);
776
+
}
777
+
break;
778
+
case '\1':
779
+
retval = actions_parse(¶ms->end_actions, optarg);
780
+
if (retval) {
781
+
err_msg("Invalid action %s\n", optarg);
782
+
exit(EXIT_FAILURE);
783
+
}
784
+
break;
782
785
default:
783
786
timerlat_top_usage("Invalid option");
784
787
}
785
788
}
789
+
790
+
if (trace_output)
791
+
actions_add_trace_output(¶ms->threshold_actions, trace_output);
786
792
787
793
if (geteuid()) {
788
794
err_msg("rtla needs root permission\n");
···
817
789
818
790
if (params->kernel_workload && params->user_workload)
819
791
timerlat_top_usage("--kernel-threads and --user-threads are mutually exclusive!");
792
+
793
+
/*
794
+
* If auto-analysis or trace output is enabled, switch from BPF mode to
795
+
* mixed mode
796
+
*/
797
+
if (params->mode == TRACING_MODE_BPF &&
798
+
(params->threshold_actions.present[ACTION_TRACE_OUTPUT] ||
799
+
params->end_actions.present[ACTION_TRACE_OUTPUT] || !params->no_aa))
800
+
params->mode = TRACING_MODE_MIXED;
820
801
821
802
return params;
822
803
}
···
918
881
static int
919
882
timerlat_top_main_loop(struct osnoise_tool *top,
920
883
struct osnoise_tool *record,
884
+
struct osnoise_tool *aa,
921
885
struct timerlat_params *params,
922
886
struct timerlat_u_params *params_u)
923
887
{
···
945
907
if (!params->quiet)
946
908
timerlat_print_stats(params, top);
947
909
948
-
if (osnoise_trace_is_off(top, record))
949
-
break;
910
+
if (osnoise_trace_is_off(top, record)) {
911
+
actions_perform(¶ms->threshold_actions);
912
+
913
+
if (!params->threshold_actions.continue_flag)
914
+
/* continue flag not set, break */
915
+
break;
916
+
917
+
/* continue action reached, re-enable tracing */
918
+
if (record)
919
+
trace_instance_start(&record->trace);
920
+
if (!params->no_aa)
921
+
trace_instance_start(&aa->trace);
922
+
trace_instance_start(trace);
923
+
}
950
924
951
925
/* is there still any user-threads ? */
952
926
if (params->user_workload) {
···
978
928
static int
979
929
timerlat_top_bpf_main_loop(struct osnoise_tool *top,
980
930
struct osnoise_tool *record,
931
+
struct osnoise_tool *aa,
981
932
struct timerlat_params *params,
982
933
struct timerlat_u_params *params_u)
983
934
{
···
990
939
return 0;
991
940
}
992
941
993
-
if (params->quiet) {
994
-
/* Quiet mode: wait for stop and then, print results */
995
-
timerlat_bpf_wait(-1);
996
-
997
-
retval = timerlat_top_bpf_pull_data(top);
998
-
if (retval) {
999
-
err_msg("Error pulling BPF data\n");
1000
-
return retval;
1001
-
}
1002
-
1003
-
return 0;
1004
-
}
1005
-
1006
942
/* Pull and display data in a loop */
1007
943
while (!stop_tracing) {
1008
-
wait_retval = timerlat_bpf_wait(params->sleep_time);
944
+
wait_retval = timerlat_bpf_wait(params->quiet ? -1 : params->sleep_time);
1009
945
1010
946
retval = timerlat_top_bpf_pull_data(top);
1011
947
if (retval) {
···
1000
962
return retval;
1001
963
}
1002
964
1003
-
timerlat_print_stats(params, top);
965
+
if (!params->quiet)
966
+
timerlat_print_stats(params, top);
1004
967
1005
-
if (wait_retval == 1)
968
+
if (wait_retval == 1) {
1006
969
/* Stopping requested by tracer */
1007
-
break;
970
+
actions_perform(¶ms->threshold_actions);
971
+
972
+
if (!params->threshold_actions.continue_flag)
973
+
/* continue flag not set, break */
974
+
break;
975
+
976
+
/* continue action reached, re-enable tracing */
977
+
if (record)
978
+
trace_instance_start(&record->trace);
979
+
if (!params->no_aa)
980
+
trace_instance_start(&aa->trace);
981
+
timerlat_bpf_restart_tracing();
982
+
}
1008
983
1009
984
/* is there still any user-threads ? */
1010
985
if (params->user_workload) {
···
1045
994
char *max_lat;
1046
995
int retval;
1047
996
int nr_cpus, i;
1048
-
bool no_bpf = false;
1049
997
1050
998
params = timerlat_top_parse_args(argc, argv);
1051
999
if (!params)
···
1056
1006
goto out_exit;
1057
1007
}
1058
1008
1009
+
trace = &top->trace;
1010
+
/*
1011
+
* Save trace instance into global variable so that SIGINT can stop
1012
+
* the timerlat tracer.
1013
+
* Otherwise, rtla could loop indefinitely when overloaded.
1014
+
*/
1015
+
top_inst = trace;
1016
+
1017
+
/*
1018
+
* Try to enable BPF, unless disabled explicitly.
1019
+
* If BPF enablement fails, fall back to tracefs mode.
1020
+
*/
1021
+
if (getenv("RTLA_NO_BPF") && strncmp(getenv("RTLA_NO_BPF"), "1", 2) == 0) {
1022
+
debug_msg("RTLA_NO_BPF set, disabling BPF\n");
1023
+
params->mode = TRACING_MODE_TRACEFS;
1024
+
} else if (!tep_find_event_by_name(trace->tep, "osnoise", "timerlat_sample")) {
1025
+
debug_msg("osnoise:timerlat_sample missing, disabling BPF\n");
1026
+
params->mode = TRACING_MODE_TRACEFS;
1027
+
} else {
1028
+
retval = timerlat_bpf_init(params);
1029
+
if (retval) {
1030
+
debug_msg("Could not enable BPF\n");
1031
+
params->mode = TRACING_MODE_TRACEFS;
1032
+
}
1033
+
}
1034
+
1059
1035
retval = timerlat_top_apply_config(top, params);
1060
1036
if (retval) {
1061
1037
err_msg("Could not apply config\n");
1062
1038
goto out_free;
1063
-
}
1064
-
1065
-
trace = &top->trace;
1066
-
/*
1067
-
* Save trace instance into global variable so that SIGINT can stop
1068
-
* the timerlat tracer.
1069
-
* Otherwise, rtla could loop indefinitely when overloaded.
1070
-
*/
1071
-
top_inst = trace;
1072
-
1073
-
if (getenv("RTLA_NO_BPF") && strncmp(getenv("RTLA_NO_BPF"), "1", 2) == 0) {
1074
-
debug_msg("RTLA_NO_BPF set, disabling BPF\n");
1075
-
no_bpf = true;
1076
-
}
1077
-
1078
-
if (!no_bpf && !tep_find_event_by_name(trace->tep, "osnoise", "timerlat_sample")) {
1079
-
debug_msg("osnoise:timerlat_sample missing, disabling BPF\n");
1080
-
no_bpf = true;
1081
-
}
1082
-
1083
-
if (!no_bpf) {
1084
-
retval = timerlat_bpf_init(params);
1085
-
if (retval) {
1086
-
debug_msg("Could not enable BPF\n");
1087
-
no_bpf = true;
1088
-
}
1089
1039
}
1090
1040
1091
1041
retval = enable_timerlat(trace);
···
1140
1090
}
1141
1091
}
1142
1092
1143
-
if (params->trace_output) {
1093
+
if (params->threshold_actions.present[ACTION_TRACE_OUTPUT] ||
1094
+
params->end_actions.present[ACTION_TRACE_OUTPUT]) {
1144
1095
record = osnoise_init_trace_tool("timerlat");
1145
1096
if (!record) {
1146
1097
err_msg("Failed to enable the trace instance\n");
1147
1098
goto out_free;
1148
1099
}
1100
+
params->threshold_actions.trace_output_inst = record->trace.inst;
1101
+
params->end_actions.trace_output_inst = record->trace.inst;
1149
1102
1150
1103
if (params->events) {
1151
1104
retval = trace_events_enable(&record->trace, params->events);
···
1215
1162
* tracing while enabling other instances. The trace instance is the
1216
1163
* one with most valuable information.
1217
1164
*/
1218
-
if (params->trace_output)
1165
+
if (record)
1219
1166
trace_instance_start(&record->trace);
1220
1167
if (!params->no_aa)
1221
1168
trace_instance_start(&aa->trace);
1222
-
if (no_bpf) {
1169
+
if (params->mode == TRACING_MODE_TRACEFS) {
1223
1170
trace_instance_start(trace);
1224
1171
} else {
1225
1172
retval = timerlat_bpf_attach();
···
1232
1179
top->start_time = time(NULL);
1233
1180
timerlat_top_set_signals(params);
1234
1181
1235
-
if (no_bpf)
1236
-
retval = timerlat_top_main_loop(top, record, params, ¶ms_u);
1182
+
if (params->mode == TRACING_MODE_TRACEFS)
1183
+
retval = timerlat_top_main_loop(top, record, aa, params, ¶ms_u);
1237
1184
else
1238
-
retval = timerlat_top_bpf_main_loop(top, record, params, ¶ms_u);
1185
+
retval = timerlat_top_bpf_main_loop(top, record, aa, params, ¶ms_u);
1239
1186
1240
1187
if (retval)
1241
1188
goto out_top;
1242
1189
1243
-
if (!no_bpf)
1190
+
if (params->mode != TRACING_MODE_TRACEFS)
1244
1191
timerlat_bpf_detach();
1245
1192
1246
1193
if (params->user_workload && !params_u.stopped_running) {
···
1250
1197
1251
1198
timerlat_print_stats(params, top);
1252
1199
1200
+
actions_perform(¶ms->end_actions);
1201
+
1253
1202
return_value = PASSED;
1254
1203
1255
1204
if (osnoise_trace_is_off(top, record) && !stop_tracing) {
···
1260
1205
if (!params->no_aa)
1261
1206
timerlat_auto_analysis(params->stop_us, params->stop_total_us);
1262
1207
1263
-
save_trace_to_file(record ? record->trace.inst : NULL,
1264
-
params->trace_output);
1265
1208
return_value = FAILED;
1266
1209
} else if (params->aa_only) {
1267
1210
/*
···
1292
1239
osnoise_destroy_tool(aa);
1293
1240
osnoise_destroy_tool(record);
1294
1241
osnoise_destroy_tool(top);
1242
+
actions_destroy(¶ms->threshold_actions);
1243
+
actions_destroy(¶ms->end_actions);
1244
+
if (params->mode != TRACING_MODE_TRACEFS)
1245
+
timerlat_bpf_destroy();
1295
1246
free(params);
1296
1247
free_cpu_idle_disable_states();
1297
1248
out_exit:
+17
-4
tools/tracing/rtla/tests/engine.sh
+17
-4
tools/tracing/rtla/tests/engine.sh
···
11
11
reset_osnoise() {
12
12
# Reset osnoise options to default and remove any dangling instances created
13
13
# by improperly exited rtla runs.
14
-
pushd /sys/kernel/tracing || return 1
14
+
pushd /sys/kernel/tracing >/dev/null || return 1
15
15
16
16
# Remove dangling instances created by previous rtla run
17
17
echo 0 > tracing_thresh
···
35
35
echo 0 > stop_tracing_us
36
36
echo 1000 > timerlat_period_us
37
37
38
-
popd
38
+
popd >/dev/null
39
39
}
40
40
41
41
check() {
42
+
test_name=$0
43
+
tested_command=$1
42
44
expected_exitcode=${3:-0}
45
+
expected_output=$4
43
46
# Simple check: run rtla with given arguments and test exit code.
44
47
# If TEST_COUNT is set, run the test. Otherwise, just count.
45
48
ctr=$(($ctr + 1))
···
52
49
[ "$NO_RESET_OSNOISE" == 1 ] || reset_osnoise
53
50
# Run rtla; in case of failure, include its output as comment
54
51
# in the test results.
55
-
result=$(stdbuf -oL $TIMEOUT "$RTLA" $2 2>&1); exitcode=$?
56
-
if [ $exitcode -eq $expected_exitcode ]
52
+
result=$(eval stdbuf -oL $TIMEOUT "$RTLA" $2 2>&1); exitcode=$?
53
+
# Test if the results matches if requested
54
+
if [ -n "$expected_output" ]
55
+
then
56
+
grep -E "$expected_output" <<< "$result" > /dev/null; grep_result=$?
57
+
else
58
+
grep_result=0
59
+
fi
60
+
61
+
if [ $exitcode -eq $expected_exitcode ] && [ $grep_result -eq 0 ]
57
62
then
58
63
echo "ok $ctr - $1"
59
64
else
···
69
58
# Add rtla output and exit code as comments in case of failure
70
59
echo "$result" | col -b | while read line; do echo "# $line"; done
71
60
printf "#\n# exit code %s\n" $exitcode
61
+
[ -n "$expected_output" ] && [ $grep_result -ne 0 ] && \
62
+
printf "# Output match failed: \"%s\"\n" "$expected_output"
72
63
fi
73
64
fi
74
65
}
+7
-6
tools/tracing/rtla/tests/hwnoise.t
+7
-6
tools/tracing/rtla/tests/hwnoise.t
···
6
6
set_timeout 2m
7
7
8
8
check "verify help page" \
9
-
"hwnoise --help"
9
+
"hwnoise --help" 0 "summary of hardware-related noise"
10
10
check "detect noise higher than one microsecond" \
11
-
"hwnoise -c 0 -T 1 -d 5s -q"
11
+
"hwnoise -c 0 -T 1 -d 5s -q" 0
12
12
check "set the automatic trace mode" \
13
-
"hwnoise -a 5 -d 30s" 2
13
+
"hwnoise -a 5 -d 10s" 2 "osnoise hit stop tracing"
14
14
check "set scheduling param to the osnoise tracer threads" \
15
-
"hwnoise -P F:1 -c 0 -r 900000 -d 1M -q"
15
+
"hwnoise -P F:1 -c 0 -r 900000 -d 10s -q"
16
16
check "stop the trace if a single sample is higher than 1 us" \
17
-
"hwnoise -s 1 -T 1 -t -d 30s" 2
17
+
"hwnoise -s 1 -T 1 -t -d 10s" 2 "Saving trace to osnoise_trace.txt"
18
18
check "enable a trace event trigger" \
19
-
"hwnoise -t -e osnoise:irq_noise trigger=\"hist:key=desc,duration:sort=desc,duration:vals=hitcount\" -d 1m"
19
+
"hwnoise -t -e osnoise:irq_noise --trigger=\"hist:key=desc,duration:sort=desc,duration:vals=hitcount\" -d 10s" \
20
+
0 "Saving event osnoise:irq_noise hist to osnoise_irq_noise_hist.txt"
20
21
21
22
test_end
+5
-5
tools/tracing/rtla/tests/osnoise.t
+5
-5
tools/tracing/rtla/tests/osnoise.t
···
6
6
set_timeout 2m
7
7
8
8
check "verify help page" \
9
-
"osnoise --help"
9
+
"osnoise --help" 0 "osnoise version"
10
10
check "verify the --priority/-P param" \
11
-
"osnoise top -P F:1 -c 0 -r 900000 -d 1M -q"
11
+
"osnoise top -P F:1 -c 0 -r 900000 -d 10s -q"
12
12
check "verify the --stop/-s param" \
13
-
"osnoise top -s 30 -T 1 -t" 2
13
+
"osnoise top -s 30 -T 1" 2 "osnoise hit stop tracing"
14
14
check "verify the --trace param" \
15
-
"osnoise hist -s 30 -T 1 -t" 2
15
+
"osnoise hist -s 30 -T 1 -t" 2 "Saving trace to osnoise_trace.txt"
16
16
check "verify the --entries/-E param" \
17
-
"osnoise hist -P F:1 -c 0 -r 900000 -d 1M -b 10 -E 25"
17
+
"osnoise hist -P F:1 -c 0 -r 900000 -d 10s -b 10 -E 25"
18
18
19
19
# Test setting default period by putting an absurdly high period
20
20
# and stopping on threshold.
+8
tools/tracing/rtla/tests/scripts/check-priority.sh
+8
tools/tracing/rtla/tests/scripts/check-priority.sh
···
1
+
#!/bin/bash
2
+
# SPDX-License-Identifier: GPL-2.0
3
+
pids="$(pgrep ^$1)" || exit 1
4
+
for pid in $pids
5
+
do
6
+
chrt -p $pid | cut -d ':' -f 2 | head -n1 | grep "^ $2\$" >/dev/null
7
+
chrt -p $pid | cut -d ':' -f 2 | tail -n1 | grep "^ $3\$" >/dev/null
8
+
done && echo "Priorities are set correctly"
+38
-7
tools/tracing/rtla/tests/timerlat.t
+38
-7
tools/tracing/rtla/tests/timerlat.t
···
18
18
for option in $no_bpf_options
19
19
do
20
20
export RTLA_NO_BPF=$option
21
+
22
+
# Basic tests
21
23
check "verify help page" \
22
-
"timerlat --help"
24
+
"timerlat --help" 0 "timerlat version"
23
25
check "verify -s/--stack" \
24
-
"timerlat top -s 3 -T 10 -t" 2
26
+
"timerlat top -s 3 -T 10 -t" 2 "Blocking thread stack trace"
25
27
check "verify -P/--priority" \
26
-
"timerlat top -P F:1 -c 0 -d 1M -q"
28
+
"timerlat top -P F:1 -c 0 -d 10s -q -T 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh timerlatu/ SCHED_FIFO 1\"" \
29
+
2 "Priorities are set correctly"
27
30
check "test in nanoseconds" \
28
-
"timerlat top -i 2 -c 0 -n -d 30s" 2
31
+
"timerlat top -i 2 -c 0 -n -d 10s" 2 "ns"
29
32
check "set the automatic trace mode" \
30
-
"timerlat top -a 5 --dump-tasks" 2
33
+
"timerlat top -a 5" 2 "analyzing it"
34
+
check "dump tasks" \
35
+
"timerlat top -a 5 --dump-tasks" 2 "Printing CPU tasks"
31
36
check "print the auto-analysis if hits the stop tracing condition" \
32
37
"timerlat top --aa-only 5" 2
33
38
check "disable auto-analysis" \
34
39
"timerlat top -s 3 -T 10 -t --no-aa" 2
35
40
check "verify -c/--cpus" \
36
-
"timerlat hist -c 0 -d 30s"
41
+
"timerlat hist -c 0 -d 10s"
37
42
check "hist test in nanoseconds" \
38
-
"timerlat hist -i 2 -c 0 -n -d 30s" 2
43
+
"timerlat hist -i 2 -c 0 -n -d 10s" 2 "ns"
44
+
45
+
# Actions tests
46
+
check "trace output through -t" \
47
+
"timerlat hist -T 2 -t" 2 "^ Saving trace to timerlat_trace.txt$"
48
+
check "trace output through -t with custom filename" \
49
+
"timerlat hist -T 2 -t custom_filename.txt" 2 "^ Saving trace to custom_filename.txt$"
50
+
check "trace output through -A trace" \
51
+
"timerlat hist -T 2 --on-threshold trace" 2 "^ Saving trace to timerlat_trace.txt$"
52
+
check "trace output through -A trace with custom filename" \
53
+
"timerlat hist -T 2 --on-threshold trace,file=custom_filename.txt" 2 "^ Saving trace to custom_filename.txt$"
54
+
check "exec command" \
55
+
"timerlat hist -T 2 --on-threshold shell,command='echo TestOutput'" 2 "^TestOutput$"
56
+
check "multiple actions" \
57
+
"timerlat hist -T 2 --on-threshold shell,command='echo -n 1' --on-threshold shell,command='echo 2'" 2 "^12$"
58
+
check "hist stop at failed action" \
59
+
"timerlat hist -T 2 --on-threshold shell,command='echo -n 1; false' --on-threshold shell,command='echo -n 2'" 2 "^1# RTLA timerlat histogram$"
60
+
check "top stop at failed action" \
61
+
"timerlat top -T 2 --on-threshold shell,command='echo -n 1; false' --on-threshold shell,command='echo -n 2'" 2 "^1ALL"
62
+
check "hist with continue" \
63
+
"timerlat hist -T 2 -d 1s --on-threshold shell,command='echo TestOutput' --on-threshold continue" 0 "^TestOutput$"
64
+
check "top with continue" \
65
+
"timerlat top -q -T 2 -d 1s --on-threshold shell,command='echo TestOutput' --on-threshold continue" 0 "^TestOutput$"
66
+
check "hist with trace output at end" \
67
+
"timerlat hist -d 1s --on-end trace" 0 "^ Saving trace to timerlat_trace.txt$"
68
+
check "top with trace output at end" \
69
+
"timerlat top -d 1s --on-end trace" 0 "^ Saving trace to timerlat_trace.txt$"
39
70
done
40
71
41
72
test_end