Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
tracing/histogram: Update the documentation for the buckets modifier
Update both the tracefs README file as well as the histogram.rst to
include an explanation of what the buckets modifier is and how to use it.
Include an example with the wakeup_latency example for both log2 and the
buckets modifiers as there was no existing log2 example.
Link: https://lkml.kernel.org/r/20210707213922.167218794@goodmis.org
Acked-by: Namhyung Kim <namhyung@kernel.org>
Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
+87
-6
+86
-6
Documentation/trace/histogram.rst
+86
-6
Documentation/trace/histogram.rst
···
77
77
.syscall display a syscall id as a system call name
78
78
.execname display a common_pid as a program name
79
79
.log2 display log2 value rather than raw number
80
+
.buckets=size display grouping of values rather than raw number
80
81
.usecs display a common_timestamp in microseconds
81
82
=========== ==========================================
82
83
···
229
228
that lists the total number of bytes requested for each function in
230
229
the kernel that made one or more calls to kmalloc::
231
230
232
-
# echo 'hist:key=call_site:val=bytes_req' > \
231
+
# echo 'hist:key=call_site:val=bytes_req.buckets=32' > \
233
232
/sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
234
233
235
234
This tells the tracing system to create a 'hist' trigger using the
···
1824
1823
how that is done using hist trigger 'onmatch' action). Once that is
1825
1824
done, the 'wakeup_latency' synthetic event instance is created.
1826
1825
1827
-
A histogram can now be defined for the new synthetic event::
1828
-
1829
-
# echo 'hist:keys=pid,prio,lat.log2:sort=pid,lat' >> \
1830
-
/sys/kernel/debug/tracing/events/synthetic/wakeup_latency/trigger
1831
-
1832
1826
The new event is created under the tracing/events/synthetic/ directory
1833
1827
and looks and behaves just like any other event::
1834
1828
1835
1829
# ls /sys/kernel/debug/tracing/events/synthetic/wakeup_latency
1836
1830
enable filter format hist id trigger
1837
1831
1832
+
A histogram can now be defined for the new synthetic event::
1833
+
1834
+
# echo 'hist:keys=pid,prio,lat.log2:sort=lat' >> \
1835
+
/sys/kernel/debug/tracing/events/synthetic/wakeup_latency/trigger
1836
+
1837
+
The above shows the latency "lat" in a power of 2 grouping.
1838
+
1838
1839
Like any other event, once a histogram is enabled for the event, the
1839
1840
output can be displayed by reading the event's 'hist' file.
1841
+
1842
+
# cat /sys/kernel/debug/tracing/events/synthetic/wakeup_latency/hist
1843
+
1844
+
# event histogram
1845
+
#
1846
+
# trigger info: hist:keys=pid,prio,lat.log2:vals=hitcount:sort=lat.log2:size=2048 [active]
1847
+
#
1848
+
1849
+
{ pid: 2035, prio: 9, lat: ~ 2^2 } hitcount: 43
1850
+
{ pid: 2034, prio: 9, lat: ~ 2^2 } hitcount: 60
1851
+
{ pid: 2029, prio: 9, lat: ~ 2^2 } hitcount: 965
1852
+
{ pid: 2034, prio: 120, lat: ~ 2^2 } hitcount: 9
1853
+
{ pid: 2033, prio: 120, lat: ~ 2^2 } hitcount: 5
1854
+
{ pid: 2030, prio: 9, lat: ~ 2^2 } hitcount: 335
1855
+
{ pid: 2030, prio: 120, lat: ~ 2^2 } hitcount: 10
1856
+
{ pid: 2032, prio: 120, lat: ~ 2^2 } hitcount: 1
1857
+
{ pid: 2035, prio: 120, lat: ~ 2^2 } hitcount: 2
1858
+
{ pid: 2031, prio: 9, lat: ~ 2^2 } hitcount: 176
1859
+
{ pid: 2028, prio: 120, lat: ~ 2^2 } hitcount: 15
1860
+
{ pid: 2033, prio: 9, lat: ~ 2^2 } hitcount: 91
1861
+
{ pid: 2032, prio: 9, lat: ~ 2^2 } hitcount: 125
1862
+
{ pid: 2029, prio: 120, lat: ~ 2^2 } hitcount: 4
1863
+
{ pid: 2031, prio: 120, lat: ~ 2^2 } hitcount: 3
1864
+
{ pid: 2029, prio: 120, lat: ~ 2^3 } hitcount: 2
1865
+
{ pid: 2035, prio: 9, lat: ~ 2^3 } hitcount: 41
1866
+
{ pid: 2030, prio: 120, lat: ~ 2^3 } hitcount: 1
1867
+
{ pid: 2032, prio: 9, lat: ~ 2^3 } hitcount: 32
1868
+
{ pid: 2031, prio: 9, lat: ~ 2^3 } hitcount: 44
1869
+
{ pid: 2034, prio: 9, lat: ~ 2^3 } hitcount: 40
1870
+
{ pid: 2030, prio: 9, lat: ~ 2^3 } hitcount: 29
1871
+
{ pid: 2033, prio: 9, lat: ~ 2^3 } hitcount: 31
1872
+
{ pid: 2029, prio: 9, lat: ~ 2^3 } hitcount: 31
1873
+
{ pid: 2028, prio: 120, lat: ~ 2^3 } hitcount: 18
1874
+
{ pid: 2031, prio: 120, lat: ~ 2^3 } hitcount: 2
1875
+
{ pid: 2028, prio: 120, lat: ~ 2^4 } hitcount: 1
1876
+
{ pid: 2029, prio: 9, lat: ~ 2^4 } hitcount: 4
1877
+
{ pid: 2031, prio: 120, lat: ~ 2^7 } hitcount: 1
1878
+
{ pid: 2032, prio: 120, lat: ~ 2^7 } hitcount: 1
1879
+
1880
+
Totals:
1881
+
Hits: 2122
1882
+
Entries: 30
1883
+
Dropped: 0
1884
+
1885
+
1886
+
The latency values can also be grouped linearly by a given size with
1887
+
the ".buckets" modifier and specify a size (in this case groups of 10).
1888
+
1889
+
# echo 'hist:keys=pid,prio,lat.buckets=10:sort=lat' >> \
1890
+
/sys/kernel/debug/tracing/events/synthetic/wakeup_latency/trigger
1891
+
1892
+
# event histogram
1893
+
#
1894
+
# trigger info: hist:keys=pid,prio,lat.buckets=10:vals=hitcount:sort=lat.buckets=10:size=2048 [active]
1895
+
#
1896
+
1897
+
{ pid: 2067, prio: 9, lat: ~ 0-9 } hitcount: 220
1898
+
{ pid: 2068, prio: 9, lat: ~ 0-9 } hitcount: 157
1899
+
{ pid: 2070, prio: 9, lat: ~ 0-9 } hitcount: 100
1900
+
{ pid: 2067, prio: 120, lat: ~ 0-9 } hitcount: 6
1901
+
{ pid: 2065, prio: 120, lat: ~ 0-9 } hitcount: 2
1902
+
{ pid: 2066, prio: 120, lat: ~ 0-9 } hitcount: 2
1903
+
{ pid: 2069, prio: 9, lat: ~ 0-9 } hitcount: 122
1904
+
{ pid: 2069, prio: 120, lat: ~ 0-9 } hitcount: 8
1905
+
{ pid: 2070, prio: 120, lat: ~ 0-9 } hitcount: 1
1906
+
{ pid: 2068, prio: 120, lat: ~ 0-9 } hitcount: 7
1907
+
{ pid: 2066, prio: 9, lat: ~ 0-9 } hitcount: 365
1908
+
{ pid: 2064, prio: 120, lat: ~ 0-9 } hitcount: 35
1909
+
{ pid: 2065, prio: 9, lat: ~ 0-9 } hitcount: 998
1910
+
{ pid: 2071, prio: 9, lat: ~ 0-9 } hitcount: 85
1911
+
{ pid: 2065, prio: 9, lat: ~ 10-19 } hitcount: 2
1912
+
{ pid: 2064, prio: 120, lat: ~ 10-19 } hitcount: 2
1913
+
1914
+
Totals:
1915
+
Hits: 2112
1916
+
Entries: 16
1917
+
Dropped: 0
1840
1918
1841
1919
2.2.3 Hist trigger 'handlers' and 'actions'
1842
1920
-------------------------------------------
+1
kernel/trace/trace.c
+1
kernel/trace/trace.c
···
5654
5654
"\t .execname display a common_pid as a program name\n"
5655
5655
"\t .syscall display a syscall id as a syscall name\n"
5656
5656
"\t .log2 display log2 value rather than raw number\n"
5657
+
"\t .buckets=size display values in groups of size rather than raw number\n"
5657
5658
"\t .usecs display a common_timestamp in microseconds\n\n"
5658
5659
"\t The 'pause' parameter can be used to pause an existing hist\n"
5659
5660
"\t trigger or to start a hist trigger but not log any events\n"