Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Merge branch 'mauro' into docs-mw
Mauro says:
This series replace get_abi.pl with a Python version.
I originally started it due to some issues I noticed when searching for
ABI symbols. While I could just go ahead and fix the already existing
script, I noticed that the script maintainance didn't have much care over
all those years, probably because it is easier to find Python programmers
those days.
Also, the code is complex and was not using modules or classes and
were using lots of global variables.
So, I decided to rewrite it in Python. I started with a manual conversion
for each function. Yet, to avoid future maintainership issues, I opted to
divide the main code on three classes, each on a sepaparate file.
Just like the original RFC, I opted to keep the Sphinx kernel-abi module
on three different phases:
- call get_abi.py as an exec file;
- import AbiParser on a minimal integration scenario;
- cleanup the code to avoid needing to parse line numbers from the text.
This way, if something goes wrong, it would be easier to just revert any
offending patches, It also provides a better rationale about what each
logical change is doing.
The initial patches on this series do some preparation work and
cleans some ABI symbol bugs that lack ":" delimiter.
+1829
-1321
+1
-1
Documentation/ABI/removed/sysfs-class-rfkill
+1
-1
Documentation/ABI/removed/sysfs-class-rfkill
···
4
4
5
5
What: /sys/class/rfkill/rfkill[0-9]+/claim
6
6
Date: 09-Jul-2007
7
-
KernelVersion v2.6.22
7
+
KernelVersion: v2.6.22
8
8
Contact: linux-wireless@vger.kernel.org
9
9
Description: This file was deprecated because there no longer was a way to
10
10
claim just control over a single rfkill instance.
+6
-6
Documentation/ABI/stable/sysfs-class-rfkill
+6
-6
Documentation/ABI/stable/sysfs-class-rfkill
···
16
16
17
17
What: /sys/class/rfkill/rfkill[0-9]+/name
18
18
Date: 09-Jul-2007
19
-
KernelVersion v2.6.22
19
+
KernelVersion: v2.6.22
20
20
Contact: linux-wireless@vger.kernel.org
21
21
Description: Name assigned by driver to this key (interface or driver name).
22
22
Values: arbitrary string.
···
24
24
25
25
What: /sys/class/rfkill/rfkill[0-9]+/type
26
26
Date: 09-Jul-2007
27
-
KernelVersion v2.6.22
27
+
KernelVersion: v2.6.22
28
28
Contact: linux-wireless@vger.kernel.org
29
29
Description: Driver type string ("wlan", "bluetooth", etc).
30
30
Values: See include/linux/rfkill.h.
···
32
32
33
33
What: /sys/class/rfkill/rfkill[0-9]+/persistent
34
34
Date: 09-Jul-2007
35
-
KernelVersion v2.6.22
35
+
KernelVersion: v2.6.22
36
36
Contact: linux-wireless@vger.kernel.org
37
37
Description: Whether the soft blocked state is initialised from non-volatile
38
38
storage at startup.
···
44
44
45
45
What: /sys/class/rfkill/rfkill[0-9]+/state
46
46
Date: 09-Jul-2007
47
-
KernelVersion v2.6.22
47
+
KernelVersion: v2.6.22
48
48
Contact: linux-wireless@vger.kernel.org
49
49
Description: Current state of the transmitter.
50
50
This file was scheduled to be removed in 2014, but due to its
···
67
67
68
68
What: /sys/class/rfkill/rfkill[0-9]+/hard
69
69
Date: 12-March-2010
70
-
KernelVersion v2.6.34
70
+
KernelVersion: v2.6.34
71
71
Contact: linux-wireless@vger.kernel.org
72
72
Description: Current hardblock state. This file is read only.
73
73
Values: A numeric value.
···
81
81
82
82
What: /sys/class/rfkill/rfkill[0-9]+/soft
83
83
Date: 12-March-2010
84
-
KernelVersion v2.6.34
84
+
KernelVersion: v2.6.34
85
85
Contact: linux-wireless@vger.kernel.org
86
86
Description: Current softblock state. This file is read and write.
87
87
Values: A numeric value.
+2
-2
Documentation/ABI/stable/sysfs-driver-dma-idxd
+2
-2
Documentation/ABI/stable/sysfs-driver-dma-idxd
···
246
246
capability.
247
247
248
248
What: /sys/bus/dsa/devices/wq<m>.<n>/occupancy
249
-
Date May 25, 2021
249
+
Date: May 25, 2021
250
250
KernelVersion: 5.14.0
251
251
Contact: dmaengine@vger.kernel.org
252
252
Description: Show the current number of entries in this WQ if WQ Occupancy
253
253
Support bit WQ capabilities is 1.
254
254
255
255
What: /sys/bus/dsa/devices/wq<m>.<n>/enqcmds_retries
256
-
Date Oct 29, 2021
256
+
Date: Oct 29, 2021
257
257
KernelVersion: 5.17.0
258
258
Contact: dmaengine@vger.kernel.org
259
259
Description: Indicate the number of retires for an enqcmds submission on a sharedwq.
+39
-39
Documentation/ABI/testing/sysfs-bus-coresight-devices-cti
+39
-39
Documentation/ABI/testing/sysfs-bus-coresight-devices-cti
···
1
1
What: /sys/bus/coresight/devices/<cti-name>/enable
2
2
Date: March 2020
3
-
KernelVersion 5.7
3
+
KernelVersion: 5.7
4
4
Contact: Mike Leach or Mathieu Poirier
5
5
Description: (RW) Enable/Disable the CTI hardware.
6
6
7
7
What: /sys/bus/coresight/devices/<cti-name>/powered
8
8
Date: March 2020
9
-
KernelVersion 5.7
9
+
KernelVersion: 5.7
10
10
Contact: Mike Leach or Mathieu Poirier
11
11
Description: (Read) Indicate if the CTI hardware is powered.
12
12
13
13
What: /sys/bus/coresight/devices/<cti-name>/ctmid
14
14
Date: March 2020
15
-
KernelVersion 5.7
15
+
KernelVersion: 5.7
16
16
Contact: Mike Leach or Mathieu Poirier
17
17
Description: (Read) Display the associated CTM ID
18
18
19
19
What: /sys/bus/coresight/devices/<cti-name>/nr_trigger_cons
20
20
Date: March 2020
21
-
KernelVersion 5.7
21
+
KernelVersion: 5.7
22
22
Contact: Mike Leach or Mathieu Poirier
23
23
Description: (Read) Number of devices connected to triggers on this CTI
24
24
25
25
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/name
26
26
Date: March 2020
27
-
KernelVersion 5.7
27
+
KernelVersion: 5.7
28
28
Contact: Mike Leach or Mathieu Poirier
29
29
Description: (Read) Name of connected device <N>
30
30
31
31
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/in_signals
32
32
Date: March 2020
33
-
KernelVersion 5.7
33
+
KernelVersion: 5.7
34
34
Contact: Mike Leach or Mathieu Poirier
35
35
Description: (Read) Input trigger signals from connected device <N>
36
36
37
37
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/in_types
38
38
Date: March 2020
39
-
KernelVersion 5.7
39
+
KernelVersion: 5.7
40
40
Contact: Mike Leach or Mathieu Poirier
41
41
Description: (Read) Functional types for the input trigger signals
42
42
from connected device <N>
43
43
44
44
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/out_signals
45
45
Date: March 2020
46
-
KernelVersion 5.7
46
+
KernelVersion: 5.7
47
47
Contact: Mike Leach or Mathieu Poirier
48
48
Description: (Read) Output trigger signals to connected device <N>
49
49
50
50
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/out_types
51
51
Date: March 2020
52
-
KernelVersion 5.7
52
+
KernelVersion: 5.7
53
53
Contact: Mike Leach or Mathieu Poirier
54
54
Description: (Read) Functional types for the output trigger signals
55
55
to connected device <N>
56
56
57
57
What: /sys/bus/coresight/devices/<cti-name>/regs/inout_sel
58
58
Date: March 2020
59
-
KernelVersion 5.7
59
+
KernelVersion: 5.7
60
60
Contact: Mike Leach or Mathieu Poirier
61
61
Description: (RW) Select the index for inen and outen registers.
62
62
63
63
What: /sys/bus/coresight/devices/<cti-name>/regs/inen
64
64
Date: March 2020
65
-
KernelVersion 5.7
65
+
KernelVersion: 5.7
66
66
Contact: Mike Leach or Mathieu Poirier
67
67
Description: (RW) Read or write the CTIINEN register selected by inout_sel.
68
68
69
69
What: /sys/bus/coresight/devices/<cti-name>/regs/outen
70
70
Date: March 2020
71
-
KernelVersion 5.7
71
+
KernelVersion: 5.7
72
72
Contact: Mike Leach or Mathieu Poirier
73
73
Description: (RW) Read or write the CTIOUTEN register selected by inout_sel.
74
74
75
75
What: /sys/bus/coresight/devices/<cti-name>/regs/gate
76
76
Date: March 2020
77
-
KernelVersion 5.7
77
+
KernelVersion: 5.7
78
78
Contact: Mike Leach or Mathieu Poirier
79
79
Description: (RW) Read or write CTIGATE register.
80
80
81
81
What: /sys/bus/coresight/devices/<cti-name>/regs/asicctl
82
82
Date: March 2020
83
-
KernelVersion 5.7
83
+
KernelVersion: 5.7
84
84
Contact: Mike Leach or Mathieu Poirier
85
85
Description: (RW) Read or write ASICCTL register.
86
86
87
87
What: /sys/bus/coresight/devices/<cti-name>/regs/intack
88
88
Date: March 2020
89
-
KernelVersion 5.7
89
+
KernelVersion: 5.7
90
90
Contact: Mike Leach or Mathieu Poirier
91
91
Description: (Write) Write the INTACK register.
92
92
93
93
What: /sys/bus/coresight/devices/<cti-name>/regs/appset
94
94
Date: March 2020
95
-
KernelVersion 5.7
95
+
KernelVersion: 5.7
96
96
Contact: Mike Leach or Mathieu Poirier
97
97
Description: (RW) Set CTIAPPSET register to activate channel. Read back to
98
98
determine current value of register.
99
99
100
100
What: /sys/bus/coresight/devices/<cti-name>/regs/appclear
101
101
Date: March 2020
102
-
KernelVersion 5.7
102
+
KernelVersion: 5.7
103
103
Contact: Mike Leach or Mathieu Poirier
104
104
Description: (Write) Write APPCLEAR register to deactivate channel.
105
105
106
106
What: /sys/bus/coresight/devices/<cti-name>/regs/apppulse
107
107
Date: March 2020
108
-
KernelVersion 5.7
108
+
KernelVersion: 5.7
109
109
Contact: Mike Leach or Mathieu Poirier
110
110
Description: (Write) Write APPPULSE to pulse a channel active for one clock
111
111
cycle.
112
112
113
113
What: /sys/bus/coresight/devices/<cti-name>/regs/chinstatus
114
114
Date: March 2020
115
-
KernelVersion 5.7
115
+
KernelVersion: 5.7
116
116
Contact: Mike Leach or Mathieu Poirier
117
117
Description: (Read) Read current status of channel inputs.
118
118
119
119
What: /sys/bus/coresight/devices/<cti-name>/regs/choutstatus
120
120
Date: March 2020
121
-
KernelVersion 5.7
121
+
KernelVersion: 5.7
122
122
Contact: Mike Leach or Mathieu Poirier
123
123
Description: (Read) read current status of channel outputs.
124
124
125
125
What: /sys/bus/coresight/devices/<cti-name>/regs/triginstatus
126
126
Date: March 2020
127
-
KernelVersion 5.7
127
+
KernelVersion: 5.7
128
128
Contact: Mike Leach or Mathieu Poirier
129
129
Description: (Read) read current status of input trigger signals
130
130
131
131
What: /sys/bus/coresight/devices/<cti-name>/regs/trigoutstatus
132
132
Date: March 2020
133
-
KernelVersion 5.7
133
+
KernelVersion: 5.7
134
134
Contact: Mike Leach or Mathieu Poirier
135
135
Description: (Read) read current status of output trigger signals.
136
136
137
137
What: /sys/bus/coresight/devices/<cti-name>/channels/trigin_attach
138
138
Date: March 2020
139
-
KernelVersion 5.7
139
+
KernelVersion: 5.7
140
140
Contact: Mike Leach or Mathieu Poirier
141
141
Description: (Write) Attach a CTI input trigger to a CTM channel.
142
142
143
143
What: /sys/bus/coresight/devices/<cti-name>/channels/trigin_detach
144
144
Date: March 2020
145
-
KernelVersion 5.7
145
+
KernelVersion: 5.7
146
146
Contact: Mike Leach or Mathieu Poirier
147
147
Description: (Write) Detach a CTI input trigger from a CTM channel.
148
148
149
149
What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_attach
150
150
Date: March 2020
151
-
KernelVersion 5.7
151
+
KernelVersion: 5.7
152
152
Contact: Mike Leach or Mathieu Poirier
153
153
Description: (Write) Attach a CTI output trigger to a CTM channel.
154
154
155
155
What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_detach
156
156
Date: March 2020
157
-
KernelVersion 5.7
157
+
KernelVersion: 5.7
158
158
Contact: Mike Leach or Mathieu Poirier
159
159
Description: (Write) Detach a CTI output trigger from a CTM channel.
160
160
161
161
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_gate_enable
162
162
Date: March 2020
163
-
KernelVersion 5.7
163
+
KernelVersion: 5.7
164
164
Contact: Mike Leach or Mathieu Poirier
165
165
Description: (RW) Enable CTIGATE for single channel (Write) or list enabled
166
166
channels through the gate (R).
167
167
168
168
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_gate_disable
169
169
Date: March 2020
170
-
KernelVersion 5.7
170
+
KernelVersion: 5.7
171
171
Contact: Mike Leach or Mathieu Poirier
172
172
Description: (Write) Disable CTIGATE for single channel.
173
173
174
174
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_set
175
175
Date: March 2020
176
-
KernelVersion 5.7
176
+
KernelVersion: 5.7
177
177
Contact: Mike Leach or Mathieu Poirier
178
178
Description: (Write) Activate a single channel.
179
179
180
180
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_clear
181
181
Date: March 2020
182
-
KernelVersion 5.7
182
+
KernelVersion: 5.7
183
183
Contact: Mike Leach or Mathieu Poirier
184
184
Description: (Write) Deactivate a single channel.
185
185
186
186
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_pulse
187
187
Date: March 2020
188
-
KernelVersion 5.7
188
+
KernelVersion: 5.7
189
189
Contact: Mike Leach or Mathieu Poirier
190
190
Description: (Write) Pulse a single channel - activate for a single clock cycle.
191
191
192
192
What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_filtered
193
193
Date: March 2020
194
-
KernelVersion 5.7
194
+
KernelVersion: 5.7
195
195
Contact: Mike Leach or Mathieu Poirier
196
196
Description: (Read) List of output triggers filtered across all connections.
197
197
198
198
What: /sys/bus/coresight/devices/<cti-name>/channels/trig_filter_enable
199
199
Date: March 2020
200
-
KernelVersion 5.7
200
+
KernelVersion: 5.7
201
201
Contact: Mike Leach or Mathieu Poirier
202
202
Description: (RW) Enable or disable trigger output signal filtering.
203
203
204
204
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_inuse
205
205
Date: March 2020
206
-
KernelVersion 5.7
206
+
KernelVersion: 5.7
207
207
Contact: Mike Leach or Mathieu Poirier
208
208
Description: (Read) show channels with at least one attached trigger signal.
209
209
210
210
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_free
211
211
Date: March 2020
212
-
KernelVersion 5.7
212
+
KernelVersion: 5.7
213
213
Contact: Mike Leach or Mathieu Poirier
214
214
Description: (Read) show channels with no attached trigger signals.
215
215
216
216
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_sel
217
217
Date: March 2020
218
-
KernelVersion 5.7
218
+
KernelVersion: 5.7
219
219
Contact: Mike Leach or Mathieu Poirier
220
220
Description: (RW) Write channel number to select a channel to view, read to
221
221
see selected channel number.
222
222
223
223
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_in
224
224
Date: March 2020
225
-
KernelVersion 5.7
225
+
KernelVersion: 5.7
226
226
Contact: Mike Leach or Mathieu Poirier
227
227
Description: (Read) Read to see input triggers connected to selected view
228
228
channel.
229
229
230
230
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_out
231
231
Date: March 2020
232
-
KernelVersion 5.7
232
+
KernelVersion: 5.7
233
233
Contact: Mike Leach or Mathieu Poirier
234
234
Description: (Read) Read to see output triggers connected to selected view
235
235
channel.
236
236
237
237
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_reset
238
238
Date: March 2020
239
-
KernelVersion 5.7
239
+
KernelVersion: 5.7
240
240
Contact: Mike Leach or Mathieu Poirier
241
241
Description: (Write) Clear all channel / trigger programming.
+26
-26
Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm
+26
-26
Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm
···
1
1
What: /sys/bus/coresight/devices/<tpdm-name>/integration_test
2
2
Date: January 2023
3
-
KernelVersion 6.2
3
+
KernelVersion: 6.2
4
4
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
5
5
Description:
6
6
(Write) Run integration test for tpdm. Integration test
···
14
14
15
15
What: /sys/bus/coresight/devices/<tpdm-name>/reset_dataset
16
16
Date: March 2023
17
-
KernelVersion 6.7
17
+
KernelVersion: 6.7
18
18
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
19
19
Description:
20
20
(Write) Reset the dataset of the tpdm.
···
24
24
25
25
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_type
26
26
Date: March 2023
27
-
KernelVersion 6.7
27
+
KernelVersion: 6.7
28
28
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
29
29
Description:
30
30
(RW) Set/Get the trigger type of the DSB for tpdm.
···
35
35
36
36
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_ts
37
37
Date: March 2023
38
-
KernelVersion 6.7
38
+
KernelVersion: 6.7
39
39
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
40
40
Description:
41
41
(RW) Set/Get the trigger timestamp of the DSB for tpdm.
···
46
46
47
47
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_mode
48
48
Date: March 2023
49
-
KernelVersion 6.7
49
+
KernelVersion: 6.7
50
50
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
51
51
Description:
52
52
(RW) Set/Get the programming mode of the DSB for tpdm.
···
60
60
61
61
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_idx
62
62
Date: March 2023
63
-
KernelVersion 6.7
63
+
KernelVersion: 6.7
64
64
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
65
65
Description:
66
66
(RW) Set/Get the index number of the edge detection for the DSB
···
69
69
70
70
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_val
71
71
Date: March 2023
72
-
KernelVersion 6.7
72
+
KernelVersion: 6.7
73
73
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
74
74
Description:
75
75
Write a data to control the edge detection corresponding to
···
85
85
86
86
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_mask
87
87
Date: March 2023
88
-
KernelVersion 6.7
88
+
KernelVersion: 6.7
89
89
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
90
90
Description:
91
91
Write a data to mask the edge detection corresponding to the index
···
97
97
98
98
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/edcr[0:15]
99
99
Date: March 2023
100
-
KernelVersion 6.7
100
+
KernelVersion: 6.7
101
101
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
102
102
Description:
103
103
Read a set of the edge control value of the DSB in TPDM.
104
104
105
105
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/edcmr[0:7]
106
106
Date: March 2023
107
-
KernelVersion 6.7
107
+
KernelVersion: 6.7
108
108
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
109
109
Description:
110
110
Read a set of the edge control mask of the DSB in TPDM.
111
111
112
112
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_patt/xpr[0:7]
113
113
Date: March 2023
114
-
KernelVersion 6.7
114
+
KernelVersion: 6.7
115
115
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
116
116
Description:
117
117
(RW) Set/Get the value of the trigger pattern for the DSB
···
119
119
120
120
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_patt/xpmr[0:7]
121
121
Date: March 2023
122
-
KernelVersion 6.7
122
+
KernelVersion: 6.7
123
123
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
124
124
Description:
125
125
(RW) Set/Get the mask of the trigger pattern for the DSB
···
127
127
128
128
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpr[0:7]
129
129
Date: March 2023
130
-
KernelVersion 6.7
130
+
KernelVersion: 6.7
131
131
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
132
132
Description:
133
133
(RW) Set/Get the value of the pattern for the DSB subunit TPDM.
134
134
135
135
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpmr[0:7]
136
136
Date: March 2023
137
-
KernelVersion 6.7
137
+
KernelVersion: 6.7
138
138
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
139
139
Description:
140
140
(RW) Set/Get the mask of the pattern for the DSB subunit TPDM.
141
141
142
142
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/enable_ts
143
143
Date: March 2023
144
-
KernelVersion 6.7
144
+
KernelVersion: 6.7
145
145
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
146
146
Description:
147
147
(Write) Set the pattern timestamp of DSB tpdm. Read
···
153
153
154
154
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/set_type
155
155
Date: March 2023
156
-
KernelVersion 6.7
156
+
KernelVersion: 6.7
157
157
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
158
158
Description:
159
159
(Write) Set the pattern type of DSB tpdm. Read
···
165
165
166
166
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_msr/msr[0:31]
167
167
Date: March 2023
168
-
KernelVersion 6.7
168
+
KernelVersion: 6.7
169
169
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
170
170
Description:
171
171
(RW) Set/Get the MSR(mux select register) for the DSB subunit
···
173
173
174
174
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_mode
175
175
Date: January 2024
176
-
KernelVersion 6.9
176
+
KernelVersion: 6.9
177
177
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
178
178
Description: (Write) Set the data collection mode of CMB tpdm. Continuous
179
179
change creates CMB data set elements on every CMBCLK edge.
···
187
187
188
188
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_patt/xpr[0:1]
189
189
Date: January 2024
190
-
KernelVersion 6.9
190
+
KernelVersion: 6.9
191
191
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
192
192
Description:
193
193
(RW) Set/Get the value of the trigger pattern for the CMB
···
195
195
196
196
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_patt/xpmr[0:1]
197
197
Date: January 2024
198
-
KernelVersion 6.9
198
+
KernelVersion: 6.9
199
199
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
200
200
Description:
201
201
(RW) Set/Get the mask of the trigger pattern for the CMB
···
203
203
204
204
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpr[0:1]
205
205
Date: January 2024
206
-
KernelVersion 6.9
206
+
KernelVersion: 6.9
207
207
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
208
208
Description:
209
209
(RW) Set/Get the value of the pattern for the CMB subunit TPDM.
210
210
211
211
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpmr[0:1]
212
212
Date: January 2024
213
-
KernelVersion 6.9
213
+
KernelVersion: 6.9
214
214
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
215
215
Description:
216
216
(RW) Set/Get the mask of the pattern for the CMB subunit TPDM.
217
217
218
218
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_patt/enable_ts
219
219
Date: January 2024
220
-
KernelVersion 6.9
220
+
KernelVersion: 6.9
221
221
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
222
222
Description:
223
223
(Write) Set the pattern timestamp of CMB tpdm. Read
···
229
229
230
230
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_ts
231
231
Date: January 2024
232
-
KernelVersion 6.9
232
+
KernelVersion: 6.9
233
233
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
234
234
Description:
235
235
(RW) Set/Get the trigger timestamp of the CMB for tpdm.
···
240
240
241
241
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_ts_all
242
242
Date: January 2024
243
-
KernelVersion 6.9
243
+
KernelVersion: 6.9
244
244
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
245
245
Description:
246
246
(RW) Read or write the status of timestamp upon all interface.
···
252
252
253
253
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_msr/msr[0:31]
254
254
Date: January 2024
255
-
KernelVersion 6.9
255
+
KernelVersion: 6.9
256
256
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
257
257
Description:
258
258
(RW) Set/Get the MSR(mux select register) for the CMB subunit
+2
-2
Documentation/ABI/testing/sysfs-fs-f2fs
+2
-2
Documentation/ABI/testing/sysfs-fs-f2fs
···
347
347
- [c] means add/del cold file extension
348
348
349
349
What: /sys/fs/f2fs/<disk>/unusable
350
-
Date April 2019
350
+
Date: April 2019
351
351
Contact: "Daniel Rosenberg" <drosen@google.com>
352
352
Description: If checkpoint=disable, it displays the number of blocks that
353
353
are unusable.
···
355
355
would be unusable if checkpoint=disable were to be set.
356
356
357
357
What: /sys/fs/f2fs/<disk>/encoding
358
-
Date July 2019
358
+
Date: July 2019
359
359
Contact: "Daniel Rosenberg" <drosen@google.com>
360
360
Description: Displays name and version of the encoding set for the filesystem.
361
361
If no encoding is set, displays (none)
+1
-1
Documentation/ABI/testing/sysfs-power
+1
-1
Documentation/ABI/testing/sysfs-power
···
131
131
CAUTION: Using it will cause your machine's real-time (CMOS)
132
132
clock to be set to a random invalid time after a resume.
133
133
134
-
What; /sys/power/pm_trace_dev_match
134
+
What: /sys/power/pm_trace_dev_match
135
135
Date: October 2010
136
136
Contact: James Hogan <jhogan@kernel.org>
137
137
Description:
+7
Documentation/admin-guide/abi-obsolete-files.rst
+7
Documentation/admin-guide/abi-obsolete-files.rst
+4
-2
Documentation/admin-guide/abi-obsolete.rst
+4
-2
Documentation/admin-guide/abi-obsolete.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
1
3
ABI obsolete symbols
2
4
====================
3
5
···
9
7
The description of the interface will document the reason why it is
10
8
obsolete and when it can be expected to be removed.
11
9
12
-
.. kernel-abi:: ABI/obsolete
13
-
:rst:
10
+
.. kernel-abi:: obsolete
11
+
:no-files:
+6
Documentation/admin-guide/abi-readme-file.rst
+6
Documentation/admin-guide/abi-readme-file.rst
+7
Documentation/admin-guide/abi-removed-files.rst
+7
Documentation/admin-guide/abi-removed-files.rst
+4
-2
Documentation/admin-guide/abi-removed.rst
+4
-2
Documentation/admin-guide/abi-removed.rst
+7
Documentation/admin-guide/abi-stable-files.rst
+7
Documentation/admin-guide/abi-stable-files.rst
+4
-2
Documentation/admin-guide/abi-stable.rst
+4
-2
Documentation/admin-guide/abi-stable.rst
+7
Documentation/admin-guide/abi-testing-files.rst
+7
Documentation/admin-guide/abi-testing-files.rst
+4
-2
Documentation/admin-guide/abi-testing.rst
+4
-2
Documentation/admin-guide/abi-testing.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
1
3
ABI testing symbols
2
4
===================
3
5
···
18
16
name to the description of these interfaces, so that the kernel
19
17
developers can easily notify them if any changes occur.
20
18
21
-
.. kernel-abi:: ABI/testing
22
-
:rst:
19
+
.. kernel-abi:: testing
20
+
:no-files:
+17
Documentation/admin-guide/abi.rst
+17
Documentation/admin-guide/abi.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
1
3
=====================
2
4
Linux ABI description
3
5
=====================
6
+
7
+
ABI symbols
8
+
-----------
4
9
5
10
.. toctree::
6
11
:maxdepth: 2
···
14
9
abi-testing
15
10
abi-obsolete
16
11
abi-removed
12
+
13
+
ABI files
14
+
---------
15
+
16
+
.. toctree::
17
+
:maxdepth: 2
18
+
19
+
abi-readme-file
20
+
abi-stable-files
21
+
abi-testing-files
22
+
abi-obsolete-files
23
+
abi-removed-files
+47
Documentation/sphinx/automarkup.py
+47
Documentation/sphinx/automarkup.py
···
11
11
import re
12
12
from itertools import chain
13
13
14
+
from kernel_abi import get_kernel_abi
15
+
14
16
#
15
17
# Python 2 lacks re.ASCII...
16
18
#
···
50
48
# an optional extension
51
49
#
52
50
RE_doc = re.compile(r'(\bDocumentation/)?((\.\./)*[\w\-/]+)\.(rst|txt)')
51
+
RE_abi_file = re.compile(r'(\bDocumentation/ABI/[\w\-/]+)')
52
+
RE_abi_symbol = re.compile(r'(\b/(sys|config|proc)/[\w\-/]+)')
53
53
54
54
RE_namespace = re.compile(r'^\s*..\s*c:namespace::\s*(\S+)\s*$')
55
55
···
88
84
# Associate each regex with the function that will markup its matches
89
85
#
90
86
markup_func_sphinx2 = {RE_doc: markup_doc_ref,
87
+
RE_abi_file: markup_abi_ref,
88
+
RE_abi_symbol: markup_abi_ref,
91
89
RE_function: markup_c_ref,
92
90
RE_generic_type: markup_c_ref}
93
91
94
92
markup_func_sphinx3 = {RE_doc: markup_doc_ref,
93
+
RE_abi_file: markup_abi_ref,
94
+
RE_abi_symbol: markup_abi_ref,
95
95
RE_function: markup_func_ref_sphinx3,
96
96
RE_struct: markup_c_ref,
97
97
RE_union: markup_c_ref,
···
267
259
#
268
260
try:
269
261
xref = stddom.resolve_xref(app.env, docname, app.builder, 'doc',
262
+
target, pxref, None)
263
+
except NoUri:
264
+
xref = None
265
+
#
266
+
# Return the xref if we got it; otherwise just return the plain text.
267
+
#
268
+
if xref:
269
+
return xref
270
+
else:
271
+
return nodes.Text(match.group(0))
272
+
273
+
#
274
+
# Try to replace a documentation reference of the form Documentation/ABI/...
275
+
# with a cross reference to that page
276
+
#
277
+
def markup_abi_ref(docname, app, match):
278
+
stddom = app.env.domains['std']
279
+
#
280
+
# Go through the dance of getting an xref out of the std domain
281
+
#
282
+
kernel_abi = get_kernel_abi()
283
+
284
+
fname = match.group(1)
285
+
target = kernel_abi.xref(fname)
286
+
287
+
# Kernel ABI doesn't describe such file or symbol
288
+
if not target:
289
+
return nodes.Text(match.group(0))
290
+
291
+
pxref = addnodes.pending_xref('', refdomain = 'std', reftype = 'ref',
292
+
reftarget = target, modname = None,
293
+
classname = None, refexplicit = False)
294
+
295
+
#
296
+
# XXX The Latex builder will throw NoUri exceptions here,
297
+
# work around that by ignoring them.
298
+
#
299
+
try:
300
+
xref = stddom.resolve_xref(app.env, docname, app.builder, 'ref',
270
301
target, pxref, None)
271
302
except NoUri:
272
303
xref = None
+95
-65
Documentation/sphinx/kernel_abi.py
+95
-65
Documentation/sphinx/kernel_abi.py
···
14
14
:license: GPL Version 2, June 1991 see Linux/COPYING for details.
15
15
16
16
The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
17
-
scripts/get_abi.pl script to parse the Kernel ABI files.
17
+
scripts/get_abi.py script to parse the Kernel ABI files.
18
18
19
19
Overview of directive's argument and options.
20
20
···
32
32
33
33
"""
34
34
35
-
import codecs
36
35
import os
37
-
import subprocess
38
-
import sys
39
36
import re
40
-
import kernellog
37
+
import sys
41
38
42
39
from docutils import nodes, statemachine
43
40
from docutils.statemachine import ViewList
44
41
from docutils.parsers.rst import directives, Directive
45
-
from docutils.utils.error_reporting import ErrorString
46
42
from sphinx.util.docutils import switch_source_input
43
+
from sphinx.util import logging
47
44
48
-
__version__ = '1.0'
45
+
srctree = os.path.abspath(os.environ["srctree"])
46
+
sys.path.insert(0, os.path.join(srctree, "scripts/lib/abi"))
47
+
48
+
from abi_parser import AbiParser
49
+
50
+
__version__ = "1.0"
51
+
52
+
logger = logging.getLogger('kernel_abi')
53
+
path = os.path.join(srctree, "Documentation/ABI")
54
+
55
+
_kernel_abi = None
56
+
57
+
def get_kernel_abi():
58
+
u"""
59
+
Initialize kernel_abi global var, if not initialized yet.
60
+
61
+
This is needed to avoid warnings during Sphinx module initialization.
62
+
"""
63
+
global _kernel_abi
64
+
65
+
if not _kernel_abi:
66
+
# Parse ABI symbols only once
67
+
_kernel_abi = AbiParser(path, logger=logger)
68
+
_kernel_abi.parse_abi()
69
+
_kernel_abi.check_issues()
70
+
71
+
return _kernel_abi
49
72
50
73
def setup(app):
51
74
52
75
app.add_directive("kernel-abi", KernelCmd)
53
-
return dict(
54
-
version = __version__
55
-
, parallel_read_safe = True
56
-
, parallel_write_safe = True
57
-
)
76
+
return {
77
+
"version": __version__,
78
+
"parallel_read_safe": True,
79
+
"parallel_write_safe": True
80
+
}
81
+
58
82
59
83
class KernelCmd(Directive):
60
-
61
84
u"""KernelABI (``kernel-abi``) directive"""
62
85
63
86
required_arguments = 1
64
-
optional_arguments = 2
87
+
optional_arguments = 3
65
88
has_content = False
66
89
final_argument_whitespace = True
90
+
parser = None
67
91
68
92
option_spec = {
69
-
"debug" : directives.flag,
70
-
"rst" : directives.unchanged
93
+
"debug": directives.flag,
94
+
"no-symbols": directives.flag,
95
+
"no-files": directives.flag,
71
96
}
72
97
73
98
def run(self):
99
+
kernel_abi = get_kernel_abi()
100
+
74
101
doc = self.state.document
75
102
if not doc.settings.file_insertion_enabled:
76
103
raise self.warning("docutils: file insertion disabled")
77
104
78
-
srctree = os.path.abspath(os.environ["srctree"])
79
-
80
-
args = [
81
-
os.path.join(srctree, 'scripts/get_abi.pl'),
82
-
'rest',
83
-
'--enable-lineno',
84
-
'--dir', os.path.join(srctree, 'Documentation', self.arguments[0]),
85
-
]
86
-
87
-
if 'rst' in self.options:
88
-
args.append('--rst-source')
89
-
90
-
lines = subprocess.check_output(args, cwd=os.path.dirname(doc.current_source)).decode('utf-8')
91
-
nodeList = self.nestedParse(lines, self.arguments[0])
92
-
return nodeList
93
-
94
-
def nestedParse(self, lines, fname):
95
105
env = self.state.document.settings.env
96
106
content = ViewList()
97
107
node = nodes.section()
98
108
99
-
if "debug" in self.options:
100
-
code_block = "\n\n.. code-block:: rst\n :linenos:\n"
101
-
for l in lines.split("\n"):
102
-
code_block += "\n " + l
103
-
lines = code_block + "\n\n"
109
+
abi_type = self.arguments[0]
104
110
105
-
line_regex = re.compile(r"^\.\. LINENO (\S+)\#([0-9]+)$")
106
-
ln = 0
111
+
if "no-symbols" in self.options:
112
+
show_symbols = False
113
+
else:
114
+
show_symbols = True
115
+
116
+
if "no-files" in self.options:
117
+
show_file = False
118
+
else:
119
+
show_file = True
120
+
121
+
tab_width = self.options.get('tab-width',
122
+
self.state.document.settings.tab_width)
123
+
124
+
old_f = None
107
125
n = 0
108
-
f = fname
109
-
110
-
for line in lines.split("\n"):
111
-
n = n + 1
112
-
match = line_regex.search(line)
113
-
if match:
114
-
new_f = match.group(1)
115
-
116
-
# Sphinx parser is lazy: it stops parsing contents in the
117
-
# middle, if it is too big. So, handle it per input file
118
-
if new_f != f and content:
119
-
self.do_parse(content, node)
120
-
content = ViewList()
121
-
122
-
# Add the file to Sphinx build dependencies
123
-
env.note_dependency(os.path.abspath(f))
124
-
125
-
f = new_f
126
-
127
-
# sphinx counts lines from 0
128
-
ln = int(match.group(2)) - 1
126
+
n_sym = 0
127
+
for msg, f, ln in kernel_abi.doc(show_file=show_file,
128
+
show_symbols=show_symbols,
129
+
filter_path=abi_type):
130
+
n_sym += 1
131
+
msg_list = statemachine.string2lines(msg, tab_width,
132
+
convert_whitespace=True)
133
+
if "debug" in self.options:
134
+
lines = [
135
+
"", "", ".. code-block:: rst",
136
+
" :linenos:", ""
137
+
]
138
+
for m in msg_list:
139
+
lines.append(" " + m)
129
140
else:
130
-
content.append(line, f, ln)
141
+
lines = msg_list
131
142
132
-
kernellog.info(self.state.document.settings.env.app, "%s: parsed %i lines" % (fname, n))
143
+
for line in lines:
144
+
# sphinx counts lines from 0
145
+
content.append(line, f, ln - 1)
146
+
n += 1
133
147
134
-
if content:
135
-
self.do_parse(content, node)
148
+
if f != old_f:
149
+
# Add the file to Sphinx build dependencies
150
+
env.note_dependency(os.path.abspath(f))
151
+
152
+
old_f = f
153
+
154
+
# Sphinx doesn't like to parse big messages. So, let's
155
+
# add content symbol by symbol
156
+
if content:
157
+
self.do_parse(content, node)
158
+
content = ViewList()
159
+
160
+
if show_symbols and not show_file:
161
+
logger.verbose("%s ABI: %i symbols (%i ReST lines)" % (abi_type, n_sym, n))
162
+
elif not show_symbols and show_file:
163
+
logger.verbose("%s ABI: %i files (%i ReST lines)" % (abi_type, n_sym, n))
164
+
else:
165
+
logger.verbose("%s ABI: %i data (%i ReST lines)" % (abi_type, n_sym, n))
136
166
137
167
return node.children
138
168
+7
-7
Documentation/sphinx/kerneldoc.py
+7
-7
Documentation/sphinx/kerneldoc.py
···
39
39
from docutils.parsers.rst import directives, Directive
40
40
import sphinx
41
41
from sphinx.util.docutils import switch_source_input
42
-
import kernellog
42
+
from sphinx.util import logging
43
43
44
44
__version__ = '1.0'
45
45
···
56
56
'functions': directives.unchanged,
57
57
}
58
58
has_content = False
59
+
logger = logging.getLogger('kerneldoc')
59
60
60
61
def run(self):
61
62
env = self.state.document.settings.env
···
110
109
cmd += [filename]
111
110
112
111
try:
113
-
kernellog.verbose(env.app,
114
-
'calling kernel-doc \'%s\'' % (" ".join(cmd)))
112
+
self.logger.verbose("calling kernel-doc '%s'" % (" ".join(cmd)))
115
113
116
114
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
117
115
out, err = p.communicate()
···
120
120
if p.returncode != 0:
121
121
sys.stderr.write(err)
122
122
123
-
kernellog.warn(env.app,
124
-
'kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode))
123
+
self.logger.warning("kernel-doc '%s' failed with return code %d"
124
+
% (" ".join(cmd), p.returncode))
125
125
return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
126
126
elif env.config.kerneldoc_verbosity > 0:
127
127
sys.stderr.write(err)
···
148
148
return node.children
149
149
150
150
except Exception as e: # pylint: disable=W0703
151
-
kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with: %s' %
152
-
(" ".join(cmd), str(e)))
151
+
self.logger.warning("kernel-doc '%s' processing failed with: %s" %
152
+
(" ".join(cmd), str(e)))
153
153
return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
154
154
155
155
def do_parse(self, result, node):
-22
Documentation/sphinx/kernellog.py
-22
Documentation/sphinx/kernellog.py
···
1
-
# SPDX-License-Identifier: GPL-2.0
2
-
#
3
-
# Sphinx has deprecated its older logging interface, but the replacement
4
-
# only goes back to 1.6. So here's a wrapper layer to keep around for
5
-
# as long as we support 1.4.
6
-
#
7
-
# We don't support 1.4 anymore, but we'll keep the wrappers around until
8
-
# we change all the code to not use them anymore :)
9
-
#
10
-
import sphinx
11
-
from sphinx.util import logging
12
-
13
-
logger = logging.getLogger('kerneldoc')
14
-
15
-
def warn(app, message):
16
-
logger.warning(message)
17
-
18
-
def verbose(app, message):
19
-
logger.verbose(message)
20
-
21
-
def info(app, message):
22
-
logger.info(message)
+43
-38
Documentation/sphinx/kfigure.py
+43
-38
Documentation/sphinx/kfigure.py
···
59
59
from docutils.parsers.rst.directives import images
60
60
import sphinx
61
61
from sphinx.util.nodes import clean_astext
62
-
import kernellog
62
+
from sphinx.util import logging
63
63
64
64
Figure = images.Figure
65
65
66
66
__version__ = '1.0.0'
67
+
68
+
logger = logging.getLogger('kfigure')
67
69
68
70
# simple helper
69
71
# -------------
···
172
170
"""
173
171
global dot_cmd, dot_Tpdf, convert_cmd, rsvg_convert_cmd # pylint: disable=W0603
174
172
global inkscape_cmd, inkscape_ver_one # pylint: disable=W0603
175
-
kernellog.verbose(app, "kfigure: check installed tools ...")
173
+
logger.verbose("kfigure: check installed tools ...")
176
174
177
175
dot_cmd = which('dot')
178
176
convert_cmd = which('convert')
···
180
178
inkscape_cmd = which('inkscape')
181
179
182
180
if dot_cmd:
183
-
kernellog.verbose(app, "use dot(1) from: " + dot_cmd)
181
+
logger.verbose("use dot(1) from: " + dot_cmd)
184
182
185
183
try:
186
184
dot_Thelp_list = subprocess.check_output([dot_cmd, '-Thelp'],
···
192
190
dot_Tpdf_ptn = b'pdf'
193
191
dot_Tpdf = re.search(dot_Tpdf_ptn, dot_Thelp_list)
194
192
else:
195
-
kernellog.warn(app, "dot(1) not found, for better output quality install "
196
-
"graphviz from https://www.graphviz.org")
193
+
logger.warning(
194
+
"dot(1) not found, for better output quality install graphviz from https://www.graphviz.org"
195
+
)
197
196
if inkscape_cmd:
198
-
kernellog.verbose(app, "use inkscape(1) from: " + inkscape_cmd)
197
+
logger.verbose("use inkscape(1) from: " + inkscape_cmd)
199
198
inkscape_ver = subprocess.check_output([inkscape_cmd, '--version'],
200
199
stderr=subprocess.DEVNULL)
201
200
ver_one_ptn = b'Inkscape 1'
···
207
204
208
205
else:
209
206
if convert_cmd:
210
-
kernellog.verbose(app, "use convert(1) from: " + convert_cmd)
207
+
logger.verbose("use convert(1) from: " + convert_cmd)
211
208
else:
212
-
kernellog.verbose(app,
209
+
logger.verbose(
213
210
"Neither inkscape(1) nor convert(1) found.\n"
214
-
"For SVG to PDF conversion, "
215
-
"install either Inkscape (https://inkscape.org/) (preferred) or\n"
216
-
"ImageMagick (https://www.imagemagick.org)")
211
+
"For SVG to PDF conversion, install either Inkscape (https://inkscape.org/) (preferred) or\n"
212
+
"ImageMagick (https://www.imagemagick.org)"
213
+
)
217
214
218
215
if rsvg_convert_cmd:
219
-
kernellog.verbose(app, "use rsvg-convert(1) from: " + rsvg_convert_cmd)
220
-
kernellog.verbose(app, "use 'dot -Tsvg' and rsvg-convert(1) for DOT -> PDF conversion")
216
+
logger.verbose("use rsvg-convert(1) from: " + rsvg_convert_cmd)
217
+
logger.verbose("use 'dot -Tsvg' and rsvg-convert(1) for DOT -> PDF conversion")
221
218
dot_Tpdf = False
222
219
else:
223
-
kernellog.verbose(app,
220
+
logger.verbose(
224
221
"rsvg-convert(1) not found.\n"
225
-
" SVG rendering of convert(1) is done by ImageMagick-native renderer.")
222
+
" SVG rendering of convert(1) is done by ImageMagick-native renderer."
223
+
)
226
224
if dot_Tpdf:
227
-
kernellog.verbose(app, "use 'dot -Tpdf' for DOT -> PDF conversion")
225
+
logger.verbose("use 'dot -Tpdf' for DOT -> PDF conversion")
228
226
else:
229
-
kernellog.verbose(app, "use 'dot -Tsvg' and convert(1) for DOT -> PDF conversion")
227
+
logger.verbose("use 'dot -Tsvg' and convert(1) for DOT -> PDF conversion")
230
228
231
229
232
230
# integrate conversion tools
···
261
257
262
258
# in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
263
259
264
-
kernellog.verbose(app, 'assert best format for: ' + img_node['uri'])
260
+
logger.verbose('assert best format for: ' + img_node['uri'])
265
261
266
262
if in_ext == '.dot':
267
263
268
264
if not dot_cmd:
269
-
kernellog.verbose(app,
270
-
"dot from graphviz not available / include DOT raw.")
265
+
logger.verbose("dot from graphviz not available / include DOT raw.")
271
266
img_node.replace_self(file2literal(src_fname))
272
267
273
268
elif translator.builder.format == 'latex':
···
293
290
294
291
if translator.builder.format == 'latex':
295
292
if not inkscape_cmd and convert_cmd is None:
296
-
kernellog.warn(app,
297
-
"no SVG to PDF conversion available / include SVG raw."
298
-
"\nIncluding large raw SVGs can cause xelatex error."
299
-
"\nInstall Inkscape (preferred) or ImageMagick.")
293
+
logger.warning(
294
+
"no SVG to PDF conversion available / include SVG raw.\n"
295
+
"Including large raw SVGs can cause xelatex error.\n"
296
+
"Install Inkscape (preferred) or ImageMagick."
297
+
)
300
298
img_node.replace_self(file2literal(src_fname))
301
299
else:
302
300
dst_fname = path.join(translator.builder.outdir, fname + '.pdf')
···
310
306
_name = dst_fname[len(str(translator.builder.outdir)) + 1:]
311
307
312
308
if isNewer(dst_fname, src_fname):
313
-
kernellog.verbose(app,
314
-
"convert: {out}/%s already exists and is newer" % _name)
309
+
logger.verbose("convert: {out}/%s already exists and is newer" % _name)
315
310
316
311
else:
317
312
ok = False
318
313
mkdir(path.dirname(dst_fname))
319
314
320
315
if in_ext == '.dot':
321
-
kernellog.verbose(app, 'convert DOT to: {out}/' + _name)
316
+
logger.verbose('convert DOT to: {out}/' + _name)
322
317
if translator.builder.format == 'latex' and not dot_Tpdf:
323
318
svg_fname = path.join(translator.builder.outdir, fname + '.svg')
324
319
ok1 = dot2format(app, src_fname, svg_fname)
···
328
325
ok = dot2format(app, src_fname, dst_fname)
329
326
330
327
elif in_ext == '.svg':
331
-
kernellog.verbose(app, 'convert SVG to: {out}/' + _name)
328
+
logger.verbose('convert SVG to: {out}/' + _name)
332
329
ok = svg2pdf(app, src_fname, dst_fname)
333
330
334
331
if not ok:
···
357
354
with open(out_fname, "w") as out:
358
355
exit_code = subprocess.call(cmd, stdout = out)
359
356
if exit_code != 0:
360
-
kernellog.warn(app,
357
+
logger.warning(
361
358
"Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
362
359
return bool(exit_code == 0)
363
360
···
391
388
pass
392
389
393
390
if exit_code != 0:
394
-
kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
391
+
logger.warning("Error #%d when calling: %s" %
392
+
(exit_code, " ".join(cmd)))
395
393
if warning_msg:
396
-
kernellog.warn(app, "Warning msg from %s: %s"
397
-
% (cmd_name, str(warning_msg, 'utf-8')))
394
+
logger.warning( "Warning msg from %s: %s" %
395
+
(cmd_name, str(warning_msg, 'utf-8')))
398
396
elif warning_msg:
399
-
kernellog.verbose(app, "Warning msg from %s (likely harmless):\n%s"
400
-
% (cmd_name, str(warning_msg, 'utf-8')))
397
+
logger.verbose("Warning msg from %s (likely harmless):\n%s" %
398
+
(cmd_name, str(warning_msg, 'utf-8')))
401
399
402
400
return bool(exit_code == 0)
403
401
···
422
418
# use stdout and stderr from parent
423
419
exit_code = subprocess.call(cmd)
424
420
if exit_code != 0:
425
-
kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
421
+
logger.warning("Error #%d when calling: %s" %
422
+
(exit_code, " ".join(cmd)))
426
423
ok = bool(exit_code == 0)
427
424
428
425
return ok
···
518
513
app = self.builder.app
519
514
srclang = node.get('srclang')
520
515
521
-
kernellog.verbose(app, 'visit kernel-render node lang: "%s"' % (srclang))
516
+
logger.verbose('visit kernel-render node lang: "%s"' % srclang)
522
517
523
518
tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
524
519
if tmp_ext is None:
525
-
kernellog.warn(app, 'kernel-render: "%s" unknown / include raw.' % (srclang))
520
+
logger.warning( 'kernel-render: "%s" unknown / include raw.' % srclang)
526
521
return
527
522
528
523
if not dot_cmd and tmp_ext == '.dot':
529
-
kernellog.verbose(app, "dot from graphviz not available / include raw.")
524
+
logger.verbose("dot from graphviz not available / include raw.")
530
525
return
531
526
532
527
literal_block = node[0]
+1
-1
scripts/documentation-file-ref-check
+1
-1
scripts/documentation-file-ref-check
···
92
92
next if ($f =~ m,^Next/,);
93
93
94
94
# Makefiles and scripts contain nasty expressions to parse docs
95
-
next if ($f =~ m/Makefile/ || $f =~ m/\.sh$/);
95
+
next if ($f =~ m/Makefile/ || $f =~ m/\.(sh|py|pl|~|rej|org|orig)$/);
96
96
97
97
# It doesn't make sense to parse hidden files
98
98
next if ($f =~ m#/\.#);
-1103
scripts/get_abi.pl
-1103
scripts/get_abi.pl
···
1
-
#!/usr/bin/env perl
2
-
# SPDX-License-Identifier: GPL-2.0
3
-
4
-
BEGIN { $Pod::Usage::Formatter = 'Pod::Text::Termcap'; }
5
-
6
-
use strict;
7
-
use warnings;
8
-
use utf8;
9
-
use Pod::Usage qw(pod2usage);
10
-
use Getopt::Long;
11
-
use File::Find;
12
-
use IO::Handle;
13
-
use Fcntl ':mode';
14
-
use Cwd 'abs_path';
15
-
use Data::Dumper;
16
-
17
-
my $help = 0;
18
-
my $hint = 0;
19
-
my $man = 0;
20
-
my $debug = 0;
21
-
my $enable_lineno = 0;
22
-
my $show_warnings = 1;
23
-
my $prefix="Documentation/ABI";
24
-
my $sysfs_prefix="/sys";
25
-
my $search_string;
26
-
27
-
# Debug options
28
-
my $dbg_what_parsing = 1;
29
-
my $dbg_what_open = 2;
30
-
my $dbg_dump_abi_structs = 4;
31
-
my $dbg_undefined = 8;
32
-
33
-
$Data::Dumper::Indent = 1;
34
-
$Data::Dumper::Terse = 1;
35
-
36
-
#
37
-
# If true, assumes that the description is formatted with ReST
38
-
#
39
-
my $description_is_rst = 1;
40
-
41
-
GetOptions(
42
-
"debug=i" => \$debug,
43
-
"enable-lineno" => \$enable_lineno,
44
-
"rst-source!" => \$description_is_rst,
45
-
"dir=s" => \$prefix,
46
-
'help|?' => \$help,
47
-
"show-hints" => \$hint,
48
-
"search-string=s" => \$search_string,
49
-
man => \$man
50
-
) or pod2usage(2);
51
-
52
-
pod2usage(1) if $help;
53
-
pod2usage(-exitstatus => 0, -noperldoc, -verbose => 2) if $man;
54
-
55
-
pod2usage(2) if (scalar @ARGV < 1 || @ARGV > 2);
56
-
57
-
my ($cmd, $arg) = @ARGV;
58
-
59
-
pod2usage(2) if ($cmd ne "search" && $cmd ne "rest" && $cmd ne "validate" && $cmd ne "undefined");
60
-
pod2usage(2) if ($cmd eq "search" && !$arg);
61
-
62
-
require Data::Dumper if ($debug & $dbg_dump_abi_structs);
63
-
64
-
my %data;
65
-
my %symbols;
66
-
67
-
#
68
-
# Displays an error message, printing file name and line
69
-
#
70
-
sub parse_error($$$$) {
71
-
my ($file, $ln, $msg, $data) = @_;
72
-
73
-
return if (!$show_warnings);
74
-
75
-
$data =~ s/\s+$/\n/;
76
-
77
-
print STDERR "Warning: file $file#$ln:\n\t$msg";
78
-
79
-
if ($data ne "") {
80
-
print STDERR ". Line\n\t\t$data";
81
-
} else {
82
-
print STDERR "\n";
83
-
}
84
-
}
85
-
86
-
#
87
-
# Parse an ABI file, storing its contents at %data
88
-
#
89
-
sub parse_abi {
90
-
my $file = $File::Find::name;
91
-
92
-
my $mode = (stat($file))[2];
93
-
return if ($mode & S_IFDIR);
94
-
return if ($file =~ m,/README,);
95
-
return if ($file =~ m,/\.,);
96
-
return if ($file =~ m,\.(rej|org|orig|bak)$,);
97
-
98
-
my $name = $file;
99
-
$name =~ s,.*/,,;
100
-
101
-
my $fn = $file;
102
-
$fn =~ s,.*Documentation/ABI/,,;
103
-
104
-
my $nametag = "File $fn";
105
-
$data{$nametag}->{what} = "File $name";
106
-
$data{$nametag}->{type} = "File";
107
-
$data{$nametag}->{file} = $name;
108
-
$data{$nametag}->{filepath} = $file;
109
-
$data{$nametag}->{is_file} = 1;
110
-
$data{$nametag}->{line_no} = 1;
111
-
112
-
my $type = $file;
113
-
$type =~ s,.*/(.*)/.*,$1,;
114
-
115
-
my $what;
116
-
my $new_what;
117
-
my $tag = "";
118
-
my $ln;
119
-
my $xrefs;
120
-
my $space;
121
-
my @labels;
122
-
my $label = "";
123
-
124
-
print STDERR "Opening $file\n" if ($debug & $dbg_what_open);
125
-
open IN, $file;
126
-
while(<IN>) {
127
-
$ln++;
128
-
if (m/^(\S+)(:\s*)(.*)/i) {
129
-
my $new_tag = lc($1);
130
-
my $sep = $2;
131
-
my $content = $3;
132
-
133
-
if (!($new_tag =~ m/(what|where|date|kernelversion|contact|description|users)/)) {
134
-
if ($tag eq "description") {
135
-
# New "tag" is actually part of
136
-
# description. Don't consider it a tag
137
-
$new_tag = "";
138
-
} elsif ($tag ne "") {
139
-
parse_error($file, $ln, "tag '$tag' is invalid", $_);
140
-
}
141
-
}
142
-
143
-
# Invalid, but it is a common mistake
144
-
if ($new_tag eq "where") {
145
-
parse_error($file, $ln, "tag 'Where' is invalid. Should be 'What:' instead", "");
146
-
$new_tag = "what";
147
-
}
148
-
149
-
if ($new_tag =~ m/what/) {
150
-
$space = "";
151
-
$content =~ s/[,.;]$//;
152
-
153
-
push @{$symbols{$content}->{file}}, " $file:" . ($ln - 1);
154
-
155
-
if ($tag =~ m/what/) {
156
-
$what .= "\xac" . $content;
157
-
} else {
158
-
if ($what) {
159
-
parse_error($file, $ln, "What '$what' doesn't have a description", "") if (!$data{$what}->{description});
160
-
161
-
foreach my $w(split /\xac/, $what) {
162
-
$symbols{$w}->{xref} = $what;
163
-
};
164
-
}
165
-
166
-
$what = $content;
167
-
$label = $content;
168
-
$new_what = 1;
169
-
}
170
-
push @labels, [($content, $label)];
171
-
$tag = $new_tag;
172
-
173
-
push @{$data{$nametag}->{symbols}}, $content if ($data{$nametag}->{what});
174
-
next;
175
-
}
176
-
177
-
if ($tag ne "" && $new_tag) {
178
-
$tag = $new_tag;
179
-
180
-
if ($new_what) {
181
-
@{$data{$what}->{label_list}} = @labels if ($data{$nametag}->{what});
182
-
@labels = ();
183
-
$label = "";
184
-
$new_what = 0;
185
-
186
-
$data{$what}->{type} = $type;
187
-
if (!defined($data{$what}->{file})) {
188
-
$data{$what}->{file} = $name;
189
-
$data{$what}->{filepath} = $file;
190
-
} else {
191
-
$data{$what}->{description} .= "\n\n" if (defined($data{$what}->{description}));
192
-
if ($name ne $data{$what}->{file}) {
193
-
$data{$what}->{file} .= " " . $name;
194
-
$data{$what}->{filepath} .= " " . $file;
195
-
}
196
-
}
197
-
print STDERR "\twhat: $what\n" if ($debug & $dbg_what_parsing);
198
-
$data{$what}->{line_no} = $ln;
199
-
} else {
200
-
$data{$what}->{line_no} = $ln if (!defined($data{$what}->{line_no}));
201
-
}
202
-
203
-
if (!$what) {
204
-
parse_error($file, $ln, "'What:' should come first:", $_);
205
-
next;
206
-
}
207
-
if ($new_tag eq "description") {
208
-
$sep =~ s,:, ,;
209
-
$content = ' ' x length($new_tag) . $sep . $content;
210
-
while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
211
-
if ($content =~ m/^(\s*)(\S.*)$/) {
212
-
# Preserve initial spaces for the first line
213
-
$space = $1;
214
-
$content = "$2\n";
215
-
$data{$what}->{$tag} .= $content;
216
-
} else {
217
-
undef($space);
218
-
}
219
-
220
-
} else {
221
-
$data{$what}->{$tag} = $content;
222
-
}
223
-
next;
224
-
}
225
-
}
226
-
227
-
# Store any contents before tags at the database
228
-
if (!$tag && $data{$nametag}->{what}) {
229
-
$data{$nametag}->{description} .= $_;
230
-
next;
231
-
}
232
-
233
-
if ($tag eq "description") {
234
-
my $content = $_;
235
-
while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
236
-
if (m/^\s*\n/) {
237
-
$data{$what}->{$tag} .= "\n";
238
-
next;
239
-
}
240
-
241
-
if (!defined($space)) {
242
-
# Preserve initial spaces for the first line
243
-
if ($content =~ m/^(\s*)(\S.*)$/) {
244
-
$space = $1;
245
-
$content = "$2\n";
246
-
}
247
-
} else {
248
-
$space = "" if (!($content =~ s/^($space)//));
249
-
}
250
-
$data{$what}->{$tag} .= $content;
251
-
252
-
next;
253
-
}
254
-
if (m/^\s*(.*)/) {
255
-
$data{$what}->{$tag} .= "\n$1";
256
-
$data{$what}->{$tag} =~ s/\n+$//;
257
-
next;
258
-
}
259
-
260
-
# Everything else is error
261
-
parse_error($file, $ln, "Unexpected content", $_);
262
-
}
263
-
$data{$nametag}->{description} =~ s/^\n+// if ($data{$nametag}->{description});
264
-
if ($what) {
265
-
parse_error($file, $ln, "What '$what' doesn't have a description", "") if (!$data{$what}->{description});
266
-
267
-
foreach my $w(split /\xac/,$what) {
268
-
$symbols{$w}->{xref} = $what;
269
-
};
270
-
}
271
-
close IN;
272
-
}
273
-
274
-
sub create_labels {
275
-
my %labels;
276
-
277
-
foreach my $what (keys %data) {
278
-
next if ($data{$what}->{file} eq "File");
279
-
280
-
foreach my $p (@{$data{$what}->{label_list}}) {
281
-
my ($content, $label) = @{$p};
282
-
$label = "abi_" . $label . " ";
283
-
$label =~ tr/A-Z/a-z/;
284
-
285
-
# Convert special chars to "_"
286
-
$label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g;
287
-
$label =~ s,_+,_,g;
288
-
$label =~ s,_$,,;
289
-
290
-
# Avoid duplicated labels
291
-
while (defined($labels{$label})) {
292
-
my @chars = ("A".."Z", "a".."z");
293
-
$label .= $chars[rand @chars];
294
-
}
295
-
$labels{$label} = 1;
296
-
297
-
$data{$what}->{label} = $label;
298
-
299
-
# only one label is enough
300
-
last;
301
-
}
302
-
}
303
-
}
304
-
305
-
#
306
-
# Outputs the book on ReST format
307
-
#
308
-
309
-
# \b doesn't work well with paths. So, we need to define something else:
310
-
# Boundaries are punct characters, spaces and end-of-line
311
-
my $start = qr {(^|\s|\() }x;
312
-
my $bondary = qr { ([,.:;\)\s]|\z) }x;
313
-
my $xref_match = qr { $start(\/(sys|config|proc|dev|kvd)\/[^,.:;\)\s]+)$bondary }x;
314
-
my $symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff]) }x;
315
-
316
-
sub output_rest {
317
-
create_labels();
318
-
319
-
my $part = "";
320
-
321
-
foreach my $what (sort {
322
-
($data{$a}->{type} eq "File") cmp ($data{$b}->{type} eq "File") ||
323
-
$a cmp $b
324
-
} keys %data) {
325
-
my $type = $data{$what}->{type};
326
-
327
-
my @file = split / /, $data{$what}->{file};
328
-
my @filepath = split / /, $data{$what}->{filepath};
329
-
330
-
if ($enable_lineno) {
331
-
printf ".. LINENO %s%s#%s\n\n",
332
-
$prefix, $file[0],
333
-
$data{$what}->{line_no};
334
-
}
335
-
336
-
my $w = $what;
337
-
338
-
if ($type ne "File") {
339
-
my $cur_part = $what;
340
-
if ($what =~ '/') {
341
-
if ($what =~ m#^(\/?(?:[\w\-]+\/?){1,2})#) {
342
-
$cur_part = "Symbols under $1";
343
-
$cur_part =~ s,/$,,;
344
-
}
345
-
}
346
-
347
-
if ($cur_part ne "" && $part ne $cur_part) {
348
-
$part = $cur_part;
349
-
my $bar = $part;
350
-
$bar =~ s/./-/g;
351
-
print "$part\n$bar\n\n";
352
-
}
353
-
354
-
printf ".. _%s:\n\n", $data{$what}->{label};
355
-
356
-
my @names = split /\xac/,$w;
357
-
my $len = 0;
358
-
359
-
foreach my $name (@names) {
360
-
$name =~ s/$symbols/\\$1/g;
361
-
$name = "**$name**";
362
-
$len = length($name) if (length($name) > $len);
363
-
}
364
-
365
-
print "+-" . "-" x $len . "-+\n";
366
-
foreach my $name (@names) {
367
-
printf "| %s", $name . " " x ($len - length($name)) . " |\n";
368
-
print "+-" . "-" x $len . "-+\n";
369
-
}
370
-
371
-
print "\n";
372
-
}
373
-
374
-
for (my $i = 0; $i < scalar(@filepath); $i++) {
375
-
my $path = $filepath[$i];
376
-
my $f = $file[$i];
377
-
378
-
$path =~ s,.*/(.*/.*),$1,;;
379
-
$path =~ s,[/\-],_,g;;
380
-
my $fileref = "abi_file_".$path;
381
-
382
-
if ($type eq "File") {
383
-
print ".. _$fileref:\n\n";
384
-
} else {
385
-
print "Defined on file :ref:`$f <$fileref>`\n\n";
386
-
}
387
-
}
388
-
389
-
if ($type eq "File") {
390
-
my $bar = $w;
391
-
$bar =~ s/./-/g;
392
-
print "$w\n$bar\n\n";
393
-
}
394
-
395
-
my $desc = "";
396
-
$desc = $data{$what}->{description} if (defined($data{$what}->{description}));
397
-
$desc =~ s/\s+$/\n/;
398
-
399
-
if (!($desc =~ /^\s*$/)) {
400
-
if ($description_is_rst) {
401
-
# Remove title markups from the description
402
-
# Having titles inside ABI files will only work if extra
403
-
# care would be taken in order to strictly follow the same
404
-
# level order for each markup.
405
-
$desc =~ s/\n[\-\*\=\^\~]+\n/\n\n/g;
406
-
407
-
# Enrich text by creating cross-references
408
-
409
-
my $new_desc = "";
410
-
my $init_indent = -1;
411
-
my $literal_indent = -1;
412
-
413
-
open(my $fh, "+<", \$desc);
414
-
while (my $d = <$fh>) {
415
-
my $indent = $d =~ m/^(\s+)/;
416
-
my $spaces = length($indent);
417
-
$init_indent = $indent if ($init_indent < 0);
418
-
if ($literal_indent >= 0) {
419
-
if ($spaces > $literal_indent) {
420
-
$new_desc .= $d;
421
-
next;
422
-
} else {
423
-
$literal_indent = -1;
424
-
}
425
-
} else {
426
-
if ($d =~ /()::$/ && !($d =~ /^\s*\.\./)) {
427
-
$literal_indent = $spaces;
428
-
}
429
-
}
430
-
431
-
$d =~ s,Documentation/(?!devicetree)(\S+)\.rst,:doc:`/$1`,g;
432
-
433
-
my @matches = $d =~ m,Documentation/ABI/([\w\/\-]+),g;
434
-
foreach my $f (@matches) {
435
-
my $xref = $f;
436
-
my $path = $f;
437
-
$path =~ s,.*/(.*/.*),$1,;;
438
-
$path =~ s,[/\-],_,g;;
439
-
$xref .= " <abi_file_" . $path . ">";
440
-
$d =~ s,\bDocumentation/ABI/$f\b,:ref:`$xref`,g;
441
-
}
442
-
443
-
# Seek for cross reference symbols like /sys/...
444
-
@matches = $d =~ m/$xref_match/g;
445
-
446
-
foreach my $s (@matches) {
447
-
next if (!($s =~ m,/,));
448
-
if (defined($data{$s}) && defined($data{$s}->{label})) {
449
-
my $xref = $s;
450
-
451
-
$xref =~ s/$symbols/\\$1/g;
452
-
$xref = ":ref:`$xref <" . $data{$s}->{label} . ">`";
453
-
454
-
$d =~ s,$start$s$bondary,$1$xref$2,g;
455
-
}
456
-
}
457
-
$new_desc .= $d;
458
-
}
459
-
close $fh;
460
-
461
-
462
-
print "$new_desc\n\n";
463
-
} else {
464
-
$desc =~ s/^\s+//;
465
-
466
-
# Remove title markups from the description, as they won't work
467
-
$desc =~ s/\n[\-\*\=\^\~]+\n/\n\n/g;
468
-
469
-
if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
470
-
# put everything inside a code block
471
-
$desc =~ s/\n/\n /g;
472
-
473
-
print "::\n\n";
474
-
print " $desc\n\n";
475
-
} else {
476
-
# Escape any special chars from description
477
-
$desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
478
-
print "$desc\n\n";
479
-
}
480
-
}
481
-
} else {
482
-
print "DESCRIPTION MISSING for $what\n\n" if (!$data{$what}->{is_file});
483
-
}
484
-
485
-
if ($data{$what}->{symbols}) {
486
-
printf "Has the following ABI:\n\n";
487
-
488
-
foreach my $content(@{$data{$what}->{symbols}}) {
489
-
my $label = $data{$symbols{$content}->{xref}}->{label};
490
-
491
-
# Escape special chars from content
492
-
$content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
493
-
494
-
print "- :ref:`$content <$label>`\n\n";
495
-
}
496
-
}
497
-
498
-
if (defined($data{$what}->{users})) {
499
-
my $users = $data{$what}->{users};
500
-
501
-
$users =~ s/\n/\n\t/g;
502
-
printf "Users:\n\t%s\n\n", $users if ($users ne "");
503
-
}
504
-
505
-
}
506
-
}
507
-
508
-
#
509
-
# Searches for ABI symbols
510
-
#
511
-
sub search_symbols {
512
-
foreach my $what (sort keys %data) {
513
-
next if (!($what =~ m/($arg)/));
514
-
515
-
my $type = $data{$what}->{type};
516
-
next if ($type eq "File");
517
-
518
-
my $file = $data{$what}->{filepath};
519
-
520
-
$what =~ s/\xac/, /g;
521
-
my $bar = $what;
522
-
$bar =~ s/./-/g;
523
-
524
-
print "\n$what\n$bar\n\n";
525
-
526
-
my $kernelversion = $data{$what}->{kernelversion} if (defined($data{$what}->{kernelversion}));
527
-
my $contact = $data{$what}->{contact} if (defined($data{$what}->{contact}));
528
-
my $users = $data{$what}->{users} if (defined($data{$what}->{users}));
529
-
my $date = $data{$what}->{date} if (defined($data{$what}->{date}));
530
-
my $desc = $data{$what}->{description} if (defined($data{$what}->{description}));
531
-
532
-
$kernelversion =~ s/^\s+// if ($kernelversion);
533
-
$contact =~ s/^\s+// if ($contact);
534
-
if ($users) {
535
-
$users =~ s/^\s+//;
536
-
$users =~ s/\n//g;
537
-
}
538
-
$date =~ s/^\s+// if ($date);
539
-
$desc =~ s/^\s+// if ($desc);
540
-
541
-
printf "Kernel version:\t\t%s\n", $kernelversion if ($kernelversion);
542
-
printf "Date:\t\t\t%s\n", $date if ($date);
543
-
printf "Contact:\t\t%s\n", $contact if ($contact);
544
-
printf "Users:\t\t\t%s\n", $users if ($users);
545
-
print "Defined on file(s):\t$file\n\n";
546
-
print "Description:\n\n$desc";
547
-
}
548
-
}
549
-
550
-
# Exclude /sys/kernel/debug and /sys/kernel/tracing from the search path
551
-
sub dont_parse_special_attributes {
552
-
if (($File::Find::dir =~ m,^/sys/kernel,)) {
553
-
return grep {!/(debug|tracing)/ } @_;
554
-
}
555
-
556
-
if (($File::Find::dir =~ m,^/sys/fs,)) {
557
-
return grep {!/(pstore|bpf|fuse)/ } @_;
558
-
}
559
-
560
-
return @_
561
-
}
562
-
563
-
my %leaf;
564
-
my %aliases;
565
-
my @files;
566
-
my %root;
567
-
568
-
sub graph_add_file {
569
-
my $file = shift;
570
-
my $type = shift;
571
-
572
-
my $dir = $file;
573
-
$dir =~ s,^(.*/).*,$1,;
574
-
$file =~ s,.*/,,;
575
-
576
-
my $name;
577
-
my $file_ref = \%root;
578
-
foreach my $edge(split "/", $dir) {
579
-
$name .= "$edge/";
580
-
if (!defined ${$file_ref}{$edge}) {
581
-
${$file_ref}{$edge} = { };
582
-
}
583
-
$file_ref = \%{$$file_ref{$edge}};
584
-
${$file_ref}{"__name"} = [ $name ];
585
-
}
586
-
$name .= "$file";
587
-
${$file_ref}{$file} = {
588
-
"__name" => [ $name ]
589
-
};
590
-
591
-
return \%{$$file_ref{$file}};
592
-
}
593
-
594
-
sub graph_add_link {
595
-
my $file = shift;
596
-
my $link = shift;
597
-
598
-
# Traverse graph to find the reference
599
-
my $file_ref = \%root;
600
-
foreach my $edge(split "/", $file) {
601
-
$file_ref = \%{$$file_ref{$edge}} || die "Missing node!";
602
-
}
603
-
604
-
# do a BFS
605
-
606
-
my @queue;
607
-
my %seen;
608
-
my $st;
609
-
610
-
push @queue, $file_ref;
611
-
$seen{$start}++;
612
-
613
-
while (@queue) {
614
-
my $v = shift @queue;
615
-
my @child = keys(%{$v});
616
-
617
-
foreach my $c(@child) {
618
-
next if $seen{$$v{$c}};
619
-
next if ($c eq "__name");
620
-
621
-
if (!defined($$v{$c}{"__name"})) {
622
-
printf STDERR "Error: Couldn't find a non-empty name on a children of $file/.*: ";
623
-
print STDERR Dumper(%{$v});
624
-
exit;
625
-
}
626
-
627
-
# Add new name
628
-
my $name = @{$$v{$c}{"__name"}}[0];
629
-
if ($name =~ s#^$file/#$link/#) {
630
-
push @{$$v{$c}{"__name"}}, $name;
631
-
}
632
-
# Add child to the queue and mark as seen
633
-
push @queue, $$v{$c};
634
-
$seen{$c}++;
635
-
}
636
-
}
637
-
}
638
-
639
-
my $escape_symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x29\x2b-\x2d\x3a-\x40\x7b-\xfe]) }x;
640
-
sub parse_existing_sysfs {
641
-
my $file = $File::Find::name;
642
-
643
-
my $mode = (lstat($file))[2];
644
-
my $abs_file = abs_path($file);
645
-
646
-
my @tmp;
647
-
push @tmp, $file;
648
-
push @tmp, $abs_file if ($abs_file ne $file);
649
-
650
-
foreach my $f(@tmp) {
651
-
# Ignore cgroup, as this is big and has zero docs under ABI
652
-
return if ($f =~ m#^/sys/fs/cgroup/#);
653
-
654
-
# Ignore firmware as it is documented elsewhere
655
-
# Either ACPI or under Documentation/devicetree/bindings/
656
-
return if ($f =~ m#^/sys/firmware/#);
657
-
658
-
# Ignore some sysfs nodes that aren't actually part of ABI
659
-
return if ($f =~ m#/sections|notes/#);
660
-
661
-
# Would need to check at
662
-
# Documentation/admin-guide/kernel-parameters.txt, but this
663
-
# is not easily parseable.
664
-
return if ($f =~ m#/parameters/#);
665
-
}
666
-
667
-
if (S_ISLNK($mode)) {
668
-
$aliases{$file} = $abs_file;
669
-
return;
670
-
}
671
-
672
-
return if (S_ISDIR($mode));
673
-
674
-
# Trivial: file is defined exactly the same way at ABI What:
675
-
return if (defined($data{$file}));
676
-
return if (defined($data{$abs_file}));
677
-
678
-
push @files, graph_add_file($abs_file, "file");
679
-
}
680
-
681
-
sub get_leave($)
682
-
{
683
-
my $what = shift;
684
-
my $leave;
685
-
686
-
my $l = $what;
687
-
my $stop = 1;
688
-
689
-
$leave = $l;
690
-
$leave =~ s,/$,,;
691
-
$leave =~ s,.*/,,;
692
-
$leave =~ s/[\(\)]//g;
693
-
694
-
# $leave is used to improve search performance at
695
-
# check_undefined_symbols, as the algorithm there can seek
696
-
# for a small number of "what". It also allows giving a
697
-
# hint about a leave with the same name somewhere else.
698
-
# However, there are a few occurences where the leave is
699
-
# either a wildcard or a number. Just group such cases
700
-
# altogether.
701
-
if ($leave =~ m/\.\*/ || $leave eq "" || $leave =~ /\\d/) {
702
-
$leave = "others";
703
-
}
704
-
705
-
return $leave;
706
-
}
707
-
708
-
my @not_found;
709
-
710
-
sub check_file($$)
711
-
{
712
-
my $file_ref = shift;
713
-
my $names_ref = shift;
714
-
my @names = @{$names_ref};
715
-
my $file = $names[0];
716
-
717
-
my $found_string;
718
-
719
-
my $leave = get_leave($file);
720
-
if (!defined($leaf{$leave})) {
721
-
$leave = "others";
722
-
}
723
-
my @expr = @{$leaf{$leave}->{expr}};
724
-
die ("\rmissing rules for $leave") if (!defined($leaf{$leave}));
725
-
726
-
my $path = $file;
727
-
$path =~ s,(.*/).*,$1,;
728
-
729
-
if ($search_string) {
730
-
return if (!($file =~ m#$search_string#));
731
-
$found_string = 1;
732
-
}
733
-
734
-
for (my $i = 0; $i < @names; $i++) {
735
-
if ($found_string && $hint) {
736
-
if (!$i) {
737
-
print STDERR "--> $names[$i]\n";
738
-
} else {
739
-
print STDERR " $names[$i]\n";
740
-
}
741
-
}
742
-
foreach my $re (@expr) {
743
-
print STDERR "$names[$i] =~ /^$re\$/\n" if ($debug && $dbg_undefined);
744
-
if ($names[$i] =~ $re) {
745
-
return;
746
-
}
747
-
}
748
-
}
749
-
750
-
if ($leave ne "others") {
751
-
my @expr = @{$leaf{"others"}->{expr}};
752
-
for (my $i = 0; $i < @names; $i++) {
753
-
foreach my $re (@expr) {
754
-
print STDERR "$names[$i] =~ /^$re\$/\n" if ($debug && $dbg_undefined);
755
-
if ($names[$i] =~ $re) {
756
-
return;
757
-
}
758
-
}
759
-
}
760
-
}
761
-
762
-
push @not_found, $file if (!$search_string || $found_string);
763
-
764
-
if ($hint && (!$search_string || $found_string)) {
765
-
my $what = $leaf{$leave}->{what};
766
-
$what =~ s/\xac/\n\t/g;
767
-
if ($leave ne "others") {
768
-
print STDERR "\r more likely regexes:\n\t$what\n";
769
-
} else {
770
-
print STDERR "\r tested regexes:\n\t$what\n";
771
-
}
772
-
}
773
-
}
774
-
775
-
sub check_undefined_symbols {
776
-
my $num_files = scalar @files;
777
-
my $next_i = 0;
778
-
my $start_time = times;
779
-
780
-
@files = sort @files;
781
-
782
-
my $last_time = $start_time;
783
-
784
-
# When either debug or hint is enabled, there's no sense showing
785
-
# progress, as the progress will be overriden.
786
-
if ($hint || ($debug && $dbg_undefined)) {
787
-
$next_i = $num_files;
788
-
}
789
-
790
-
my $is_console;
791
-
$is_console = 1 if (-t STDERR);
792
-
793
-
for (my $i = 0; $i < $num_files; $i++) {
794
-
my $file_ref = $files[$i];
795
-
my @names = @{$$file_ref{"__name"}};
796
-
797
-
check_file($file_ref, \@names);
798
-
799
-
my $cur_time = times;
800
-
801
-
if ($i == $next_i || $cur_time > $last_time + 1) {
802
-
my $percent = $i * 100 / $num_files;
803
-
804
-
my $tm = $cur_time - $start_time;
805
-
my $time = sprintf "%d:%02d", int($tm), 60 * ($tm - int($tm));
806
-
807
-
printf STDERR "\33[2K\r", if ($is_console);
808
-
printf STDERR "%s: processing sysfs files... %i%%: $names[0]", $time, $percent;
809
-
printf STDERR "\n", if (!$is_console);
810
-
STDERR->flush();
811
-
812
-
$next_i = int (($percent + 1) * $num_files / 100);
813
-
$last_time = $cur_time;
814
-
}
815
-
}
816
-
817
-
my $cur_time = times;
818
-
my $tm = $cur_time - $start_time;
819
-
my $time = sprintf "%d:%02d", int($tm), 60 * ($tm - int($tm));
820
-
821
-
printf STDERR "\33[2K\r", if ($is_console);
822
-
printf STDERR "%s: processing sysfs files... done\n", $time;
823
-
824
-
foreach my $file (@not_found) {
825
-
print "$file not found.\n";
826
-
}
827
-
}
828
-
829
-
sub undefined_symbols {
830
-
print STDERR "Reading $sysfs_prefix directory contents...";
831
-
find({
832
-
wanted =>\&parse_existing_sysfs,
833
-
preprocess =>\&dont_parse_special_attributes,
834
-
no_chdir => 1
835
-
}, $sysfs_prefix);
836
-
print STDERR "done.\n";
837
-
838
-
$leaf{"others"}->{what} = "";
839
-
840
-
print STDERR "Converting ABI What fields into regexes...";
841
-
foreach my $w (sort keys %data) {
842
-
foreach my $what (split /\xac/,$w) {
843
-
next if (!($what =~ m/^$sysfs_prefix/));
844
-
845
-
# Convert what into regular expressions
846
-
847
-
# Escape dot characters
848
-
$what =~ s/\./\xf6/g;
849
-
850
-
# Temporarily change [0-9]+ type of patterns
851
-
$what =~ s/\[0\-9\]\+/\xff/g;
852
-
853
-
# Temporarily change [\d+-\d+] type of patterns
854
-
$what =~ s/\[0\-\d+\]/\xff/g;
855
-
$what =~ s/\[(\d+)\]/\xf4$1\xf5/g;
856
-
857
-
# Temporarily change [0-9] type of patterns
858
-
$what =~ s/\[(\d)\-(\d)\]/\xf4$1-$2\xf5/g;
859
-
860
-
# Handle multiple option patterns
861
-
$what =~ s/[\{\<\[]([\w_]+)(?:[,|]+([\w_]+)){1,}[\}\>\]]/($1|$2)/g;
862
-
863
-
# Handle wildcards
864
-
$what =~ s,\*,.*,g;
865
-
$what =~ s,/\xf6..,/.*,g;
866
-
$what =~ s/\<[^\>]+\>/.*/g;
867
-
$what =~ s/\{[^\}]+\}/.*/g;
868
-
$what =~ s/\[[^\]]+\]/.*/g;
869
-
870
-
$what =~ s/[XYZ]/.*/g;
871
-
872
-
# Recover [0-9] type of patterns
873
-
$what =~ s/\xf4/[/g;
874
-
$what =~ s/\xf5/]/g;
875
-
876
-
# Remove duplicated spaces
877
-
$what =~ s/\s+/ /g;
878
-
879
-
# Special case: this ABI has a parenthesis on it
880
-
$what =~ s/sqrt\(x^2\+y^2\+z^2\)/sqrt\(x^2\+y^2\+z^2\)/;
881
-
882
-
# Special case: drop comparition as in:
883
-
# What: foo = <something>
884
-
# (this happens on a few IIO definitions)
885
-
$what =~ s,\s*\=.*$,,;
886
-
887
-
# Escape all other symbols
888
-
$what =~ s/$escape_symbols/\\$1/g;
889
-
$what =~ s/\\\\/\\/g;
890
-
$what =~ s/\\([\[\]\(\)\|])/$1/g;
891
-
$what =~ s/(\d+)\\(-\d+)/$1$2/g;
892
-
893
-
$what =~ s/\xff/\\d+/g;
894
-
895
-
# Special case: IIO ABI which a parenthesis.
896
-
$what =~ s/sqrt(.*)/sqrt\(.*\)/;
897
-
898
-
# Simplify regexes with multiple .*
899
-
$what =~ s#(?:\.\*){2,}##g;
900
-
# $what =~ s#\.\*/\.\*#.*#g;
901
-
902
-
# Recover dot characters
903
-
$what =~ s/\xf6/\./g;
904
-
905
-
my $leave = get_leave($what);
906
-
907
-
my $added = 0;
908
-
foreach my $l (split /\|/, $leave) {
909
-
if (defined($leaf{$l})) {
910
-
next if ($leaf{$l}->{what} =~ m/\b$what\b/);
911
-
$leaf{$l}->{what} .= "\xac" . $what;
912
-
$added = 1;
913
-
} else {
914
-
$leaf{$l}->{what} = $what;
915
-
$added = 1;
916
-
}
917
-
}
918
-
if ($search_string && $added) {
919
-
print STDERR "What: $what\n" if ($what =~ m#$search_string#);
920
-
}
921
-
922
-
}
923
-
}
924
-
# Compile regexes
925
-
foreach my $l (sort keys %leaf) {
926
-
my @expr;
927
-
foreach my $w(sort split /\xac/, $leaf{$l}->{what}) {
928
-
push @expr, qr /^$w$/;
929
-
}
930
-
$leaf{$l}->{expr} = \@expr;
931
-
}
932
-
933
-
# Take links into account
934
-
foreach my $link (sort keys %aliases) {
935
-
my $abs_file = $aliases{$link};
936
-
graph_add_link($abs_file, $link);
937
-
}
938
-
print STDERR "done.\n";
939
-
940
-
check_undefined_symbols;
941
-
}
942
-
943
-
# Ensure that the prefix will always end with a slash
944
-
# While this is not needed for find, it makes the patch nicer
945
-
# with --enable-lineno
946
-
$prefix =~ s,/?$,/,;
947
-
948
-
if ($cmd eq "undefined" || $cmd eq "search") {
949
-
$show_warnings = 0;
950
-
}
951
-
#
952
-
# Parses all ABI files located at $prefix dir
953
-
#
954
-
find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
955
-
956
-
print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug & $dbg_dump_abi_structs);
957
-
958
-
#
959
-
# Handles the command
960
-
#
961
-
if ($cmd eq "undefined") {
962
-
undefined_symbols;
963
-
} elsif ($cmd eq "search") {
964
-
search_symbols;
965
-
} else {
966
-
if ($cmd eq "rest") {
967
-
output_rest;
968
-
}
969
-
970
-
# Warn about duplicated ABI entries
971
-
foreach my $what(sort keys %symbols) {
972
-
my @files = @{$symbols{$what}->{file}};
973
-
974
-
next if (scalar(@files) == 1);
975
-
976
-
printf STDERR "Warning: $what is defined %d times: @files\n",
977
-
scalar(@files);
978
-
}
979
-
}
980
-
981
-
__END__
982
-
983
-
=head1 NAME
984
-
985
-
get_abi.pl - parse the Linux ABI files and produce a ReST book.
986
-
987
-
=head1 SYNOPSIS
988
-
989
-
B<get_abi.pl> [--debug <level>] [--enable-lineno] [--man] [--help]
990
-
[--(no-)rst-source] [--dir=<dir>] [--show-hints]
991
-
[--search-string <regex>]
992
-
<COMMAND> [<ARGUMENT>]
993
-
994
-
Where B<COMMAND> can be:
995
-
996
-
=over 8
997
-
998
-
B<search> I<SEARCH_REGEX> - search for I<SEARCH_REGEX> inside ABI
999
-
1000
-
B<rest> - output the ABI in ReST markup language
1001
-
1002
-
B<validate> - validate the ABI contents
1003
-
1004
-
B<undefined> - existing symbols at the system that aren't
1005
-
defined at Documentation/ABI
1006
-
1007
-
=back
1008
-
1009
-
=head1 OPTIONS
1010
-
1011
-
=over 8
1012
-
1013
-
=item B<--dir>
1014
-
1015
-
Changes the location of the ABI search. By default, it uses
1016
-
the Documentation/ABI directory.
1017
-
1018
-
=item B<--rst-source> and B<--no-rst-source>
1019
-
1020
-
The input file may be using ReST syntax or not. Those two options allow
1021
-
selecting between a rst-compliant source ABI (B<--rst-source>), or a
1022
-
plain text that may be violating ReST spec, so it requres some escaping
1023
-
logic (B<--no-rst-source>).
1024
-
1025
-
=item B<--enable-lineno>
1026
-
1027
-
Enable output of .. LINENO lines.
1028
-
1029
-
=item B<--debug> I<debug level>
1030
-
1031
-
Print debug information according with the level, which is given by the
1032
-
following bitmask:
1033
-
1034
-
- 1: Debug parsing What entries from ABI files;
1035
-
- 2: Shows what files are opened from ABI files;
1036
-
- 4: Dump the structs used to store the contents of the ABI files.
1037
-
1038
-
=item B<--show-hints>
1039
-
1040
-
Show hints about possible definitions for the missing ABI symbols.
1041
-
Used only when B<undefined>.
1042
-
1043
-
=item B<--search-string> I<regex string>
1044
-
1045
-
Show only occurences that match a search string.
1046
-
Used only when B<undefined>.
1047
-
1048
-
=item B<--help>
1049
-
1050
-
Prints a brief help message and exits.
1051
-
1052
-
=item B<--man>
1053
-
1054
-
Prints the manual page and exits.
1055
-
1056
-
=back
1057
-
1058
-
=head1 DESCRIPTION
1059
-
1060
-
Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI),
1061
-
allowing to search for ABI symbols or to produce a ReST book containing
1062
-
the Linux ABI documentation.
1063
-
1064
-
=head1 EXAMPLES
1065
-
1066
-
Search for all stable symbols with the word "usb":
1067
-
1068
-
=over 8
1069
-
1070
-
$ scripts/get_abi.pl search usb --dir Documentation/ABI/stable
1071
-
1072
-
=back
1073
-
1074
-
Search for all symbols that match the regex expression "usb.*cap":
1075
-
1076
-
=over 8
1077
-
1078
-
$ scripts/get_abi.pl search usb.*cap
1079
-
1080
-
=back
1081
-
1082
-
Output all obsoleted symbols in ReST format
1083
-
1084
-
=over 8
1085
-
1086
-
$ scripts/get_abi.pl rest --dir Documentation/ABI/obsolete
1087
-
1088
-
=back
1089
-
1090
-
=head1 BUGS
1091
-
1092
-
Report bugs to Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
1093
-
1094
-
=head1 COPYRIGHT
1095
-
1096
-
Copyright (c) 2016-2021 by Mauro Carvalho Chehab <mchehab+huawei@kernel.org>.
1097
-
1098
-
License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
1099
-
1100
-
This is free software: you are free to change and redistribute it.
1101
-
There is NO WARRANTY, to the extent permitted by law.
1102
-
1103
-
=cut
+214
scripts/get_abi.py
+214
scripts/get_abi.py
···
1
+
#!/usr/bin/env python3
2
+
# pylint: disable=R0903
3
+
# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
4
+
# SPDX-License-Identifier: GPL-2.0
5
+
6
+
"""
7
+
Parse ABI documentation and produce results from it.
8
+
"""
9
+
10
+
import argparse
11
+
import logging
12
+
import os
13
+
import sys
14
+
15
+
# Import Python modules
16
+
17
+
LIB_DIR = "lib/abi"
18
+
SRC_DIR = os.path.dirname(os.path.realpath(__file__))
19
+
20
+
sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
21
+
22
+
from abi_parser import AbiParser # pylint: disable=C0413
23
+
from abi_regex import AbiRegex # pylint: disable=C0413
24
+
from helpers import ABI_DIR, DEBUG_HELP # pylint: disable=C0413
25
+
from system_symbols import SystemSymbols # pylint: disable=C0413
26
+
27
+
# Command line classes
28
+
29
+
30
+
REST_DESC = """
31
+
Produce output in ReST format.
32
+
33
+
The output is done on two sections:
34
+
35
+
- Symbols: show all parsed symbols in alphabetic order;
36
+
- Files: cross reference the content of each file with the symbols on it.
37
+
"""
38
+
39
+
class AbiRest:
40
+
"""Initialize an argparse subparser for rest output"""
41
+
42
+
def __init__(self, subparsers):
43
+
"""Initialize argparse subparsers"""
44
+
45
+
parser = subparsers.add_parser("rest",
46
+
formatter_class=argparse.RawTextHelpFormatter,
47
+
description=REST_DESC)
48
+
49
+
parser.add_argument("--enable-lineno", action="store_true",
50
+
help="enable lineno")
51
+
parser.add_argument("--raw", action="store_true",
52
+
help="output text as contained in the ABI files. "
53
+
"It not used, output will contain dynamically"
54
+
" generated cross references when possible.")
55
+
parser.add_argument("--no-file", action="store_true",
56
+
help="Don't the files section")
57
+
parser.add_argument("--show-hints", help="Show-hints")
58
+
59
+
parser.set_defaults(func=self.run)
60
+
61
+
def run(self, args):
62
+
"""Run subparser"""
63
+
64
+
parser = AbiParser(args.dir, debug=args.debug)
65
+
parser.parse_abi()
66
+
parser.check_issues()
67
+
68
+
for t in parser.doc(args.raw, not args.no_file):
69
+
if args.enable_lineno:
70
+
print (f".. LINENO {t[1]}#{t[2]}\n\n")
71
+
72
+
print(t[0])
73
+
74
+
class AbiValidate:
75
+
"""Initialize an argparse subparser for ABI validation"""
76
+
77
+
def __init__(self, subparsers):
78
+
"""Initialize argparse subparsers"""
79
+
80
+
parser = subparsers.add_parser("validate",
81
+
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
82
+
description="list events")
83
+
84
+
parser.set_defaults(func=self.run)
85
+
86
+
def run(self, args):
87
+
"""Run subparser"""
88
+
89
+
parser = AbiParser(args.dir, debug=args.debug)
90
+
parser.parse_abi()
91
+
parser.check_issues()
92
+
93
+
94
+
class AbiSearch:
95
+
"""Initialize an argparse subparser for ABI search"""
96
+
97
+
def __init__(self, subparsers):
98
+
"""Initialize argparse subparsers"""
99
+
100
+
parser = subparsers.add_parser("search",
101
+
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
102
+
description="Search ABI using a regular expression")
103
+
104
+
parser.add_argument("expression",
105
+
help="Case-insensitive search pattern for the ABI symbol")
106
+
107
+
parser.set_defaults(func=self.run)
108
+
109
+
def run(self, args):
110
+
"""Run subparser"""
111
+
112
+
parser = AbiParser(args.dir, debug=args.debug)
113
+
parser.parse_abi()
114
+
parser.search_symbols(args.expression)
115
+
116
+
UNDEFINED_DESC="""
117
+
Check undefined ABIs on local machine.
118
+
119
+
Read sysfs devnodes and check if the devnodes there are defined inside
120
+
ABI documentation.
121
+
122
+
The search logic tries to minimize the number of regular expressions to
123
+
search per each symbol.
124
+
125
+
By default, it runs on a single CPU, as Python support for CPU threads
126
+
is still experimental, and multi-process runs on Python is very slow.
127
+
128
+
On experimental tests, if the number of ABI symbols to search per devnode
129
+
is contained on a limit of ~150 regular expressions, using a single CPU
130
+
is a lot faster than using multiple processes. However, if the number of
131
+
regular expressions to check is at the order of ~30000, using multiple
132
+
CPUs speeds up the check.
133
+
"""
134
+
135
+
class AbiUndefined:
136
+
"""
137
+
Initialize an argparse subparser for logic to check undefined ABI at
138
+
the current machine's sysfs
139
+
"""
140
+
141
+
def __init__(self, subparsers):
142
+
"""Initialize argparse subparsers"""
143
+
144
+
parser = subparsers.add_parser("undefined",
145
+
formatter_class=argparse.RawTextHelpFormatter,
146
+
description=UNDEFINED_DESC)
147
+
148
+
parser.add_argument("-S", "--sysfs-dir", default="/sys",
149
+
help="directory where sysfs is mounted")
150
+
parser.add_argument("-s", "--search-string",
151
+
help="search string regular expression to limit symbol search")
152
+
parser.add_argument("-H", "--show-hints", action="store_true",
153
+
help="Hints about definitions for missing ABI symbols.")
154
+
parser.add_argument("-j", "--jobs", "--max-workers", type=int, default=1,
155
+
help="If bigger than one, enables multiprocessing.")
156
+
parser.add_argument("-c", "--max-chunk-size", type=int, default=50,
157
+
help="Maximum number of chunk size")
158
+
parser.add_argument("-f", "--found", action="store_true",
159
+
help="Also show found items. "
160
+
"Helpful to debug the parser."),
161
+
parser.add_argument("-d", "--dry-run", action="store_true",
162
+
help="Don't actually search for undefined. "
163
+
"Helpful to debug the parser."),
164
+
165
+
parser.set_defaults(func=self.run)
166
+
167
+
def run(self, args):
168
+
"""Run subparser"""
169
+
170
+
abi = AbiRegex(args.dir, debug=args.debug,
171
+
search_string=args.search_string)
172
+
173
+
abi_symbols = SystemSymbols(abi=abi, hints=args.show_hints,
174
+
sysfs=args.sysfs_dir)
175
+
176
+
abi_symbols.check_undefined_symbols(dry_run=args.dry_run,
177
+
found=args.found,
178
+
max_workers=args.jobs,
179
+
chunk_size=args.max_chunk_size)
180
+
181
+
182
+
def main():
183
+
"""Main program"""
184
+
185
+
parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter)
186
+
187
+
parser.add_argument("-d", "--debug", type=int, default=0, help="debug level")
188
+
parser.add_argument("-D", "--dir", default=ABI_DIR, help=DEBUG_HELP)
189
+
190
+
subparsers = parser.add_subparsers()
191
+
192
+
AbiRest(subparsers)
193
+
AbiValidate(subparsers)
194
+
AbiSearch(subparsers)
195
+
AbiUndefined(subparsers)
196
+
197
+
args = parser.parse_args()
198
+
199
+
if args.debug:
200
+
level = logging.DEBUG
201
+
else:
202
+
level = logging.INFO
203
+
204
+
logging.basicConfig(level=level, format="[%(levelname)s] %(message)s")
205
+
206
+
if "func" in args:
207
+
args.func(args)
208
+
else:
209
+
sys.exit(f"Please specify a valid command for {sys.argv[0]}")
210
+
211
+
212
+
# Call main method
213
+
if __name__ == "__main__":
214
+
main()
+628
scripts/lib/abi/abi_parser.py
+628
scripts/lib/abi/abi_parser.py
···
1
+
#!/usr/bin/env python3
2
+
# pylint: disable=R0902,R0903,R0911,R0912,R0913,R0914,R0915,R0917,C0302
3
+
# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
4
+
# SPDX-License-Identifier: GPL-2.0
5
+
6
+
"""
7
+
Parse ABI documentation and produce results from it.
8
+
"""
9
+
10
+
from argparse import Namespace
11
+
import logging
12
+
import os
13
+
import re
14
+
15
+
from pprint import pformat
16
+
from random import randrange, seed
17
+
18
+
# Import Python modules
19
+
20
+
from helpers import AbiDebug, ABI_DIR
21
+
22
+
23
+
class AbiParser:
24
+
"""Main class to parse ABI files"""
25
+
26
+
TAGS = r"(what|where|date|kernelversion|contact|description|users)"
27
+
XREF = r"(?:^|\s|\()(\/(?:sys|config|proc|dev|kvd)\/[^,.:;\)\s]+)(?:[,.:;\)\s]|\Z)"
28
+
29
+
def __init__(self, directory, logger=None,
30
+
enable_lineno=False, show_warnings=True, debug=0):
31
+
"""Stores arguments for the class and initialize class vars"""
32
+
33
+
self.directory = directory
34
+
self.enable_lineno = enable_lineno
35
+
self.show_warnings = show_warnings
36
+
self.debug = debug
37
+
38
+
if not logger:
39
+
self.log = logging.getLogger("get_abi")
40
+
else:
41
+
self.log = logger
42
+
43
+
self.data = {}
44
+
self.what_symbols = {}
45
+
self.file_refs = {}
46
+
self.what_refs = {}
47
+
48
+
# Ignore files that contain such suffixes
49
+
self.ignore_suffixes = (".rej", ".org", ".orig", ".bak", "~")
50
+
51
+
# Regular expressions used on parser
52
+
self.re_abi_dir = re.compile(r"(.*)" + ABI_DIR)
53
+
self.re_tag = re.compile(r"(\S+)(:\s*)(.*)", re.I)
54
+
self.re_valid = re.compile(self.TAGS)
55
+
self.re_start_spc = re.compile(r"(\s*)(\S.*)")
56
+
self.re_whitespace = re.compile(r"^\s+")
57
+
58
+
# Regular used on print
59
+
self.re_what = re.compile(r"(\/?(?:[\w\-]+\/?){1,2})")
60
+
self.re_escape = re.compile(r"([\.\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])")
61
+
self.re_unprintable = re.compile(r"([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff]+)")
62
+
self.re_title_mark = re.compile(r"\n[\-\*\=\^\~]+\n")
63
+
self.re_doc = re.compile(r"Documentation/(?!devicetree)(\S+)\.rst")
64
+
self.re_abi = re.compile(r"(Documentation/ABI/)([\w\/\-]+)")
65
+
self.re_xref_node = re.compile(self.XREF)
66
+
67
+
def warn(self, fdata, msg, extra=None):
68
+
"""Displays a parse error if warning is enabled"""
69
+
70
+
if not self.show_warnings:
71
+
return
72
+
73
+
msg = f"{fdata.fname}:{fdata.ln}: {msg}"
74
+
if extra:
75
+
msg += "\n\t\t" + extra
76
+
77
+
self.log.warning(msg)
78
+
79
+
def add_symbol(self, what, fname, ln=None, xref=None):
80
+
"""Create a reference table describing where each 'what' is located"""
81
+
82
+
if what not in self.what_symbols:
83
+
self.what_symbols[what] = {"file": {}}
84
+
85
+
if fname not in self.what_symbols[what]["file"]:
86
+
self.what_symbols[what]["file"][fname] = []
87
+
88
+
if ln and ln not in self.what_symbols[what]["file"][fname]:
89
+
self.what_symbols[what]["file"][fname].append(ln)
90
+
91
+
if xref:
92
+
self.what_symbols[what]["xref"] = xref
93
+
94
+
def _parse_line(self, fdata, line):
95
+
"""Parse a single line of an ABI file"""
96
+
97
+
new_what = False
98
+
new_tag = False
99
+
content = None
100
+
101
+
match = self.re_tag.match(line)
102
+
if match:
103
+
new = match.group(1).lower()
104
+
sep = match.group(2)
105
+
content = match.group(3)
106
+
107
+
match = self.re_valid.search(new)
108
+
if match:
109
+
new_tag = match.group(1)
110
+
else:
111
+
if fdata.tag == "description":
112
+
# New "tag" is actually part of description.
113
+
# Don't consider it a tag
114
+
new_tag = False
115
+
elif fdata.tag != "":
116
+
self.warn(fdata, f"tag '{fdata.tag}' is invalid", line)
117
+
118
+
if new_tag:
119
+
# "where" is Invalid, but was a common mistake. Warn if found
120
+
if new_tag == "where":
121
+
self.warn(fdata, "tag 'Where' is invalid. Should be 'What:' instead")
122
+
new_tag = "what"
123
+
124
+
if new_tag == "what":
125
+
fdata.space = None
126
+
127
+
if content not in self.what_symbols:
128
+
self.add_symbol(what=content, fname=fdata.fname, ln=fdata.ln)
129
+
130
+
if fdata.tag == "what":
131
+
fdata.what.append(content.strip("\n"))
132
+
else:
133
+
if fdata.key:
134
+
if "description" not in self.data.get(fdata.key, {}):
135
+
self.warn(fdata, f"{fdata.key} doesn't have a description")
136
+
137
+
for w in fdata.what:
138
+
self.add_symbol(what=w, fname=fdata.fname,
139
+
ln=fdata.what_ln, xref=fdata.key)
140
+
141
+
fdata.label = content
142
+
new_what = True
143
+
144
+
key = "abi_" + content.lower()
145
+
fdata.key = self.re_unprintable.sub("_", key).strip("_")
146
+
147
+
# Avoid duplicated keys but using a defined seed, to make
148
+
# the namespace identical if there aren't changes at the
149
+
# ABI symbols
150
+
seed(42)
151
+
152
+
while fdata.key in self.data:
153
+
char = randrange(0, 51) + ord("A")
154
+
if char > ord("Z"):
155
+
char += ord("a") - ord("Z") - 1
156
+
157
+
fdata.key += chr(char)
158
+
159
+
if fdata.key and fdata.key not in self.data:
160
+
self.data[fdata.key] = {
161
+
"what": [content],
162
+
"file": [fdata.file_ref],
163
+
"path": fdata.ftype,
164
+
"line_no": fdata.ln,
165
+
}
166
+
167
+
fdata.what = self.data[fdata.key]["what"]
168
+
169
+
self.what_refs[content] = fdata.key
170
+
fdata.tag = new_tag
171
+
fdata.what_ln = fdata.ln
172
+
173
+
if fdata.nametag["what"]:
174
+
t = (content, fdata.key)
175
+
if t not in fdata.nametag["symbols"]:
176
+
fdata.nametag["symbols"].append(t)
177
+
178
+
return
179
+
180
+
if fdata.tag and new_tag:
181
+
fdata.tag = new_tag
182
+
183
+
if new_what:
184
+
fdata.label = ""
185
+
186
+
if "description" in self.data[fdata.key]:
187
+
self.data[fdata.key]["description"] += "\n\n"
188
+
189
+
if fdata.file_ref not in self.data[fdata.key]["file"]:
190
+
self.data[fdata.key]["file"].append(fdata.file_ref)
191
+
192
+
if self.debug == AbiDebug.WHAT_PARSING:
193
+
self.log.debug("what: %s", fdata.what)
194
+
195
+
if not fdata.what:
196
+
self.warn(fdata, "'What:' should come first:", line)
197
+
return
198
+
199
+
if new_tag == "description":
200
+
fdata.space = None
201
+
202
+
if content:
203
+
sep = sep.replace(":", " ")
204
+
205
+
c = " " * len(new_tag) + sep + content
206
+
c = c.expandtabs()
207
+
208
+
match = self.re_start_spc.match(c)
209
+
if match:
210
+
# Preserve initial spaces for the first line
211
+
fdata.space = match.group(1)
212
+
content = match.group(2) + "\n"
213
+
214
+
self.data[fdata.key][fdata.tag] = content
215
+
216
+
return
217
+
218
+
# Store any contents before tags at the database
219
+
if not fdata.tag and "what" in fdata.nametag:
220
+
fdata.nametag["description"] += line
221
+
return
222
+
223
+
if fdata.tag == "description":
224
+
content = line.expandtabs()
225
+
226
+
if self.re_whitespace.sub("", content) == "":
227
+
self.data[fdata.key][fdata.tag] += "\n"
228
+
return
229
+
230
+
if fdata.space is None:
231
+
match = self.re_start_spc.match(content)
232
+
if match:
233
+
# Preserve initial spaces for the first line
234
+
fdata.space = match.group(1)
235
+
236
+
content = match.group(2) + "\n"
237
+
else:
238
+
if content.startswith(fdata.space):
239
+
content = content[len(fdata.space):]
240
+
241
+
else:
242
+
fdata.space = ""
243
+
244
+
if fdata.tag == "what":
245
+
w = content.strip("\n")
246
+
if w:
247
+
self.data[fdata.key][fdata.tag].append(w)
248
+
else:
249
+
self.data[fdata.key][fdata.tag] += content
250
+
return
251
+
252
+
content = line.strip()
253
+
if fdata.tag:
254
+
if fdata.tag == "what":
255
+
w = content.strip("\n")
256
+
if w:
257
+
self.data[fdata.key][fdata.tag].append(w)
258
+
else:
259
+
self.data[fdata.key][fdata.tag] += "\n" + content.rstrip("\n")
260
+
return
261
+
262
+
# Everything else is error
263
+
if content:
264
+
self.warn(fdata, "Unexpected content", line)
265
+
266
+
def parse_readme(self, nametag, fname):
267
+
"""Parse ABI README file"""
268
+
269
+
nametag["what"] = ["ABI file contents"]
270
+
nametag["path"] = "README"
271
+
with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp:
272
+
for line in fp:
273
+
match = self.re_tag.match(line)
274
+
if match:
275
+
new = match.group(1).lower()
276
+
277
+
match = self.re_valid.search(new)
278
+
if match:
279
+
nametag["description"] += "\n:" + line
280
+
continue
281
+
282
+
nametag["description"] += line
283
+
284
+
def parse_file(self, fname, path, basename):
285
+
"""Parse a single file"""
286
+
287
+
ref = f"abi_file_{path}_{basename}"
288
+
ref = self.re_unprintable.sub("_", ref).strip("_")
289
+
290
+
# Store per-file state into a namespace variable. This will be used
291
+
# by the per-line parser state machine and by the warning function.
292
+
fdata = Namespace
293
+
294
+
fdata.fname = fname
295
+
fdata.name = basename
296
+
297
+
pos = fname.find(ABI_DIR)
298
+
if pos > 0:
299
+
f = fname[pos:]
300
+
else:
301
+
f = fname
302
+
303
+
fdata.file_ref = (f, ref)
304
+
self.file_refs[f] = ref
305
+
306
+
fdata.ln = 0
307
+
fdata.what_ln = 0
308
+
fdata.tag = ""
309
+
fdata.label = ""
310
+
fdata.what = []
311
+
fdata.key = None
312
+
fdata.xrefs = None
313
+
fdata.space = None
314
+
fdata.ftype = path.split("/")[0]
315
+
316
+
fdata.nametag = {}
317
+
fdata.nametag["what"] = [f"ABI file {path}/{basename}"]
318
+
fdata.nametag["type"] = "File"
319
+
fdata.nametag["path"] = fdata.ftype
320
+
fdata.nametag["file"] = [fdata.file_ref]
321
+
fdata.nametag["line_no"] = 1
322
+
fdata.nametag["description"] = ""
323
+
fdata.nametag["symbols"] = []
324
+
325
+
self.data[ref] = fdata.nametag
326
+
327
+
if self.debug & AbiDebug.WHAT_OPEN:
328
+
self.log.debug("Opening file %s", fname)
329
+
330
+
if basename == "README":
331
+
self.parse_readme(fdata.nametag, fname)
332
+
return
333
+
334
+
with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp:
335
+
for line in fp:
336
+
fdata.ln += 1
337
+
338
+
self._parse_line(fdata, line)
339
+
340
+
if "description" in fdata.nametag:
341
+
fdata.nametag["description"] = fdata.nametag["description"].lstrip("\n")
342
+
343
+
if fdata.key:
344
+
if "description" not in self.data.get(fdata.key, {}):
345
+
self.warn(fdata, f"{fdata.key} doesn't have a description")
346
+
347
+
for w in fdata.what:
348
+
self.add_symbol(what=w, fname=fname, xref=fdata.key)
349
+
350
+
def _parse_abi(self, root=None):
351
+
"""Internal function to parse documentation ABI recursively"""
352
+
353
+
if not root:
354
+
root = self.directory
355
+
356
+
with os.scandir(root) as obj:
357
+
for entry in obj:
358
+
name = os.path.join(root, entry.name)
359
+
360
+
if entry.is_dir():
361
+
self._parse_abi(name)
362
+
continue
363
+
364
+
if not entry.is_file():
365
+
continue
366
+
367
+
basename = os.path.basename(name)
368
+
369
+
if basename.startswith("."):
370
+
continue
371
+
372
+
if basename.endswith(self.ignore_suffixes):
373
+
continue
374
+
375
+
path = self.re_abi_dir.sub("", os.path.dirname(name))
376
+
377
+
self.parse_file(name, path, basename)
378
+
379
+
def parse_abi(self, root=None):
380
+
"""Parse documentation ABI"""
381
+
382
+
self._parse_abi(root)
383
+
384
+
if self.debug & AbiDebug.DUMP_ABI_STRUCTS:
385
+
self.log.debug(pformat(self.data))
386
+
387
+
def desc_txt(self, desc):
388
+
"""Print description as found inside ABI files"""
389
+
390
+
desc = desc.strip(" \t\n")
391
+
392
+
return desc + "\n\n"
393
+
394
+
def xref(self, fname):
395
+
"""
396
+
Converts a Documentation/ABI + basename into a ReST cross-reference
397
+
"""
398
+
399
+
xref = self.file_refs.get(fname)
400
+
if not xref:
401
+
return None
402
+
else:
403
+
return xref
404
+
405
+
def desc_rst(self, desc):
406
+
"""Enrich ReST output by creating cross-references"""
407
+
408
+
# Remove title markups from the description
409
+
# Having titles inside ABI files will only work if extra
410
+
# care would be taken in order to strictly follow the same
411
+
# level order for each markup.
412
+
desc = self.re_title_mark.sub("\n\n", "\n" + desc)
413
+
desc = desc.rstrip(" \t\n").lstrip("\n")
414
+
415
+
# Python's regex performance for non-compiled expressions is a lot
416
+
# than Perl, as Perl automatically caches them at their
417
+
# first usage. Here, we'll need to do the same, as otherwise the
418
+
# performance penalty is be high
419
+
420
+
new_desc = ""
421
+
for d in desc.split("\n"):
422
+
if d == "":
423
+
new_desc += "\n"
424
+
continue
425
+
426
+
# Use cross-references for doc files where needed
427
+
d = self.re_doc.sub(r":doc:`/\1`", d)
428
+
429
+
# Use cross-references for ABI generated docs where needed
430
+
matches = self.re_abi.findall(d)
431
+
for m in matches:
432
+
abi = m[0] + m[1]
433
+
434
+
xref = self.file_refs.get(abi)
435
+
if not xref:
436
+
# This may happen if ABI is on a separate directory,
437
+
# like parsing ABI testing and symbol is at stable.
438
+
# The proper solution is to move this part of the code
439
+
# for it to be inside sphinx/kernel_abi.py
440
+
self.log.info("Didn't find ABI reference for '%s'", abi)
441
+
else:
442
+
new = self.re_escape.sub(r"\\\1", m[1])
443
+
d = re.sub(fr"\b{abi}\b", f":ref:`{new} <{xref}>`", d)
444
+
445
+
# Seek for cross reference symbols like /sys/...
446
+
# Need to be careful to avoid doing it on a code block
447
+
if d[0] not in [" ", "\t"]:
448
+
matches = self.re_xref_node.findall(d)
449
+
for m in matches:
450
+
# Finding ABI here is more complex due to wildcards
451
+
xref = self.what_refs.get(m)
452
+
if xref:
453
+
new = self.re_escape.sub(r"\\\1", m)
454
+
d = re.sub(fr"\b{m}\b", f":ref:`{new} <{xref}>`", d)
455
+
456
+
new_desc += d + "\n"
457
+
458
+
return new_desc + "\n\n"
459
+
460
+
def doc(self, output_in_txt=False, show_symbols=True, show_file=True,
461
+
filter_path=None):
462
+
"""Print ABI at stdout"""
463
+
464
+
part = None
465
+
for key, v in sorted(self.data.items(),
466
+
key=lambda x: (x[1].get("type", ""),
467
+
x[1].get("what"))):
468
+
469
+
wtype = v.get("type", "Symbol")
470
+
file_ref = v.get("file")
471
+
names = v.get("what", [""])
472
+
473
+
if wtype == "File":
474
+
if not show_file:
475
+
continue
476
+
else:
477
+
if not show_symbols:
478
+
continue
479
+
480
+
if filter_path:
481
+
if v.get("path") != filter_path:
482
+
continue
483
+
484
+
msg = ""
485
+
486
+
if wtype != "File":
487
+
cur_part = names[0]
488
+
if cur_part.find("/") >= 0:
489
+
match = self.re_what.match(cur_part)
490
+
if match:
491
+
symbol = match.group(1).rstrip("/")
492
+
cur_part = "Symbols under " + symbol
493
+
494
+
if cur_part and cur_part != part:
495
+
part = cur_part
496
+
msg += part + "\n"+ "-" * len(part) +"\n\n"
497
+
498
+
msg += f".. _{key}:\n\n"
499
+
500
+
max_len = 0
501
+
for i in range(0, len(names)): # pylint: disable=C0200
502
+
names[i] = "**" + self.re_escape.sub(r"\\\1", names[i]) + "**"
503
+
504
+
max_len = max(max_len, len(names[i]))
505
+
506
+
msg += "+-" + "-" * max_len + "-+\n"
507
+
for name in names:
508
+
msg += f"| {name}" + " " * (max_len - len(name)) + " |\n"
509
+
msg += "+-" + "-" * max_len + "-+\n"
510
+
msg += "\n"
511
+
512
+
for ref in file_ref:
513
+
if wtype == "File":
514
+
msg += f".. _{ref[1]}:\n\n"
515
+
else:
516
+
base = os.path.basename(ref[0])
517
+
msg += f"Defined on file :ref:`{base} <{ref[1]}>`\n\n"
518
+
519
+
if wtype == "File":
520
+
msg += names[0] +"\n" + "-" * len(names[0]) +"\n\n"
521
+
522
+
desc = v.get("description")
523
+
if not desc and wtype != "File":
524
+
msg += f"DESCRIPTION MISSING for {names[0]}\n\n"
525
+
526
+
if desc:
527
+
if output_in_txt:
528
+
msg += self.desc_txt(desc)
529
+
else:
530
+
msg += self.desc_rst(desc)
531
+
532
+
symbols = v.get("symbols")
533
+
if symbols:
534
+
msg += "Has the following ABI:\n\n"
535
+
536
+
for w, label in symbols:
537
+
# Escape special chars from content
538
+
content = self.re_escape.sub(r"\\\1", w)
539
+
540
+
msg += f"- :ref:`{content} <{label}>`\n\n"
541
+
542
+
users = v.get("users")
543
+
if users and users.strip(" \t\n"):
544
+
users = users.strip("\n").replace('\n', '\n\t')
545
+
msg += f"Users:\n\t{users}\n\n"
546
+
547
+
ln = v.get("line_no", 1)
548
+
549
+
yield (msg, file_ref[0][0], ln)
550
+
551
+
def check_issues(self):
552
+
"""Warn about duplicated ABI entries"""
553
+
554
+
for what, v in self.what_symbols.items():
555
+
files = v.get("file")
556
+
if not files:
557
+
# Should never happen if the parser works properly
558
+
self.log.warning("%s doesn't have a file associated", what)
559
+
continue
560
+
561
+
if len(files) == 1:
562
+
continue
563
+
564
+
f = []
565
+
for fname, lines in sorted(files.items()):
566
+
if not lines:
567
+
f.append(f"{fname}")
568
+
elif len(lines) == 1:
569
+
f.append(f"{fname}:{lines[0]}")
570
+
else:
571
+
m = fname + "lines "
572
+
m += ", ".join(str(x) for x in lines)
573
+
f.append(m)
574
+
575
+
self.log.warning("%s is defined %d times: %s", what, len(f), "; ".join(f))
576
+
577
+
def search_symbols(self, expr):
578
+
""" Searches for ABI symbols """
579
+
580
+
regex = re.compile(expr, re.I)
581
+
582
+
found_keys = 0
583
+
for t in sorted(self.data.items(), key=lambda x: [0]):
584
+
v = t[1]
585
+
586
+
wtype = v.get("type", "")
587
+
if wtype == "File":
588
+
continue
589
+
590
+
for what in v.get("what", [""]):
591
+
if regex.search(what):
592
+
found_keys += 1
593
+
594
+
kernelversion = v.get("kernelversion", "").strip(" \t\n")
595
+
date = v.get("date", "").strip(" \t\n")
596
+
contact = v.get("contact", "").strip(" \t\n")
597
+
users = v.get("users", "").strip(" \t\n")
598
+
desc = v.get("description", "").strip(" \t\n")
599
+
600
+
files = []
601
+
for f in v.get("file", ()):
602
+
files.append(f[0])
603
+
604
+
what = str(found_keys) + ". " + what
605
+
title_tag = "-" * len(what)
606
+
607
+
print(f"\n{what}\n{title_tag}\n")
608
+
609
+
if kernelversion:
610
+
print(f"Kernel version:\t\t{kernelversion}")
611
+
612
+
if date:
613
+
print(f"Date:\t\t\t{date}")
614
+
615
+
if contact:
616
+
print(f"Contact:\t\t{contact}")
617
+
618
+
if users:
619
+
print(f"Users:\t\t\t{users}")
620
+
621
+
print("Defined on file(s):\t" + ", ".join(files))
622
+
623
+
if desc:
624
+
desc = desc.strip("\n")
625
+
print(f"\n{desc}\n")
626
+
627
+
if not found_keys:
628
+
print(f"Regular expression /{expr}/ not found.")
+234
scripts/lib/abi/abi_regex.py
+234
scripts/lib/abi/abi_regex.py
···
1
+
#!/usr/bin/env python3
2
+
# xxpylint: disable=R0903
3
+
# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
4
+
# SPDX-License-Identifier: GPL-2.0
5
+
6
+
"""
7
+
Convert ABI what into regular expressions
8
+
"""
9
+
10
+
import re
11
+
import sys
12
+
13
+
from pprint import pformat
14
+
15
+
from abi_parser import AbiParser
16
+
from helpers import AbiDebug
17
+
18
+
class AbiRegex(AbiParser):
19
+
"""Extends AbiParser to search ABI nodes with regular expressions"""
20
+
21
+
# Escape only ASCII visible characters
22
+
escape_symbols = r"([\x21-\x29\x2b-\x2d\x3a-\x40\x5c\x60\x7b-\x7e])"
23
+
leave_others = "others"
24
+
25
+
# Tuples with regular expressions to be compiled and replacement data
26
+
re_whats = [
27
+
# Drop escape characters that might exist
28
+
(re.compile("\\\\"), ""),
29
+
30
+
# Temporarily escape dot characters
31
+
(re.compile(r"\."), "\xf6"),
32
+
33
+
# Temporarily change [0-9]+ type of patterns
34
+
(re.compile(r"\[0\-9\]\+"), "\xff"),
35
+
36
+
# Temporarily change [\d+-\d+] type of patterns
37
+
(re.compile(r"\[0\-\d+\]"), "\xff"),
38
+
(re.compile(r"\[0:\d+\]"), "\xff"),
39
+
(re.compile(r"\[(\d+)\]"), "\xf4\\\\d+\xf5"),
40
+
41
+
# Temporarily change [0-9] type of patterns
42
+
(re.compile(r"\[(\d)\-(\d)\]"), "\xf4\1-\2\xf5"),
43
+
44
+
# Handle multiple option patterns
45
+
(re.compile(r"[\{\<\[]([\w_]+)(?:[,|]+([\w_]+)){1,}[\}\>\]]"), r"(\1|\2)"),
46
+
47
+
# Handle wildcards
48
+
(re.compile(r"([^\/])\*"), "\\1\\\\w\xf7"),
49
+
(re.compile(r"/\*/"), "/.*/"),
50
+
(re.compile(r"/\xf6\xf6\xf6"), "/.*"),
51
+
(re.compile(r"\<[^\>]+\>"), "\\\\w\xf7"),
52
+
(re.compile(r"\{[^\}]+\}"), "\\\\w\xf7"),
53
+
(re.compile(r"\[[^\]]+\]"), "\\\\w\xf7"),
54
+
55
+
(re.compile(r"XX+"), "\\\\w\xf7"),
56
+
(re.compile(r"([^A-Z])[XYZ]([^A-Z])"), "\\1\\\\w\xf7\\2"),
57
+
(re.compile(r"([^A-Z])[XYZ]$"), "\\1\\\\w\xf7"),
58
+
(re.compile(r"_[AB]_"), "_\\\\w\xf7_"),
59
+
60
+
# Recover [0-9] type of patterns
61
+
(re.compile(r"\xf4"), "["),
62
+
(re.compile(r"\xf5"), "]"),
63
+
64
+
# Remove duplicated spaces
65
+
(re.compile(r"\s+"), r" "),
66
+
67
+
# Special case: drop comparison as in:
68
+
# What: foo = <something>
69
+
# (this happens on a few IIO definitions)
70
+
(re.compile(r"\s*\=.*$"), ""),
71
+
72
+
# Escape all other symbols
73
+
(re.compile(escape_symbols), r"\\\1"),
74
+
(re.compile(r"\\\\"), r"\\"),
75
+
(re.compile(r"\\([\[\]\(\)\|])"), r"\1"),
76
+
(re.compile(r"(\d+)\\(-\d+)"), r"\1\2"),
77
+
78
+
(re.compile(r"\xff"), r"\\d+"),
79
+
80
+
# Special case: IIO ABI which a parenthesis.
81
+
(re.compile(r"sqrt(.*)"), r"sqrt(.*)"),
82
+
83
+
# Simplify regexes with multiple .*
84
+
(re.compile(r"(?:\.\*){2,}"), ""),
85
+
86
+
# Recover dot characters
87
+
(re.compile(r"\xf6"), "\\."),
88
+
# Recover plus characters
89
+
(re.compile(r"\xf7"), "+"),
90
+
]
91
+
re_has_num = re.compile(r"\\d")
92
+
93
+
# Symbol name after escape_chars that are considered a devnode basename
94
+
re_symbol_name = re.compile(r"(\w|\\[\.\-\:])+$")
95
+
96
+
# List of popular group names to be skipped to minimize regex group size
97
+
# Use AbiDebug.SUBGROUP_SIZE to detect those
98
+
skip_names = set(["devices", "hwmon"])
99
+
100
+
def regex_append(self, what, new):
101
+
"""
102
+
Get a search group for a subset of regular expressions.
103
+
104
+
As ABI may have thousands of symbols, using a for to search all
105
+
regular expressions is at least O(n^2). When there are wildcards,
106
+
the complexity increases substantially, eventually becoming exponential.
107
+
108
+
To avoid spending too much time on them, use a logic to split
109
+
them into groups. The smaller the group, the better, as it would
110
+
mean that searches will be confined to a small number of regular
111
+
expressions.
112
+
113
+
The conversion to a regex subset is tricky, as we need something
114
+
that can be easily obtained from the sysfs symbol and from the
115
+
regular expression. So, we need to discard nodes that have
116
+
wildcards.
117
+
118
+
If it can't obtain a subgroup, place the regular expression inside
119
+
a special group (self.leave_others).
120
+
"""
121
+
122
+
search_group = None
123
+
124
+
for search_group in reversed(new.split("/")):
125
+
if not search_group or search_group in self.skip_names:
126
+
continue
127
+
if self.re_symbol_name.match(search_group):
128
+
break
129
+
130
+
if not search_group:
131
+
search_group = self.leave_others
132
+
133
+
if self.debug & AbiDebug.SUBGROUP_MAP:
134
+
self.log.debug("%s: mapped as %s", what, search_group)
135
+
136
+
try:
137
+
if search_group not in self.regex_group:
138
+
self.regex_group[search_group] = []
139
+
140
+
self.regex_group[search_group].append(re.compile(new))
141
+
if self.search_string:
142
+
if what.find(self.search_string) >= 0:
143
+
print(f"What: {what}")
144
+
except re.PatternError:
145
+
self.log.warning("Ignoring '%s' as it produced an invalid regex:\n"
146
+
" '%s'", what, new)
147
+
148
+
def get_regexes(self, what):
149
+
"""
150
+
Given an ABI devnode, return a list of all regular expressions that
151
+
may match it, based on the sub-groups created by regex_append()
152
+
"""
153
+
154
+
re_list = []
155
+
156
+
patches = what.split("/")
157
+
patches.reverse()
158
+
patches.append(self.leave_others)
159
+
160
+
for search_group in patches:
161
+
if search_group in self.regex_group:
162
+
re_list += self.regex_group[search_group]
163
+
164
+
return re_list
165
+
166
+
def __init__(self, *args, **kwargs):
167
+
"""
168
+
Override init method to get verbose argument
169
+
"""
170
+
171
+
self.regex_group = None
172
+
self.search_string = None
173
+
self.re_string = None
174
+
175
+
if "search_string" in kwargs:
176
+
self.search_string = kwargs.get("search_string")
177
+
del kwargs["search_string"]
178
+
179
+
if self.search_string:
180
+
181
+
try:
182
+
self.re_string = re.compile(self.search_string)
183
+
except re.PatternError as e:
184
+
msg = f"{self.search_string} is not a valid regular expression"
185
+
raise ValueError(msg) from e
186
+
187
+
super().__init__(*args, **kwargs)
188
+
189
+
def parse_abi(self, *args, **kwargs):
190
+
191
+
super().parse_abi(*args, **kwargs)
192
+
193
+
self.regex_group = {}
194
+
195
+
print("Converting ABI What fields into regexes...", file=sys.stderr)
196
+
197
+
for t in sorted(self.data.items(), key=lambda x: x[0]):
198
+
v = t[1]
199
+
if v.get("type") == "File":
200
+
continue
201
+
202
+
v["regex"] = []
203
+
204
+
for what in v.get("what", []):
205
+
if not what.startswith("/sys"):
206
+
continue
207
+
208
+
new = what
209
+
for r, s in self.re_whats:
210
+
try:
211
+
new = r.sub(s, new)
212
+
except re.PatternError as e:
213
+
# Help debugging troubles with new regexes
214
+
raise re.PatternError(f"{e}\nwhile re.sub('{r.pattern}', {s}, str)") from e
215
+
216
+
v["regex"].append(new)
217
+
218
+
if self.debug & AbiDebug.REGEX:
219
+
self.log.debug("%-90s <== %s", new, what)
220
+
221
+
# Store regex into a subgroup to speedup searches
222
+
self.regex_append(what, new)
223
+
224
+
if self.debug & AbiDebug.SUBGROUP_DICT:
225
+
self.log.debug("%s", pformat(self.regex_group))
226
+
227
+
if self.debug & AbiDebug.SUBGROUP_SIZE:
228
+
biggestd_keys = sorted(self.regex_group.keys(),
229
+
key= lambda k: len(self.regex_group[k]),
230
+
reverse=True)
231
+
232
+
print("Top regex subgroups:", file=sys.stderr)
233
+
for k in biggestd_keys[:10]:
234
+
print(f"{k} has {len(self.regex_group[k])} elements", file=sys.stderr)
+38
scripts/lib/abi/helpers.py
+38
scripts/lib/abi/helpers.py
···
1
+
#!/usr/bin/env python3
2
+
# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
3
+
# pylint: disable=R0903
4
+
# SPDX-License-Identifier: GPL-2.0
5
+
6
+
"""
7
+
Helper classes for ABI parser
8
+
"""
9
+
10
+
ABI_DIR = "Documentation/ABI/"
11
+
12
+
13
+
class AbiDebug:
14
+
"""Debug levels"""
15
+
16
+
WHAT_PARSING = 1
17
+
WHAT_OPEN = 2
18
+
DUMP_ABI_STRUCTS = 4
19
+
UNDEFINED = 8
20
+
REGEX = 16
21
+
SUBGROUP_MAP = 32
22
+
SUBGROUP_DICT = 64
23
+
SUBGROUP_SIZE = 128
24
+
GRAPH = 256
25
+
26
+
27
+
DEBUG_HELP = """
28
+
1 - enable debug parsing logic
29
+
2 - enable debug messages on file open
30
+
4 - enable debug for ABI parse data
31
+
8 - enable extra debug information to identify troubles
32
+
with ABI symbols found at the local machine that
33
+
weren't found on ABI documentation (used only for
34
+
undefined subcommand)
35
+
16 - enable debug for what to regex conversion
36
+
32 - enable debug for symbol regex subgroups
37
+
64 - enable debug for sysfs graph tree variable
38
+
"""
+378
scripts/lib/abi/system_symbols.py
+378
scripts/lib/abi/system_symbols.py
···
1
+
#!/usr/bin/env python3
2
+
# pylint: disable=R0902,R0912,R0914,R0915,R1702
3
+
# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
4
+
# SPDX-License-Identifier: GPL-2.0
5
+
6
+
"""
7
+
Parse ABI documentation and produce results from it.
8
+
"""
9
+
10
+
import os
11
+
import re
12
+
import sys
13
+
14
+
from concurrent import futures
15
+
from datetime import datetime
16
+
from random import shuffle
17
+
18
+
from helpers import AbiDebug
19
+
20
+
class SystemSymbols:
21
+
"""Stores arguments for the class and initialize class vars"""
22
+
23
+
def graph_add_file(self, path, link=None):
24
+
"""
25
+
add a file path to the sysfs graph stored at self.root
26
+
"""
27
+
28
+
if path in self.files:
29
+
return
30
+
31
+
name = ""
32
+
ref = self.root
33
+
for edge in path.split("/"):
34
+
name += edge + "/"
35
+
if edge not in ref:
36
+
ref[edge] = {"__name": [name.rstrip("/")]}
37
+
38
+
ref = ref[edge]
39
+
40
+
if link and link not in ref["__name"]:
41
+
ref["__name"].append(link.rstrip("/"))
42
+
43
+
self.files.add(path)
44
+
45
+
def print_graph(self, root_prefix="", root=None, level=0):
46
+
"""Prints a reference tree graph using UTF-8 characters"""
47
+
48
+
if not root:
49
+
root = self.root
50
+
level = 0
51
+
52
+
# Prevent endless traverse
53
+
if level > 5:
54
+
return
55
+
56
+
if level > 0:
57
+
prefix = "├──"
58
+
last_prefix = "└──"
59
+
else:
60
+
prefix = ""
61
+
last_prefix = ""
62
+
63
+
items = list(root.items())
64
+
65
+
names = root.get("__name", [])
66
+
for k, edge in items:
67
+
if k == "__name":
68
+
continue
69
+
70
+
if not k:
71
+
k = "/"
72
+
73
+
if len(names) > 1:
74
+
k += " links: " + ",".join(names[1:])
75
+
76
+
if edge == items[-1][1]:
77
+
print(root_prefix + last_prefix + k)
78
+
p = root_prefix
79
+
if level > 0:
80
+
p += " "
81
+
self.print_graph(p, edge, level + 1)
82
+
else:
83
+
print(root_prefix + prefix + k)
84
+
p = root_prefix + "│ "
85
+
self.print_graph(p, edge, level + 1)
86
+
87
+
def _walk(self, root):
88
+
"""
89
+
Walk through sysfs to get all devnodes that aren't ignored.
90
+
91
+
By default, uses /sys as sysfs mounting point. If another
92
+
directory is used, it replaces them to /sys at the patches.
93
+
"""
94
+
95
+
with os.scandir(root) as obj:
96
+
for entry in obj:
97
+
path = os.path.join(root, entry.name)
98
+
if self.sysfs:
99
+
p = path.replace(self.sysfs, "/sys", count=1)
100
+
else:
101
+
p = path
102
+
103
+
if self.re_ignore.search(p):
104
+
return
105
+
106
+
# Handle link first to avoid directory recursion
107
+
if entry.is_symlink():
108
+
real = os.path.realpath(path)
109
+
if not self.sysfs:
110
+
self.aliases[path] = real
111
+
else:
112
+
real = real.replace(self.sysfs, "/sys", count=1)
113
+
114
+
# Add absfile location to graph if it doesn't exist
115
+
if not self.re_ignore.search(real):
116
+
# Add link to the graph
117
+
self.graph_add_file(real, p)
118
+
119
+
elif entry.is_file():
120
+
self.graph_add_file(p)
121
+
122
+
elif entry.is_dir():
123
+
self._walk(path)
124
+
125
+
def __init__(self, abi, sysfs="/sys", hints=False):
126
+
"""
127
+
Initialize internal variables and get a list of all files inside
128
+
sysfs that can currently be parsed.
129
+
130
+
Please notice that there are several entries on sysfs that aren't
131
+
documented as ABI. Ignore those.
132
+
133
+
The real paths will be stored under self.files. Aliases will be
134
+
stored in separate, as self.aliases.
135
+
"""
136
+
137
+
self.abi = abi
138
+
self.log = abi.log
139
+
140
+
if sysfs != "/sys":
141
+
self.sysfs = sysfs.rstrip("/")
142
+
else:
143
+
self.sysfs = None
144
+
145
+
self.hints = hints
146
+
147
+
self.root = {}
148
+
self.aliases = {}
149
+
self.files = set()
150
+
151
+
dont_walk = [
152
+
# Those require root access and aren't documented at ABI
153
+
f"^{sysfs}/kernel/debug",
154
+
f"^{sysfs}/kernel/tracing",
155
+
f"^{sysfs}/fs/pstore",
156
+
f"^{sysfs}/fs/bpf",
157
+
f"^{sysfs}/fs/fuse",
158
+
159
+
# This is not documented at ABI
160
+
f"^{sysfs}/module",
161
+
162
+
f"^{sysfs}/fs/cgroup", # this is big and has zero docs under ABI
163
+
f"^{sysfs}/firmware", # documented elsewhere: ACPI, DT bindings
164
+
"sections|notes", # aren't actually part of ABI
165
+
166
+
# kernel-parameters.txt - not easy to parse
167
+
"parameters",
168
+
]
169
+
170
+
self.re_ignore = re.compile("|".join(dont_walk))
171
+
172
+
print(f"Reading {sysfs} directory contents...", file=sys.stderr)
173
+
self._walk(sysfs)
174
+
175
+
def check_file(self, refs, found):
176
+
"""Check missing ABI symbols for a given sysfs file"""
177
+
178
+
res_list = []
179
+
180
+
try:
181
+
for names in refs:
182
+
fname = names[0]
183
+
184
+
res = {
185
+
"found": False,
186
+
"fname": fname,
187
+
"msg": "",
188
+
}
189
+
res_list.append(res)
190
+
191
+
re_what = self.abi.get_regexes(fname)
192
+
if not re_what:
193
+
self.abi.log.warning(f"missing rules for {fname}")
194
+
continue
195
+
196
+
for name in names:
197
+
for r in re_what:
198
+
if self.abi.debug & AbiDebug.UNDEFINED:
199
+
self.log.debug("check if %s matches '%s'", name, r.pattern)
200
+
if r.match(name):
201
+
res["found"] = True
202
+
if found:
203
+
res["msg"] += f" {fname}: regex:\n\t"
204
+
continue
205
+
206
+
if self.hints and not res["found"]:
207
+
res["msg"] += f" {fname} not found. Tested regexes:\n"
208
+
for r in re_what:
209
+
res["msg"] += " " + r.pattern + "\n"
210
+
211
+
except KeyboardInterrupt:
212
+
pass
213
+
214
+
return res_list
215
+
216
+
def _ref_interactor(self, root):
217
+
"""Recursive function to interact over the sysfs tree"""
218
+
219
+
for k, v in root.items():
220
+
if isinstance(v, dict):
221
+
yield from self._ref_interactor(v)
222
+
223
+
if root == self.root or k == "__name":
224
+
continue
225
+
226
+
if self.abi.re_string:
227
+
fname = v["__name"][0]
228
+
if self.abi.re_string.search(fname):
229
+
yield v
230
+
else:
231
+
yield v
232
+
233
+
234
+
def get_fileref(self, all_refs, chunk_size):
235
+
"""Interactor to group refs into chunks"""
236
+
237
+
n = 0
238
+
refs = []
239
+
240
+
for ref in all_refs:
241
+
refs.append(ref)
242
+
243
+
n += 1
244
+
if n >= chunk_size:
245
+
yield refs
246
+
n = 0
247
+
refs = []
248
+
249
+
yield refs
250
+
251
+
def check_undefined_symbols(self, max_workers=None, chunk_size=50,
252
+
found=None, dry_run=None):
253
+
"""Seach ABI for sysfs symbols missing documentation"""
254
+
255
+
self.abi.parse_abi()
256
+
257
+
if self.abi.debug & AbiDebug.GRAPH:
258
+
self.print_graph()
259
+
260
+
all_refs = []
261
+
for ref in self._ref_interactor(self.root):
262
+
all_refs.append(ref["__name"])
263
+
264
+
if dry_run:
265
+
print("Would check", file=sys.stderr)
266
+
for ref in all_refs:
267
+
print(", ".join(ref))
268
+
269
+
return
270
+
271
+
print("Starting to search symbols (it may take several minutes):",
272
+
file=sys.stderr)
273
+
start = datetime.now()
274
+
old_elapsed = None
275
+
276
+
# Python doesn't support multithreading due to limitations on its
277
+
# global lock (GIL). While Python 3.13 finally made GIL optional,
278
+
# there are still issues related to it. Also, we want to have
279
+
# backward compatibility with older versions of Python.
280
+
#
281
+
# So, use instead multiprocess. However, Python is very slow passing
282
+
# data from/to multiple processes. Also, it may consume lots of memory
283
+
# if the data to be shared is not small. So, we need to group workload
284
+
# in chunks that are big enough to generate performance gains while
285
+
# not being so big that would cause out-of-memory.
286
+
287
+
num_refs = len(all_refs)
288
+
print(f"Number of references to parse: {num_refs}", file=sys.stderr)
289
+
290
+
if not max_workers:
291
+
max_workers = os.cpu_count()
292
+
elif max_workers > os.cpu_count():
293
+
max_workers = os.cpu_count()
294
+
295
+
max_workers = max(max_workers, 1)
296
+
297
+
max_chunk_size = int((num_refs + max_workers - 1) / max_workers)
298
+
chunk_size = min(chunk_size, max_chunk_size)
299
+
chunk_size = max(1, chunk_size)
300
+
301
+
if max_workers > 1:
302
+
executor = futures.ProcessPoolExecutor
303
+
304
+
# Place references in a random order. This may help improving
305
+
# performance, by mixing complex/simple expressions when creating
306
+
# chunks
307
+
shuffle(all_refs)
308
+
else:
309
+
# Python has a high overhead with processes. When there's just
310
+
# one worker, it is faster to not create a new process.
311
+
# Yet, User still deserves to have a progress print. So, use
312
+
# python's "thread", which is actually a single process, using
313
+
# an internal schedule to switch between tasks. No performance
314
+
# gains for non-IO tasks, but still it can be quickly interrupted
315
+
# from time to time to display progress.
316
+
executor = futures.ThreadPoolExecutor
317
+
318
+
not_found = []
319
+
f_list = []
320
+
with executor(max_workers=max_workers) as exe:
321
+
for refs in self.get_fileref(all_refs, chunk_size):
322
+
if refs:
323
+
try:
324
+
f_list.append(exe.submit(self.check_file, refs, found))
325
+
326
+
except KeyboardInterrupt:
327
+
return
328
+
329
+
total = len(f_list)
330
+
331
+
if not total:
332
+
if self.abi.re_string:
333
+
print(f"No ABI symbol matches {self.abi.search_string}")
334
+
else:
335
+
self.abi.log.warning("No ABI symbols found")
336
+
return
337
+
338
+
print(f"{len(f_list):6d} jobs queued on {max_workers} workers",
339
+
file=sys.stderr)
340
+
341
+
while f_list:
342
+
try:
343
+
t = futures.wait(f_list, timeout=1,
344
+
return_when=futures.FIRST_COMPLETED)
345
+
346
+
done = t[0]
347
+
348
+
for fut in done:
349
+
res_list = fut.result()
350
+
351
+
for res in res_list:
352
+
if not res["found"]:
353
+
not_found.append(res["fname"])
354
+
if res["msg"]:
355
+
print(res["msg"])
356
+
357
+
f_list.remove(fut)
358
+
except KeyboardInterrupt:
359
+
return
360
+
361
+
except RuntimeError as e:
362
+
self.abi.log.warning(f"Future: {e}")
363
+
break
364
+
365
+
if sys.stderr.isatty():
366
+
elapsed = str(datetime.now() - start).split(".", maxsplit=1)[0]
367
+
if len(f_list) < total:
368
+
elapsed += f" ({total - len(f_list)}/{total} jobs completed). "
369
+
if elapsed != old_elapsed:
370
+
print(elapsed + "\r", end="", flush=True,
371
+
file=sys.stderr)
372
+
old_elapsed = elapsed
373
+
374
+
elapsed = str(datetime.now() - start).split(".", maxsplit=1)[0]
375
+
print(elapsed, file=sys.stderr)
376
+
377
+
for f in sorted(not_found):
378
+
print(f"{f} not found.")