Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
Cross-merge networking fixes after downstream PR.
No conflicts.
Adjacent changes:
drivers/net/ethernet/broadcom/bnxt/bnxt.h
c948c0973df5 ("bnxt_en: Don't clear ntuple filters and rss contexts during ethtool ops")
f2878cdeb754 ("bnxt_en: Add support to call FW to update a VNIC")
Link: https://patch.msgid.link/20240822210125.1542769-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
+4957
-2920
+2
-1
Documentation/ABI/testing/sysfs-devices-system-cpu
+2
-1
Documentation/ABI/testing/sysfs-devices-system-cpu
···
562
562
================ =========================================
563
563
564
564
If control status is "forceoff" or "notsupported" writes
565
-
are rejected.
565
+
are rejected. Note that enabling SMT on PowerPC skips
566
+
offline cores.
566
567
567
568
What: /sys/devices/system/cpu/cpuX/power/energy_perf_bias
568
569
Date: March 2019
+8
-7
Documentation/admin-guide/device-mapper/dm-crypt.rst
+8
-7
Documentation/admin-guide/device-mapper/dm-crypt.rst
···
162
162
163
163
164
164
Module parameters::
165
-
max_read_size
166
-
max_write_size
167
-
Maximum size of read or write requests. When a request larger than this size
168
-
is received, dm-crypt will split the request. The splitting improves
169
-
concurrency (the split requests could be encrypted in parallel by multiple
170
-
cores), but it also causes overhead. The user should tune these parameters to
171
-
fit the actual workload.
165
+
166
+
max_read_size
167
+
max_write_size
168
+
Maximum size of read or write requests. When a request larger than this size
169
+
is received, dm-crypt will split the request. The splitting improves
170
+
concurrency (the split requests could be encrypted in parallel by multiple
171
+
cores), but it also causes overhead. The user should tune these parameters to
172
+
fit the actual workload.
172
173
173
174
174
175
Example scripts
+22
-14
Documentation/arch/riscv/hwprobe.rst
+22
-14
Documentation/arch/riscv/hwprobe.rst
···
239
239
ratified in commit 98918c844281 ("Merge pull request #1217 from
240
240
riscv/zawrs") of riscv-isa-manual.
241
241
242
-
* :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: A bitmask that contains performance
243
-
information about the selected set of processors.
242
+
* :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: Deprecated. Returns similar values to
243
+
:c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF`, but the key was
244
+
mistakenly classified as a bitmask rather than a value.
244
245
245
-
* :c:macro:`RISCV_HWPROBE_MISALIGNED_UNKNOWN`: The performance of misaligned
246
-
accesses is unknown.
246
+
* :c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF`: An enum value describing
247
+
the performance of misaligned scalar native word accesses on the selected set
248
+
of processors.
247
249
248
-
* :c:macro:`RISCV_HWPROBE_MISALIGNED_EMULATED`: Misaligned accesses are
249
-
emulated via software, either in or below the kernel. These accesses are
250
-
always extremely slow.
250
+
* :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN`: The performance of
251
+
misaligned scalar accesses is unknown.
251
252
252
-
* :c:macro:`RISCV_HWPROBE_MISALIGNED_SLOW`: Misaligned accesses are slower
253
-
than equivalent byte accesses. Misaligned accesses may be supported
254
-
directly in hardware, or trapped and emulated by software.
253
+
* :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED`: Misaligned scalar
254
+
accesses are emulated via software, either in or below the kernel. These
255
+
accesses are always extremely slow.
255
256
256
-
* :c:macro:`RISCV_HWPROBE_MISALIGNED_FAST`: Misaligned accesses are faster
257
-
than equivalent byte accesses.
257
+
* :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW`: Misaligned scalar native
258
+
word sized accesses are slower than the equivalent quantity of byte
259
+
accesses. Misaligned accesses may be supported directly in hardware, or
260
+
trapped and emulated by software.
258
261
259
-
* :c:macro:`RISCV_HWPROBE_MISALIGNED_UNSUPPORTED`: Misaligned accesses are
260
-
not supported at all and will generate a misaligned address fault.
262
+
* :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_FAST`: Misaligned scalar native
263
+
word sized accesses are faster than the equivalent quantity of byte
264
+
accesses.
265
+
266
+
* :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_UNSUPPORTED`: Misaligned scalar
267
+
accesses are not supported at all and will generate a misaligned address
268
+
fault.
261
269
262
270
* :c:macro:`RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE`: An unsigned int which
263
271
represents the size of the Zicboz block in bytes.
+1
-1
Documentation/devicetree/bindings/clock/qcom,dispcc-sm6350.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,dispcc-sm6350.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,gcc-msm8994.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,gcc-msm8994.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,gcc-sm6125.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,gcc-sm6125.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,gcc-sm6350.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,gcc-sm6350.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6115-gpucc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6115-gpucc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6125-gpucc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6125-gpucc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6350-camcc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6350-camcc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6375-dispcc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6375-dispcc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6375-gcc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6375-gcc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6375-gpucc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm6375-gpucc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm8350-videocc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm8350-videocc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm8450-gpucc.yaml
+1
-1
Documentation/devicetree/bindings/clock/qcom,sm8450-gpucc.yaml
+1
-1
Documentation/devicetree/bindings/display/msm/qcom,sm6375-mdss.yaml
+1
-1
Documentation/devicetree/bindings/display/msm/qcom,sm6375-mdss.yaml
+1
-1
Documentation/devicetree/bindings/display/panel/asus,z00t-tm5p5-nt35596.yaml
+1
-1
Documentation/devicetree/bindings/display/panel/asus,z00t-tm5p5-nt35596.yaml
+6
-6
Documentation/devicetree/bindings/display/panel/samsung,atna33xc20.yaml
+6
-6
Documentation/devicetree/bindings/display/panel/samsung,atna33xc20.yaml
···
18
18
# Samsung 13.3" FHD (1920x1080 pixels) eDP AMOLED panel
19
19
- const: samsung,atna33xc20
20
20
- items:
21
-
- enum:
22
-
# Samsung 14.5" WQXGA+ (2880x1800 pixels) eDP AMOLED panel
23
-
- samsung,atna45af01
24
-
# Samsung 14.5" 3K (2944x1840 pixels) eDP AMOLED panel
25
-
- samsung,atna45dc02
26
-
- const: samsung,atna33xc20
21
+
- enum:
22
+
# Samsung 14.5" WQXGA+ (2880x1800 pixels) eDP AMOLED panel
23
+
- samsung,atna45af01
24
+
# Samsung 14.5" 3K (2944x1840 pixels) eDP AMOLED panel
25
+
- samsung,atna45dc02
26
+
- const: samsung,atna33xc20
27
27
28
28
enable-gpios: true
29
29
port: true
+1
-1
Documentation/devicetree/bindings/display/panel/sony,td4353-jdi.yaml
+1
-1
Documentation/devicetree/bindings/display/panel/sony,td4353-jdi.yaml
+1
Documentation/devicetree/bindings/eeprom/at25.yaml
+1
Documentation/devicetree/bindings/eeprom/at25.yaml
+1
-1
Documentation/devicetree/bindings/interconnect/qcom,sc7280-rpmh.yaml
+1
-1
Documentation/devicetree/bindings/interconnect/qcom,sc7280-rpmh.yaml
+1
-1
Documentation/devicetree/bindings/interconnect/qcom,sc8280xp-rpmh.yaml
+1
-1
Documentation/devicetree/bindings/interconnect/qcom,sc8280xp-rpmh.yaml
+1
-1
Documentation/devicetree/bindings/interconnect/qcom,sm8450-rpmh.yaml
+1
-1
Documentation/devicetree/bindings/interconnect/qcom,sm8450-rpmh.yaml
+1
-1
Documentation/devicetree/bindings/iommu/qcom,iommu.yaml
+1
-1
Documentation/devicetree/bindings/iommu/qcom,iommu.yaml
+1
-1
Documentation/devicetree/bindings/pinctrl/qcom,mdm9607-tlmm.yaml
+1
-1
Documentation/devicetree/bindings/pinctrl/qcom,mdm9607-tlmm.yaml
+1
-1
Documentation/devicetree/bindings/pinctrl/qcom,sm6350-tlmm.yaml
+1
-1
Documentation/devicetree/bindings/pinctrl/qcom,sm6350-tlmm.yaml
+1
-1
Documentation/devicetree/bindings/pinctrl/qcom,sm6375-tlmm.yaml
+1
-1
Documentation/devicetree/bindings/pinctrl/qcom,sm6375-tlmm.yaml
+1
-1
Documentation/devicetree/bindings/remoteproc/qcom,rpm-proc.yaml
+1
-1
Documentation/devicetree/bindings/remoteproc/qcom,rpm-proc.yaml
+1
-1
Documentation/devicetree/bindings/soc/qcom/qcom,rpm-master-stats.yaml
+1
-1
Documentation/devicetree/bindings/soc/qcom/qcom,rpm-master-stats.yaml
+1
-1
Documentation/filesystems/erofs.rst
+1
-1
Documentation/filesystems/erofs.rst
···
75
75
76
76
- Support merging tail-end data into a special inode as fragments.
77
77
78
-
- Support large folios for uncompressed files.
78
+
- Support large folios to make use of THPs (Transparent Hugepages);
79
79
80
80
- Support direct I/O on uncompressed files to avoid double caching for loop
81
81
devices;
+13
-13
Documentation/filesystems/smb/ksmbd.rst
+13
-13
Documentation/filesystems/smb/ksmbd.rst
···
13
13
The subset of performance related operations belong in kernelspace and
14
14
the other subset which belong to operations which are not really related with
15
15
performance in userspace. So, DCE/RPC management that has historically resulted
16
-
into number of buffer overflow issues and dangerous security bugs and user
16
+
into a number of buffer overflow issues and dangerous security bugs and user
17
17
account management are implemented in user space as ksmbd.mountd.
18
18
File operations that are related with performance (open/read/write/close etc.)
19
19
in kernel space (ksmbd). This also allows for easier integration with VFS
···
24
24
25
25
When the server daemon is started, It starts up a forker thread
26
26
(ksmbd/interface name) at initialization time and open a dedicated port 445
27
-
for listening to SMB requests. Whenever new clients make request, Forker
28
-
thread will accept the client connection and fork a new thread for dedicated
27
+
for listening to SMB requests. Whenever new clients make a request, the Forker
28
+
thread will accept the client connection and fork a new thread for a dedicated
29
29
communication channel between the client and the server. It allows for parallel
30
30
processing of SMB requests(commands) from clients as well as allowing for new
31
31
clients to make new connections. Each instance is named ksmbd/1~n(port number)
···
34
34
currently DCE/RPC commands are identified to be handled through the user space.
35
35
To further utilize the linux kernel, it has been chosen to process the commands
36
36
as workitems and to be executed in the handlers of the ksmbd-io kworker threads.
37
-
It allows for multiplexing of the handlers as the kernel take care of initiating
37
+
It allows for multiplexing of the handlers as the kernel takes care of initiating
38
38
extra worker threads if the load is increased and vice versa, if the load is
39
-
decreased it destroys the extra worker threads. So, after connection is
40
-
established with client. Dedicated ksmbd/1..n(port number) takes complete
39
+
decreased it destroys the extra worker threads. So, after the connection is
40
+
established with the client. Dedicated ksmbd/1..n(port number) takes complete
41
41
ownership of receiving/parsing of SMB commands. Each received command is worked
42
-
in parallel i.e., There can be multiple clients commands which are worked in
42
+
in parallel i.e., there can be multiple client commands which are worked in
43
43
parallel. After receiving each command a separated kernel workitem is prepared
44
44
for each command which is further queued to be handled by ksmbd-io kworkers.
45
45
So, each SMB workitem is queued to the kworkers. This allows the benefit of load
···
49
49
ksmbd.mountd (user space daemon)
50
50
--------------------------------
51
51
52
-
ksmbd.mountd is userspace process to, transfer user account and password that
52
+
ksmbd.mountd is a userspace process to, transfer the user account and password that
53
53
are registered using ksmbd.adduser (part of utils for user space). Further it
54
-
allows sharing information parameters that parsed from smb.conf to ksmbd in
54
+
allows sharing information parameters that are parsed from smb.conf to ksmbd in
55
55
kernel. For the execution part it has a daemon which is continuously running
56
56
and connected to the kernel interface using netlink socket, it waits for the
57
57
requests (dcerpc and share/user info). It handles RPC calls (at a minimum few
···
124
124
1. Download ksmbd-tools(https://github.com/cifsd-team/ksmbd-tools/releases) and
125
125
compile them.
126
126
127
-
- Refer README(https://github.com/cifsd-team/ksmbd-tools/blob/master/README.md)
127
+
- Refer to README(https://github.com/cifsd-team/ksmbd-tools/blob/master/README.md)
128
128
to know how to use ksmbd.mountd/adduser/addshare/control utils
129
129
130
130
$ ./autogen.sh
···
133
133
134
134
2. Create /usr/local/etc/ksmbd/ksmbd.conf file, add SMB share in ksmbd.conf file.
135
135
136
-
- Refer ksmbd.conf.example in ksmbd-utils, See ksmbd.conf manpage
136
+
- Refer to ksmbd.conf.example in ksmbd-utils, See ksmbd.conf manpage
137
137
for details to configure shares.
138
138
139
139
$ man ksmbd.conf
···
145
145
$ man ksmbd.adduser
146
146
$ sudo ksmbd.adduser -a <Enter USERNAME for SMB share access>
147
147
148
-
4. Insert ksmbd.ko module after build your kernel. No need to load module
148
+
4. Insert the ksmbd.ko module after you build your kernel. No need to load the module
149
149
if ksmbd is built into the kernel.
150
150
151
151
- Set ksmbd in menuconfig(e.g. $ make menuconfig)
···
175
175
1. Enable all component prints
176
176
# sudo ksmbd.control -d "all"
177
177
178
-
2. Enable one of components (smb, auth, vfs, oplock, ipc, conn, rdma)
178
+
2. Enable one of the components (smb, auth, vfs, oplock, ipc, conn, rdma)
179
179
# sudo ksmbd.control -d "smb"
180
180
181
181
3. Show what prints are enabled.
+1
-1
Documentation/kbuild/llvm.rst
+1
-1
Documentation/kbuild/llvm.rst
···
126
126
127
127
``ccache`` can be used with ``clang`` to improve subsequent builds, (though
128
128
KBUILD_BUILD_TIMESTAMP_ should be set to a deterministic value between builds
129
-
in order to avoid 100% cache misses, see Reproducible_builds_ for more info):
129
+
in order to avoid 100% cache misses, see Reproducible_builds_ for more info)::
130
130
131
131
KBUILD_BUILD_TIMESTAMP='' make LLVM=1 CC="ccache clang"
132
132
+27
-6
MAINTAINERS
+27
-6
MAINTAINERS
···
3504
3504
W: http://linux-atm.sourceforge.net
3505
3505
F: drivers/atm/
3506
3506
F: include/linux/atm*
3507
+
F: include/linux/sonet.h
3507
3508
F: include/uapi/linux/atm*
3509
+
F: include/uapi/linux/sonet.h
3508
3510
3509
3511
ATMEL MACB ETHERNET DRIVER
3510
3512
M: Nicolas Ferre <nicolas.ferre@microchip.com>
···
11995
11993
JME NETWORK DRIVER
11996
11994
M: Guo-Fu Tseng <cooldavid@cooldavid.org>
11997
11995
L: netdev@vger.kernel.org
11998
-
S: Maintained
11996
+
S: Odd Fixes
11999
11997
F: drivers/net/ethernet/jme.*
12000
11998
12001
11999
JOURNALLING FLASH FILE SYSTEM V2 (JFFS2)
···
15879
15877
F: include/dt-bindings/net/
15880
15878
F: include/linux/cn_proc.h
15881
15879
F: include/linux/etherdevice.h
15880
+
F: include/linux/ethtool_netlink.h
15882
15881
F: include/linux/fcdevice.h
15883
15882
F: include/linux/fddidevice.h
15884
15883
F: include/linux/hippidevice.h
15885
15884
F: include/linux/if_*
15886
15885
F: include/linux/inetdevice.h
15887
-
F: include/linux/netdevice.h
15886
+
F: include/linux/netdev*
15887
+
F: include/linux/platform_data/wiznet.h
15888
15888
F: include/uapi/linux/cn_proc.h
15889
+
F: include/uapi/linux/ethtool_netlink.h
15889
15890
F: include/uapi/linux/if_*
15890
-
F: include/uapi/linux/netdevice.h
15891
+
F: include/uapi/linux/netdev*
15892
+
F: tools/testing/selftests/drivers/net/
15891
15893
X: drivers/net/wireless/
15892
15894
15893
15895
NETWORKING DRIVERS (WIRELESS)
···
15942
15936
F: include/linux/framer/framer.h
15943
15937
F: include/linux/in.h
15944
15938
F: include/linux/indirect_call_wrapper.h
15939
+
F: include/linux/inet.h
15940
+
F: include/linux/inet_diag.h
15945
15941
F: include/linux/net.h
15946
-
F: include/linux/netdevice.h
15947
-
F: include/linux/skbuff.h
15942
+
F: include/linux/netdev*
15943
+
F: include/linux/netlink.h
15944
+
F: include/linux/netpoll.h
15945
+
F: include/linux/rtnetlink.h
15946
+
F: include/linux/seq_file_net.h
15947
+
F: include/linux/skbuff*
15948
15948
F: include/net/
15949
+
F: include/uapi/linux/genetlink.h
15950
+
F: include/uapi/linux/hsr_netlink.h
15949
15951
F: include/uapi/linux/in.h
15952
+
F: include/uapi/linux/inet_diag.h
15953
+
F: include/uapi/linux/nbd-netlink.h
15950
15954
F: include/uapi/linux/net.h
15951
15955
F: include/uapi/linux/net_namespace.h
15952
-
F: include/uapi/linux/netdevice.h
15956
+
F: include/uapi/linux/netconf.h
15957
+
F: include/uapi/linux/netdev*
15958
+
F: include/uapi/linux/netlink.h
15959
+
F: include/uapi/linux/netlink_diag.h
15960
+
F: include/uapi/linux/rtnetlink.h
15953
15961
F: lib/net_utils.c
15954
15962
F: lib/random32.c
15955
15963
F: net/
···
21075
21055
M: Willem de Bruijn <willemdebruijn.kernel@gmail.com>
21076
21056
S: Maintained
21077
21057
F: Documentation/networking/timestamping.rst
21058
+
F: include/linux/net_tstamp.h
21078
21059
F: include/uapi/linux/net_tstamp.h
21079
21060
F: tools/testing/selftests/net/so_txtime.c
21080
21061
+3
-3
Makefile
+3
-3
Makefile
···
2
2
VERSION = 6
3
3
PATCHLEVEL = 11
4
4
SUBLEVEL = 0
5
-
EXTRAVERSION = -rc3
5
+
EXTRAVERSION = -rc4
6
6
NAME = Baby Opossum Posse
7
7
8
8
# *DOCUMENTATION*
···
1963
1963
# Protocol).
1964
1964
PHONY += rust-analyzer
1965
1965
rust-analyzer:
1966
-
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh
1966
+
+$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh
1967
1967
$(Q)$(MAKE) $(build)=rust $@
1968
1968
1969
1969
# Script to generate missing namespace dependencies
···
1980
1980
quiet_cmd_gen_compile_commands = GEN $@
1981
1981
cmd_gen_compile_commands = $(PYTHON3) $< -a $(AR) -o $@ $(filter-out $<, $(real-prereqs))
1982
1982
1983
-
$(extmod_prefix)compile_commands.json: scripts/clang-tools/gen_compile_commands.py \
1983
+
$(extmod_prefix)compile_commands.json: $(srctree)/scripts/clang-tools/gen_compile_commands.py \
1984
1984
$(if $(KBUILD_EXTMOD),, vmlinux.a $(KBUILD_VMLINUX_LIBS)) \
1985
1985
$(if $(CONFIG_MODULES), $(MODORDER)) FORCE
1986
1986
$(call if_changed,gen_compile_commands)
+1
-1
arch/arm/mach-rpc/ecard.c
+1
-1
arch/arm/mach-rpc/ecard.c
···
1109
1109
driver_unregister(&drv->drv);
1110
1110
}
1111
1111
1112
-
static int ecard_match(struct device *_dev, struct device_driver *_drv)
1112
+
static int ecard_match(struct device *_dev, const struct device_driver *_drv)
1113
1113
{
1114
1114
struct expansion_card *ec = ECARD_DEV(_dev);
1115
1115
struct ecard_driver *drv = ECARD_DRV(_drv);
+1
-1
arch/arm64/include/asm/uaccess.h
+1
-1
arch/arm64/include/asm/uaccess.h
···
188
188
#define __get_mem_asm(load, reg, x, addr, label, type) \
189
189
asm_goto_output( \
190
190
"1: " load " " reg "0, [%1]\n" \
191
-
_ASM_EXTABLE_##type##ACCESS_ERR(1b, %l2, %w0) \
191
+
_ASM_EXTABLE_##type##ACCESS(1b, %l2) \
192
192
: "=r" (x) \
193
193
: "r" (addr) : : label)
194
194
#else
+1
-1
arch/arm64/kernel/acpi_numa.c
+1
-1
arch/arm64/kernel/acpi_numa.c
-3
arch/arm64/kernel/setup.c
-3
arch/arm64/kernel/setup.c
+2
arch/arm64/kernel/smp.c
+2
arch/arm64/kernel/smp.c
+1
-1
arch/mips/sgi-ip22/ip22-gio.c
+1
-1
arch/mips/sgi-ip22/ip22-gio.c
···
111
111
}
112
112
EXPORT_SYMBOL_GPL(gio_device_unregister);
113
113
114
-
static int gio_bus_match(struct device *dev, struct device_driver *drv)
114
+
static int gio_bus_match(struct device *dev, const struct device_driver *drv)
115
115
{
116
116
struct gio_device *gio_dev = to_gio_device(dev);
117
117
struct gio_driver *gio_drv = to_gio_driver(drv);
+13
arch/powerpc/include/asm/topology.h
+13
arch/powerpc/include/asm/topology.h
···
145
145
146
146
#ifdef CONFIG_HOTPLUG_SMT
147
147
#include <linux/cpu_smt.h>
148
+
#include <linux/cpumask.h>
148
149
#include <asm/cputhreads.h>
149
150
150
151
static inline bool topology_is_primary_thread(unsigned int cpu)
···
156
155
static inline bool topology_smt_thread_allowed(unsigned int cpu)
157
156
{
158
157
return cpu_thread_in_core(cpu) < cpu_smt_num_threads;
158
+
}
159
+
160
+
#define topology_is_core_online topology_is_core_online
161
+
static inline bool topology_is_core_online(unsigned int cpu)
162
+
{
163
+
int i, first_cpu = cpu_first_thread_sibling(cpu);
164
+
165
+
for (i = first_cpu; i < first_cpu + threads_per_core; ++i) {
166
+
if (cpu_online(i))
167
+
return true;
168
+
}
169
+
return false;
159
170
}
160
171
#endif
161
172
+1
arch/powerpc/kernel/setup-common.c
+1
arch/powerpc/kernel/setup-common.c
+2
-2
arch/powerpc/mm/init-common.c
+2
-2
arch/powerpc/mm/init-common.c
···
73
73
74
74
#define CTOR(shift) static void ctor_##shift(void *addr) \
75
75
{ \
76
-
memset(addr, 0, sizeof(void *) << (shift)); \
76
+
memset(addr, 0, sizeof(pgd_t) << (shift)); \
77
77
}
78
78
79
79
CTOR(0); CTOR(1); CTOR(2); CTOR(3); CTOR(4); CTOR(5); CTOR(6); CTOR(7);
···
117
117
void pgtable_cache_add(unsigned int shift)
118
118
{
119
119
char *name;
120
-
unsigned long table_size = sizeof(void *) << shift;
120
+
unsigned long table_size = sizeof(pgd_t) << shift;
121
121
unsigned long align = table_size;
122
122
123
123
/* When batching pgtable pointers for RCU freeing, we store
-2
arch/powerpc/mm/mem.c
-2
arch/powerpc/mm/mem.c
+1
-1
arch/riscv/include/asm/hwprobe.h
+1
-1
arch/riscv/include/asm/hwprobe.h
+6
arch/riscv/include/uapi/asm/hwprobe.h
+6
arch/riscv/include/uapi/asm/hwprobe.h
···
82
82
#define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6
83
83
#define RISCV_HWPROBE_KEY_HIGHEST_VIRT_ADDRESS 7
84
84
#define RISCV_HWPROBE_KEY_TIME_CSR_FREQ 8
85
+
#define RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF 9
86
+
#define RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN 0
87
+
#define RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED 1
88
+
#define RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW 2
89
+
#define RISCV_HWPROBE_MISALIGNED_SCALAR_FAST 3
90
+
#define RISCV_HWPROBE_MISALIGNED_SCALAR_UNSUPPORTED 4
85
91
/* Increase RISCV_HWPROBE_MAX_KEY when adding items. */
86
92
87
93
/* Flags */
+1
-1
arch/riscv/kernel/acpi_numa.c
+1
-1
arch/riscv/kernel/acpi_numa.c
+4
arch/riscv/kernel/patch.c
+4
arch/riscv/kernel/patch.c
···
205
205
int ret;
206
206
207
207
ret = patch_insn_set(addr, c, len);
208
+
if (!ret)
209
+
flush_icache_range((uintptr_t)addr, (uintptr_t)addr + len);
208
210
209
211
return ret;
210
212
}
···
241
239
int ret;
242
240
243
241
ret = patch_insn_write(addr, insns, len);
242
+
if (!ret)
243
+
flush_icache_range((uintptr_t)addr, (uintptr_t)addr + len);
244
244
245
245
return ret;
246
246
}
+6
-5
arch/riscv/kernel/sys_hwprobe.c
+6
-5
arch/riscv/kernel/sys_hwprobe.c
···
178
178
perf = this_perf;
179
179
180
180
if (perf != this_perf) {
181
-
perf = RISCV_HWPROBE_MISALIGNED_UNKNOWN;
181
+
perf = RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN;
182
182
break;
183
183
}
184
184
}
185
185
186
186
if (perf == -1ULL)
187
-
return RISCV_HWPROBE_MISALIGNED_UNKNOWN;
187
+
return RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN;
188
188
189
189
return perf;
190
190
}
···
192
192
static u64 hwprobe_misaligned(const struct cpumask *cpus)
193
193
{
194
194
if (IS_ENABLED(CONFIG_RISCV_EFFICIENT_UNALIGNED_ACCESS))
195
-
return RISCV_HWPROBE_MISALIGNED_FAST;
195
+
return RISCV_HWPROBE_MISALIGNED_SCALAR_FAST;
196
196
197
197
if (IS_ENABLED(CONFIG_RISCV_EMULATED_UNALIGNED_ACCESS) && unaligned_ctl_available())
198
-
return RISCV_HWPROBE_MISALIGNED_EMULATED;
198
+
return RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED;
199
199
200
-
return RISCV_HWPROBE_MISALIGNED_SLOW;
200
+
return RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW;
201
201
}
202
202
#endif
203
203
···
225
225
break;
226
226
227
227
case RISCV_HWPROBE_KEY_CPUPERF_0:
228
+
case RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF:
228
229
pair->value = hwprobe_misaligned(cpus);
229
230
break;
230
231
+2
-2
arch/riscv/kernel/traps.c
+2
-2
arch/riscv/kernel/traps.c
···
319
319
320
320
regs->epc += 4;
321
321
regs->orig_a0 = regs->a0;
322
+
regs->a0 = -ENOSYS;
322
323
323
324
riscv_v_vstate_discard(regs);
324
325
···
329
328
330
329
if (syscall >= 0 && syscall < NR_syscalls)
331
330
syscall_handler(regs, syscall);
332
-
else if (syscall != -1)
333
-
regs->a0 = -ENOSYS;
331
+
334
332
/*
335
333
* Ultimately, this value will get limited by KSTACK_OFFSET_MAX(),
336
334
* so the maximum stack offset is 1k bytes (10 bits).
+3
-3
arch/riscv/kernel/traps_misaligned.c
+3
-3
arch/riscv/kernel/traps_misaligned.c
···
338
338
perf_sw_event(PERF_COUNT_SW_ALIGNMENT_FAULTS, 1, regs, addr);
339
339
340
340
#ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS
341
-
*this_cpu_ptr(&misaligned_access_speed) = RISCV_HWPROBE_MISALIGNED_EMULATED;
341
+
*this_cpu_ptr(&misaligned_access_speed) = RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED;
342
342
#endif
343
343
344
344
if (!unaligned_enabled)
···
532
532
unsigned long tmp_var, tmp_val;
533
533
bool misaligned_emu_detected;
534
534
535
-
*mas_ptr = RISCV_HWPROBE_MISALIGNED_UNKNOWN;
535
+
*mas_ptr = RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN;
536
536
537
537
__asm__ __volatile__ (
538
538
" "REG_L" %[tmp], 1(%[ptr])\n"
539
539
: [tmp] "=r" (tmp_val) : [ptr] "r" (&tmp_var) : "memory");
540
540
541
-
misaligned_emu_detected = (*mas_ptr == RISCV_HWPROBE_MISALIGNED_EMULATED);
541
+
misaligned_emu_detected = (*mas_ptr == RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED);
542
542
/*
543
543
* If unaligned_ctl is already set, this means that we detected that all
544
544
* CPUS uses emulated misaligned access at boot time. If that changed
+6
-6
arch/riscv/kernel/unaligned_access_speed.c
+6
-6
arch/riscv/kernel/unaligned_access_speed.c
···
34
34
struct page *page = param;
35
35
void *dst;
36
36
void *src;
37
-
long speed = RISCV_HWPROBE_MISALIGNED_SLOW;
37
+
long speed = RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW;
38
38
39
-
if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_UNKNOWN)
39
+
if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN)
40
40
return 0;
41
41
42
42
/* Make an unaligned destination buffer. */
···
95
95
}
96
96
97
97
if (word_cycles < byte_cycles)
98
-
speed = RISCV_HWPROBE_MISALIGNED_FAST;
98
+
speed = RISCV_HWPROBE_MISALIGNED_SCALAR_FAST;
99
99
100
100
ratio = div_u64((byte_cycles * 100), word_cycles);
101
101
pr_info("cpu%d: Ratio of byte access time to unaligned word access is %d.%02d, unaligned accesses are %s\n",
102
102
cpu,
103
103
ratio / 100,
104
104
ratio % 100,
105
-
(speed == RISCV_HWPROBE_MISALIGNED_FAST) ? "fast" : "slow");
105
+
(speed == RISCV_HWPROBE_MISALIGNED_SCALAR_FAST) ? "fast" : "slow");
106
106
107
107
per_cpu(misaligned_access_speed, cpu) = speed;
108
108
···
110
110
* Set the value of fast_misaligned_access of a CPU. These operations
111
111
* are atomic to avoid race conditions.
112
112
*/
113
-
if (speed == RISCV_HWPROBE_MISALIGNED_FAST)
113
+
if (speed == RISCV_HWPROBE_MISALIGNED_SCALAR_FAST)
114
114
cpumask_set_cpu(cpu, &fast_misaligned_access);
115
115
else
116
116
cpumask_clear_cpu(cpu, &fast_misaligned_access);
···
188
188
static struct page *buf;
189
189
190
190
/* We are already set since the last check */
191
-
if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_UNKNOWN)
191
+
if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN)
192
192
goto exit;
193
193
194
194
buf = alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER);
+1
-1
arch/riscv/kernel/vendor_extensions.c
+1
-1
arch/riscv/kernel/vendor_extensions.c
···
38
38
#ifdef CONFIG_RISCV_ISA_VENDOR_EXT_ANDES
39
39
case ANDES_VENDOR_ID:
40
40
bmap = &riscv_isa_vendor_ext_list_andes.all_harts_isa_bitmap;
41
-
cpu_bmap = &riscv_isa_vendor_ext_list_andes.per_hart_isa_bitmap[cpu];
41
+
cpu_bmap = riscv_isa_vendor_ext_list_andes.per_hart_isa_bitmap;
42
42
break;
43
43
#endif
44
44
default:
+2
-2
arch/riscv/mm/init.c
+2
-2
arch/riscv/mm/init.c
···
927
927
PMD_SIZE, PAGE_KERNEL_EXEC);
928
928
929
929
/* Map the data in RAM */
930
-
end_va = kernel_map.virt_addr + XIP_OFFSET + kernel_map.size;
930
+
end_va = kernel_map.virt_addr + kernel_map.size;
931
931
for (va = kernel_map.virt_addr + XIP_OFFSET; va < end_va; va += PMD_SIZE)
932
932
create_pgd_mapping(pgdir, va,
933
933
kernel_map.phys_addr + (va - (kernel_map.virt_addr + XIP_OFFSET)),
···
1096
1096
1097
1097
phys_ram_base = CONFIG_PHYS_RAM_BASE;
1098
1098
kernel_map.phys_addr = (uintptr_t)CONFIG_PHYS_RAM_BASE;
1099
-
kernel_map.size = (uintptr_t)(&_end) - (uintptr_t)(&_sdata);
1099
+
kernel_map.size = (uintptr_t)(&_end) - (uintptr_t)(&_start);
1100
1100
1101
1101
kernel_map.va_kernel_xip_pa_offset = kernel_map.virt_addr - kernel_map.xiprom;
1102
1102
#else
+3
-2
block/blk-mq-tag.c
+3
-2
block/blk-mq-tag.c
···
38
38
void __blk_mq_tag_busy(struct blk_mq_hw_ctx *hctx)
39
39
{
40
40
unsigned int users;
41
+
unsigned long flags;
41
42
struct blk_mq_tags *tags = hctx->tags;
42
43
43
44
/*
···
57
56
return;
58
57
}
59
58
60
-
spin_lock_irq(&tags->lock);
59
+
spin_lock_irqsave(&tags->lock, flags);
61
60
users = tags->active_queues + 1;
62
61
WRITE_ONCE(tags->active_queues, users);
63
62
blk_mq_update_wake_batch(tags, users);
64
-
spin_unlock_irq(&tags->lock);
63
+
spin_unlock_irqrestore(&tags->lock, flags);
65
64
}
66
65
67
66
/*
-1
drivers/accessibility/speakup/genmap.c
-1
drivers/accessibility/speakup/genmap.c
-1
drivers/accessibility/speakup/makemapdata.c
-1
drivers/accessibility/speakup/makemapdata.c
+1
-5
drivers/acpi/acpica/acevents.h
+1
-5
drivers/acpi/acpica/acevents.h
···
188
188
u8 acpi_ns_is_locked);
189
189
190
190
void
191
-
acpi_ev_execute_reg_methods(struct acpi_namespace_node *node,
191
+
acpi_ev_execute_reg_methods(struct acpi_namespace_node *node, u32 max_depth,
192
192
acpi_adr_space_type space_id, u32 function);
193
-
194
-
void
195
-
acpi_ev_execute_orphan_reg_method(struct acpi_namespace_node *node,
196
-
acpi_adr_space_type space_id);
197
193
198
194
acpi_status
199
195
acpi_ev_execute_reg_method(union acpi_operand_object *region_obj, u32 function);
+9
-3
drivers/acpi/acpica/evregion.c
+9
-3
drivers/acpi/acpica/evregion.c
···
20
20
21
21
/* Local prototypes */
22
22
23
+
static void
24
+
acpi_ev_execute_orphan_reg_method(struct acpi_namespace_node *device_node,
25
+
acpi_adr_space_type space_id);
26
+
23
27
static acpi_status
24
28
acpi_ev_reg_run(acpi_handle obj_handle,
25
29
u32 level, void *context, void **return_value);
···
65
61
acpi_gbl_default_address_spaces
66
62
[i])) {
67
63
acpi_ev_execute_reg_methods(acpi_gbl_root_node,
64
+
ACPI_UINT32_MAX,
68
65
acpi_gbl_default_address_spaces
69
66
[i], ACPI_REG_CONNECT);
70
67
}
···
673
668
* FUNCTION: acpi_ev_execute_reg_methods
674
669
*
675
670
* PARAMETERS: node - Namespace node for the device
671
+
* max_depth - Depth to which search for _REG
676
672
* space_id - The address space ID
677
673
* function - Passed to _REG: On (1) or Off (0)
678
674
*
···
685
679
******************************************************************************/
686
680
687
681
void
688
-
acpi_ev_execute_reg_methods(struct acpi_namespace_node *node,
682
+
acpi_ev_execute_reg_methods(struct acpi_namespace_node *node, u32 max_depth,
689
683
acpi_adr_space_type space_id, u32 function)
690
684
{
691
685
struct acpi_reg_walk_info info;
···
719
713
* regions and _REG methods. (i.e. handlers must be installed for all
720
714
* regions of this Space ID before we can run any _REG methods)
721
715
*/
722
-
(void)acpi_ns_walk_namespace(ACPI_TYPE_ANY, node, ACPI_UINT32_MAX,
716
+
(void)acpi_ns_walk_namespace(ACPI_TYPE_ANY, node, max_depth,
723
717
ACPI_NS_WALK_UNLOCK, acpi_ev_reg_run, NULL,
724
718
&info, NULL);
725
719
···
820
814
*
821
815
******************************************************************************/
822
816
823
-
void
817
+
static void
824
818
acpi_ev_execute_orphan_reg_method(struct acpi_namespace_node *device_node,
825
819
acpi_adr_space_type space_id)
826
820
{
+7
-57
drivers/acpi/acpica/evxfregn.c
+7
-57
drivers/acpi/acpica/evxfregn.c
···
85
85
/* Run all _REG methods for this address space */
86
86
87
87
if (run_reg) {
88
-
acpi_ev_execute_reg_methods(node, space_id, ACPI_REG_CONNECT);
88
+
acpi_ev_execute_reg_methods(node, ACPI_UINT32_MAX, space_id,
89
+
ACPI_REG_CONNECT);
89
90
}
90
91
91
92
unlock_and_exit:
···
264
263
* FUNCTION: acpi_execute_reg_methods
265
264
*
266
265
* PARAMETERS: device - Handle for the device
266
+
* max_depth - Depth to which search for _REG
267
267
* space_id - The address space ID
268
268
*
269
269
* RETURN: Status
···
273
271
*
274
272
******************************************************************************/
275
273
acpi_status
276
-
acpi_execute_reg_methods(acpi_handle device, acpi_adr_space_type space_id)
274
+
acpi_execute_reg_methods(acpi_handle device, u32 max_depth,
275
+
acpi_adr_space_type space_id)
277
276
{
278
277
struct acpi_namespace_node *node;
279
278
acpi_status status;
···
299
296
300
297
/* Run all _REG methods for this address space */
301
298
302
-
acpi_ev_execute_reg_methods(node, space_id, ACPI_REG_CONNECT);
299
+
acpi_ev_execute_reg_methods(node, max_depth, space_id,
300
+
ACPI_REG_CONNECT);
303
301
} else {
304
302
status = AE_BAD_PARAMETER;
305
303
}
···
310
306
}
311
307
312
308
ACPI_EXPORT_SYMBOL(acpi_execute_reg_methods)
313
-
314
-
/*******************************************************************************
315
-
*
316
-
* FUNCTION: acpi_execute_orphan_reg_method
317
-
*
318
-
* PARAMETERS: device - Handle for the device
319
-
* space_id - The address space ID
320
-
*
321
-
* RETURN: Status
322
-
*
323
-
* DESCRIPTION: Execute an "orphan" _REG method that appears under an ACPI
324
-
* device. This is a _REG method that has no corresponding region
325
-
* within the device's scope.
326
-
*
327
-
******************************************************************************/
328
-
acpi_status
329
-
acpi_execute_orphan_reg_method(acpi_handle device, acpi_adr_space_type space_id)
330
-
{
331
-
struct acpi_namespace_node *node;
332
-
acpi_status status;
333
-
334
-
ACPI_FUNCTION_TRACE(acpi_execute_orphan_reg_method);
335
-
336
-
/* Parameter validation */
337
-
338
-
if (!device) {
339
-
return_ACPI_STATUS(AE_BAD_PARAMETER);
340
-
}
341
-
342
-
status = acpi_ut_acquire_mutex(ACPI_MTX_NAMESPACE);
343
-
if (ACPI_FAILURE(status)) {
344
-
return_ACPI_STATUS(status);
345
-
}
346
-
347
-
/* Convert and validate the device handle */
348
-
349
-
node = acpi_ns_validate_handle(device);
350
-
if (node) {
351
-
352
-
/*
353
-
* If an "orphan" _REG method is present in the device's scope
354
-
* for the given address space ID, run it.
355
-
*/
356
-
357
-
acpi_ev_execute_orphan_reg_method(node, space_id);
358
-
} else {
359
-
status = AE_BAD_PARAMETER;
360
-
}
361
-
362
-
(void)acpi_ut_release_mutex(ACPI_MTX_NAMESPACE);
363
-
return_ACPI_STATUS(status);
364
-
}
365
-
366
-
ACPI_EXPORT_SYMBOL(acpi_execute_orphan_reg_method)
+9
-5
drivers/acpi/ec.c
+9
-5
drivers/acpi/ec.c
···
1487
1487
static int ec_install_handlers(struct acpi_ec *ec, struct acpi_device *device,
1488
1488
bool call_reg)
1489
1489
{
1490
-
acpi_handle scope_handle = ec == first_ec ? ACPI_ROOT_OBJECT : ec->handle;
1491
1490
acpi_status status;
1492
1491
1493
1492
acpi_ec_start(ec, false);
1494
1493
1495
1494
if (!test_bit(EC_FLAGS_EC_HANDLER_INSTALLED, &ec->flags)) {
1495
+
acpi_handle scope_handle = ec == first_ec ? ACPI_ROOT_OBJECT : ec->handle;
1496
+
1496
1497
acpi_ec_enter_noirq(ec);
1497
1498
status = acpi_install_address_space_handler_no_reg(scope_handle,
1498
1499
ACPI_ADR_SPACE_EC,
···
1507
1506
}
1508
1507
1509
1508
if (call_reg && !test_bit(EC_FLAGS_EC_REG_CALLED, &ec->flags)) {
1510
-
acpi_execute_reg_methods(scope_handle, ACPI_ADR_SPACE_EC);
1511
-
if (scope_handle != ec->handle)
1512
-
acpi_execute_orphan_reg_method(ec->handle, ACPI_ADR_SPACE_EC);
1513
-
1509
+
acpi_execute_reg_methods(ec->handle, ACPI_UINT32_MAX, ACPI_ADR_SPACE_EC);
1514
1510
set_bit(EC_FLAGS_EC_REG_CALLED, &ec->flags);
1515
1511
}
1516
1512
···
1720
1722
ec_remove_handlers(ec);
1721
1723
acpi_ec_free(ec);
1722
1724
}
1725
+
}
1726
+
1727
+
void acpi_ec_register_opregions(struct acpi_device *adev)
1728
+
{
1729
+
if (first_ec && first_ec->handle != adev->handle)
1730
+
acpi_execute_reg_methods(adev->handle, 1, ACPI_ADR_SPACE_EC);
1723
1731
}
1724
1732
1725
1733
static acpi_status
+1
drivers/acpi/internal.h
+1
drivers/acpi/internal.h
+2
drivers/acpi/scan.c
+2
drivers/acpi/scan.c
-10
drivers/bluetooth/btintel.c
-10
drivers/bluetooth/btintel.c
···
2945
2945
INTEL_ROM_LEGACY_NO_WBS_SUPPORT))
2946
2946
set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED,
2947
2947
&hdev->quirks);
2948
-
if (ver.hw_variant == 0x08 && ver.fw_variant == 0x22)
2949
-
set_bit(HCI_QUIRK_VALID_LE_STATES,
2950
-
&hdev->quirks);
2951
2948
2952
2949
err = btintel_legacy_rom_setup(hdev, &ver);
2953
2950
break;
···
2953
2956
case 0x12: /* ThP */
2954
2957
case 0x13: /* HrP */
2955
2958
case 0x14: /* CcP */
2956
-
set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
2957
2959
fallthrough;
2958
2960
case 0x0c: /* WsP */
2959
2961
/* Apply the device specific HCI quirks
···
3044
3048
/* These variants don't seem to support LE Coded PHY */
3045
3049
set_bit(HCI_QUIRK_BROKEN_LE_CODED, &hdev->quirks);
3046
3050
3047
-
/* Set Valid LE States quirk */
3048
-
set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
3049
-
3050
3051
/* Setup MSFT Extension support */
3051
3052
btintel_set_msft_opcode(hdev, ver.hw_variant);
3052
3053
···
3068
3075
* All TLV based devices support WBS
3069
3076
*/
3070
3077
set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks);
3071
-
3072
-
/* Apply LE States quirk from solar onwards */
3073
-
set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
3074
3078
3075
3079
/* Setup MSFT Extension support */
3076
3080
btintel_set_msft_opcode(hdev,
-3
drivers/bluetooth/btintel_pcie.c
-3
drivers/bluetooth/btintel_pcie.c
···
1180
1180
*/
1181
1181
set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks);
1182
1182
1183
-
/* Apply LE States quirk from solar onwards */
1184
-
set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
1185
-
1186
1183
/* Setup MSFT Extension support */
1187
1184
btintel_set_msft_opcode(hdev,
1188
1185
INTEL_HW_VARIANT(ver_tlv.cnvi_bt));
-3
drivers/bluetooth/btmtksdio.c
-3
drivers/bluetooth/btmtksdio.c
-1
drivers/bluetooth/btrtl.c
-1
drivers/bluetooth/btrtl.c
···
1287
1287
case CHIP_ID_8852C:
1288
1288
case CHIP_ID_8851B:
1289
1289
case CHIP_ID_8852BT:
1290
-
set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
1291
1290
set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks);
1292
1291
1293
1292
/* RTL8852C needs to transmit mSBC data continuously without
+2
-2
drivers/bluetooth/btusb.c
+2
-2
drivers/bluetooth/btusb.c
···
3956
3956
if (id->driver_info & BTUSB_WIDEBAND_SPEECH)
3957
3957
set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks);
3958
3958
3959
-
if (id->driver_info & BTUSB_VALID_LE_STATES)
3960
-
set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
3959
+
if (!(id->driver_info & BTUSB_VALID_LE_STATES))
3960
+
set_bit(HCI_QUIRK_BROKEN_LE_STATES, &hdev->quirks);
3961
3961
3962
3962
if (id->driver_info & BTUSB_DIGIANSWER) {
3963
3963
data->cmdreq_type = USB_TYPE_VENDOR;
+2
-2
drivers/bluetooth/hci_qca.c
+2
-2
drivers/bluetooth/hci_qca.c
···
2474
2474
set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED,
2475
2475
&hdev->quirks);
2476
2476
2477
-
if (data->capabilities & QCA_CAP_VALID_LE_STATES)
2478
-
set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks);
2477
+
if (!(data->capabilities & QCA_CAP_VALID_LE_STATES))
2478
+
set_bit(HCI_QUIRK_BROKEN_LE_STATES, &hdev->quirks);
2479
2479
}
2480
2480
2481
2481
return 0;
-2
drivers/bluetooth/hci_vhci.c
-2
drivers/bluetooth/hci_vhci.c
+34
-8
drivers/char/xillybus/xillyusb.c
+34
-8
drivers/char/xillybus/xillyusb.c
···
50
50
static const char xillyname[] = "xillyusb";
51
51
52
52
static unsigned int fifo_buf_order;
53
+
static struct workqueue_struct *wakeup_wq;
53
54
54
55
#define USB_VENDOR_ID_XILINX 0x03fd
55
56
#define USB_VENDOR_ID_ALTERA 0x09fb
···
570
569
* errors if executed. The mechanism relies on that xdev->error is assigned
571
570
* a non-zero value by report_io_error() prior to queueing wakeup_all(),
572
571
* which prevents bulk_in_work() from calling process_bulk_in().
573
-
*
574
-
* The fact that wakeup_all() and bulk_in_work() are queued on the same
575
-
* workqueue makes their concurrent execution very unlikely, however the
576
-
* kernel's API doesn't seem to ensure this strictly.
577
572
*/
578
573
579
574
static void wakeup_all(struct work_struct *work)
···
624
627
625
628
if (do_once) {
626
629
kref_get(&xdev->kref); /* xdev is used by work item */
627
-
queue_work(xdev->workq, &xdev->wakeup_workitem);
630
+
queue_work(wakeup_wq, &xdev->wakeup_workitem);
628
631
}
629
632
}
630
633
···
1903
1906
1904
1907
static int xillyusb_setup_base_eps(struct xillyusb_dev *xdev)
1905
1908
{
1909
+
struct usb_device *udev = xdev->udev;
1910
+
1911
+
/* Verify that device has the two fundamental bulk in/out endpoints */
1912
+
if (usb_pipe_type_check(udev, usb_sndbulkpipe(udev, MSG_EP_NUM)) ||
1913
+
usb_pipe_type_check(udev, usb_rcvbulkpipe(udev, IN_EP_NUM)))
1914
+
return -ENODEV;
1915
+
1906
1916
xdev->msg_ep = endpoint_alloc(xdev, MSG_EP_NUM | USB_DIR_OUT,
1907
1917
bulk_out_work, 1, 2);
1908
1918
if (!xdev->msg_ep)
···
1939
1935
__le16 *chandesc,
1940
1936
int num_channels)
1941
1937
{
1942
-
struct xillyusb_channel *chan;
1938
+
struct usb_device *udev = xdev->udev;
1939
+
struct xillyusb_channel *chan, *new_channels;
1943
1940
int i;
1944
1941
1945
1942
chan = kcalloc(num_channels, sizeof(*chan), GFP_KERNEL);
1946
1943
if (!chan)
1947
1944
return -ENOMEM;
1948
1945
1949
-
xdev->channels = chan;
1946
+
new_channels = chan;
1950
1947
1951
1948
for (i = 0; i < num_channels; i++, chan++) {
1952
1949
unsigned int in_desc = le16_to_cpu(*chandesc++);
···
1976
1971
*/
1977
1972
1978
1973
if ((out_desc & 0x80) && i < 14) { /* Entry is valid */
1974
+
if (usb_pipe_type_check(udev,
1975
+
usb_sndbulkpipe(udev, i + 2))) {
1976
+
dev_err(xdev->dev,
1977
+
"Missing BULK OUT endpoint %d\n",
1978
+
i + 2);
1979
+
kfree(new_channels);
1980
+
return -ENODEV;
1981
+
}
1982
+
1979
1983
chan->writable = 1;
1980
1984
chan->out_synchronous = !!(out_desc & 0x40);
1981
1985
chan->out_seekable = !!(out_desc & 0x20);
···
1994
1980
}
1995
1981
}
1996
1982
1983
+
xdev->channels = new_channels;
1997
1984
return 0;
1998
1985
}
1999
1986
···
2111
2096
* just after responding with the IDT, there is no reason for any
2112
2097
* work item to be running now. To be sure that xdev->channels
2113
2098
* is updated on anything that might run in parallel, flush the
2114
-
* workqueue, which rarely does anything.
2099
+
* device's workqueue and the wakeup work item. This rarely
2100
+
* does anything.
2115
2101
*/
2116
2102
flush_workqueue(xdev->workq);
2103
+
flush_work(&xdev->wakeup_workitem);
2117
2104
2118
2105
xdev->num_channels = num_channels;
2119
2106
···
2275
2258
{
2276
2259
int rc = 0;
2277
2260
2261
+
wakeup_wq = alloc_workqueue(xillyname, 0, 0);
2262
+
if (!wakeup_wq)
2263
+
return -ENOMEM;
2264
+
2278
2265
if (LOG2_INITIAL_FIFO_BUF_SIZE > PAGE_SHIFT)
2279
2266
fifo_buf_order = LOG2_INITIAL_FIFO_BUF_SIZE - PAGE_SHIFT;
2280
2267
else
···
2286
2265
2287
2266
rc = usb_register(&xillyusb_driver);
2288
2267
2268
+
if (rc)
2269
+
destroy_workqueue(wakeup_wq);
2270
+
2289
2271
return rc;
2290
2272
}
2291
2273
2292
2274
static void __exit xillyusb_exit(void)
2293
2275
{
2294
2276
usb_deregister(&xillyusb_driver);
2277
+
2278
+
destroy_workqueue(wakeup_wq);
2295
2279
}
2296
2280
2297
2281
module_init(xillyusb_init);
+1
-1
drivers/clk/thead/clk-th1520-ap.c
+1
-1
drivers/clk/thead/clk-th1520-ap.c
+6
-4
drivers/cxl/core/pci.c
+6
-4
drivers/cxl/core/pci.c
···
834
834
void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport)
835
835
{
836
836
struct device *dport_dev = dport->dport_dev;
837
-
struct pci_host_bridge *host_bridge;
838
837
839
-
host_bridge = to_pci_host_bridge(dport_dev);
840
-
if (host_bridge->native_aer)
841
-
dport->rcrb.aer_cap = cxl_rcrb_to_aer(dport_dev, dport->rcrb.base);
838
+
if (dport->rch) {
839
+
struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport_dev);
840
+
841
+
if (host_bridge->native_aer)
842
+
dport->rcrb.aer_cap = cxl_rcrb_to_aer(dport_dev, dport->rcrb.base);
843
+
}
842
844
843
845
dport->reg_map.host = host;
844
846
cxl_dport_map_regs(dport);
+14
drivers/gpio/gpio-mlxbf3.c
+14
drivers/gpio/gpio-mlxbf3.c
···
39
39
#define MLXBF_GPIO_CAUSE_OR_EVTEN0 0x14
40
40
#define MLXBF_GPIO_CAUSE_OR_CLRCAUSE 0x18
41
41
42
+
#define MLXBF_GPIO_CLR_ALL_INTS GENMASK(31, 0)
43
+
42
44
struct mlxbf3_gpio_context {
43
45
struct gpio_chip gc;
44
46
···
84
82
val = readl(gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_EVTEN0);
85
83
val &= ~BIT(offset);
86
84
writel(val, gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_EVTEN0);
85
+
86
+
writel(BIT(offset), gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_CLRCAUSE);
87
87
raw_spin_unlock_irqrestore(&gs->gc.bgpio_lock, flags);
88
88
89
89
gpiochip_disable_irq(gc, offset);
···
257
253
return 0;
258
254
}
259
255
256
+
static void mlxbf3_gpio_shutdown(struct platform_device *pdev)
257
+
{
258
+
struct mlxbf3_gpio_context *gs = platform_get_drvdata(pdev);
259
+
260
+
/* Disable and clear all interrupts */
261
+
writel(0, gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_EVTEN0);
262
+
writel(MLXBF_GPIO_CLR_ALL_INTS, gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_CLRCAUSE);
263
+
}
264
+
260
265
static const struct acpi_device_id mlxbf3_gpio_acpi_match[] = {
261
266
{ "MLNXBF33", 0 },
262
267
{}
···
278
265
.acpi_match_table = mlxbf3_gpio_acpi_match,
279
266
},
280
267
.probe = mlxbf3_gpio_probe,
268
+
.shutdown = mlxbf3_gpio_shutdown,
281
269
};
282
270
module_platform_driver(mlxbf3_gpio_driver);
283
271
+3
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+3
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+8
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
+8
drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
···
685
685
686
686
switch (args->in.op) {
687
687
case AMDGPU_CTX_OP_ALLOC_CTX:
688
+
if (args->in.flags)
689
+
return -EINVAL;
688
690
r = amdgpu_ctx_alloc(adev, fpriv, filp, priority, &id);
689
691
args->out.alloc.ctx_id = id;
690
692
break;
691
693
case AMDGPU_CTX_OP_FREE_CTX:
694
+
if (args->in.flags)
695
+
return -EINVAL;
692
696
r = amdgpu_ctx_free(fpriv, id);
693
697
break;
694
698
case AMDGPU_CTX_OP_QUERY_STATE:
699
+
if (args->in.flags)
700
+
return -EINVAL;
695
701
r = amdgpu_ctx_query(adev, fpriv, id, &args->out);
696
702
break;
697
703
case AMDGPU_CTX_OP_QUERY_STATE2:
704
+
if (args->in.flags)
705
+
return -EINVAL;
698
706
r = amdgpu_ctx_query2(adev, fpriv, id, &args->out);
699
707
break;
700
708
case AMDGPU_CTX_OP_GET_STABLE_PSTATE:
+24
-2
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+24
-2
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
···
509
509
int i, r = 0;
510
510
int j;
511
511
512
+
if (adev->enable_mes) {
513
+
for (i = 0; i < adev->gfx.num_compute_rings; i++) {
514
+
j = i + xcc_id * adev->gfx.num_compute_rings;
515
+
amdgpu_mes_unmap_legacy_queue(adev,
516
+
&adev->gfx.compute_ring[j],
517
+
RESET_QUEUES, 0, 0);
518
+
}
519
+
return 0;
520
+
}
521
+
512
522
if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues)
513
523
return -EINVAL;
514
524
···
560
550
struct amdgpu_ring *kiq_ring = &kiq->ring;
561
551
int i, r = 0;
562
552
int j;
553
+
554
+
if (adev->enable_mes) {
555
+
if (amdgpu_gfx_is_master_xcc(adev, xcc_id)) {
556
+
for (i = 0; i < adev->gfx.num_gfx_rings; i++) {
557
+
j = i + xcc_id * adev->gfx.num_gfx_rings;
558
+
amdgpu_mes_unmap_legacy_queue(adev,
559
+
&adev->gfx.gfx_ring[j],
560
+
PREEMPT_QUEUES, 0, 0);
561
+
}
562
+
}
563
+
return 0;
564
+
}
563
565
564
566
if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues)
565
567
return -EINVAL;
···
1017
995
if (amdgpu_device_skip_hw_access(adev))
1018
996
return 0;
1019
997
1020
-
if (adev->mes.ring.sched.ready)
998
+
if (adev->mes.ring[0].sched.ready)
1021
999
return amdgpu_mes_rreg(adev, reg);
1022
1000
1023
1001
BUG_ON(!ring->funcs->emit_rreg);
···
1087
1065
if (amdgpu_device_skip_hw_access(adev))
1088
1066
return;
1089
1067
1090
-
if (adev->mes.ring.sched.ready) {
1068
+
if (adev->mes.ring[0].sched.ready) {
1091
1069
amdgpu_mes_wreg(adev, reg, v);
1092
1070
return;
1093
1071
}
+3
-2
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+3
-2
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
···
589
589
ring = adev->rings[i];
590
590
vmhub = ring->vm_hub;
591
591
592
-
if (ring == &adev->mes.ring ||
592
+
if (ring == &adev->mes.ring[0] ||
593
+
ring == &adev->mes.ring[1] ||
593
594
ring == &adev->umsch_mm.ring)
594
595
continue;
595
596
···
762
761
unsigned long flags;
763
762
uint32_t seq;
764
763
765
-
if (adev->mes.ring.sched.ready) {
764
+
if (adev->mes.ring[0].sched.ready) {
766
765
amdgpu_mes_reg_write_reg_wait(adev, reg0, reg1,
767
766
ref, mask);
768
767
return;
+51
-32
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c
+51
-32
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c
···
135
135
idr_init(&adev->mes.queue_id_idr);
136
136
ida_init(&adev->mes.doorbell_ida);
137
137
spin_lock_init(&adev->mes.queue_id_lock);
138
-
spin_lock_init(&adev->mes.ring_lock);
139
138
mutex_init(&adev->mes.mutex_hidden);
139
+
140
+
for (i = 0; i < AMDGPU_MAX_MES_PIPES; i++)
141
+
spin_lock_init(&adev->mes.ring_lock[i]);
140
142
141
143
adev->mes.total_max_queue = AMDGPU_FENCE_MES_QUEUE_ID_MASK;
142
144
adev->mes.vmid_mask_mmhub = 0xffffff00;
···
165
163
adev->mes.sdma_hqd_mask[i] = 0xfc;
166
164
}
167
165
168
-
r = amdgpu_device_wb_get(adev, &adev->mes.sch_ctx_offs);
169
-
if (r) {
170
-
dev_err(adev->dev,
171
-
"(%d) ring trail_fence_offs wb alloc failed\n", r);
172
-
goto error_ids;
173
-
}
174
-
adev->mes.sch_ctx_gpu_addr =
175
-
adev->wb.gpu_addr + (adev->mes.sch_ctx_offs * 4);
176
-
adev->mes.sch_ctx_ptr =
177
-
(uint64_t *)&adev->wb.wb[adev->mes.sch_ctx_offs];
166
+
for (i = 0; i < AMDGPU_MAX_MES_PIPES; i++) {
167
+
r = amdgpu_device_wb_get(adev, &adev->mes.sch_ctx_offs[i]);
168
+
if (r) {
169
+
dev_err(adev->dev,
170
+
"(%d) ring trail_fence_offs wb alloc failed\n",
171
+
r);
172
+
goto error;
173
+
}
174
+
adev->mes.sch_ctx_gpu_addr[i] =
175
+
adev->wb.gpu_addr + (adev->mes.sch_ctx_offs[i] * 4);
176
+
adev->mes.sch_ctx_ptr[i] =
177
+
(uint64_t *)&adev->wb.wb[adev->mes.sch_ctx_offs[i]];
178
178
179
-
r = amdgpu_device_wb_get(adev, &adev->mes.query_status_fence_offs);
180
-
if (r) {
181
-
amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
182
-
dev_err(adev->dev,
183
-
"(%d) query_status_fence_offs wb alloc failed\n", r);
184
-
goto error_ids;
179
+
r = amdgpu_device_wb_get(adev,
180
+
&adev->mes.query_status_fence_offs[i]);
181
+
if (r) {
182
+
dev_err(adev->dev,
183
+
"(%d) query_status_fence_offs wb alloc failed\n",
184
+
r);
185
+
goto error;
186
+
}
187
+
adev->mes.query_status_fence_gpu_addr[i] = adev->wb.gpu_addr +
188
+
(adev->mes.query_status_fence_offs[i] * 4);
189
+
adev->mes.query_status_fence_ptr[i] =
190
+
(uint64_t *)&adev->wb.wb[adev->mes.query_status_fence_offs[i]];
185
191
}
186
-
adev->mes.query_status_fence_gpu_addr =
187
-
adev->wb.gpu_addr + (adev->mes.query_status_fence_offs * 4);
188
-
adev->mes.query_status_fence_ptr =
189
-
(uint64_t *)&adev->wb.wb[adev->mes.query_status_fence_offs];
190
192
191
193
r = amdgpu_device_wb_get(adev, &adev->mes.read_val_offs);
192
194
if (r) {
193
-
amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
194
-
amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs);
195
195
dev_err(adev->dev,
196
196
"(%d) read_val_offs alloc failed\n", r);
197
-
goto error_ids;
197
+
goto error;
198
198
}
199
199
adev->mes.read_val_gpu_addr =
200
200
adev->wb.gpu_addr + (adev->mes.read_val_offs * 4);
···
216
212
error_doorbell:
217
213
amdgpu_mes_doorbell_free(adev);
218
214
error:
219
-
amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
220
-
amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs);
221
-
amdgpu_device_wb_free(adev, adev->mes.read_val_offs);
222
-
error_ids:
215
+
for (i = 0; i < AMDGPU_MAX_MES_PIPES; i++) {
216
+
if (adev->mes.sch_ctx_ptr[i])
217
+
amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs[i]);
218
+
if (adev->mes.query_status_fence_ptr[i])
219
+
amdgpu_device_wb_free(adev,
220
+
adev->mes.query_status_fence_offs[i]);
221
+
}
222
+
if (adev->mes.read_val_ptr)
223
+
amdgpu_device_wb_free(adev, adev->mes.read_val_offs);
224
+
223
225
idr_destroy(&adev->mes.pasid_idr);
224
226
idr_destroy(&adev->mes.gang_id_idr);
225
227
idr_destroy(&adev->mes.queue_id_idr);
···
236
226
237
227
void amdgpu_mes_fini(struct amdgpu_device *adev)
238
228
{
229
+
int i;
230
+
239
231
amdgpu_bo_free_kernel(&adev->mes.event_log_gpu_obj,
240
232
&adev->mes.event_log_gpu_addr,
241
233
&adev->mes.event_log_cpu_addr);
242
234
243
-
amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
244
-
amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs);
245
-
amdgpu_device_wb_free(adev, adev->mes.read_val_offs);
235
+
for (i = 0; i < AMDGPU_MAX_MES_PIPES; i++) {
236
+
if (adev->mes.sch_ctx_ptr[i])
237
+
amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs[i]);
238
+
if (adev->mes.query_status_fence_ptr[i])
239
+
amdgpu_device_wb_free(adev,
240
+
adev->mes.query_status_fence_offs[i]);
241
+
}
242
+
if (adev->mes.read_val_ptr)
243
+
amdgpu_device_wb_free(adev, adev->mes.read_val_offs);
244
+
246
245
amdgpu_mes_doorbell_free(adev);
247
246
248
247
idr_destroy(&adev->mes.pasid_idr);
···
1518
1499
1519
1500
amdgpu_ucode_ip_version_decode(adev, GC_HWIP, ucode_prefix,
1520
1501
sizeof(ucode_prefix));
1521
-
if (adev->enable_uni_mes && pipe == AMDGPU_MES_SCHED_PIPE) {
1502
+
if (adev->enable_uni_mes) {
1522
1503
snprintf(fw_name, sizeof(fw_name),
1523
1504
"amdgpu/%s_uni_mes.bin", ucode_prefix);
1524
1505
} else if (amdgpu_ip_version(adev, GC_HWIP, 0) >= IP_VERSION(11, 0, 0) &&
+8
-8
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h
+8
-8
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h
···
82
82
uint64_t default_process_quantum;
83
83
uint64_t default_gang_quantum;
84
84
85
-
struct amdgpu_ring ring;
86
-
spinlock_t ring_lock;
85
+
struct amdgpu_ring ring[AMDGPU_MAX_MES_PIPES];
86
+
spinlock_t ring_lock[AMDGPU_MAX_MES_PIPES];
87
87
88
88
const struct firmware *fw[AMDGPU_MAX_MES_PIPES];
89
89
···
112
112
uint32_t gfx_hqd_mask[AMDGPU_MES_MAX_GFX_PIPES];
113
113
uint32_t sdma_hqd_mask[AMDGPU_MES_MAX_SDMA_PIPES];
114
114
uint32_t aggregated_doorbells[AMDGPU_MES_PRIORITY_NUM_LEVELS];
115
-
uint32_t sch_ctx_offs;
116
-
uint64_t sch_ctx_gpu_addr;
117
-
uint64_t *sch_ctx_ptr;
118
-
uint32_t query_status_fence_offs;
119
-
uint64_t query_status_fence_gpu_addr;
120
-
uint64_t *query_status_fence_ptr;
115
+
uint32_t sch_ctx_offs[AMDGPU_MAX_MES_PIPES];
116
+
uint64_t sch_ctx_gpu_addr[AMDGPU_MAX_MES_PIPES];
117
+
uint64_t *sch_ctx_ptr[AMDGPU_MAX_MES_PIPES];
118
+
uint32_t query_status_fence_offs[AMDGPU_MAX_MES_PIPES];
119
+
uint64_t query_status_fence_gpu_addr[AMDGPU_MAX_MES_PIPES];
120
+
uint64_t *query_status_fence_ptr[AMDGPU_MAX_MES_PIPES];
121
121
uint32_t read_val_offs;
122
122
uint64_t read_val_gpu_addr;
123
123
uint32_t *read_val_ptr;
+2
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
+2
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
···
212
212
*/
213
213
if (ring->funcs->type == AMDGPU_RING_TYPE_KIQ)
214
214
sched_hw_submission = max(sched_hw_submission, 256);
215
+
if (ring->funcs->type == AMDGPU_RING_TYPE_MES)
216
+
sched_hw_submission = 8;
215
217
else if (ring == &adev->sdma.instance[0].page)
216
218
sched_hw_submission = 256;
217
219
+4
-1
drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
+4
-1
drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
···
461
461
struct amdgpu_fw_shared_unified_queue_struct sq;
462
462
uint8_t pad1[8];
463
463
struct amdgpu_fw_shared_fw_logging fw_log;
464
+
uint8_t pad2[20];
464
465
struct amdgpu_fw_shared_rb_setup rb_setup;
465
-
uint8_t pad2[4];
466
+
struct amdgpu_fw_shared_smu_interface_info smu_dpm_interface;
467
+
struct amdgpu_fw_shared_drm_key_wa drm_key_wa;
468
+
uint8_t pad3[9];
466
469
};
467
470
468
471
#define VCN_BLOCK_ENCODE_DISABLE_MASK 0x80
+1
-1
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+1
-1
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+1
-26
drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+1
-26
drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
···
3546
3546
return r;
3547
3547
}
3548
3548
3549
-
static int gfx_v12_0_kiq_disable_kgq(struct amdgpu_device *adev)
3550
-
{
3551
-
struct amdgpu_kiq *kiq = &adev->gfx.kiq[0];
3552
-
struct amdgpu_ring *kiq_ring = &kiq->ring;
3553
-
int i, r = 0;
3554
-
3555
-
if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues)
3556
-
return -EINVAL;
3557
-
3558
-
if (amdgpu_ring_alloc(kiq_ring, kiq->pmf->unmap_queues_size *
3559
-
adev->gfx.num_gfx_rings))
3560
-
return -ENOMEM;
3561
-
3562
-
for (i = 0; i < adev->gfx.num_gfx_rings; i++)
3563
-
kiq->pmf->kiq_unmap_queues(kiq_ring, &adev->gfx.gfx_ring[i],
3564
-
PREEMPT_QUEUES, 0, 0);
3565
-
3566
-
if (adev->gfx.kiq[0].ring.sched.ready)
3567
-
r = amdgpu_ring_test_helper(kiq_ring);
3568
-
3569
-
return r;
3570
-
}
3571
-
3572
3549
static int gfx_v12_0_hw_fini(void *handle)
3573
3550
{
3574
3551
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
3575
-
int r;
3576
3552
uint32_t tmp;
3577
3553
3578
3554
amdgpu_irq_put(adev, &adev->gfx.priv_reg_irq, 0);
···
3556
3580
3557
3581
if (!adev->no_hw_access) {
3558
3582
if (amdgpu_async_gfx_ring) {
3559
-
r = gfx_v12_0_kiq_disable_kgq(adev);
3560
-
if (r)
3583
+
if (amdgpu_gfx_disable_kgq(adev, 0))
3561
3584
DRM_ERROR("KGQ disable failed\n");
3562
3585
}
3563
3586
+1
-1
drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
+1
-1
drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
···
231
231
/* This is necessary for SRIOV as well as for GFXOFF to function
232
232
* properly under bare metal
233
233
*/
234
-
if ((adev->gfx.kiq[0].ring.sched.ready || adev->mes.ring.sched.ready) &&
234
+
if ((adev->gfx.kiq[0].ring.sched.ready || adev->mes.ring[0].sched.ready) &&
235
235
(amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev))) {
236
236
amdgpu_gmc_fw_reg_write_reg_wait(adev, req, ack, inv_req,
237
237
1 << vmid, GET_INST(GC, 0));
+1
-1
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
+1
-1
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
···
299
299
/* This is necessary for SRIOV as well as for GFXOFF to function
300
300
* properly under bare metal
301
301
*/
302
-
if ((adev->gfx.kiq[0].ring.sched.ready || adev->mes.ring.sched.ready) &&
302
+
if ((adev->gfx.kiq[0].ring.sched.ready || adev->mes.ring[0].sched.ready) &&
303
303
(amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev))) {
304
304
struct amdgpu_vmhub *hub = &adev->vmhub[vmhub];
305
305
const unsigned eng = 17;
+2
-2
drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c
+2
-2
drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c
···
538
538
539
539
amdgpu_ring_write(ring, PACKETJ(mmUVD_LMI_JRBC_IB_VMID_INTERNAL_OFFSET,
540
540
0, 0, PACKETJ_TYPE0));
541
-
amdgpu_ring_write(ring, (vmid | (vmid << 4)));
541
+
amdgpu_ring_write(ring, (vmid | (vmid << 4) | (vmid << 8)));
542
542
543
543
amdgpu_ring_write(ring, PACKETJ(mmUVD_LMI_JPEG_VMID_INTERNAL_OFFSET,
544
544
0, 0, PACKETJ_TYPE0));
545
-
amdgpu_ring_write(ring, (vmid | (vmid << 4)));
545
+
amdgpu_ring_write(ring, (vmid | (vmid << 4) | (vmid << 8)));
546
546
547
547
amdgpu_ring_write(ring, PACKETJ(mmUVD_LMI_JRBC_IB_64BIT_BAR_LOW_INTERNAL_OFFSET,
548
548
0, 0, PACKETJ_TYPE0));
+61
-2
drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c
+61
-2
drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c
···
23
23
24
24
#include "amdgpu.h"
25
25
#include "amdgpu_jpeg.h"
26
+
#include "amdgpu_cs.h"
26
27
#include "soc15.h"
27
28
#include "soc15d.h"
28
29
#include "jpeg_v4_0_3.h"
···
783
782
784
783
amdgpu_ring_write(ring, PACKETJ(regUVD_LMI_JRBC_IB_VMID_INTERNAL_OFFSET,
785
784
0, 0, PACKETJ_TYPE0));
786
-
amdgpu_ring_write(ring, (vmid | (vmid << 4)));
785
+
786
+
if (ring->funcs->parse_cs)
787
+
amdgpu_ring_write(ring, 0);
788
+
else
789
+
amdgpu_ring_write(ring, (vmid | (vmid << 4) | (vmid << 8)));
787
790
788
791
amdgpu_ring_write(ring, PACKETJ(regUVD_LMI_JPEG_VMID_INTERNAL_OFFSET,
789
792
0, 0, PACKETJ_TYPE0));
790
-
amdgpu_ring_write(ring, (vmid | (vmid << 4)));
793
+
amdgpu_ring_write(ring, (vmid | (vmid << 4) | (vmid << 8)));
791
794
792
795
amdgpu_ring_write(ring, PACKETJ(regUVD_LMI_JRBC_IB_64BIT_BAR_LOW_INTERNAL_OFFSET,
793
796
0, 0, PACKETJ_TYPE0));
···
1089
1084
.get_rptr = jpeg_v4_0_3_dec_ring_get_rptr,
1090
1085
.get_wptr = jpeg_v4_0_3_dec_ring_get_wptr,
1091
1086
.set_wptr = jpeg_v4_0_3_dec_ring_set_wptr,
1087
+
.parse_cs = jpeg_v4_0_3_dec_ring_parse_cs,
1092
1088
.emit_frame_size =
1093
1089
SOC15_FLUSH_GPU_TLB_NUM_WREG * 6 +
1094
1090
SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 8 +
···
1253
1247
static void jpeg_v4_0_3_set_ras_funcs(struct amdgpu_device *adev)
1254
1248
{
1255
1249
adev->jpeg.ras = &jpeg_v4_0_3_ras;
1250
+
}
1251
+
1252
+
/**
1253
+
* jpeg_v4_0_3_dec_ring_parse_cs - command submission parser
1254
+
*
1255
+
* @parser: Command submission parser context
1256
+
* @job: the job to parse
1257
+
* @ib: the IB to parse
1258
+
*
1259
+
* Parse the command stream, return -EINVAL for invalid packet,
1260
+
* 0 otherwise
1261
+
*/
1262
+
int jpeg_v4_0_3_dec_ring_parse_cs(struct amdgpu_cs_parser *parser,
1263
+
struct amdgpu_job *job,
1264
+
struct amdgpu_ib *ib)
1265
+
{
1266
+
uint32_t i, reg, res, cond, type;
1267
+
struct amdgpu_device *adev = parser->adev;
1268
+
1269
+
for (i = 0; i < ib->length_dw ; i += 2) {
1270
+
reg = CP_PACKETJ_GET_REG(ib->ptr[i]);
1271
+
res = CP_PACKETJ_GET_RES(ib->ptr[i]);
1272
+
cond = CP_PACKETJ_GET_COND(ib->ptr[i]);
1273
+
type = CP_PACKETJ_GET_TYPE(ib->ptr[i]);
1274
+
1275
+
if (res) /* only support 0 at the moment */
1276
+
return -EINVAL;
1277
+
1278
+
switch (type) {
1279
+
case PACKETJ_TYPE0:
1280
+
if (cond != PACKETJ_CONDITION_CHECK0 || reg < JPEG_REG_RANGE_START || reg > JPEG_REG_RANGE_END) {
1281
+
dev_err(adev->dev, "Invalid packet [0x%08x]!\n", ib->ptr[i]);
1282
+
return -EINVAL;
1283
+
}
1284
+
break;
1285
+
case PACKETJ_TYPE3:
1286
+
if (cond != PACKETJ_CONDITION_CHECK3 || reg < JPEG_REG_RANGE_START || reg > JPEG_REG_RANGE_END) {
1287
+
dev_err(adev->dev, "Invalid packet [0x%08x]!\n", ib->ptr[i]);
1288
+
return -EINVAL;
1289
+
}
1290
+
break;
1291
+
case PACKETJ_TYPE6:
1292
+
if (ib->ptr[i] == CP_PACKETJ_NOP)
1293
+
continue;
1294
+
dev_err(adev->dev, "Invalid packet [0x%08x]!\n", ib->ptr[i]);
1295
+
return -EINVAL;
1296
+
default:
1297
+
dev_err(adev->dev, "Unknown packet type %d !\n", type);
1298
+
return -EINVAL;
1299
+
}
1300
+
}
1301
+
1302
+
return 0;
1256
1303
}
+6
-1
drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.h
+6
-1
drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.h
···
46
46
47
47
#define JRBC_DEC_EXTERNAL_REG_WRITE_ADDR 0x18000
48
48
49
+
#define JPEG_REG_RANGE_START 0x4000
50
+
#define JPEG_REG_RANGE_END 0x41c2
51
+
49
52
extern const struct amdgpu_ip_block_version jpeg_v4_0_3_ip_block;
50
53
51
54
void jpeg_v4_0_3_dec_ring_emit_ib(struct amdgpu_ring *ring,
···
65
62
void jpeg_v4_0_3_dec_ring_emit_wreg(struct amdgpu_ring *ring, uint32_t reg, uint32_t val);
66
63
void jpeg_v4_0_3_dec_ring_emit_reg_wait(struct amdgpu_ring *ring, uint32_t reg,
67
64
uint32_t val, uint32_t mask);
68
-
65
+
int jpeg_v4_0_3_dec_ring_parse_cs(struct amdgpu_cs_parser *parser,
66
+
struct amdgpu_job *job,
67
+
struct amdgpu_ib *ib);
69
68
#endif /* __JPEG_V4_0_3_H__ */
+1
drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c
+1
drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c
···
646
646
.get_rptr = jpeg_v5_0_0_dec_ring_get_rptr,
647
647
.get_wptr = jpeg_v5_0_0_dec_ring_get_wptr,
648
648
.set_wptr = jpeg_v5_0_0_dec_ring_set_wptr,
649
+
.parse_cs = jpeg_v4_0_3_dec_ring_parse_cs,
649
650
.emit_frame_size =
650
651
SOC15_FLUSH_GPU_TLB_NUM_WREG * 6 +
651
652
SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 8 +
+33
-26
drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+33
-26
drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
···
162
162
union MESAPI__QUERY_MES_STATUS mes_status_pkt;
163
163
signed long timeout = 3000000; /* 3000 ms */
164
164
struct amdgpu_device *adev = mes->adev;
165
-
struct amdgpu_ring *ring = &mes->ring;
165
+
struct amdgpu_ring *ring = &mes->ring[0];
166
166
struct MES_API_STATUS *api_status;
167
167
union MESAPI__MISC *x_pkt = pkt;
168
168
const char *op_str, *misc_op_str;
169
169
unsigned long flags;
170
170
u64 status_gpu_addr;
171
-
u32 status_offset;
171
+
u32 seq, status_offset;
172
172
u64 *status_ptr;
173
173
signed long r;
174
174
int ret;
···
191
191
status_ptr = (u64 *)&adev->wb.wb[status_offset];
192
192
*status_ptr = 0;
193
193
194
-
spin_lock_irqsave(&mes->ring_lock, flags);
194
+
spin_lock_irqsave(&mes->ring_lock[0], flags);
195
195
r = amdgpu_ring_alloc(ring, (size + sizeof(mes_status_pkt)) / 4);
196
196
if (r)
197
197
goto error_unlock_free;
198
+
199
+
seq = ++ring->fence_drv.sync_seq;
200
+
r = amdgpu_fence_wait_polling(ring,
201
+
seq - ring->fence_drv.num_fences_mask,
202
+
timeout);
203
+
if (r < 1)
204
+
goto error_undo;
198
205
199
206
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
200
207
api_status->api_completion_fence_addr = status_gpu_addr;
···
215
208
mes_status_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS;
216
209
mes_status_pkt.api_status.api_completion_fence_addr =
217
210
ring->fence_drv.gpu_addr;
218
-
mes_status_pkt.api_status.api_completion_fence_value =
219
-
++ring->fence_drv.sync_seq;
211
+
mes_status_pkt.api_status.api_completion_fence_value = seq;
220
212
221
213
amdgpu_ring_write_multiple(ring, &mes_status_pkt,
222
214
sizeof(mes_status_pkt) / 4);
223
215
224
216
amdgpu_ring_commit(ring);
225
-
spin_unlock_irqrestore(&mes->ring_lock, flags);
217
+
spin_unlock_irqrestore(&mes->ring_lock[0], flags);
226
218
227
219
op_str = mes_v11_0_get_op_string(x_pkt);
228
220
misc_op_str = mes_v11_0_get_misc_op_string(x_pkt);
···
235
229
dev_dbg(adev->dev, "MES msg=%d was emitted\n",
236
230
x_pkt->header.opcode);
237
231
238
-
r = amdgpu_fence_wait_polling(ring, ring->fence_drv.sync_seq, timeout);
232
+
r = amdgpu_fence_wait_polling(ring, seq, timeout);
239
233
if (r < 1 || !*status_ptr) {
240
234
241
235
if (misc_op_str)
···
258
252
amdgpu_device_wb_free(adev, status_offset);
259
253
return 0;
260
254
255
+
error_undo:
256
+
dev_err(adev->dev, "MES ring buffer is full.\n");
257
+
amdgpu_ring_undo(ring);
258
+
261
259
error_unlock_free:
262
-
spin_unlock_irqrestore(&mes->ring_lock, flags);
260
+
spin_unlock_irqrestore(&mes->ring_lock[0], flags);
263
261
264
262
error_wb_free:
265
263
amdgpu_device_wb_free(adev, status_offset);
···
522
512
mes_set_hw_res_pkt.vmid_mask_gfxhub = mes->vmid_mask_gfxhub;
523
513
mes_set_hw_res_pkt.gds_size = adev->gds.gds_size;
524
514
mes_set_hw_res_pkt.paging_vmid = 0;
525
-
mes_set_hw_res_pkt.g_sch_ctx_gpu_mc_ptr = mes->sch_ctx_gpu_addr;
515
+
mes_set_hw_res_pkt.g_sch_ctx_gpu_mc_ptr = mes->sch_ctx_gpu_addr[0];
526
516
mes_set_hw_res_pkt.query_status_fence_gpu_mc_ptr =
527
-
mes->query_status_fence_gpu_addr;
517
+
mes->query_status_fence_gpu_addr[0];
528
518
529
519
for (i = 0; i < MAX_COMPUTE_PIPES; i++)
530
520
mes_set_hw_res_pkt.compute_hqd_mask[i] =
···
1025
1015
return r;
1026
1016
}
1027
1017
1028
-
kiq->pmf->kiq_map_queues(kiq_ring, &adev->mes.ring);
1018
+
kiq->pmf->kiq_map_queues(kiq_ring, &adev->mes.ring[0]);
1029
1019
1030
1020
return amdgpu_ring_test_helper(kiq_ring);
1031
1021
}
···
1039
1029
if (pipe == AMDGPU_MES_KIQ_PIPE)
1040
1030
ring = &adev->gfx.kiq[0].ring;
1041
1031
else if (pipe == AMDGPU_MES_SCHED_PIPE)
1042
-
ring = &adev->mes.ring;
1032
+
ring = &adev->mes.ring[0];
1043
1033
else
1044
1034
BUG();
1045
1035
···
1081
1071
{
1082
1072
struct amdgpu_ring *ring;
1083
1073
1084
-
ring = &adev->mes.ring;
1074
+
ring = &adev->mes.ring[0];
1085
1075
1086
1076
ring->funcs = &mes_v11_0_ring_funcs;
1087
1077
···
1134
1124
if (pipe == AMDGPU_MES_KIQ_PIPE)
1135
1125
ring = &adev->gfx.kiq[0].ring;
1136
1126
else if (pipe == AMDGPU_MES_SCHED_PIPE)
1137
-
ring = &adev->mes.ring;
1127
+
ring = &adev->mes.ring[0];
1138
1128
else
1139
1129
BUG();
1140
1130
···
1210
1200
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
1211
1201
int pipe;
1212
1202
1213
-
amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
1214
-
amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs);
1215
-
1216
1203
for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) {
1217
1204
kfree(adev->mes.mqd_backup[pipe]);
1218
1205
···
1223
1216
&adev->gfx.kiq[0].ring.mqd_gpu_addr,
1224
1217
&adev->gfx.kiq[0].ring.mqd_ptr);
1225
1218
1226
-
amdgpu_bo_free_kernel(&adev->mes.ring.mqd_obj,
1227
-
&adev->mes.ring.mqd_gpu_addr,
1228
-
&adev->mes.ring.mqd_ptr);
1219
+
amdgpu_bo_free_kernel(&adev->mes.ring[0].mqd_obj,
1220
+
&adev->mes.ring[0].mqd_gpu_addr,
1221
+
&adev->mes.ring[0].mqd_ptr);
1229
1222
1230
1223
amdgpu_ring_fini(&adev->gfx.kiq[0].ring);
1231
-
amdgpu_ring_fini(&adev->mes.ring);
1224
+
amdgpu_ring_fini(&adev->mes.ring[0]);
1232
1225
1233
1226
if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) {
1234
1227
mes_v11_0_free_ucode_buffers(adev, AMDGPU_MES_KIQ_PIPE);
···
1339
1332
1340
1333
static int mes_v11_0_kiq_hw_fini(struct amdgpu_device *adev)
1341
1334
{
1342
-
if (adev->mes.ring.sched.ready) {
1343
-
mes_v11_0_kiq_dequeue(&adev->mes.ring);
1344
-
adev->mes.ring.sched.ready = false;
1335
+
if (adev->mes.ring[0].sched.ready) {
1336
+
mes_v11_0_kiq_dequeue(&adev->mes.ring[0]);
1337
+
adev->mes.ring[0].sched.ready = false;
1345
1338
}
1346
1339
1347
1340
if (amdgpu_sriov_vf(adev)) {
···
1359
1352
int r;
1360
1353
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
1361
1354
1362
-
if (adev->mes.ring.sched.ready)
1355
+
if (adev->mes.ring[0].sched.ready)
1363
1356
goto out;
1364
1357
1365
1358
if (!adev->enable_mes_kiq) {
···
1404
1397
* with MES enabled.
1405
1398
*/
1406
1399
adev->gfx.kiq[0].ring.sched.ready = false;
1407
-
adev->mes.ring.sched.ready = true;
1400
+
adev->mes.ring[0].sched.ready = true;
1408
1401
1409
1402
return 0;
1410
1403
+161
-135
drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
+161
-135
drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
···
142
142
}
143
143
144
144
static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
145
-
void *pkt, int size,
146
-
int api_status_off)
145
+
int pipe, void *pkt, int size,
146
+
int api_status_off)
147
147
{
148
148
union MESAPI__QUERY_MES_STATUS mes_status_pkt;
149
149
signed long timeout = 3000000; /* 3000 ms */
150
150
struct amdgpu_device *adev = mes->adev;
151
-
struct amdgpu_ring *ring = &mes->ring;
151
+
struct amdgpu_ring *ring = &mes->ring[pipe];
152
+
spinlock_t *ring_lock = &mes->ring_lock[pipe];
152
153
struct MES_API_STATUS *api_status;
153
154
union MESAPI__MISC *x_pkt = pkt;
154
155
const char *op_str, *misc_op_str;
155
156
unsigned long flags;
156
157
u64 status_gpu_addr;
157
-
u32 status_offset;
158
+
u32 seq, status_offset;
158
159
u64 *status_ptr;
159
160
signed long r;
160
161
int ret;
···
178
177
status_ptr = (u64 *)&adev->wb.wb[status_offset];
179
178
*status_ptr = 0;
180
179
181
-
spin_lock_irqsave(&mes->ring_lock, flags);
180
+
spin_lock_irqsave(ring_lock, flags);
182
181
r = amdgpu_ring_alloc(ring, (size + sizeof(mes_status_pkt)) / 4);
183
182
if (r)
184
183
goto error_unlock_free;
184
+
185
+
seq = ++ring->fence_drv.sync_seq;
186
+
r = amdgpu_fence_wait_polling(ring,
187
+
seq - ring->fence_drv.num_fences_mask,
188
+
timeout);
189
+
if (r < 1)
190
+
goto error_undo;
185
191
186
192
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
187
193
api_status->api_completion_fence_addr = status_gpu_addr;
···
202
194
mes_status_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS;
203
195
mes_status_pkt.api_status.api_completion_fence_addr =
204
196
ring->fence_drv.gpu_addr;
205
-
mes_status_pkt.api_status.api_completion_fence_value =
206
-
++ring->fence_drv.sync_seq;
197
+
mes_status_pkt.api_status.api_completion_fence_value = seq;
207
198
208
199
amdgpu_ring_write_multiple(ring, &mes_status_pkt,
209
200
sizeof(mes_status_pkt) / 4);
210
201
211
202
amdgpu_ring_commit(ring);
212
-
spin_unlock_irqrestore(&mes->ring_lock, flags);
203
+
spin_unlock_irqrestore(ring_lock, flags);
213
204
214
205
op_str = mes_v12_0_get_op_string(x_pkt);
215
206
misc_op_str = mes_v12_0_get_misc_op_string(x_pkt);
216
207
217
208
if (misc_op_str)
218
-
dev_dbg(adev->dev, "MES msg=%s (%s) was emitted\n", op_str,
219
-
misc_op_str);
209
+
dev_dbg(adev->dev, "MES(%d) msg=%s (%s) was emitted\n",
210
+
pipe, op_str, misc_op_str);
220
211
else if (op_str)
221
-
dev_dbg(adev->dev, "MES msg=%s was emitted\n", op_str);
212
+
dev_dbg(adev->dev, "MES(%d) msg=%s was emitted\n",
213
+
pipe, op_str);
222
214
else
223
-
dev_dbg(adev->dev, "MES msg=%d was emitted\n",
224
-
x_pkt->header.opcode);
215
+
dev_dbg(adev->dev, "MES(%d) msg=%d was emitted\n",
216
+
pipe, x_pkt->header.opcode);
225
217
226
-
r = amdgpu_fence_wait_polling(ring, ring->fence_drv.sync_seq, timeout);
218
+
r = amdgpu_fence_wait_polling(ring, seq, timeout);
227
219
if (r < 1 || !*status_ptr) {
228
220
229
221
if (misc_op_str)
230
-
dev_err(adev->dev, "MES failed to respond to msg=%s (%s)\n",
231
-
op_str, misc_op_str);
222
+
dev_err(adev->dev, "MES(%d) failed to respond to msg=%s (%s)\n",
223
+
pipe, op_str, misc_op_str);
232
224
else if (op_str)
233
-
dev_err(adev->dev, "MES failed to respond to msg=%s\n",
234
-
op_str);
225
+
dev_err(adev->dev, "MES(%d) failed to respond to msg=%s\n",
226
+
pipe, op_str);
235
227
else
236
-
dev_err(adev->dev, "MES failed to respond to msg=%d\n",
237
-
x_pkt->header.opcode);
228
+
dev_err(adev->dev, "MES(%d) failed to respond to msg=%d\n",
229
+
pipe, x_pkt->header.opcode);
238
230
239
231
while (halt_if_hws_hang)
240
232
schedule();
···
246
238
amdgpu_device_wb_free(adev, status_offset);
247
239
return 0;
248
240
241
+
error_undo:
242
+
dev_err(adev->dev, "MES ring buffer is full.\n");
243
+
amdgpu_ring_undo(ring);
244
+
249
245
error_unlock_free:
250
-
spin_unlock_irqrestore(&mes->ring_lock, flags);
246
+
spin_unlock_irqrestore(ring_lock, flags);
251
247
252
248
error_wb_free:
253
249
amdgpu_device_wb_free(adev, status_offset);
···
266
254
return MES_QUEUE_TYPE_COMPUTE;
267
255
else if (queue_type == AMDGPU_RING_TYPE_SDMA)
268
256
return MES_QUEUE_TYPE_SDMA;
257
+
else if (queue_type == AMDGPU_RING_TYPE_MES)
258
+
return MES_QUEUE_TYPE_SCHQ;
269
259
else
270
260
BUG();
271
261
return -1;
···
325
311
mes_add_queue_pkt.gds_size = input->queue_size;
326
312
327
313
return mes_v12_0_submit_pkt_and_poll_completion(mes,
314
+
AMDGPU_MES_SCHED_PIPE,
328
315
&mes_add_queue_pkt, sizeof(mes_add_queue_pkt),
329
316
offsetof(union MESAPI__ADD_QUEUE, api_status));
330
317
}
···
345
330
mes_remove_queue_pkt.gang_context_addr = input->gang_context_addr;
346
331
347
332
return mes_v12_0_submit_pkt_and_poll_completion(mes,
333
+
AMDGPU_MES_SCHED_PIPE,
348
334
&mes_remove_queue_pkt, sizeof(mes_remove_queue_pkt),
349
335
offsetof(union MESAPI__REMOVE_QUEUE, api_status));
350
336
}
···
354
338
struct mes_map_legacy_queue_input *input)
355
339
{
356
340
union MESAPI__ADD_QUEUE mes_add_queue_pkt;
341
+
int pipe;
357
342
358
343
memset(&mes_add_queue_pkt, 0, sizeof(mes_add_queue_pkt));
359
344
···
371
354
convert_to_mes_queue_type(input->queue_type);
372
355
mes_add_queue_pkt.map_legacy_kq = 1;
373
356
374
-
return mes_v12_0_submit_pkt_and_poll_completion(mes,
357
+
if (mes->adev->enable_uni_mes)
358
+
pipe = AMDGPU_MES_KIQ_PIPE;
359
+
else
360
+
pipe = AMDGPU_MES_SCHED_PIPE;
361
+
362
+
return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe,
375
363
&mes_add_queue_pkt, sizeof(mes_add_queue_pkt),
376
364
offsetof(union MESAPI__ADD_QUEUE, api_status));
377
365
}
···
385
363
struct mes_unmap_legacy_queue_input *input)
386
364
{
387
365
union MESAPI__REMOVE_QUEUE mes_remove_queue_pkt;
366
+
int pipe;
388
367
389
368
memset(&mes_remove_queue_pkt, 0, sizeof(mes_remove_queue_pkt));
390
369
···
410
387
convert_to_mes_queue_type(input->queue_type);
411
388
}
412
389
413
-
return mes_v12_0_submit_pkt_and_poll_completion(mes,
390
+
if (mes->adev->enable_uni_mes)
391
+
pipe = AMDGPU_MES_KIQ_PIPE;
392
+
else
393
+
pipe = AMDGPU_MES_SCHED_PIPE;
394
+
395
+
return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe,
414
396
&mes_remove_queue_pkt, sizeof(mes_remove_queue_pkt),
415
397
offsetof(union MESAPI__REMOVE_QUEUE, api_status));
416
398
}
···
432
404
return 0;
433
405
}
434
406
435
-
static int mes_v12_0_query_sched_status(struct amdgpu_mes *mes)
407
+
static int mes_v12_0_query_sched_status(struct amdgpu_mes *mes, int pipe)
436
408
{
437
409
union MESAPI__QUERY_MES_STATUS mes_status_pkt;
438
410
···
442
414
mes_status_pkt.header.opcode = MES_SCH_API_QUERY_SCHEDULER_STATUS;
443
415
mes_status_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS;
444
416
445
-
return mes_v12_0_submit_pkt_and_poll_completion(mes,
417
+
return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe,
446
418
&mes_status_pkt, sizeof(mes_status_pkt),
447
419
offsetof(union MESAPI__QUERY_MES_STATUS, api_status));
448
420
}
···
451
423
struct mes_misc_op_input *input)
452
424
{
453
425
union MESAPI__MISC misc_pkt;
426
+
int pipe;
454
427
455
428
memset(&misc_pkt, 0, sizeof(misc_pkt));
456
429
···
504
475
return -EINVAL;
505
476
}
506
477
507
-
return mes_v12_0_submit_pkt_and_poll_completion(mes,
478
+
if (mes->adev->enable_uni_mes)
479
+
pipe = AMDGPU_MES_KIQ_PIPE;
480
+
else
481
+
pipe = AMDGPU_MES_SCHED_PIPE;
482
+
483
+
return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe,
508
484
&misc_pkt, sizeof(misc_pkt),
509
485
offsetof(union MESAPI__MISC, api_status));
510
486
}
511
487
512
-
static int mes_v12_0_set_hw_resources_1(struct amdgpu_mes *mes)
488
+
static int mes_v12_0_set_hw_resources_1(struct amdgpu_mes *mes, int pipe)
513
489
{
514
490
union MESAPI_SET_HW_RESOURCES_1 mes_set_hw_res_1_pkt;
515
491
···
525
491
mes_set_hw_res_1_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS;
526
492
mes_set_hw_res_1_pkt.mes_kiq_unmap_timeout = 100;
527
493
528
-
return mes_v12_0_submit_pkt_and_poll_completion(mes,
494
+
return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe,
529
495
&mes_set_hw_res_1_pkt, sizeof(mes_set_hw_res_1_pkt),
530
496
offsetof(union MESAPI_SET_HW_RESOURCES_1, api_status));
531
497
}
532
498
533
-
static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes)
499
+
static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes, int pipe)
534
500
{
535
501
int i;
536
502
struct amdgpu_device *adev = mes->adev;
···
542
508
mes_set_hw_res_pkt.header.opcode = MES_SCH_API_SET_HW_RSRC;
543
509
mes_set_hw_res_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS;
544
510
545
-
mes_set_hw_res_pkt.vmid_mask_mmhub = mes->vmid_mask_mmhub;
546
-
mes_set_hw_res_pkt.vmid_mask_gfxhub = mes->vmid_mask_gfxhub;
547
-
mes_set_hw_res_pkt.gds_size = adev->gds.gds_size;
548
-
mes_set_hw_res_pkt.paging_vmid = 0;
549
-
mes_set_hw_res_pkt.g_sch_ctx_gpu_mc_ptr = mes->sch_ctx_gpu_addr;
511
+
if (pipe == AMDGPU_MES_SCHED_PIPE) {
512
+
mes_set_hw_res_pkt.vmid_mask_mmhub = mes->vmid_mask_mmhub;
513
+
mes_set_hw_res_pkt.vmid_mask_gfxhub = mes->vmid_mask_gfxhub;
514
+
mes_set_hw_res_pkt.gds_size = adev->gds.gds_size;
515
+
mes_set_hw_res_pkt.paging_vmid = 0;
516
+
517
+
for (i = 0; i < MAX_COMPUTE_PIPES; i++)
518
+
mes_set_hw_res_pkt.compute_hqd_mask[i] =
519
+
mes->compute_hqd_mask[i];
520
+
521
+
for (i = 0; i < MAX_GFX_PIPES; i++)
522
+
mes_set_hw_res_pkt.gfx_hqd_mask[i] =
523
+
mes->gfx_hqd_mask[i];
524
+
525
+
for (i = 0; i < MAX_SDMA_PIPES; i++)
526
+
mes_set_hw_res_pkt.sdma_hqd_mask[i] =
527
+
mes->sdma_hqd_mask[i];
528
+
529
+
for (i = 0; i < AMD_PRIORITY_NUM_LEVELS; i++)
530
+
mes_set_hw_res_pkt.aggregated_doorbells[i] =
531
+
mes->aggregated_doorbells[i];
532
+
}
533
+
534
+
mes_set_hw_res_pkt.g_sch_ctx_gpu_mc_ptr =
535
+
mes->sch_ctx_gpu_addr[pipe];
550
536
mes_set_hw_res_pkt.query_status_fence_gpu_mc_ptr =
551
-
mes->query_status_fence_gpu_addr;
552
-
553
-
for (i = 0; i < MAX_COMPUTE_PIPES; i++)
554
-
mes_set_hw_res_pkt.compute_hqd_mask[i] =
555
-
mes->compute_hqd_mask[i];
556
-
557
-
for (i = 0; i < MAX_GFX_PIPES; i++)
558
-
mes_set_hw_res_pkt.gfx_hqd_mask[i] = mes->gfx_hqd_mask[i];
559
-
560
-
for (i = 0; i < MAX_SDMA_PIPES; i++)
561
-
mes_set_hw_res_pkt.sdma_hqd_mask[i] = mes->sdma_hqd_mask[i];
562
-
563
-
for (i = 0; i < AMD_PRIORITY_NUM_LEVELS; i++)
564
-
mes_set_hw_res_pkt.aggregated_doorbells[i] =
565
-
mes->aggregated_doorbells[i];
537
+
mes->query_status_fence_gpu_addr[pipe];
566
538
567
539
for (i = 0; i < 5; i++) {
568
540
mes_set_hw_res_pkt.gc_base[i] = adev->reg_offset[GC_HWIP][0][i];
···
596
556
mes_set_hw_res_pkt.event_intr_history_gpu_mc_ptr = mes->event_log_gpu_addr;
597
557
}
598
558
599
-
return mes_v12_0_submit_pkt_and_poll_completion(mes,
559
+
return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe,
600
560
&mes_set_hw_res_pkt, sizeof(mes_set_hw_res_pkt),
601
561
offsetof(union MESAPI_SET_HW_RESOURCES, api_status));
602
562
}
···
774
734
if (enable) {
775
735
data = RREG32_SOC15(GC, 0, regCP_MES_CNTL);
776
736
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE0_RESET, 1);
777
-
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_RESET,
778
-
(!adev->enable_uni_mes && adev->enable_mes_kiq) ? 1 : 0);
737
+
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_RESET, 1);
779
738
WREG32_SOC15(GC, 0, regCP_MES_CNTL, data);
780
739
781
740
mutex_lock(&adev->srbm_mutex);
782
741
for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) {
783
-
if ((!adev->enable_mes_kiq || adev->enable_uni_mes) &&
784
-
pipe == AMDGPU_MES_KIQ_PIPE)
785
-
continue;
786
-
787
742
soc21_grbm_select(adev, 3, pipe, 0, 0);
788
743
789
744
ucode_addr = adev->mes.uc_start_addr[pipe] >> 2;
···
792
757
793
758
/* unhalt MES and activate pipe0 */
794
759
data = REG_SET_FIELD(0, CP_MES_CNTL, MES_PIPE0_ACTIVE, 1);
795
-
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_ACTIVE,
796
-
(!adev->enable_uni_mes && adev->enable_mes_kiq) ? 1 : 0);
760
+
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_ACTIVE, 1);
797
761
WREG32_SOC15(GC, 0, regCP_MES_CNTL, data);
798
762
799
763
if (amdgpu_emu_mode)
···
808
774
data = REG_SET_FIELD(data, CP_MES_CNTL,
809
775
MES_INVALIDATE_ICACHE, 1);
810
776
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE0_RESET, 1);
811
-
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_RESET,
812
-
(!adev->enable_uni_mes && adev->enable_mes_kiq) ? 1 : 0);
777
+
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_RESET, 1);
813
778
data = REG_SET_FIELD(data, CP_MES_CNTL, MES_HALT, 1);
814
779
WREG32_SOC15(GC, 0, regCP_MES_CNTL, data);
815
780
}
···
823
790
824
791
mutex_lock(&adev->srbm_mutex);
825
792
for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) {
826
-
if ((!adev->enable_mes_kiq || adev->enable_uni_mes) &&
827
-
pipe == AMDGPU_MES_KIQ_PIPE)
828
-
continue;
829
-
830
793
/* me=3, queue=0 */
831
794
soc21_grbm_select(adev, 3, pipe, 0, 0);
832
795
···
1114
1085
return r;
1115
1086
}
1116
1087
1117
-
kiq->pmf->kiq_map_queues(kiq_ring, &adev->mes.ring);
1088
+
kiq->pmf->kiq_map_queues(kiq_ring, &adev->mes.ring[0]);
1118
1089
1119
1090
r = amdgpu_ring_test_ring(kiq_ring);
1120
1091
if (r) {
···
1130
1101
struct amdgpu_ring *ring;
1131
1102
int r;
1132
1103
1133
-
if (pipe == AMDGPU_MES_KIQ_PIPE)
1104
+
if (!adev->enable_uni_mes && pipe == AMDGPU_MES_KIQ_PIPE)
1134
1105
ring = &adev->gfx.kiq[0].ring;
1135
-
else if (pipe == AMDGPU_MES_SCHED_PIPE)
1136
-
ring = &adev->mes.ring;
1137
1106
else
1138
-
BUG();
1107
+
ring = &adev->mes.ring[pipe];
1139
1108
1140
-
if ((pipe == AMDGPU_MES_SCHED_PIPE) &&
1109
+
if ((adev->enable_uni_mes || pipe == AMDGPU_MES_SCHED_PIPE) &&
1141
1110
(amdgpu_in_reset(adev) || adev->in_suspend)) {
1142
1111
*(ring->wptr_cpu_addr) = 0;
1143
1112
*(ring->rptr_cpu_addr) = 0;
···
1147
1120
return r;
1148
1121
1149
1122
if (pipe == AMDGPU_MES_SCHED_PIPE) {
1150
-
if (adev->enable_uni_mes) {
1151
-
mes_v12_0_queue_init_register(ring);
1152
-
} else {
1123
+
if (adev->enable_uni_mes)
1124
+
r = amdgpu_mes_map_legacy_queue(adev, ring);
1125
+
else
1153
1126
r = mes_v12_0_kiq_enable_queue(adev);
1154
-
if (r)
1155
-
return r;
1156
-
}
1127
+
if (r)
1128
+
return r;
1157
1129
} else {
1158
1130
mes_v12_0_queue_init_register(ring);
1159
1131
}
···
1172
1146
return 0;
1173
1147
}
1174
1148
1175
-
static int mes_v12_0_ring_init(struct amdgpu_device *adev)
1149
+
static int mes_v12_0_ring_init(struct amdgpu_device *adev, int pipe)
1176
1150
{
1177
1151
struct amdgpu_ring *ring;
1178
1152
1179
-
ring = &adev->mes.ring;
1153
+
ring = &adev->mes.ring[pipe];
1180
1154
1181
1155
ring->funcs = &mes_v12_0_ring_funcs;
1182
1156
1183
1157
ring->me = 3;
1184
-
ring->pipe = 0;
1158
+
ring->pipe = pipe;
1185
1159
ring->queue = 0;
1186
1160
1187
1161
ring->ring_obj = NULL;
1188
1162
ring->use_doorbell = true;
1189
-
ring->doorbell_index = adev->doorbell_index.mes_ring0 << 1;
1190
-
ring->eop_gpu_addr = adev->mes.eop_gpu_addr[AMDGPU_MES_SCHED_PIPE];
1163
+
ring->eop_gpu_addr = adev->mes.eop_gpu_addr[pipe];
1191
1164
ring->no_scheduler = true;
1192
1165
sprintf(ring->name, "mes_%d.%d.%d", ring->me, ring->pipe, ring->queue);
1166
+
1167
+
if (pipe == AMDGPU_MES_SCHED_PIPE)
1168
+
ring->doorbell_index = adev->doorbell_index.mes_ring0 << 1;
1169
+
else
1170
+
ring->doorbell_index = adev->doorbell_index.mes_ring1 << 1;
1193
1171
1194
1172
return amdgpu_ring_init(adev, ring, 1024, NULL, 0,
1195
1173
AMDGPU_RING_PRIO_DEFAULT, NULL);
···
1208
1178
ring = &adev->gfx.kiq[0].ring;
1209
1179
1210
1180
ring->me = 3;
1211
-
ring->pipe = adev->enable_uni_mes ? 0 : 1;
1181
+
ring->pipe = 1;
1212
1182
ring->queue = 0;
1213
1183
1214
1184
ring->adev = NULL;
···
1230
1200
int r, mqd_size = sizeof(struct v12_compute_mqd);
1231
1201
struct amdgpu_ring *ring;
1232
1202
1233
-
if (pipe == AMDGPU_MES_KIQ_PIPE)
1203
+
if (!adev->enable_uni_mes && pipe == AMDGPU_MES_KIQ_PIPE)
1234
1204
ring = &adev->gfx.kiq[0].ring;
1235
-
else if (pipe == AMDGPU_MES_SCHED_PIPE)
1236
-
ring = &adev->mes.ring;
1237
1205
else
1238
-
BUG();
1206
+
ring = &adev->mes.ring[pipe];
1239
1207
1240
1208
if (ring->mqd_obj)
1241
1209
return 0;
···
1274
1246
return r;
1275
1247
1276
1248
for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) {
1277
-
if (!adev->enable_mes_kiq && pipe == AMDGPU_MES_KIQ_PIPE)
1278
-
continue;
1279
-
1280
1249
r = mes_v12_0_allocate_eop_buf(adev, pipe);
1281
1250
if (r)
1282
1251
return r;
···
1281
1256
r = mes_v12_0_mqd_sw_init(adev, pipe);
1282
1257
if (r)
1283
1258
return r;
1284
-
}
1285
1259
1286
-
if (adev->enable_mes_kiq) {
1287
-
r = mes_v12_0_kiq_ring_init(adev);
1260
+
if (!adev->enable_uni_mes && pipe == AMDGPU_MES_KIQ_PIPE)
1261
+
r = mes_v12_0_kiq_ring_init(adev);
1262
+
else
1263
+
r = mes_v12_0_ring_init(adev, pipe);
1288
1264
if (r)
1289
1265
return r;
1290
1266
}
1291
-
1292
-
r = mes_v12_0_ring_init(adev);
1293
-
if (r)
1294
-
return r;
1295
1267
1296
1268
return 0;
1297
1269
}
···
1298
1276
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
1299
1277
int pipe;
1300
1278
1301
-
amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
1302
-
amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs);
1303
-
1304
1279
for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) {
1305
1280
kfree(adev->mes.mqd_backup[pipe]);
1306
1281
···
1305
1286
&adev->mes.eop_gpu_addr[pipe],
1306
1287
NULL);
1307
1288
amdgpu_ucode_release(&adev->mes.fw[pipe]);
1289
+
1290
+
if (adev->enable_uni_mes || pipe == AMDGPU_MES_SCHED_PIPE) {
1291
+
amdgpu_bo_free_kernel(&adev->mes.ring[pipe].mqd_obj,
1292
+
&adev->mes.ring[pipe].mqd_gpu_addr,
1293
+
&adev->mes.ring[pipe].mqd_ptr);
1294
+
amdgpu_ring_fini(&adev->mes.ring[pipe]);
1295
+
}
1308
1296
}
1309
1297
1310
-
amdgpu_bo_free_kernel(&adev->gfx.kiq[0].ring.mqd_obj,
1311
-
&adev->gfx.kiq[0].ring.mqd_gpu_addr,
1312
-
&adev->gfx.kiq[0].ring.mqd_ptr);
1313
-
1314
-
amdgpu_bo_free_kernel(&adev->mes.ring.mqd_obj,
1315
-
&adev->mes.ring.mqd_gpu_addr,
1316
-
&adev->mes.ring.mqd_ptr);
1317
-
1318
-
amdgpu_ring_fini(&adev->gfx.kiq[0].ring);
1319
-
amdgpu_ring_fini(&adev->mes.ring);
1298
+
if (!adev->enable_uni_mes) {
1299
+
amdgpu_bo_free_kernel(&adev->gfx.kiq[0].ring.mqd_obj,
1300
+
&adev->gfx.kiq[0].ring.mqd_gpu_addr,
1301
+
&adev->gfx.kiq[0].ring.mqd_ptr);
1302
+
amdgpu_ring_fini(&adev->gfx.kiq[0].ring);
1303
+
}
1320
1304
1321
1305
if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) {
1322
1306
mes_v12_0_free_ucode_buffers(adev, AMDGPU_MES_KIQ_PIPE);
···
1363
1341
soc21_grbm_select(adev, 0, 0, 0, 0);
1364
1342
mutex_unlock(&adev->srbm_mutex);
1365
1343
1366
-
adev->mes.ring.sched.ready = false;
1344
+
adev->mes.ring[0].sched.ready = false;
1367
1345
}
1368
1346
1369
1347
static void mes_v12_0_kiq_setting(struct amdgpu_ring *ring)
···
1384
1362
{
1385
1363
int r = 0;
1386
1364
1387
-
mes_v12_0_kiq_setting(&adev->gfx.kiq[0].ring);
1388
-
1389
1365
if (adev->enable_uni_mes)
1390
-
return mes_v12_0_hw_init(adev);
1366
+
mes_v12_0_kiq_setting(&adev->mes.ring[AMDGPU_MES_KIQ_PIPE]);
1367
+
else
1368
+
mes_v12_0_kiq_setting(&adev->gfx.kiq[0].ring);
1391
1369
1392
1370
if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) {
1393
1371
···
1414
1392
if (r)
1415
1393
goto failure;
1416
1394
1395
+
if (adev->enable_uni_mes) {
1396
+
r = mes_v12_0_set_hw_resources(&adev->mes, AMDGPU_MES_KIQ_PIPE);
1397
+
if (r)
1398
+
goto failure;
1399
+
1400
+
mes_v12_0_set_hw_resources_1(&adev->mes, AMDGPU_MES_KIQ_PIPE);
1401
+
}
1402
+
1417
1403
r = mes_v12_0_hw_init(adev);
1418
1404
if (r)
1419
1405
goto failure;
···
1435
1405
1436
1406
static int mes_v12_0_kiq_hw_fini(struct amdgpu_device *adev)
1437
1407
{
1438
-
if (adev->mes.ring.sched.ready) {
1439
-
mes_v12_0_kiq_dequeue_sched(adev);
1440
-
adev->mes.ring.sched.ready = false;
1408
+
if (adev->mes.ring[0].sched.ready) {
1409
+
if (adev->enable_uni_mes)
1410
+
amdgpu_mes_unmap_legacy_queue(adev,
1411
+
&adev->mes.ring[AMDGPU_MES_SCHED_PIPE],
1412
+
RESET_QUEUES, 0, 0);
1413
+
else
1414
+
mes_v12_0_kiq_dequeue_sched(adev);
1415
+
1416
+
adev->mes.ring[0].sched.ready = false;
1441
1417
}
1442
1418
1443
1419
mes_v12_0_enable(adev, false);
···
1456
1420
int r;
1457
1421
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
1458
1422
1459
-
if (adev->mes.ring.sched.ready)
1423
+
if (adev->mes.ring[0].sched.ready)
1460
1424
goto out;
1461
1425
1462
-
if (!adev->enable_mes_kiq || adev->enable_uni_mes) {
1426
+
if (!adev->enable_mes_kiq) {
1463
1427
if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) {
1464
1428
r = mes_v12_0_load_microcode(adev,
1465
1429
AMDGPU_MES_SCHED_PIPE, true);
···
1479
1443
mes_v12_0_enable(adev, true);
1480
1444
}
1481
1445
1446
+
/* Enable the MES to handle doorbell ring on unmapped queue */
1447
+
mes_v12_0_enable_unmapped_doorbell_handling(&adev->mes, true);
1448
+
1482
1449
r = mes_v12_0_queue_init(adev, AMDGPU_MES_SCHED_PIPE);
1483
1450
if (r)
1484
1451
goto failure;
1485
1452
1486
-
r = mes_v12_0_set_hw_resources(&adev->mes);
1453
+
r = mes_v12_0_set_hw_resources(&adev->mes, AMDGPU_MES_SCHED_PIPE);
1487
1454
if (r)
1488
1455
goto failure;
1489
1456
1490
1457
if (adev->enable_uni_mes)
1491
-
mes_v12_0_set_hw_resources_1(&adev->mes);
1458
+
mes_v12_0_set_hw_resources_1(&adev->mes, AMDGPU_MES_SCHED_PIPE);
1492
1459
1493
1460
mes_v12_0_init_aggregated_doorbell(&adev->mes);
1494
1461
1495
-
/* Enable the MES to handle doorbell ring on unmapped queue */
1496
-
mes_v12_0_enable_unmapped_doorbell_handling(&adev->mes, true);
1497
-
1498
-
r = mes_v12_0_query_sched_status(&adev->mes);
1462
+
r = mes_v12_0_query_sched_status(&adev->mes, AMDGPU_MES_SCHED_PIPE);
1499
1463
if (r) {
1500
1464
DRM_ERROR("MES is busy\n");
1501
1465
goto failure;
···
1508
1472
* with MES enabled.
1509
1473
*/
1510
1474
adev->gfx.kiq[0].ring.sched.ready = false;
1511
-
adev->mes.ring.sched.ready = true;
1475
+
adev->mes.ring[0].sched.ready = true;
1512
1476
1513
1477
return 0;
1514
1478
···
1551
1515
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
1552
1516
int pipe, r;
1553
1517
1554
-
if (adev->enable_uni_mes) {
1555
-
r = amdgpu_mes_init_microcode(adev, AMDGPU_MES_SCHED_PIPE);
1556
-
if (!r)
1557
-
return 0;
1558
-
1559
-
adev->enable_uni_mes = false;
1560
-
}
1561
-
1562
1518
for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) {
1563
-
if (!adev->enable_mes_kiq && pipe == AMDGPU_MES_KIQ_PIPE)
1564
-
continue;
1565
1519
r = amdgpu_mes_init_microcode(adev, pipe);
1566
1520
if (r)
1567
1521
return r;
+6
drivers/gpu/drm/amd/amdgpu/soc15d.h
+6
drivers/gpu/drm/amd/amdgpu/soc15d.h
···
76
76
((cond & 0xF) << 24) | \
77
77
((type & 0xF) << 28))
78
78
79
+
#define CP_PACKETJ_NOP 0x60000000
80
+
#define CP_PACKETJ_GET_REG(x) ((x) & 0x3FFFF)
81
+
#define CP_PACKETJ_GET_RES(x) (((x) >> 18) & 0x3F)
82
+
#define CP_PACKETJ_GET_COND(x) (((x) >> 24) & 0xF)
83
+
#define CP_PACKETJ_GET_TYPE(x) (((x) >> 28) & 0xF)
84
+
79
85
/* Packet 3 types */
80
86
#define PACKET3_NOP 0x10
81
87
#define PACKET3_SET_BASE 0x11
+2
drivers/gpu/drm/amd/amdgpu/soc24.c
+2
drivers/gpu/drm/amd/amdgpu/soc24.c
···
406
406
AMD_CG_SUPPORT_ATHUB_MGCG |
407
407
AMD_CG_SUPPORT_ATHUB_LS |
408
408
AMD_CG_SUPPORT_MC_MGCG |
409
+
AMD_CG_SUPPORT_HDP_SD |
409
410
AMD_CG_SUPPORT_MC_LS;
410
411
adev->pg_flags = AMD_PG_SUPPORT_VCN |
411
412
AMD_PG_SUPPORT_JPEG |
···
425
424
AMD_CG_SUPPORT_ATHUB_MGCG |
426
425
AMD_CG_SUPPORT_ATHUB_LS |
427
426
AMD_CG_SUPPORT_MC_MGCG |
427
+
AMD_CG_SUPPORT_HDP_SD |
428
428
AMD_CG_SUPPORT_MC_LS;
429
429
430
430
adev->pg_flags = AMD_PG_SUPPORT_VCN |
+3
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+3
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
···
2893
2893
2894
2894
hpd_rx_irq_work_suspend(dm);
2895
2895
2896
+
if (adev->dm.dc->caps.ips_support)
2897
+
dc_allow_idle_optimizations(adev->dm.dc, true);
2898
+
2896
2899
dc_set_power_state(dm->dc, DC_ACPI_CM_POWER_STATE_D3);
2897
2900
dc_dmub_srv_set_power_state(dm->dc->ctx->dmub_srv, DC_ACPI_CM_POWER_STATE_D3);
2898
2901
+24
-9
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
+24
-9
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
···
804
804
};
805
805
806
806
#if defined(CONFIG_DRM_AMD_DC_FP)
807
-
static int kbps_to_peak_pbn(int kbps)
807
+
static uint16_t get_fec_overhead_multiplier(struct dc_link *dc_link)
808
+
{
809
+
u8 link_coding_cap;
810
+
uint16_t fec_overhead_multiplier_x1000 = PBN_FEC_OVERHEAD_MULTIPLIER_8B_10B;
811
+
812
+
link_coding_cap = dc_link_dp_mst_decide_link_encoding_format(dc_link);
813
+
if (link_coding_cap == DP_128b_132b_ENCODING)
814
+
fec_overhead_multiplier_x1000 = PBN_FEC_OVERHEAD_MULTIPLIER_128B_132B;
815
+
816
+
return fec_overhead_multiplier_x1000;
817
+
}
818
+
819
+
static int kbps_to_peak_pbn(int kbps, uint16_t fec_overhead_multiplier_x1000)
808
820
{
809
821
u64 peak_kbps = kbps;
810
822
811
823
peak_kbps *= 1006;
812
-
peak_kbps = div_u64(peak_kbps, 1000);
824
+
peak_kbps *= fec_overhead_multiplier_x1000;
825
+
peak_kbps = div_u64(peak_kbps, 1000 * 1000);
813
826
return (int) DIV64_U64_ROUND_UP(peak_kbps * 64, (54 * 8 * 1000));
814
827
}
815
828
···
923
910
int link_timeslots_used;
924
911
int fair_pbn_alloc;
925
912
int ret = 0;
913
+
uint16_t fec_overhead_multiplier_x1000 = get_fec_overhead_multiplier(dc_link);
926
914
927
915
for (i = 0; i < count; i++) {
928
916
if (vars[i + k].dsc_enabled) {
929
917
initial_slack[i] =
930
-
kbps_to_peak_pbn(params[i].bw_range.max_kbps) - vars[i + k].pbn;
918
+
kbps_to_peak_pbn(params[i].bw_range.max_kbps, fec_overhead_multiplier_x1000) - vars[i + k].pbn;
931
919
bpp_increased[i] = false;
932
920
remaining_to_increase += 1;
933
921
} else {
···
1024
1010
int next_index;
1025
1011
int remaining_to_try = 0;
1026
1012
int ret;
1013
+
uint16_t fec_overhead_multiplier_x1000 = get_fec_overhead_multiplier(dc_link);
1027
1014
1028
1015
for (i = 0; i < count; i++) {
1029
1016
if (vars[i + k].dsc_enabled
···
1054
1039
if (next_index == -1)
1055
1040
break;
1056
1041
1057
-
vars[next_index].pbn = kbps_to_peak_pbn(params[next_index].bw_range.stream_kbps);
1042
+
vars[next_index].pbn = kbps_to_peak_pbn(params[next_index].bw_range.stream_kbps, fec_overhead_multiplier_x1000);
1058
1043
ret = drm_dp_atomic_find_time_slots(state,
1059
1044
params[next_index].port->mgr,
1060
1045
params[next_index].port,
···
1067
1052
vars[next_index].dsc_enabled = false;
1068
1053
vars[next_index].bpp_x16 = 0;
1069
1054
} else {
1070
-
vars[next_index].pbn = kbps_to_peak_pbn(
1071
-
params[next_index].bw_range.max_kbps);
1055
+
vars[next_index].pbn = kbps_to_peak_pbn(params[next_index].bw_range.stream_kbps, fec_overhead_multiplier_x1000);
1072
1056
ret = drm_dp_atomic_find_time_slots(state,
1073
1057
params[next_index].port->mgr,
1074
1058
params[next_index].port,
···
1096
1082
int count = 0;
1097
1083
int i, k, ret;
1098
1084
bool debugfs_overwrite = false;
1085
+
uint16_t fec_overhead_multiplier_x1000 = get_fec_overhead_multiplier(dc_link);
1099
1086
1100
1087
memset(params, 0, sizeof(params));
1101
1088
···
1161
1146
/* Try no compression */
1162
1147
for (i = 0; i < count; i++) {
1163
1148
vars[i + k].aconnector = params[i].aconnector;
1164
-
vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.stream_kbps);
1149
+
vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.stream_kbps, fec_overhead_multiplier_x1000);
1165
1150
vars[i + k].dsc_enabled = false;
1166
1151
vars[i + k].bpp_x16 = 0;
1167
1152
ret = drm_dp_atomic_find_time_slots(state, params[i].port->mgr, params[i].port,
···
1180
1165
/* Try max compression */
1181
1166
for (i = 0; i < count; i++) {
1182
1167
if (params[i].compression_possible && params[i].clock_force_enable != DSC_CLK_FORCE_DISABLE) {
1183
-
vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.min_kbps);
1168
+
vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.min_kbps, fec_overhead_multiplier_x1000);
1184
1169
vars[i + k].dsc_enabled = true;
1185
1170
vars[i + k].bpp_x16 = params[i].bw_range.min_target_bpp_x16;
1186
1171
ret = drm_dp_atomic_find_time_slots(state, params[i].port->mgr,
···
1188
1173
if (ret < 0)
1189
1174
return ret;
1190
1175
} else {
1191
-
vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.stream_kbps);
1176
+
vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.stream_kbps, fec_overhead_multiplier_x1000);
1192
1177
vars[i + k].dsc_enabled = false;
1193
1178
vars[i + k].bpp_x16 = 0;
1194
1179
ret = drm_dp_atomic_find_time_slots(state, params[i].port->mgr,
+3
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.h
+3
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.h
···
46
46
#define SYNAPTICS_CASCADED_HUB_ID 0x5A
47
47
#define IS_SYNAPTICS_CASCADED_PANAMERA(devName, data) ((IS_SYNAPTICS_PANAMERA(devName) && ((int)data[2] == SYNAPTICS_CASCADED_HUB_ID)) ? 1 : 0)
48
48
49
+
#define PBN_FEC_OVERHEAD_MULTIPLIER_8B_10B 1031
50
+
#define PBN_FEC_OVERHEAD_MULTIPLIER_128B_132B 1000
51
+
49
52
enum mst_msg_ready_type {
50
53
NONE_MSG_RDY_EVENT = 0,
51
54
DOWN_REP_MSG_RDY_EVENT = 1,
+2
-2
drivers/gpu/drm/amd/display/dc/hwss/dcn10/dcn10_hwseq.c
+2
-2
drivers/gpu/drm/amd/display/dc/hwss/dcn10/dcn10_hwseq.c
···
3589
3589
(int)hubp->curs_attr.width || pos_cpy.x
3590
3590
<= (int)hubp->curs_attr.width +
3591
3591
pipe_ctx->plane_state->src_rect.x) {
3592
-
pos_cpy.x = temp_x + viewport_width;
3592
+
pos_cpy.x = 2 * viewport_width - temp_x;
3593
3593
}
3594
3594
}
3595
3595
} else {
···
3682
3682
(int)hubp->curs_attr.width || pos_cpy.x
3683
3683
<= (int)hubp->curs_attr.width +
3684
3684
pipe_ctx->plane_state->src_rect.x) {
3685
-
pos_cpy.x = 2 * viewport_width - temp_x;
3685
+
pos_cpy.x = temp_x + viewport_width;
3686
3686
}
3687
3687
}
3688
3688
} else {
+3
drivers/gpu/drm/amd/display/dc/resource/dcn321/dcn321_resource.c
+3
drivers/gpu/drm/amd/display/dc/resource/dcn321/dcn321_resource.c
···
1778
1778
dc->caps.color.mpc.ogam_rom_caps.hlg = 0;
1779
1779
dc->caps.color.mpc.ocsc = 1;
1780
1780
1781
+
/* Use pipe context based otg sync logic */
1782
+
dc->config.use_pipe_ctx_sync_logic = true;
1783
+
1781
1784
dc->config.dc_mode_clk_limit_support = true;
1782
1785
dc->config.enable_windowed_mpo_odm = true;
1783
1786
/* read VBIOS LTTPR caps */
+6
-1
drivers/gpu/drm/amd/include/mes_v12_api_def.h
+6
-1
drivers/gpu/drm/amd/include/mes_v12_api_def.h
···
97
97
MES_QUEUE_TYPE_SDMA,
98
98
99
99
MES_QUEUE_TYPE_MAX,
100
+
MES_QUEUE_TYPE_SCHQ = MES_QUEUE_TYPE_MAX,
100
101
};
101
102
102
103
struct MES_API_STATUS {
···
243
242
uint32_t send_write_data : 1;
244
243
uint32_t os_tdr_timeout_override : 1;
245
244
uint32_t use_rs64mem_for_proc_gang_ctx : 1;
245
+
uint32_t halt_on_misaligned_access : 1;
246
+
uint32_t use_add_queue_unmap_flag_addr : 1;
247
+
uint32_t enable_mes_sch_stb_log : 1;
248
+
uint32_t limit_single_process : 1;
246
249
uint32_t unmapped_doorbell_handling: 2;
247
-
uint32_t reserved : 15;
250
+
uint32_t reserved : 11;
248
251
};
249
252
uint32_t uint32_all;
250
253
};
+12
drivers/gpu/drm/drm_panel_orientation_quirks.c
+12
drivers/gpu/drm/drm_panel_orientation_quirks.c
···
208
208
DMI_MATCH(DMI_BOARD_NAME, "KUN"),
209
209
},
210
210
.driver_data = (void *)&lcd1600x2560_rightside_up,
211
+
}, { /* AYN Loki Max */
212
+
.matches = {
213
+
DMI_EXACT_MATCH(DMI_SYS_VENDOR, "ayn"),
214
+
DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "Loki Max"),
215
+
},
216
+
.driver_data = (void *)&lcd1080x1920_leftside_up,
217
+
}, { /* AYN Loki Zero */
218
+
.matches = {
219
+
DMI_EXACT_MATCH(DMI_SYS_VENDOR, "ayn"),
220
+
DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "Loki Zero"),
221
+
},
222
+
.driver_data = (void *)&lcd1080x1920_leftside_up,
211
223
}, { /* Chuwi HiBook (CWI514) */
212
224
.matches = {
213
225
DMI_MATCH(DMI_BOARD_VENDOR, "Hampoo"),
+2
-2
drivers/gpu/drm/mediatek/mtk_drm_drv.c
+2
-2
drivers/gpu/drm/mediatek/mtk_drm_drv.c
···
539
539
}
540
540
541
541
/* IGT will check if the cursor size is configured */
542
-
drm->mode_config.cursor_width = drm->mode_config.max_width;
543
-
drm->mode_config.cursor_height = drm->mode_config.max_height;
542
+
drm->mode_config.cursor_width = 512;
543
+
drm->mode_config.cursor_height = 512;
544
544
545
545
/* Use OVL device for all DMA memory allocations */
546
546
crtc = drm_crtc_from_index(drm, 0);
+1
-3
drivers/gpu/drm/rockchip/inno_hdmi.c
+1
-3
drivers/gpu/drm/rockchip/inno_hdmi.c
···
279
279
const u8 *buffer, size_t len)
280
280
{
281
281
struct inno_hdmi *hdmi = connector_to_inno_hdmi(connector);
282
-
u8 packed_frame[HDMI_MAXIMUM_INFO_FRAME_SIZE];
283
282
ssize_t i;
284
283
285
284
if (type != HDMI_INFOFRAME_TYPE_AVI) {
···
290
291
inno_hdmi_disable_frame(connector, type);
291
292
292
293
for (i = 0; i < len; i++)
293
-
hdmi_writeb(hdmi, HDMI_CONTROL_PACKET_ADDR + i,
294
-
packed_frame[i]);
294
+
hdmi_writeb(hdmi, HDMI_CONTROL_PACKET_ADDR + i, buffer[i]);
295
295
296
296
return 0;
297
297
}
+11
-3
drivers/gpu/drm/v3d/v3d_sched.c
+11
-3
drivers/gpu/drm/v3d/v3d_sched.c
···
315
315
struct v3d_dev *v3d = job->base.v3d;
316
316
struct drm_device *dev = &v3d->drm;
317
317
struct dma_fence *fence;
318
-
int i, csd_cfg0_reg, csd_cfg_reg_count;
318
+
int i, csd_cfg0_reg;
319
319
320
320
v3d->csd_job = job;
321
321
···
335
335
v3d_switch_perfmon(v3d, &job->base);
336
336
337
337
csd_cfg0_reg = V3D_CSD_QUEUED_CFG0(v3d->ver);
338
-
csd_cfg_reg_count = v3d->ver < 71 ? 6 : 7;
339
-
for (i = 1; i <= csd_cfg_reg_count; i++)
338
+
for (i = 1; i <= 6; i++)
340
339
V3D_CORE_WRITE(0, csd_cfg0_reg + 4 * i, job->args.cfg[i]);
340
+
341
+
/* Although V3D 7.1 has an eighth configuration register, we are not
342
+
* using it. Therefore, make sure it remains unused.
343
+
*
344
+
* XXX: Set the CFG7 register
345
+
*/
346
+
if (v3d->ver >= 71)
347
+
V3D_CORE_WRITE(0, V3D_V7_CSD_QUEUED_CFG7, 0);
348
+
341
349
/* CFG0 write kicks off the job. */
342
350
V3D_CORE_WRITE(0, csd_cfg0_reg, job->args.cfg[0]);
343
351
+50
-9
drivers/gpu/drm/xe/xe_device.c
+50
-9
drivers/gpu/drm/xe/xe_device.c
···
87
87
spin_unlock(&xe->clients.lock);
88
88
89
89
file->driver_priv = xef;
90
+
kref_init(&xef->refcount);
91
+
90
92
return 0;
93
+
}
94
+
95
+
static void xe_file_destroy(struct kref *ref)
96
+
{
97
+
struct xe_file *xef = container_of(ref, struct xe_file, refcount);
98
+
struct xe_device *xe = xef->xe;
99
+
100
+
xa_destroy(&xef->exec_queue.xa);
101
+
mutex_destroy(&xef->exec_queue.lock);
102
+
xa_destroy(&xef->vm.xa);
103
+
mutex_destroy(&xef->vm.lock);
104
+
105
+
spin_lock(&xe->clients.lock);
106
+
xe->clients.count--;
107
+
spin_unlock(&xe->clients.lock);
108
+
109
+
xe_drm_client_put(xef->client);
110
+
kfree(xef);
111
+
}
112
+
113
+
/**
114
+
* xe_file_get() - Take a reference to the xe file object
115
+
* @xef: Pointer to the xe file
116
+
*
117
+
* Anyone with a pointer to xef must take a reference to the xe file
118
+
* object using this call.
119
+
*
120
+
* Return: xe file pointer
121
+
*/
122
+
struct xe_file *xe_file_get(struct xe_file *xef)
123
+
{
124
+
kref_get(&xef->refcount);
125
+
return xef;
126
+
}
127
+
128
+
/**
129
+
* xe_file_put() - Drop a reference to the xe file object
130
+
* @xef: Pointer to the xe file
131
+
*
132
+
* Used to drop reference to the xef object
133
+
*/
134
+
void xe_file_put(struct xe_file *xef)
135
+
{
136
+
kref_put(&xef->refcount, xe_file_destroy);
91
137
}
92
138
93
139
static void xe_file_close(struct drm_device *dev, struct drm_file *file)
···
143
97
struct xe_vm *vm;
144
98
struct xe_exec_queue *q;
145
99
unsigned long idx;
100
+
101
+
xe_pm_runtime_get(xe);
146
102
147
103
/*
148
104
* No need for exec_queue.lock here as there is no contention for it
···
156
108
xe_exec_queue_kill(q);
157
109
xe_exec_queue_put(q);
158
110
}
159
-
xa_destroy(&xef->exec_queue.xa);
160
-
mutex_destroy(&xef->exec_queue.lock);
161
111
mutex_lock(&xef->vm.lock);
162
112
xa_for_each(&xef->vm.xa, idx, vm)
163
113
xe_vm_close_and_put(vm);
164
114
mutex_unlock(&xef->vm.lock);
165
-
xa_destroy(&xef->vm.xa);
166
-
mutex_destroy(&xef->vm.lock);
167
115
168
-
spin_lock(&xe->clients.lock);
169
-
xe->clients.count--;
170
-
spin_unlock(&xe->clients.lock);
116
+
xe_file_put(xef);
171
117
172
-
xe_drm_client_put(xef->client);
173
-
kfree(xef);
118
+
xe_pm_runtime_put(xe);
174
119
}
175
120
176
121
static const struct drm_ioctl_desc xe_ioctls[] = {
+3
drivers/gpu/drm/xe/xe_device.h
+3
drivers/gpu/drm/xe/xe_device.h
+3
drivers/gpu/drm/xe/xe_device_types.h
+3
drivers/gpu/drm/xe/xe_device_types.h
+1
-4
drivers/gpu/drm/xe/xe_drm_client.c
+1
-4
drivers/gpu/drm/xe/xe_drm_client.c
···
251
251
252
252
/* Accumulate all the exec queues from this client */
253
253
mutex_lock(&xef->exec_queue.lock);
254
-
xa_for_each(&xef->exec_queue.xa, i, q) {
254
+
xa_for_each(&xef->exec_queue.xa, i, q)
255
255
xe_exec_queue_update_run_ticks(q);
256
-
xef->run_ticks[q->class] += q->run_ticks - q->old_run_ticks;
257
-
q->old_run_ticks = q->run_ticks;
258
-
}
259
256
mutex_unlock(&xef->exec_queue.lock);
260
257
261
258
/* Get the total GPU cycles */
+9
-1
drivers/gpu/drm/xe/xe_exec_queue.c
+9
-1
drivers/gpu/drm/xe/xe_exec_queue.c
···
37
37
{
38
38
if (q->vm)
39
39
xe_vm_put(q->vm);
40
+
41
+
if (q->xef)
42
+
xe_file_put(q->xef);
43
+
40
44
kfree(q);
41
45
}
42
46
···
653
649
goto kill_exec_queue;
654
650
655
651
args->exec_queue_id = id;
652
+
q->xef = xe_file_get(xef);
656
653
657
654
return 0;
658
655
···
767
762
*/
768
763
void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q)
769
764
{
765
+
struct xe_file *xef;
770
766
struct xe_lrc *lrc;
771
767
u32 old_ts, new_ts;
772
768
···
779
773
if (!q->vm || !q->vm->xef)
780
774
return;
781
775
776
+
xef = q->vm->xef;
777
+
782
778
/*
783
779
* Only sample the first LRC. For parallel submission, all of them are
784
780
* scheduled together and we compensate that below by multiplying by
···
791
783
*/
792
784
lrc = q->lrc[0];
793
785
new_ts = xe_lrc_update_timestamp(lrc, &old_ts);
794
-
q->run_ticks += (new_ts - old_ts) * q->width;
786
+
xef->run_ticks[q->class] += (new_ts - old_ts) * q->width;
795
787
}
796
788
797
789
void xe_exec_queue_kill(struct xe_exec_queue *q)
+3
-4
drivers/gpu/drm/xe/xe_exec_queue_types.h
+3
-4
drivers/gpu/drm/xe/xe_exec_queue_types.h
···
38
38
* a kernel object.
39
39
*/
40
40
struct xe_exec_queue {
41
+
/** @xef: Back pointer to xe file if this is user created exec queue */
42
+
struct xe_file *xef;
43
+
41
44
/** @gt: graphics tile this exec queue can submit to */
42
45
struct xe_gt *gt;
43
46
/**
···
142
139
* Protected by @vm's resv. Unused if @vm == NULL.
143
140
*/
144
141
u64 tlb_flush_seqno;
145
-
/** @old_run_ticks: prior hw engine class run time in ticks for this exec queue */
146
-
u64 old_run_ticks;
147
-
/** @run_ticks: hw engine class run time in ticks for this exec queue */
148
-
u64 run_ticks;
149
142
/** @lrc: logical ring context for this exec queue */
150
143
struct xe_lrc *lrc[];
151
144
};
+8
-3
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c
+8
-3
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c
···
1927
1927
{
1928
1928
struct xe_gt *primary_gt = gt_to_tile(gt)->primary_gt;
1929
1929
struct xe_device *xe = gt_to_xe(gt);
1930
+
bool is_primary = !xe_gt_is_media_type(gt);
1930
1931
bool valid_ggtt, valid_ctxs, valid_dbs;
1931
1932
bool valid_any, valid_all;
1932
1933
···
1936
1935
valid_dbs = pf_get_vf_config_dbs(gt, vfid);
1937
1936
1938
1937
/* note that GuC doorbells are optional */
1939
-
valid_any = valid_ggtt || valid_ctxs || valid_dbs;
1940
-
valid_all = valid_ggtt && valid_ctxs;
1938
+
valid_any = valid_ctxs || valid_dbs;
1939
+
valid_all = valid_ctxs;
1940
+
1941
+
/* and GGTT/LMEM is configured on primary GT only */
1942
+
valid_all = valid_all && valid_ggtt;
1943
+
valid_any = valid_any || (valid_ggtt && is_primary);
1941
1944
1942
1945
if (IS_DGFX(xe)) {
1943
1946
bool valid_lmem = pf_get_vf_config_ggtt(primary_gt, vfid);
1944
1947
1945
-
valid_any = valid_any || valid_lmem;
1948
+
valid_any = valid_any || (valid_lmem && is_primary);
1946
1949
valid_all = valid_all && valid_lmem;
1947
1950
}
1948
1951
+1
-1
drivers/gpu/drm/xe/xe_gt_sriov_vf.c
+1
-1
drivers/gpu/drm/xe/xe_gt_sriov_vf.c
+108
-91
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c
+108
-91
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c
···
13
13
#include "xe_guc.h"
14
14
#include "xe_guc_ct.h"
15
15
#include "xe_mmio.h"
16
+
#include "xe_pm.h"
16
17
#include "xe_sriov.h"
17
18
#include "xe_trace.h"
18
19
#include "regs/xe_guc_regs.h"
20
+
21
+
#define FENCE_STACK_BIT DMA_FENCE_FLAG_USER_BITS
19
22
20
23
/*
21
24
* TLB inval depends on pending commands in the CT queue and then the real
···
36
33
return hw_tlb_timeout + 2 * delay;
37
34
}
38
35
36
+
static void
37
+
__invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fence *fence)
38
+
{
39
+
bool stack = test_bit(FENCE_STACK_BIT, &fence->base.flags);
40
+
41
+
trace_xe_gt_tlb_invalidation_fence_signal(xe, fence);
42
+
xe_gt_tlb_invalidation_fence_fini(fence);
43
+
dma_fence_signal(&fence->base);
44
+
if (!stack)
45
+
dma_fence_put(&fence->base);
46
+
}
47
+
48
+
static void
49
+
invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fence *fence)
50
+
{
51
+
list_del(&fence->link);
52
+
__invalidation_fence_signal(xe, fence);
53
+
}
39
54
40
55
static void xe_gt_tlb_fence_timeout(struct work_struct *work)
41
56
{
···
75
54
xe_gt_err(gt, "TLB invalidation fence timeout, seqno=%d recv=%d",
76
55
fence->seqno, gt->tlb_invalidation.seqno_recv);
77
56
78
-
list_del(&fence->link);
79
57
fence->base.error = -ETIME;
80
-
dma_fence_signal(&fence->base);
81
-
dma_fence_put(&fence->base);
58
+
invalidation_fence_signal(xe, fence);
82
59
}
83
60
if (!list_empty(>->tlb_invalidation.pending_fences))
84
61
queue_delayed_work(system_wq,
···
106
87
return 0;
107
88
}
108
89
109
-
static void
110
-
__invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fence *fence)
111
-
{
112
-
trace_xe_gt_tlb_invalidation_fence_signal(xe, fence);
113
-
dma_fence_signal(&fence->base);
114
-
dma_fence_put(&fence->base);
115
-
}
116
-
117
-
static void
118
-
invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fence *fence)
119
-
{
120
-
list_del(&fence->link);
121
-
__invalidation_fence_signal(xe, fence);
122
-
}
123
-
124
90
/**
125
91
* xe_gt_tlb_invalidation_reset - Initialize GT TLB invalidation reset
126
92
* @gt: graphics tile
···
115
111
void xe_gt_tlb_invalidation_reset(struct xe_gt *gt)
116
112
{
117
113
struct xe_gt_tlb_invalidation_fence *fence, *next;
118
-
struct xe_guc *guc = >->uc.guc;
119
114
int pending_seqno;
120
115
121
116
/*
···
137
134
else
138
135
pending_seqno = gt->tlb_invalidation.seqno - 1;
139
136
WRITE_ONCE(gt->tlb_invalidation.seqno_recv, pending_seqno);
140
-
wake_up_all(&guc->ct.wq);
141
137
142
138
list_for_each_entry_safe(fence, next,
143
139
>->tlb_invalidation.pending_fences, link)
···
167
165
int seqno;
168
166
int ret;
169
167
168
+
xe_gt_assert(gt, fence);
169
+
170
170
/*
171
171
* XXX: The seqno algorithm relies on TLB invalidation being processed
172
172
* in order which they currently are, if that changes the algorithm will
···
177
173
178
174
mutex_lock(&guc->ct.lock);
179
175
seqno = gt->tlb_invalidation.seqno;
180
-
if (fence) {
181
-
fence->seqno = seqno;
182
-
trace_xe_gt_tlb_invalidation_fence_send(xe, fence);
183
-
}
176
+
fence->seqno = seqno;
177
+
trace_xe_gt_tlb_invalidation_fence_send(xe, fence);
184
178
action[1] = seqno;
185
179
ret = xe_guc_ct_send_locked(&guc->ct, action, len,
186
180
G2H_LEN_DW_TLB_INVALIDATE, 1);
···
211
209
TLB_INVALIDATION_SEQNO_MAX;
212
210
if (!gt->tlb_invalidation.seqno)
213
211
gt->tlb_invalidation.seqno = 1;
214
-
ret = seqno;
215
212
}
216
213
mutex_unlock(&guc->ct.lock);
217
214
···
224
223
/**
225
224
* xe_gt_tlb_invalidation_guc - Issue a TLB invalidation on this GT for the GuC
226
225
* @gt: graphics tile
226
+
* @fence: invalidation fence which will be signal on TLB invalidation
227
+
* completion
227
228
*
228
229
* Issue a TLB invalidation for the GuC. Completion of TLB is asynchronous and
229
-
* caller can use seqno + xe_gt_tlb_invalidation_wait to wait for completion.
230
+
* caller can use the invalidation fence to wait for completion.
230
231
*
231
-
* Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success,
232
-
* negative error code on error.
232
+
* Return: 0 on success, negative error code on error
233
233
*/
234
-
static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt)
234
+
static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt,
235
+
struct xe_gt_tlb_invalidation_fence *fence)
235
236
{
236
237
u32 action[] = {
237
238
XE_GUC_ACTION_TLB_INVALIDATION,
···
241
238
MAKE_INVAL_OP(XE_GUC_TLB_INVAL_GUC),
242
239
};
243
240
244
-
return send_tlb_invalidation(>->uc.guc, NULL, action,
241
+
return send_tlb_invalidation(>->uc.guc, fence, action,
245
242
ARRAY_SIZE(action));
246
243
}
247
244
···
260
257
261
258
if (xe_guc_ct_enabled(>->uc.guc.ct) &&
262
259
gt->uc.guc.submission_state.enabled) {
263
-
int seqno;
260
+
struct xe_gt_tlb_invalidation_fence fence;
261
+
int ret;
264
262
265
-
seqno = xe_gt_tlb_invalidation_guc(gt);
266
-
if (seqno <= 0)
267
-
return seqno;
263
+
xe_gt_tlb_invalidation_fence_init(gt, &fence, true);
264
+
ret = xe_gt_tlb_invalidation_guc(gt, &fence);
265
+
if (ret < 0) {
266
+
xe_gt_tlb_invalidation_fence_fini(&fence);
267
+
return ret;
268
+
}
268
269
269
-
xe_gt_tlb_invalidation_wait(gt, seqno);
270
+
xe_gt_tlb_invalidation_fence_wait(&fence);
270
271
} else if (xe_device_uc_enabled(xe) && !xe_device_wedged(xe)) {
271
272
if (IS_SRIOV_VF(xe))
272
273
return 0;
···
297
290
*
298
291
* @gt: graphics tile
299
292
* @fence: invalidation fence which will be signal on TLB invalidation
300
-
* completion, can be NULL
293
+
* completion
301
294
* @start: start address
302
295
* @end: end address
303
296
* @asid: address space id
304
297
*
305
298
* Issue a range based TLB invalidation if supported, if not fallback to a full
306
-
* TLB invalidation. Completion of TLB is asynchronous and caller can either use
307
-
* the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for
308
-
* completion.
299
+
* TLB invalidation. Completion of TLB is asynchronous and caller can use
300
+
* the invalidation fence to wait for completion.
309
301
*
310
-
* Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success,
311
-
* negative error code on error.
302
+
* Return: Negative error code on error, 0 on success
312
303
*/
313
304
int xe_gt_tlb_invalidation_range(struct xe_gt *gt,
314
305
struct xe_gt_tlb_invalidation_fence *fence,
···
317
312
u32 action[MAX_TLB_INVALIDATION_LEN];
318
313
int len = 0;
319
314
315
+
xe_gt_assert(gt, fence);
316
+
320
317
/* Execlists not supported */
321
318
if (gt_to_xe(gt)->info.force_execlist) {
322
-
if (fence)
323
-
__invalidation_fence_signal(xe, fence);
324
-
319
+
__invalidation_fence_signal(xe, fence);
325
320
return 0;
326
321
}
327
322
···
387
382
* @vma: VMA to invalidate
388
383
*
389
384
* Issue a range based TLB invalidation if supported, if not fallback to a full
390
-
* TLB invalidation. Completion of TLB is asynchronous and caller can either use
391
-
* the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for
392
-
* completion.
385
+
* TLB invalidation. Completion of TLB is asynchronous and caller can use
386
+
* the invalidation fence to wait for completion.
393
387
*
394
-
* Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success,
395
-
* negative error code on error.
388
+
* Return: Negative error code on error, 0 on success
396
389
*/
397
390
int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,
398
391
struct xe_gt_tlb_invalidation_fence *fence,
···
401
398
return xe_gt_tlb_invalidation_range(gt, fence, xe_vma_start(vma),
402
399
xe_vma_end(vma),
403
400
xe_vma_vm(vma)->usm.asid);
404
-
}
405
-
406
-
/**
407
-
* xe_gt_tlb_invalidation_wait - Wait for TLB to complete
408
-
* @gt: graphics tile
409
-
* @seqno: seqno to wait which was returned from xe_gt_tlb_invalidation
410
-
*
411
-
* Wait for tlb_timeout_jiffies() for a TLB invalidation to complete.
412
-
*
413
-
* Return: 0 on success, -ETIME on TLB invalidation timeout
414
-
*/
415
-
int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno)
416
-
{
417
-
struct xe_guc *guc = >->uc.guc;
418
-
int ret;
419
-
420
-
/* Execlists not supported */
421
-
if (gt_to_xe(gt)->info.force_execlist)
422
-
return 0;
423
-
424
-
/*
425
-
* XXX: See above, this algorithm only works if seqno are always in
426
-
* order
427
-
*/
428
-
ret = wait_event_timeout(guc->ct.wq,
429
-
tlb_invalidation_seqno_past(gt, seqno),
430
-
tlb_timeout_jiffies(gt));
431
-
if (!ret) {
432
-
struct drm_printer p = xe_gt_err_printer(gt);
433
-
434
-
xe_gt_err(gt, "TLB invalidation time'd out, seqno=%d, recv=%d\n",
435
-
seqno, gt->tlb_invalidation.seqno_recv);
436
-
xe_guc_ct_print(&guc->ct, &p, true);
437
-
return -ETIME;
438
-
}
439
-
440
-
return 0;
441
401
}
442
402
443
403
/**
···
446
480
return 0;
447
481
}
448
482
449
-
/*
450
-
* wake_up_all() and wait_event_timeout() already have the correct
451
-
* barriers.
452
-
*/
453
483
WRITE_ONCE(gt->tlb_invalidation.seqno_recv, msg[0]);
454
-
wake_up_all(&guc->ct.wq);
455
484
456
485
list_for_each_entry_safe(fence, next,
457
486
>->tlb_invalidation.pending_fences, link) {
···
468
507
spin_unlock_irqrestore(>->tlb_invalidation.pending_lock, flags);
469
508
470
509
return 0;
510
+
}
511
+
512
+
static const char *
513
+
invalidation_fence_get_driver_name(struct dma_fence *dma_fence)
514
+
{
515
+
return "xe";
516
+
}
517
+
518
+
static const char *
519
+
invalidation_fence_get_timeline_name(struct dma_fence *dma_fence)
520
+
{
521
+
return "invalidation_fence";
522
+
}
523
+
524
+
static const struct dma_fence_ops invalidation_fence_ops = {
525
+
.get_driver_name = invalidation_fence_get_driver_name,
526
+
.get_timeline_name = invalidation_fence_get_timeline_name,
527
+
};
528
+
529
+
/**
530
+
* xe_gt_tlb_invalidation_fence_init - Initialize TLB invalidation fence
531
+
* @gt: GT
532
+
* @fence: TLB invalidation fence to initialize
533
+
* @stack: fence is stack variable
534
+
*
535
+
* Initialize TLB invalidation fence for use. xe_gt_tlb_invalidation_fence_fini
536
+
* must be called if fence is not signaled.
537
+
*/
538
+
void xe_gt_tlb_invalidation_fence_init(struct xe_gt *gt,
539
+
struct xe_gt_tlb_invalidation_fence *fence,
540
+
bool stack)
541
+
{
542
+
xe_pm_runtime_get_noresume(gt_to_xe(gt));
543
+
544
+
spin_lock_irq(>->tlb_invalidation.lock);
545
+
dma_fence_init(&fence->base, &invalidation_fence_ops,
546
+
>->tlb_invalidation.lock,
547
+
dma_fence_context_alloc(1), 1);
548
+
spin_unlock_irq(>->tlb_invalidation.lock);
549
+
INIT_LIST_HEAD(&fence->link);
550
+
if (stack)
551
+
set_bit(FENCE_STACK_BIT, &fence->base.flags);
552
+
else
553
+
dma_fence_get(&fence->base);
554
+
fence->gt = gt;
555
+
}
556
+
557
+
/**
558
+
* xe_gt_tlb_invalidation_fence_fini - Finalize TLB invalidation fence
559
+
* @fence: TLB invalidation fence to finalize
560
+
*
561
+
* Drop PM ref which fence took durinig init.
562
+
*/
563
+
void xe_gt_tlb_invalidation_fence_fini(struct xe_gt_tlb_invalidation_fence *fence)
564
+
{
565
+
xe_pm_runtime_put(gt_to_xe(fence->gt));
471
566
}
+11
-1
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h
+11
-1
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h
···
23
23
int xe_gt_tlb_invalidation_range(struct xe_gt *gt,
24
24
struct xe_gt_tlb_invalidation_fence *fence,
25
25
u64 start, u64 end, u32 asid);
26
-
int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno);
27
26
int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len);
27
+
28
+
void xe_gt_tlb_invalidation_fence_init(struct xe_gt *gt,
29
+
struct xe_gt_tlb_invalidation_fence *fence,
30
+
bool stack);
31
+
void xe_gt_tlb_invalidation_fence_fini(struct xe_gt_tlb_invalidation_fence *fence);
32
+
33
+
static inline void
34
+
xe_gt_tlb_invalidation_fence_wait(struct xe_gt_tlb_invalidation_fence *fence)
35
+
{
36
+
dma_fence_wait(&fence->base, false);
37
+
}
28
38
29
39
#endif /* _XE_GT_TLB_INVALIDATION_ */
+4
drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h
+4
drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h
···
8
8
9
9
#include <linux/dma-fence.h>
10
10
11
+
struct xe_gt;
12
+
11
13
/**
12
14
* struct xe_gt_tlb_invalidation_fence - XE GT TLB invalidation fence
13
15
*
···
19
17
struct xe_gt_tlb_invalidation_fence {
20
18
/** @base: dma fence base */
21
19
struct dma_fence base;
20
+
/** @gt: GT which fence belong to */
21
+
struct xe_gt *gt;
22
22
/** @link: link into list of pending tlb fences */
23
23
struct list_head link;
24
24
/** @seqno: seqno of TLB invalidation to signal fence one */
+9
-1
drivers/gpu/drm/xe/xe_guc_ct.c
+9
-1
drivers/gpu/drm/xe/xe_guc_ct.c
···
327
327
xe_gt_assert(ct_to_gt(ct), ct->g2h_outstanding == 0 ||
328
328
state == XE_GUC_CT_STATE_STOPPED);
329
329
330
+
if (ct->g2h_outstanding)
331
+
xe_pm_runtime_put(ct_to_xe(ct));
330
332
ct->g2h_outstanding = 0;
331
333
ct->state = state;
332
334
···
497
495
static void __g2h_reserve_space(struct xe_guc_ct *ct, u32 g2h_len, u32 num_g2h)
498
496
{
499
497
xe_gt_assert(ct_to_gt(ct), g2h_len <= ct->ctbs.g2h.info.space);
498
+
xe_gt_assert(ct_to_gt(ct), (!g2h_len && !num_g2h) ||
499
+
(g2h_len && num_g2h));
500
500
501
501
if (g2h_len) {
502
502
lockdep_assert_held(&ct->fast_lock);
503
+
504
+
if (!ct->g2h_outstanding)
505
+
xe_pm_runtime_get_noresume(ct_to_xe(ct));
503
506
504
507
ct->ctbs.g2h.info.space -= g2h_len;
505
508
ct->g2h_outstanding += num_g2h;
···
518
511
ct->ctbs.g2h.info.size - ct->ctbs.g2h.info.resv_space);
519
512
520
513
ct->ctbs.g2h.info.space += g2h_len;
521
-
--ct->g2h_outstanding;
514
+
if (!--ct->g2h_outstanding)
515
+
xe_pm_runtime_put(ct_to_xe(ct));
522
516
}
523
517
524
518
static void g2h_release_space(struct xe_guc_ct *ct, u32 g2h_len)
+4
drivers/gpu/drm/xe/xe_guc_submit.c
+4
drivers/gpu/drm/xe/xe_guc_submit.c
···
1393
1393
default:
1394
1394
XE_WARN_ON("Unknown message type");
1395
1395
}
1396
+
1397
+
xe_pm_runtime_put(guc_to_xe(exec_queue_to_guc(msg->private_data)));
1396
1398
}
1397
1399
1398
1400
static const struct drm_sched_backend_ops drm_sched_ops = {
···
1484
1482
static void guc_exec_queue_add_msg(struct xe_exec_queue *q, struct xe_sched_msg *msg,
1485
1483
u32 opcode)
1486
1484
{
1485
+
xe_pm_runtime_get_noresume(guc_to_xe(exec_queue_to_guc(q)));
1486
+
1487
1487
INIT_LIST_HEAD(&msg->link);
1488
1488
msg->opcode = opcode;
1489
1489
msg->private_data = q;
+1
-25
drivers/gpu/drm/xe/xe_pt.c
+1
-25
drivers/gpu/drm/xe/xe_pt.c
···
1115
1115
u32 asid;
1116
1116
};
1117
1117
1118
-
static const char *
1119
-
invalidation_fence_get_driver_name(struct dma_fence *dma_fence)
1120
-
{
1121
-
return "xe";
1122
-
}
1123
-
1124
-
static const char *
1125
-
invalidation_fence_get_timeline_name(struct dma_fence *dma_fence)
1126
-
{
1127
-
return "invalidation_fence";
1128
-
}
1129
-
1130
-
static const struct dma_fence_ops invalidation_fence_ops = {
1131
-
.get_driver_name = invalidation_fence_get_driver_name,
1132
-
.get_timeline_name = invalidation_fence_get_timeline_name,
1133
-
};
1134
-
1135
1118
static void invalidation_fence_cb(struct dma_fence *fence,
1136
1119
struct dma_fence_cb *cb)
1137
1120
{
···
1153
1170
1154
1171
trace_xe_gt_tlb_invalidation_fence_create(gt_to_xe(gt), &ifence->base);
1155
1172
1156
-
spin_lock_irq(>->tlb_invalidation.lock);
1157
-
dma_fence_init(&ifence->base.base, &invalidation_fence_ops,
1158
-
>->tlb_invalidation.lock,
1159
-
dma_fence_context_alloc(1), 1);
1160
-
spin_unlock_irq(>->tlb_invalidation.lock);
1173
+
xe_gt_tlb_invalidation_fence_init(gt, &ifence->base, false);
1161
1174
1162
-
INIT_LIST_HEAD(&ifence->base.link);
1163
-
1164
-
dma_fence_get(&ifence->base.base); /* Ref for caller */
1165
1175
ifence->fence = fence;
1166
1176
ifence->gt = gt;
1167
1177
ifence->start = start;
+8
-4
drivers/gpu/drm/xe/xe_sync.c
+8
-4
drivers/gpu/drm/xe/xe_sync.c
···
53
53
u64 value)
54
54
{
55
55
struct xe_user_fence *ufence;
56
+
u64 __user *ptr = u64_to_user_ptr(addr);
57
+
58
+
if (!access_ok(ptr, sizeof(ptr)))
59
+
return ERR_PTR(-EFAULT);
56
60
57
61
ufence = kmalloc(sizeof(*ufence), GFP_KERNEL);
58
62
if (!ufence)
59
-
return NULL;
63
+
return ERR_PTR(-ENOMEM);
60
64
61
65
ufence->xe = xe;
62
66
kref_init(&ufence->refcount);
63
-
ufence->addr = u64_to_user_ptr(addr);
67
+
ufence->addr = ptr;
64
68
ufence->value = value;
65
69
ufence->mm = current->mm;
66
70
mmgrab(ufence->mm);
···
187
183
} else {
188
184
sync->ufence = user_fence_create(xe, sync_in.addr,
189
185
sync_in.timeline_value);
190
-
if (XE_IOCTL_DBG(xe, !sync->ufence))
191
-
return -ENOMEM;
186
+
if (XE_IOCTL_DBG(xe, IS_ERR(sync->ufence)))
187
+
return PTR_ERR(sync->ufence);
192
188
}
193
189
194
190
break;
+23
-15
drivers/gpu/drm/xe/xe_vm.c
+23
-15
drivers/gpu/drm/xe/xe_vm.c
···
1601
1601
XE_WARN_ON(vm->pt_root[id]);
1602
1602
1603
1603
trace_xe_vm_free(vm);
1604
+
1605
+
if (vm->xef)
1606
+
xe_file_put(vm->xef);
1607
+
1604
1608
kfree(vm);
1605
1609
}
1606
1610
···
1920
1916
}
1921
1917
1922
1918
args->vm_id = id;
1923
-
vm->xef = xef;
1919
+
vm->xef = xe_file_get(xef);
1924
1920
1925
1921
/* Record BO memory for VM pagetable created against client */
1926
1922
for_each_tile(tile, xe, id)
···
3341
3337
{
3342
3338
struct xe_device *xe = xe_vma_vm(vma)->xe;
3343
3339
struct xe_tile *tile;
3340
+
struct xe_gt_tlb_invalidation_fence fence[XE_MAX_TILES_PER_DEVICE];
3344
3341
u32 tile_needs_invalidate = 0;
3345
-
int seqno[XE_MAX_TILES_PER_DEVICE];
3346
3342
u8 id;
3347
-
int ret;
3343
+
int ret = 0;
3348
3344
3349
3345
xe_assert(xe, !xe_vma_is_null(vma));
3350
3346
trace_xe_vma_invalidate(vma);
···
3369
3365
3370
3366
for_each_tile(tile, xe, id) {
3371
3367
if (xe_pt_zap_ptes(tile, vma)) {
3372
-
tile_needs_invalidate |= BIT(id);
3373
3368
xe_device_wmb(xe);
3369
+
xe_gt_tlb_invalidation_fence_init(tile->primary_gt,
3370
+
&fence[id], true);
3371
+
3374
3372
/*
3375
3373
* FIXME: We potentially need to invalidate multiple
3376
3374
* GTs within the tile
3377
3375
*/
3378
-
seqno[id] = xe_gt_tlb_invalidation_vma(tile->primary_gt, NULL, vma);
3379
-
if (seqno[id] < 0)
3380
-
return seqno[id];
3376
+
ret = xe_gt_tlb_invalidation_vma(tile->primary_gt,
3377
+
&fence[id], vma);
3378
+
if (ret < 0) {
3379
+
xe_gt_tlb_invalidation_fence_fini(&fence[id]);
3380
+
goto wait;
3381
+
}
3382
+
3383
+
tile_needs_invalidate |= BIT(id);
3381
3384
}
3382
3385
}
3383
3386
3384
-
for_each_tile(tile, xe, id) {
3385
-
if (tile_needs_invalidate & BIT(id)) {
3386
-
ret = xe_gt_tlb_invalidation_wait(tile->primary_gt, seqno[id]);
3387
-
if (ret < 0)
3388
-
return ret;
3389
-
}
3390
-
}
3387
+
wait:
3388
+
for_each_tile(tile, xe, id)
3389
+
if (tile_needs_invalidate & BIT(id))
3390
+
xe_gt_tlb_invalidation_fence_wait(&fence[id]);
3391
3391
3392
3392
vma->tile_invalidated = vma->tile_mask;
3393
3393
3394
-
return 0;
3394
+
return ret;
3395
3395
}
3396
3396
3397
3397
struct xe_vm_snapshot {
+3
-1
drivers/hid/amd-sfh-hid/amd_sfh_hid.c
+3
-1
drivers/hid/amd-sfh-hid/amd_sfh_hid.c
···
171
171
void amdtp_hid_remove(struct amdtp_cl_data *cli_data)
172
172
{
173
173
int i;
174
+
struct amdtp_hid_data *hid_data;
174
175
175
176
for (i = 0; i < cli_data->num_hid_devices; ++i) {
176
177
if (cli_data->hid_sensor_hubs[i]) {
177
-
kfree(cli_data->hid_sensor_hubs[i]->driver_data);
178
+
hid_data = cli_data->hid_sensor_hubs[i]->driver_data;
178
179
hid_destroy_device(cli_data->hid_sensor_hubs[i]);
180
+
kfree(hid_data);
179
181
cli_data->hid_sensor_hubs[i] = NULL;
180
182
}
181
183
}
+3
drivers/hid/hid-asus.c
+3
drivers/hid/hid-asus.c
···
1249
1249
USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY),
1250
1250
QUIRK_USE_KBD_BACKLIGHT | QUIRK_ROG_NKEY_KEYBOARD },
1251
1251
{ HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK,
1252
+
USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X),
1253
+
QUIRK_USE_KBD_BACKLIGHT | QUIRK_ROG_NKEY_KEYBOARD },
1254
+
{ HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK,
1252
1255
USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD),
1253
1256
QUIRK_ROG_CLAYMORE_II_KEYBOARD },
1254
1257
{ HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK,
+1
-1
drivers/hid/hid-cougar.c
+1
-1
drivers/hid/hid-cougar.c
···
106
106
static __u8 *cougar_report_fixup(struct hid_device *hdev, __u8 *rdesc,
107
107
unsigned int *rsize)
108
108
{
109
-
if (rdesc[2] == 0x09 && rdesc[3] == 0x02 &&
109
+
if (*rsize >= 117 && rdesc[2] == 0x09 && rdesc[3] == 0x02 &&
110
110
(rdesc[115] | rdesc[116] << 8) >= HID_MAX_USAGES) {
111
111
hid_info(hdev,
112
112
"usage count exceeds max: fixing up report descriptor\n");
+3
drivers/hid/hid-ids.h
+3
drivers/hid/hid-ids.h
···
210
210
#define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_KEYBOARD3 0x1a30
211
211
#define USB_DEVICE_ID_ASUSTEK_ROG_Z13_LIGHTBAR 0x18c6
212
212
#define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY 0x1abe
213
+
#define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X 0x1b4c
213
214
#define USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD 0x196b
214
215
#define USB_DEVICE_ID_ASUSTEK_FX503VD_KEYBOARD 0x1869
215
216
···
521
520
#define USB_DEVICE_ID_GENERAL_TOUCH_WIN8_PIT_E100 0xe100
522
521
523
522
#define I2C_VENDOR_ID_GOODIX 0x27c6
523
+
#define I2C_DEVICE_ID_GOODIX_01E8 0x01e8
524
+
#define I2C_DEVICE_ID_GOODIX_01E9 0x01e9
524
525
#define I2C_DEVICE_ID_GOODIX_01F0 0x01f0
525
526
526
527
#define USB_VENDOR_ID_GOODTOUCH 0x1aad
+33
drivers/hid/hid-multitouch.c
+33
drivers/hid/hid-multitouch.c
···
1441
1441
return 0;
1442
1442
}
1443
1443
1444
+
static __u8 *mt_report_fixup(struct hid_device *hdev, __u8 *rdesc,
1445
+
unsigned int *size)
1446
+
{
1447
+
if (hdev->vendor == I2C_VENDOR_ID_GOODIX &&
1448
+
(hdev->product == I2C_DEVICE_ID_GOODIX_01E8 ||
1449
+
hdev->product == I2C_DEVICE_ID_GOODIX_01E9)) {
1450
+
if (rdesc[607] == 0x15) {
1451
+
rdesc[607] = 0x25;
1452
+
dev_info(
1453
+
&hdev->dev,
1454
+
"GT7868Q report descriptor fixup is applied.\n");
1455
+
} else {
1456
+
dev_info(
1457
+
&hdev->dev,
1458
+
"The byte is not expected for fixing the report descriptor. \
1459
+
It's possible that the touchpad firmware is not suitable for applying the fix. \
1460
+
got: %x\n",
1461
+
rdesc[607]);
1462
+
}
1463
+
}
1464
+
1465
+
return rdesc;
1466
+
}
1467
+
1444
1468
static void mt_report(struct hid_device *hid, struct hid_report *report)
1445
1469
{
1446
1470
struct mt_device *td = hid_get_drvdata(hid);
···
2059
2035
MT_BT_DEVICE(USB_VENDOR_ID_FRUCTEL,
2060
2036
USB_DEVICE_ID_GAMETEL_MT_MODE) },
2061
2037
2038
+
/* Goodix GT7868Q devices */
2039
+
{ .driver_data = MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU,
2040
+
HID_DEVICE(BUS_I2C, HID_GROUP_ANY, I2C_VENDOR_ID_GOODIX,
2041
+
I2C_DEVICE_ID_GOODIX_01E8) },
2042
+
{ .driver_data = MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU,
2043
+
HID_DEVICE(BUS_I2C, HID_GROUP_ANY, I2C_VENDOR_ID_GOODIX,
2044
+
I2C_DEVICE_ID_GOODIX_01E8) },
2045
+
2062
2046
/* GoodTouch panels */
2063
2047
{ .driver_data = MT_CLS_NSMU,
2064
2048
MT_USB_DEVICE(USB_VENDOR_ID_GOODTOUCH,
···
2302
2270
.feature_mapping = mt_feature_mapping,
2303
2271
.usage_table = mt_grabbed_usages,
2304
2272
.event = mt_event,
2273
+
.report_fixup = mt_report_fixup,
2305
2274
.report = mt_report,
2306
2275
.suspend = pm_ptr(mt_suspend),
2307
2276
.reset_resume = pm_ptr(mt_reset_resume),
+3
-1
drivers/hid/wacom_wac.c
+3
-1
drivers/hid/wacom_wac.c
···
1878
1878
int fmax = field->logical_maximum;
1879
1879
unsigned int equivalent_usage = wacom_equivalent_usage(usage->hid);
1880
1880
int resolution_code = code;
1881
-
int resolution = hidinput_calc_abs_res(field, resolution_code);
1881
+
int resolution;
1882
1882
1883
1883
if (equivalent_usage == HID_DG_TWIST) {
1884
1884
resolution_code = ABS_RZ;
1885
1885
}
1886
+
1887
+
resolution = hidinput_calc_abs_res(field, resolution_code);
1886
1888
1887
1889
if (equivalent_usage == HID_GD_X) {
1888
1890
fmin += features->offset_left;
+3
-1
drivers/i2c/busses/i2c-qcom-geni.c
+3
-1
drivers/i2c/busses/i2c-qcom-geni.c
+2
-2
drivers/i2c/busses/i2c-tegra.c
+2
-2
drivers/i2c/busses/i2c-tegra.c
···
1802
1802
* domain.
1803
1803
*
1804
1804
* VI I2C device shouldn't be marked as IRQ-safe because VI I2C won't
1805
-
* be used for atomic transfers.
1805
+
* be used for atomic transfers. ACPI device is not IRQ safe also.
1806
1806
*/
1807
-
if (!IS_VI(i2c_dev))
1807
+
if (!IS_VI(i2c_dev) && !has_acpi_companion(i2c_dev->dev))
1808
1808
pm_runtime_irq_safe(i2c_dev->dev);
1809
1809
1810
1810
pm_runtime_enable(i2c_dev->dev);
+1
drivers/iommu/io-pgfault.c
+1
drivers/iommu/io-pgfault.c
+1
-1
drivers/iommu/iommufd/device.c
+1
-1
drivers/iommu/iommufd/device.c
+1
-1
drivers/iommu/iommufd/selftest.c
+1
-1
drivers/iommu/iommufd/selftest.c
+20
-2
drivers/md/dm-ioctl.c
+20
-2
drivers/md/dm-ioctl.c
···
1181
1181
suspend_flags &= ~DM_SUSPEND_LOCKFS_FLAG;
1182
1182
if (param->flags & DM_NOFLUSH_FLAG)
1183
1183
suspend_flags |= DM_SUSPEND_NOFLUSH_FLAG;
1184
-
if (!dm_suspended_md(md))
1185
-
dm_suspend(md, suspend_flags);
1184
+
if (!dm_suspended_md(md)) {
1185
+
r = dm_suspend(md, suspend_flags);
1186
+
if (r) {
1187
+
down_write(&_hash_lock);
1188
+
hc = dm_get_mdptr(md);
1189
+
if (hc && !hc->new_map) {
1190
+
hc->new_map = new_map;
1191
+
new_map = NULL;
1192
+
} else {
1193
+
r = -ENXIO;
1194
+
}
1195
+
up_write(&_hash_lock);
1196
+
if (new_map) {
1197
+
dm_sync_table(md);
1198
+
dm_table_destroy(new_map);
1199
+
}
1200
+
dm_put(md);
1201
+
return r;
1202
+
}
1203
+
}
1186
1204
1187
1205
old_size = dm_get_size(md);
1188
1206
old_map = dm_swap_table(md, new_map);
+2
-2
drivers/md/dm.c
+2
-2
drivers/md/dm.c
···
2737
2737
break;
2738
2738
2739
2739
if (signal_pending_state(task_state, current)) {
2740
-
r = -EINTR;
2740
+
r = -ERESTARTSYS;
2741
2741
break;
2742
2742
}
2743
2743
···
2762
2762
break;
2763
2763
2764
2764
if (signal_pending_state(task_state, current)) {
2765
-
r = -EINTR;
2765
+
r = -ERESTARTSYS;
2766
2766
break;
2767
2767
}
2768
2768
+2
-2
drivers/md/persistent-data/dm-space-map-metadata.c
+2
-2
drivers/md/persistent-data/dm-space-map-metadata.c
···
277
277
{
278
278
struct sm_metadata *smm = container_of(sm, struct sm_metadata, sm);
279
279
280
-
kfree(smm);
280
+
kvfree(smm);
281
281
}
282
282
283
283
static int sm_metadata_get_nr_blocks(struct dm_space_map *sm, dm_block_t *count)
···
772
772
{
773
773
struct sm_metadata *smm;
774
774
775
-
smm = kmalloc(sizeof(*smm), GFP_KERNEL);
775
+
smm = kvmalloc(sizeof(*smm), GFP_KERNEL);
776
776
if (!smm)
777
777
return ERR_PTR(-ENOMEM);
778
778
+10
-4
drivers/md/raid1.c
+10
-4
drivers/md/raid1.c
···
617
617
return -1;
618
618
}
619
619
620
+
static bool rdev_in_recovery(struct md_rdev *rdev, struct r1bio *r1_bio)
621
+
{
622
+
return !test_bit(In_sync, &rdev->flags) &&
623
+
rdev->recovery_offset < r1_bio->sector + r1_bio->sectors;
624
+
}
625
+
620
626
static int choose_bb_rdev(struct r1conf *conf, struct r1bio *r1_bio,
621
627
int *max_sectors)
622
628
{
···
641
635
642
636
rdev = conf->mirrors[disk].rdev;
643
637
if (!rdev || test_bit(Faulty, &rdev->flags) ||
638
+
rdev_in_recovery(rdev, r1_bio) ||
644
639
test_bit(WriteMostly, &rdev->flags))
645
640
continue;
646
641
···
680
673
681
674
rdev = conf->mirrors[disk].rdev;
682
675
if (!rdev || test_bit(Faulty, &rdev->flags) ||
683
-
!test_bit(WriteMostly, &rdev->flags))
676
+
!test_bit(WriteMostly, &rdev->flags) ||
677
+
rdev_in_recovery(rdev, r1_bio))
684
678
continue;
685
679
686
680
/* there are no bad blocks, we can use this disk */
···
741
733
if (!rdev || test_bit(Faulty, &rdev->flags))
742
734
return false;
743
735
744
-
/* still in recovery */
745
-
if (!test_bit(In_sync, &rdev->flags) &&
746
-
rdev->recovery_offset < r1_bio->sector + r1_bio->sectors)
736
+
if (rdev_in_recovery(rdev, r1_bio))
747
737
return false;
748
738
749
739
/* don't read from slow disk unless have to */
+3
-19
drivers/misc/fastrpc.c
+3
-19
drivers/misc/fastrpc.c
···
2085
2085
return err;
2086
2086
}
2087
2087
2088
-
static int is_attach_rejected(struct fastrpc_user *fl)
2089
-
{
2090
-
/* Check if the device node is non-secure */
2091
-
if (!fl->is_secure_dev) {
2092
-
dev_dbg(&fl->cctx->rpdev->dev, "untrusted app trying to attach to privileged DSP PD\n");
2093
-
return -EACCES;
2094
-
}
2095
-
return 0;
2096
-
}
2097
-
2098
2088
static long fastrpc_device_ioctl(struct file *file, unsigned int cmd,
2099
2089
unsigned long arg)
2100
2090
{
···
2097
2107
err = fastrpc_invoke(fl, argp);
2098
2108
break;
2099
2109
case FASTRPC_IOCTL_INIT_ATTACH:
2100
-
err = is_attach_rejected(fl);
2101
-
if (!err)
2102
-
err = fastrpc_init_attach(fl, ROOT_PD);
2110
+
err = fastrpc_init_attach(fl, ROOT_PD);
2103
2111
break;
2104
2112
case FASTRPC_IOCTL_INIT_ATTACH_SNS:
2105
-
err = is_attach_rejected(fl);
2106
-
if (!err)
2107
-
err = fastrpc_init_attach(fl, SENSORS_PD);
2113
+
err = fastrpc_init_attach(fl, SENSORS_PD);
2108
2114
break;
2109
2115
case FASTRPC_IOCTL_INIT_CREATE_STATIC:
2110
-
err = is_attach_rejected(fl);
2111
-
if (!err)
2112
-
err = fastrpc_init_create_static_process(fl, argp);
2116
+
err = fastrpc_init_create_static_process(fl, argp);
2113
2117
break;
2114
2118
case FASTRPC_IOCTL_INIT_CREATE:
2115
2119
err = fastrpc_init_create_process(fl, argp);
+16
drivers/misc/lkdtm/refcount.c
+16
drivers/misc/lkdtm/refcount.c
···
182
182
check_negative(&neg, 3);
183
183
}
184
184
185
+
/*
186
+
* A refcount_sub_and_test() by zero when the counter is at zero should act like
187
+
* refcount_sub_and_test() above when going negative.
188
+
*/
189
+
static void lkdtm_REFCOUNT_SUB_AND_TEST_ZERO(void)
190
+
{
191
+
refcount_t neg = REFCOUNT_INIT(0);
192
+
193
+
pr_info("attempting bad refcount_sub_and_test() at zero\n");
194
+
if (refcount_sub_and_test(0, &neg))
195
+
pr_warn("Weird: refcount_sub_and_test() reported zero\n");
196
+
197
+
check_negative(&neg, 0);
198
+
}
199
+
185
200
static void check_from_zero(refcount_t *ref)
186
201
{
187
202
switch (refcount_read(ref)) {
···
415
400
CRASHTYPE(REFCOUNT_DEC_NEGATIVE),
416
401
CRASHTYPE(REFCOUNT_DEC_AND_TEST_NEGATIVE),
417
402
CRASHTYPE(REFCOUNT_SUB_AND_TEST_NEGATIVE),
403
+
CRASHTYPE(REFCOUNT_SUB_AND_TEST_ZERO),
418
404
CRASHTYPE(REFCOUNT_INC_ZERO),
419
405
CRASHTYPE(REFCOUNT_ADD_ZERO),
420
406
CRASHTYPE(REFCOUNT_INC_SATURATED),
+8
-13
drivers/net/bonding/bond_main.c
+8
-13
drivers/net/bonding/bond_main.c
···
582
582
} else {
583
583
slave->dev->xfrmdev_ops->xdo_dev_state_delete(ipsec->xs);
584
584
}
585
-
ipsec->xs->xso.real_dev = NULL;
586
585
}
587
586
spin_unlock_bh(&bond->ipsec_lock);
588
587
rcu_read_unlock();
···
598
599
struct net_device *real_dev;
599
600
struct slave *curr_active;
600
601
struct bonding *bond;
601
-
int err;
602
+
bool ok = false;
602
603
603
604
bond = netdev_priv(bond_dev);
604
605
rcu_read_lock();
605
606
curr_active = rcu_dereference(bond->curr_active_slave);
607
+
if (!curr_active)
608
+
goto out;
606
609
real_dev = curr_active->dev;
607
610
608
-
if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) {
609
-
err = false;
611
+
if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP)
610
612
goto out;
611
-
}
612
613
613
-
if (!xs->xso.real_dev) {
614
-
err = false;
614
+
if (!xs->xso.real_dev)
615
615
goto out;
616
-
}
617
616
618
617
if (!real_dev->xfrmdev_ops ||
619
618
!real_dev->xfrmdev_ops->xdo_dev_offload_ok ||
620
-
netif_is_bond_master(real_dev)) {
621
-
err = false;
619
+
netif_is_bond_master(real_dev))
622
620
goto out;
623
-
}
624
621
625
-
err = real_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs);
622
+
ok = real_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs);
626
623
out:
627
624
rcu_read_unlock();
628
-
return err;
625
+
return ok;
629
626
}
630
627
631
628
static const struct xfrmdev_ops bond_xfrmdev_ops = {
+1
-1
drivers/net/bonding/bond_options.c
+1
-1
drivers/net/bonding/bond_options.c
···
936
936
/* check to see if we are clearing active */
937
937
if (!slave_dev) {
938
938
netdev_dbg(bond->dev, "Clearing current active slave\n");
939
-
RCU_INIT_POINTER(bond->curr_active_slave, NULL);
939
+
bond_change_active_slave(bond, NULL);
940
940
bond_select_active_slave(bond);
941
941
} else {
942
942
struct slave *old_active = rtnl_dereference(bond->curr_active_slave);
+1
-4
drivers/net/dsa/microchip/ksz_ptp.c
+1
-4
drivers/net/dsa/microchip/ksz_ptp.c
···
266
266
struct ksz_port *prt;
267
267
struct dsa_port *dp;
268
268
bool tag_en = false;
269
-
int ret;
270
269
271
270
dsa_switch_for_each_user_port(dp, dev->ds) {
272
271
prt = &dev->ports[dp->index];
···
276
277
}
277
278
278
279
if (tag_en) {
279
-
ret = ptp_schedule_worker(ptp_data->clock, 0);
280
-
if (ret)
281
-
return ret;
280
+
ptp_schedule_worker(ptp_data->clock, 0);
282
281
} else {
283
282
ptp_cancel_worker_sync(ptp_data->clock);
284
283
}
+2
-1
drivers/net/dsa/mv88e6xxx/global1_atu.c
+2
-1
drivers/net/dsa/mv88e6xxx/global1_atu.c
+120
-6
drivers/net/dsa/ocelot/felix.c
+120
-6
drivers/net/dsa/ocelot/felix.c
···
61
61
return cpu_dp->index;
62
62
}
63
63
64
+
/**
65
+
* felix_update_tag_8021q_rx_rule - Update VCAP ES0 tag_8021q rule after
66
+
* vlan_filtering change
67
+
* @outer_tagging_rule: Pointer to VCAP filter on which the update is performed
68
+
* @vlan_filtering: Current bridge VLAN filtering setting
69
+
*
70
+
* Source port identification for tag_8021q is done using VCAP ES0 rules on the
71
+
* CPU port(s). The ES0 tag B (inner tag from the packet) can be configured as
72
+
* either:
73
+
* - push_inner_tag=0: the inner tag is never pushed into the frame
74
+
* (and we lose info about the classified VLAN). This is
75
+
* good when the classified VLAN is a discardable quantity
76
+
* for the software RX path: it is either set to
77
+
* OCELOT_STANDALONE_PVID, or to
78
+
* ocelot_vlan_unaware_pvid(bridge).
79
+
* - push_inner_tag=1: the inner tag is always pushed. This is good when the
80
+
* classified VLAN is not a discardable quantity (the port
81
+
* is under a VLAN-aware bridge, and software needs to
82
+
* continue processing the packet in the same VLAN as the
83
+
* hardware).
84
+
* The point is that what is good for a VLAN-unaware port is not good for a
85
+
* VLAN-aware port, and vice versa. Thus, the RX tagging rules must be kept in
86
+
* sync with the VLAN filtering state of the port.
87
+
*/
88
+
static void
89
+
felix_update_tag_8021q_rx_rule(struct ocelot_vcap_filter *outer_tagging_rule,
90
+
bool vlan_filtering)
91
+
{
92
+
if (vlan_filtering)
93
+
outer_tagging_rule->action.push_inner_tag = OCELOT_ES0_TAG;
94
+
else
95
+
outer_tagging_rule->action.push_inner_tag = OCELOT_NO_ES0_TAG;
96
+
}
97
+
64
98
/* Set up VCAP ES0 rules for pushing a tag_8021q VLAN towards the CPU such that
65
99
* the tagger can perform RX source port identification.
66
100
*/
67
101
static int felix_tag_8021q_vlan_add_rx(struct dsa_switch *ds, int port,
68
-
int upstream, u16 vid)
102
+
int upstream, u16 vid,
103
+
bool vlan_filtering)
69
104
{
70
105
struct ocelot_vcap_filter *outer_tagging_rule;
71
106
struct ocelot *ocelot = ds->priv;
···
131
96
outer_tagging_rule->action.tag_a_tpid_sel = OCELOT_TAG_TPID_SEL_8021AD;
132
97
outer_tagging_rule->action.tag_a_vid_sel = 1;
133
98
outer_tagging_rule->action.vid_a_val = vid;
99
+
felix_update_tag_8021q_rx_rule(outer_tagging_rule, vlan_filtering);
100
+
outer_tagging_rule->action.tag_b_tpid_sel = OCELOT_TAG_TPID_SEL_8021Q;
101
+
/* Leave TAG_B_VID_SEL at 0 (Classified VID + VID_B_VAL). Since we also
102
+
* leave VID_B_VAL at 0, this makes ES0 tag B (the inner tag) equal to
103
+
* the classified VID, which we need to see in the DSA tagger's receive
104
+
* path. Note: the inner tag is only visible in the packet when pushed
105
+
* (push_inner_tag == OCELOT_ES0_TAG).
106
+
*/
134
107
135
108
err = ocelot_vcap_filter_add(ocelot, outer_tagging_rule, NULL);
136
109
if (err)
···
270
227
static int felix_tag_8021q_vlan_add(struct dsa_switch *ds, int port, u16 vid,
271
228
u16 flags)
272
229
{
230
+
struct dsa_port *dp = dsa_to_port(ds, port);
273
231
struct dsa_port *cpu_dp;
274
232
int err;
275
233
···
278
234
* membership, which we aren't. So we don't need to add any VCAP filter
279
235
* for the CPU port.
280
236
*/
281
-
if (!dsa_is_user_port(ds, port))
237
+
if (!dsa_port_is_user(dp))
282
238
return 0;
283
239
284
240
dsa_switch_for_each_cpu_port(cpu_dp, ds) {
285
-
err = felix_tag_8021q_vlan_add_rx(ds, port, cpu_dp->index, vid);
241
+
err = felix_tag_8021q_vlan_add_rx(ds, port, cpu_dp->index, vid,
242
+
dsa_port_is_vlan_filtering(dp));
286
243
if (err)
287
244
return err;
288
245
}
···
303
258
304
259
static int felix_tag_8021q_vlan_del(struct dsa_switch *ds, int port, u16 vid)
305
260
{
261
+
struct dsa_port *dp = dsa_to_port(ds, port);
306
262
struct dsa_port *cpu_dp;
307
263
int err;
308
264
309
-
if (!dsa_is_user_port(ds, port))
265
+
if (!dsa_port_is_user(dp))
310
266
return 0;
311
267
312
268
dsa_switch_for_each_cpu_port(cpu_dp, ds) {
···
324
278
325
279
del_tx_failed:
326
280
dsa_switch_for_each_cpu_port(cpu_dp, ds)
327
-
felix_tag_8021q_vlan_add_rx(ds, port, cpu_dp->index, vid);
281
+
felix_tag_8021q_vlan_add_rx(ds, port, cpu_dp->index, vid,
282
+
dsa_port_is_vlan_filtering(dp));
328
283
329
284
return err;
285
+
}
286
+
287
+
static int felix_update_tag_8021q_rx_rules(struct dsa_switch *ds, int port,
288
+
bool vlan_filtering)
289
+
{
290
+
struct ocelot_vcap_filter *outer_tagging_rule;
291
+
struct ocelot_vcap_block *block_vcap_es0;
292
+
struct ocelot *ocelot = ds->priv;
293
+
struct dsa_port *cpu_dp;
294
+
unsigned long cookie;
295
+
int err;
296
+
297
+
block_vcap_es0 = &ocelot->block[VCAP_ES0];
298
+
299
+
dsa_switch_for_each_cpu_port(cpu_dp, ds) {
300
+
cookie = OCELOT_VCAP_ES0_TAG_8021Q_RXVLAN(ocelot, port,
301
+
cpu_dp->index);
302
+
303
+
outer_tagging_rule = ocelot_vcap_block_find_filter_by_id(block_vcap_es0,
304
+
cookie, false);
305
+
306
+
felix_update_tag_8021q_rx_rule(outer_tagging_rule, vlan_filtering);
307
+
308
+
err = ocelot_vcap_filter_replace(ocelot, outer_tagging_rule);
309
+
if (err)
310
+
return err;
311
+
}
312
+
313
+
return 0;
330
314
}
331
315
332
316
static int felix_trap_get_cpu_port(struct dsa_switch *ds,
···
604
528
* so we need to be careful that there are no extra frames to be
605
529
* dequeued over MMIO, since we would never know to discard them.
606
530
*/
531
+
ocelot_lock_xtr_grp_bh(ocelot, 0);
607
532
ocelot_drain_cpu_queue(ocelot, 0);
533
+
ocelot_unlock_xtr_grp_bh(ocelot, 0);
534
+
535
+
/* Problem: when using push_inner_tag=1 for ES0 tag B, we lose info
536
+
* about whether the received packets were VLAN-tagged on the wire,
537
+
* since they are always tagged on egress towards the CPU port.
538
+
*
539
+
* Since using push_inner_tag=1 is unavoidable for VLAN-aware bridges,
540
+
* we must work around the fallout by untagging in software to make
541
+
* untagged reception work more or less as expected.
542
+
*/
543
+
ds->untag_vlan_aware_bridge_pvid = true;
608
544
609
545
return 0;
610
546
}
···
642
554
ocelot_port_teardown_dsa_8021q_cpu(ocelot, dp->index);
643
555
644
556
dsa_tag_8021q_unregister(ds);
557
+
558
+
ds->untag_vlan_aware_bridge_pvid = false;
645
559
}
646
560
647
561
static unsigned long felix_tag_8021q_get_host_fwd_mask(struct dsa_switch *ds)
···
1098
1008
struct netlink_ext_ack *extack)
1099
1009
{
1100
1010
struct ocelot *ocelot = ds->priv;
1011
+
bool using_tag_8021q;
1012
+
struct felix *felix;
1013
+
int err;
1101
1014
1102
-
return ocelot_port_vlan_filtering(ocelot, port, enabled, extack);
1015
+
err = ocelot_port_vlan_filtering(ocelot, port, enabled, extack);
1016
+
if (err)
1017
+
return err;
1018
+
1019
+
felix = ocelot_to_felix(ocelot);
1020
+
using_tag_8021q = felix->tag_proto == DSA_TAG_PROTO_OCELOT_8021Q;
1021
+
if (using_tag_8021q) {
1022
+
err = felix_update_tag_8021q_rx_rules(ds, port, enabled);
1023
+
if (err)
1024
+
return err;
1025
+
}
1026
+
1027
+
return 0;
1103
1028
}
1104
1029
1105
1030
static int felix_vlan_add(struct dsa_switch *ds, int port,
···
1620
1515
int port = xmit_work->dp->index;
1621
1516
int retries = 10;
1622
1517
1518
+
ocelot_lock_inj_grp(ocelot, 0);
1519
+
1623
1520
do {
1624
1521
if (ocelot_can_inject(ocelot, 0))
1625
1522
break;
···
1630
1523
} while (--retries);
1631
1524
1632
1525
if (!retries) {
1526
+
ocelot_unlock_inj_grp(ocelot, 0);
1633
1527
dev_err(ocelot->dev, "port %d failed to inject skb\n",
1634
1528
port);
1635
1529
ocelot_port_purge_txtstamp_skb(ocelot, port, skb);
···
1639
1531
}
1640
1532
1641
1533
ocelot_port_inject_frame(ocelot, port, 0, rew_op, skb);
1534
+
1535
+
ocelot_unlock_inj_grp(ocelot, 0);
1642
1536
1643
1537
consume_skb(skb);
1644
1538
kfree(xmit_work);
···
1801
1691
if (!felix->info->quirk_no_xtr_irq)
1802
1692
return false;
1803
1693
1694
+
ocelot_lock_xtr_grp(ocelot, grp);
1695
+
1804
1696
while (ocelot_read(ocelot, QS_XTR_DATA_PRESENT) & BIT(grp)) {
1805
1697
struct sk_buff *skb;
1806
1698
unsigned int type;
···
1838
1726
ERR_PTR(err));
1839
1727
ocelot_drain_cpu_queue(ocelot, 0);
1840
1728
}
1729
+
1730
+
ocelot_unlock_xtr_grp(ocelot, grp);
1841
1731
1842
1732
return true;
1843
1733
}
+2
-2
drivers/net/ethernet/broadcom/bnxt/bnxt.c
+2
-2
drivers/net/ethernet/broadcom/bnxt/bnxt.c
···
5056
5056
list_del_init(&fltr->list);
5057
5057
}
5058
5058
5059
-
void bnxt_clear_usr_fltrs(struct bnxt *bp, bool all)
5059
+
static void bnxt_clear_usr_fltrs(struct bnxt *bp, bool all)
5060
5060
{
5061
5061
struct bnxt_filter_base *usr_fltr, *tmp;
5062
5062
···
10271
10271
}
10272
10272
}
10273
10273
10274
-
void bnxt_clear_rss_ctxs(struct bnxt *bp)
10274
+
static void bnxt_clear_rss_ctxs(struct bnxt *bp)
10275
10275
{
10276
10276
struct ethtool_rxfh_context *ctx;
10277
10277
unsigned long context;
-2
drivers/net/ethernet/broadcom/bnxt/bnxt.h
-2
drivers/net/ethernet/broadcom/bnxt/bnxt.h
···
2795
2795
int bnxt_set_rx_skb_mode(struct bnxt *bp, bool page_mode);
2796
2796
void bnxt_insert_usr_fltr(struct bnxt *bp, struct bnxt_filter_base *fltr);
2797
2797
void bnxt_del_one_usr_fltr(struct bnxt *bp, struct bnxt_filter_base *fltr);
2798
-
void bnxt_clear_usr_fltrs(struct bnxt *bp, bool all);
2799
2798
int bnxt_hwrm_func_drv_rgtr(struct bnxt *bp, unsigned long *bmap,
2800
2799
int bmap_size, bool async_only);
2801
2800
int bnxt_hwrm_func_drv_unrgtr(struct bnxt *bp);
···
2848
2849
int __bnxt_setup_vnic_p5(struct bnxt *bp, struct bnxt_vnic_info *vnic);
2849
2850
void bnxt_del_one_rss_ctx(struct bnxt *bp, struct bnxt_rss_ctx *rss_ctx,
2850
2851
bool all);
2851
-
void bnxt_clear_rss_ctxs(struct bnxt *bp);
2852
2852
int bnxt_open_nic(struct bnxt *, bool, bool);
2853
2853
int bnxt_half_open_nic(struct bnxt *bp);
2854
2854
void bnxt_half_close_nic(struct bnxt *bp);
-4
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
-4
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
···
968
968
return -EINVAL;
969
969
}
970
970
971
-
bnxt_clear_usr_fltrs(bp, true);
972
-
if (BNXT_SUPPORTS_MULTI_RSS_CTX(bp))
973
-
bnxt_clear_rss_ctxs(bp);
974
971
if (netif_running(dev)) {
975
972
if (BNXT_PF(bp)) {
976
973
/* TODO CHIMP_FW: Send message to all VF's
···
1997
2000
1998
2001
bnxt_modify_rss(bp, NULL, NULL, rxfh);
1999
2002
2000
-
bnxt_clear_usr_fltrs(bp, false);
2001
2003
if (netif_running(bp->dev)) {
2002
2004
bnxt_close_nic(bp, false, false);
2003
2005
rc = bnxt_open_nic(bp, false, false);
-5
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
-5
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
···
297
297
* redirect is coming from a frame received by the
298
298
* bnxt_en driver.
299
299
*/
300
-
rx_buf = &rxr->rx_buf_ring[cons];
301
-
mapping = rx_buf->mapping - bp->rx_dma_offset;
302
-
dma_unmap_page_attrs(&pdev->dev, mapping,
303
-
BNXT_RX_PAGE_SIZE, bp->rx_dir,
304
-
DMA_ATTR_WEAK_ORDERING);
305
300
306
301
/* if we are unable to allocate a new buffer, abort and reuse */
307
302
if (bnxt_alloc_rx_data(bp, rxr, rxr->rx_prod, GFP_ATOMIC)) {
+2
-1
drivers/net/ethernet/chelsio/cxgb4/cxgb4_filter.c
+2
-1
drivers/net/ethernet/chelsio/cxgb4/cxgb4_filter.c
···
1244
1244
* in the Compressed Filter Tuple.
1245
1245
*/
1246
1246
if (tp->vlan_shift >= 0 && fs->mask.ivlan)
1247
-
ntuple |= (FT_VLAN_VLD_F | fs->val.ivlan) << tp->vlan_shift;
1247
+
ntuple |= (u64)(FT_VLAN_VLD_F |
1248
+
fs->val.ivlan) << tp->vlan_shift;
1248
1249
1249
1250
if (tp->port_shift >= 0 && fs->mask.iport)
1250
1251
ntuple |= (u64)fs->val.iport << tp->port_shift;
+4
-3
drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c
+4
-3
drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c
···
2638
2638
2639
2639
static int dpaa2_switch_seed_bp(struct ethsw_core *ethsw)
2640
2640
{
2641
-
int *count, i;
2641
+
int *count, ret, i;
2642
2642
2643
2643
for (i = 0; i < DPAA2_ETHSW_NUM_BUFS; i += BUFS_PER_CMD) {
2644
+
ret = dpaa2_switch_add_bufs(ethsw, ethsw->bpid);
2644
2645
count = ðsw->buf_count;
2645
-
*count += dpaa2_switch_add_bufs(ethsw, ethsw->bpid);
2646
+
*count += ret;
2646
2647
2647
-
if (unlikely(*count < BUFS_PER_CMD))
2648
+
if (unlikely(ret < BUFS_PER_CMD))
2648
2649
return -ENOMEM;
2649
2650
}
2650
2651
+2
-2
drivers/net/ethernet/intel/ice/devlink/devlink_port.c
+2
-2
drivers/net/ethernet/intel/ice/devlink/devlink_port.c
···
337
337
return -EIO;
338
338
339
339
attrs.flavour = DEVLINK_PORT_FLAVOUR_PHYSICAL;
340
-
attrs.phys.port_number = pf->hw.bus.func;
340
+
attrs.phys.port_number = pf->hw.pf_id;
341
341
342
342
/* As FW supports only port split options for whole device,
343
343
* set port split options only for first PF.
···
455
455
return -EINVAL;
456
456
457
457
attrs.flavour = DEVLINK_PORT_FLAVOUR_PCI_VF;
458
-
attrs.pci_vf.pf = pf->hw.bus.func;
458
+
attrs.pci_vf.pf = pf->hw.pf_id;
459
459
attrs.pci_vf.vf = vf->vf_id;
460
460
461
461
ice_devlink_set_switch_id(pf, &attrs.switch_id);
+20
-1
drivers/net/ethernet/intel/ice/ice_base.c
+20
-1
drivers/net/ethernet/intel/ice/ice_base.c
···
513
513
}
514
514
515
515
/**
516
+
* ice_get_frame_sz - calculate xdp_buff::frame_sz
517
+
* @rx_ring: the ring being configured
518
+
*
519
+
* Return frame size based on underlying PAGE_SIZE
520
+
*/
521
+
static unsigned int ice_get_frame_sz(struct ice_rx_ring *rx_ring)
522
+
{
523
+
unsigned int frame_sz;
524
+
525
+
#if (PAGE_SIZE >= 8192)
526
+
frame_sz = rx_ring->rx_buf_len;
527
+
#else
528
+
frame_sz = ice_rx_pg_size(rx_ring) / 2;
529
+
#endif
530
+
531
+
return frame_sz;
532
+
}
533
+
534
+
/**
516
535
* ice_vsi_cfg_rxq - Configure an Rx queue
517
536
* @ring: the ring being configured
518
537
*
···
595
576
}
596
577
}
597
578
598
-
xdp_init_buff(&ring->xdp, ice_rx_pg_size(ring) / 2, &ring->xdp_rxq);
579
+
xdp_init_buff(&ring->xdp, ice_get_frame_sz(ring), &ring->xdp_rxq);
599
580
ring->xdp.data = NULL;
600
581
ring->xdp_ext.pkt_ctx = &ring->pkt_ctx;
601
582
err = ice_setup_rx_ctx(ring);
+4
-43
drivers/net/ethernet/intel/ice/ice_txrx.c
+4
-43
drivers/net/ethernet/intel/ice/ice_txrx.c
···
522
522
}
523
523
524
524
/**
525
-
* ice_rx_frame_truesize
526
-
* @rx_ring: ptr to Rx ring
527
-
* @size: size
528
-
*
529
-
* calculate the truesize with taking into the account PAGE_SIZE of
530
-
* underlying arch
531
-
*/
532
-
static unsigned int
533
-
ice_rx_frame_truesize(struct ice_rx_ring *rx_ring, const unsigned int size)
534
-
{
535
-
unsigned int truesize;
536
-
537
-
#if (PAGE_SIZE < 8192)
538
-
truesize = ice_rx_pg_size(rx_ring) / 2; /* Must be power-of-2 */
539
-
#else
540
-
truesize = rx_ring->rx_offset ?
541
-
SKB_DATA_ALIGN(rx_ring->rx_offset + size) +
542
-
SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) :
543
-
SKB_DATA_ALIGN(size);
544
-
#endif
545
-
return truesize;
546
-
}
547
-
548
-
/**
549
525
* ice_run_xdp - Executes an XDP program on initialized xdp_buff
550
526
* @rx_ring: Rx ring
551
527
* @xdp: xdp_buff used as input to the XDP program
···
813
837
if (!dev_page_is_reusable(page))
814
838
return false;
815
839
816
-
#if (PAGE_SIZE < 8192)
817
840
/* if we are only owner of page we can reuse it */
818
841
if (unlikely(rx_buf->pgcnt - pagecnt_bias > 1))
819
842
return false;
820
-
#else
843
+
#if (PAGE_SIZE >= 8192)
821
844
#define ICE_LAST_OFFSET \
822
-
(SKB_WITH_OVERHEAD(PAGE_SIZE) - ICE_RXBUF_2048)
845
+
(SKB_WITH_OVERHEAD(PAGE_SIZE) - ICE_RXBUF_3072)
823
846
if (rx_buf->page_offset > ICE_LAST_OFFSET)
824
847
return false;
825
-
#endif /* PAGE_SIZE < 8192) */
848
+
#endif /* PAGE_SIZE >= 8192) */
826
849
827
850
/* If we have drained the page fragment pool we need to update
828
851
* the pagecnt_bias and page count so that we fully restock the
···
924
949
struct ice_rx_buf *rx_buf;
925
950
926
951
rx_buf = &rx_ring->rx_buf[ntc];
927
-
rx_buf->pgcnt =
928
-
#if (PAGE_SIZE < 8192)
929
-
page_count(rx_buf->page);
930
-
#else
931
-
0;
932
-
#endif
952
+
rx_buf->pgcnt = page_count(rx_buf->page);
933
953
prefetchw(rx_buf->page);
934
954
935
955
if (!size)
···
1130
1160
bool failure;
1131
1161
u32 first;
1132
1162
1133
-
/* Frame size depend on rx_ring setup when PAGE_SIZE=4K */
1134
-
#if (PAGE_SIZE < 8192)
1135
-
xdp->frame_sz = ice_rx_frame_truesize(rx_ring, 0);
1136
-
#endif
1137
-
1138
1163
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
1139
1164
if (xdp_prog) {
1140
1165
xdp_ring = rx_ring->xdp_ring;
···
1188
1223
hard_start = page_address(rx_buf->page) + rx_buf->page_offset -
1189
1224
offset;
1190
1225
xdp_prepare_buff(xdp, hard_start, offset, size, !!offset);
1191
-
#if (PAGE_SIZE > 4096)
1192
-
/* At larger PAGE_SIZE, frame_sz depend on len size */
1193
-
xdp->frame_sz = ice_rx_frame_truesize(rx_ring, size);
1194
-
#endif
1195
1226
xdp_buff_clear_frags_flag(xdp);
1196
1227
} else if (ice_add_xdp_frag(rx_ring, xdp, rx_buf, size)) {
1197
1228
break;
+1
drivers/net/ethernet/intel/igb/igb_main.c
+1
drivers/net/ethernet/intel/igb/igb_main.c
+11
-12
drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c
+11
-12
drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c
···
632
632
return ret;
633
633
}
634
634
635
-
static bool is_valid_offset(struct rvu *rvu, struct cpt_rd_wr_reg_msg *req)
635
+
static bool validate_and_update_reg_offset(struct rvu *rvu,
636
+
struct cpt_rd_wr_reg_msg *req,
637
+
u64 *reg_offset)
636
638
{
637
639
u64 offset = req->reg_offset;
638
640
int blkaddr, num_lfs, lf;
···
664
662
req->hdr.pcifunc, lf);
665
663
if (lf < 0)
666
664
return false;
665
+
666
+
/* Translate local LF's offset to global CPT LF's offset to
667
+
* access LFX register.
668
+
*/
669
+
*reg_offset = (req->reg_offset & 0xFF000) + (lf << 3);
667
670
668
671
return true;
669
672
} else if (!(req->hdr.pcifunc & RVU_PFVF_FUNC_MASK)) {
···
704
697
struct cpt_rd_wr_reg_msg *rsp)
705
698
{
706
699
u64 offset = req->reg_offset;
707
-
int blkaddr, lf;
700
+
int blkaddr;
708
701
709
702
blkaddr = validate_and_get_cpt_blkaddr(req->blkaddr);
710
703
if (blkaddr < 0)
···
715
708
!is_cpt_vf(rvu, req->hdr.pcifunc))
716
709
return CPT_AF_ERR_ACCESS_DENIED;
717
710
718
-
if (!is_valid_offset(rvu, req))
711
+
if (!validate_and_update_reg_offset(rvu, req, &offset))
719
712
return CPT_AF_ERR_ACCESS_DENIED;
720
713
721
-
/* Translate local LF used by VFs to global CPT LF */
722
-
lf = rvu_get_lf(rvu, &rvu->hw->block[blkaddr], req->hdr.pcifunc,
723
-
(offset & 0xFFF) >> 3);
724
-
725
-
/* Translate local LF's offset to global CPT LF's offset */
726
-
offset &= 0xFF000;
727
-
offset += lf << 3;
728
-
729
-
rsp->reg_offset = offset;
714
+
rsp->reg_offset = req->reg_offset;
730
715
rsp->ret_val = req->ret_val;
731
716
rsp->is_write = req->is_write;
732
717
+1
drivers/net/ethernet/mellanox/mlx5/core/en.h
+1
drivers/net/ethernet/mellanox/mlx5/core/en.h
···
998
998
bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev, u8 page_shift,
999
999
enum mlx5e_mpwrq_umr_mode umr_mode);
1000
1000
1001
+
void mlx5e_shampo_fill_umr(struct mlx5e_rq *rq, int len);
1001
1002
void mlx5e_shampo_dealloc_hd(struct mlx5e_rq *rq);
1002
1003
void mlx5e_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats);
1003
1004
void mlx5e_fold_sw_stats64(struct mlx5e_priv *priv, struct rtnl_link_stats64 *s);
+17
-6
drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+17
-6
drivers/net/ethernet/mellanox/mlx5/core/en_main.c
···
1236
1236
rq->mpwqe.actual_wq_head = wq->head;
1237
1237
rq->mpwqe.umr_in_progress = 0;
1238
1238
rq->mpwqe.umr_completed = 0;
1239
+
1240
+
if (test_bit(MLX5E_RQ_STATE_SHAMPO, &rq->state)) {
1241
+
struct mlx5e_shampo_hd *shampo = rq->mpwqe.shampo;
1242
+
u16 len;
1243
+
1244
+
len = (shampo->pi - shampo->ci) & shampo->hd_per_wq;
1245
+
mlx5e_shampo_fill_umr(rq, len);
1246
+
}
1239
1247
}
1240
1248
1241
1249
void mlx5e_free_rx_descs(struct mlx5e_rq *rq)
···
3028
3020
static void mlx5e_set_default_xps_cpumasks(struct mlx5e_priv *priv,
3029
3021
struct mlx5e_params *params)
3030
3022
{
3031
-
struct mlx5_core_dev *mdev = priv->mdev;
3032
-
int num_comp_vectors, ix, irq;
3033
-
3034
-
num_comp_vectors = mlx5_comp_vectors_max(mdev);
3023
+
int ix;
3035
3024
3036
3025
for (ix = 0; ix < params->num_channels; ix++) {
3037
-
cpumask_clear(priv->scratchpad.cpumask);
3026
+
int num_comp_vectors, irq, vec_ix;
3027
+
struct mlx5_core_dev *mdev;
3038
3028
3039
-
for (irq = ix; irq < num_comp_vectors; irq += params->num_channels) {
3029
+
mdev = mlx5_sd_ch_ix_get_dev(priv->mdev, ix);
3030
+
num_comp_vectors = mlx5_comp_vectors_max(mdev);
3031
+
cpumask_clear(priv->scratchpad.cpumask);
3032
+
vec_ix = mlx5_sd_ch_ix_get_vec_ix(mdev, ix);
3033
+
3034
+
for (irq = vec_ix; irq < num_comp_vectors; irq += params->num_channels) {
3040
3035
int cpu = mlx5_comp_vector_get_cpu(mdev, irq);
3041
3036
3042
3037
cpumask_set_cpu(cpu, priv->scratchpad.cpumask);
+16
-10
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+16
-10
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
···
735
735
ksm_entries = bitmap_find_window(shampo->bitmap,
736
736
shampo->hd_per_wqe,
737
737
shampo->hd_per_wq, shampo->pi);
738
+
ksm_entries = ALIGN_DOWN(ksm_entries, MLX5E_SHAMPO_WQ_HEADER_PER_PAGE);
738
739
if (!ksm_entries)
739
740
return 0;
740
741
···
963
962
sq->cc = sqcc;
964
963
}
965
964
966
-
static void mlx5e_handle_shampo_hd_umr(struct mlx5e_shampo_umr umr,
967
-
struct mlx5e_icosq *sq)
965
+
void mlx5e_shampo_fill_umr(struct mlx5e_rq *rq, int len)
968
966
{
969
-
struct mlx5e_channel *c = container_of(sq, struct mlx5e_channel, icosq);
970
-
struct mlx5e_shampo_hd *shampo;
971
-
/* assume 1:1 relationship between RQ and icosq */
972
-
struct mlx5e_rq *rq = &c->rq;
973
-
int end, from, len = umr.len;
967
+
struct mlx5e_shampo_hd *shampo = rq->mpwqe.shampo;
968
+
int end, from, full_len = len;
974
969
975
-
shampo = rq->mpwqe.shampo;
976
970
end = shampo->hd_per_wq;
977
971
from = shampo->ci;
978
-
if (from + len > shampo->hd_per_wq) {
972
+
if (from + len > end) {
979
973
len -= end - from;
980
974
bitmap_set(shampo->bitmap, from, end - from);
981
975
from = 0;
982
976
}
983
977
984
978
bitmap_set(shampo->bitmap, from, len);
985
-
shampo->ci = (shampo->ci + umr.len) & (shampo->hd_per_wq - 1);
979
+
shampo->ci = (shampo->ci + full_len) & (shampo->hd_per_wq - 1);
980
+
}
981
+
982
+
static void mlx5e_handle_shampo_hd_umr(struct mlx5e_shampo_umr umr,
983
+
struct mlx5e_icosq *sq)
984
+
{
985
+
struct mlx5e_channel *c = container_of(sq, struct mlx5e_channel, icosq);
986
+
/* assume 1:1 relationship between RQ and icosq */
987
+
struct mlx5e_rq *rq = &c->rq;
988
+
989
+
mlx5e_shampo_fill_umr(rq, umr.len);
986
990
}
987
991
988
992
int mlx5e_poll_ico_cq(struct mlx5e_cq *cq)
+4
-2
drivers/net/ethernet/mellanox/mlx5/core/lib/ipsec_fs_roce.c
+4
-2
drivers/net/ethernet/mellanox/mlx5/core/lib/ipsec_fs_roce.c
···
386
386
return -EOPNOTSUPP;
387
387
388
388
peer_priv = mlx5_devcom_get_next_peer_data(*ipsec_roce->devcom, &tmp);
389
-
if (!peer_priv) {
389
+
if (!peer_priv || !peer_priv->ipsec) {
390
+
mlx5_core_err(mdev, "IPsec not supported on master device\n");
390
391
err = -EOPNOTSUPP;
391
392
goto release_peer;
392
393
}
···
456
455
return -EOPNOTSUPP;
457
456
458
457
peer_priv = mlx5_devcom_get_next_peer_data(*ipsec_roce->devcom, &tmp);
459
-
if (!peer_priv) {
458
+
if (!peer_priv || !peer_priv->ipsec) {
459
+
mlx5_core_err(mdev, "IPsec not supported on master device\n");
460
460
err = -EOPNOTSUPP;
461
461
goto release_peer;
462
462
}
+261
-18
drivers/net/ethernet/mscc/ocelot.c
+261
-18
drivers/net/ethernet/mscc/ocelot.c
···
453
453
return VLAN_N_VID - bridge_num - 1;
454
454
}
455
455
456
+
/**
457
+
* ocelot_update_vlan_reclassify_rule() - Make switch aware only to bridge VLAN TPID
458
+
*
459
+
* @ocelot: Switch private data structure
460
+
* @port: Index of ingress port
461
+
*
462
+
* IEEE 802.1Q-2018 clauses "5.5 C-VLAN component conformance" and "5.6 S-VLAN
463
+
* component conformance" suggest that a C-VLAN component should only recognize
464
+
* and filter on C-Tags, and an S-VLAN component should only recognize and
465
+
* process based on C-Tags.
466
+
*
467
+
* In Linux, as per commit 1a0b20b25732 ("Merge branch 'bridge-next'"), C-VLAN
468
+
* components are largely represented by a bridge with vlan_protocol 802.1Q,
469
+
* and S-VLAN components by a bridge with vlan_protocol 802.1ad.
470
+
*
471
+
* Currently the driver only offloads vlan_protocol 802.1Q, but the hardware
472
+
* design is non-conformant, because the switch assigns each frame to a VLAN
473
+
* based on an entirely different question, as detailed in figure "Basic VLAN
474
+
* Classification Flow" from its manual and reproduced below.
475
+
*
476
+
* Set TAG_TYPE, PCP, DEI, VID to port-default values in VLAN_CFG register
477
+
* if VLAN_AWARE_ENA[port] and frame has outer tag then:
478
+
* if VLAN_INNER_TAG_ENA[port] and frame has inner tag then:
479
+
* TAG_TYPE = (Frame.InnerTPID <> 0x8100)
480
+
* Set PCP, DEI, VID to values from inner VLAN header
481
+
* else:
482
+
* TAG_TYPE = (Frame.OuterTPID <> 0x8100)
483
+
* Set PCP, DEI, VID to values from outer VLAN header
484
+
* if VID == 0 then:
485
+
* VID = VLAN_CFG.VLAN_VID
486
+
*
487
+
* Summarized, the switch will recognize both 802.1Q and 802.1ad TPIDs as VLAN
488
+
* "with equal rights", and just set the TAG_TYPE bit to 0 (if 802.1Q) or to 1
489
+
* (if 802.1ad). It will classify based on whichever of the tags is "outer", no
490
+
* matter what TPID that may have (or "inner", if VLAN_INNER_TAG_ENA[port]).
491
+
*
492
+
* In the VLAN Table, the TAG_TYPE information is not accessible - just the
493
+
* classified VID is - so it is as if each VLAN Table entry is for 2 VLANs:
494
+
* C-VLAN X, and S-VLAN X.
495
+
*
496
+
* Whereas the Linux bridge behavior is to only filter on frames with a TPID
497
+
* equal to the vlan_protocol, and treat everything else as VLAN-untagged.
498
+
*
499
+
* Consider an ingress packet tagged with 802.1ad VID=3 and 802.1Q VID=5,
500
+
* received on a bridge vlan_filtering=1 vlan_protocol=802.1Q port. This frame
501
+
* should be treated as 802.1Q-untagged, and classified to the PVID of that
502
+
* bridge port. Not to VID=3, and not to VID=5.
503
+
*
504
+
* The VCAP IS1 TCAM has everything we need to overwrite the choices made in
505
+
* the basic VLAN classification pipeline: it can match on TAG_TYPE in the key,
506
+
* and it can modify the classified VID in the action. Thus, for each port
507
+
* under a vlan_filtering bridge, we can insert a rule in VCAP IS1 lookup 0 to
508
+
* match on 802.1ad tagged frames and modify their classified VID to the 802.1Q
509
+
* PVID of the port. This effectively makes it appear to the outside world as
510
+
* if those packets were processed as VLAN-untagged.
511
+
*
512
+
* The rule needs to be updated each time the bridge PVID changes, and needs
513
+
* to be deleted if the bridge PVID is deleted, or if the port becomes
514
+
* VLAN-unaware.
515
+
*/
516
+
static int ocelot_update_vlan_reclassify_rule(struct ocelot *ocelot, int port)
517
+
{
518
+
unsigned long cookie = OCELOT_VCAP_IS1_VLAN_RECLASSIFY(ocelot, port);
519
+
struct ocelot_vcap_block *block_vcap_is1 = &ocelot->block[VCAP_IS1];
520
+
struct ocelot_port *ocelot_port = ocelot->ports[port];
521
+
const struct ocelot_bridge_vlan *pvid_vlan;
522
+
struct ocelot_vcap_filter *filter;
523
+
int err, val, pcp, dei;
524
+
bool vid_replace_ena;
525
+
u16 vid;
526
+
527
+
pvid_vlan = ocelot_port->pvid_vlan;
528
+
vid_replace_ena = ocelot_port->vlan_aware && pvid_vlan;
529
+
530
+
filter = ocelot_vcap_block_find_filter_by_id(block_vcap_is1, cookie,
531
+
false);
532
+
if (!vid_replace_ena) {
533
+
/* If the reclassification filter doesn't need to exist, delete
534
+
* it if it was previously installed, and exit doing nothing
535
+
* otherwise.
536
+
*/
537
+
if (filter)
538
+
return ocelot_vcap_filter_del(ocelot, filter);
539
+
540
+
return 0;
541
+
}
542
+
543
+
/* The reclassification rule must apply. See if it already exists
544
+
* or if it must be created.
545
+
*/
546
+
547
+
/* Treating as VLAN-untagged means using as classified VID equal to
548
+
* the bridge PVID, and PCP/DEI set to the port default QoS values.
549
+
*/
550
+
vid = pvid_vlan->vid;
551
+
val = ocelot_read_gix(ocelot, ANA_PORT_QOS_CFG, port);
552
+
pcp = ANA_PORT_QOS_CFG_QOS_DEFAULT_VAL_X(val);
553
+
dei = !!(val & ANA_PORT_QOS_CFG_DP_DEFAULT_VAL);
554
+
555
+
if (filter) {
556
+
bool changed = false;
557
+
558
+
/* Filter exists, just update it */
559
+
if (filter->action.vid != vid) {
560
+
filter->action.vid = vid;
561
+
changed = true;
562
+
}
563
+
if (filter->action.pcp != pcp) {
564
+
filter->action.pcp = pcp;
565
+
changed = true;
566
+
}
567
+
if (filter->action.dei != dei) {
568
+
filter->action.dei = dei;
569
+
changed = true;
570
+
}
571
+
572
+
if (!changed)
573
+
return 0;
574
+
575
+
return ocelot_vcap_filter_replace(ocelot, filter);
576
+
}
577
+
578
+
/* Filter doesn't exist, create it */
579
+
filter = kzalloc(sizeof(*filter), GFP_KERNEL);
580
+
if (!filter)
581
+
return -ENOMEM;
582
+
583
+
filter->key_type = OCELOT_VCAP_KEY_ANY;
584
+
filter->ingress_port_mask = BIT(port);
585
+
filter->vlan.tpid = OCELOT_VCAP_BIT_1;
586
+
filter->prio = 1;
587
+
filter->id.cookie = cookie;
588
+
filter->id.tc_offload = false;
589
+
filter->block_id = VCAP_IS1;
590
+
filter->type = OCELOT_VCAP_FILTER_OFFLOAD;
591
+
filter->lookup = 0;
592
+
filter->action.vid_replace_ena = true;
593
+
filter->action.pcp_dei_ena = true;
594
+
filter->action.vid = vid;
595
+
filter->action.pcp = pcp;
596
+
filter->action.dei = dei;
597
+
598
+
err = ocelot_vcap_filter_add(ocelot, filter, NULL);
599
+
if (err)
600
+
kfree(filter);
601
+
602
+
return err;
603
+
}
604
+
456
605
/* Default vlan to clasify for untagged frames (may be zero) */
457
-
static void ocelot_port_set_pvid(struct ocelot *ocelot, int port,
458
-
const struct ocelot_bridge_vlan *pvid_vlan)
606
+
static int ocelot_port_set_pvid(struct ocelot *ocelot, int port,
607
+
const struct ocelot_bridge_vlan *pvid_vlan)
459
608
{
460
609
struct ocelot_port *ocelot_port = ocelot->ports[port];
461
610
u16 pvid = ocelot_vlan_unaware_pvid(ocelot, ocelot_port->bridge);
···
624
475
* happens automatically), but also 802.1p traffic which gets
625
476
* classified to VLAN 0, but that is always in our RX filter, so it
626
477
* would get accepted were it not for this setting.
478
+
*
479
+
* Also, we only support the bridge 802.1Q VLAN protocol, so
480
+
* 802.1ad-tagged frames (carrying S-Tags) should be considered
481
+
* 802.1Q-untagged, and also dropped.
627
482
*/
628
483
if (!pvid_vlan && ocelot_port->vlan_aware)
629
484
val = ANA_PORT_DROP_CFG_DROP_PRIO_S_TAGGED_ENA |
630
-
ANA_PORT_DROP_CFG_DROP_PRIO_C_TAGGED_ENA;
485
+
ANA_PORT_DROP_CFG_DROP_PRIO_C_TAGGED_ENA |
486
+
ANA_PORT_DROP_CFG_DROP_S_TAGGED_ENA;
631
487
632
488
ocelot_rmw_gix(ocelot, val,
633
489
ANA_PORT_DROP_CFG_DROP_PRIO_S_TAGGED_ENA |
634
-
ANA_PORT_DROP_CFG_DROP_PRIO_C_TAGGED_ENA,
490
+
ANA_PORT_DROP_CFG_DROP_PRIO_C_TAGGED_ENA |
491
+
ANA_PORT_DROP_CFG_DROP_S_TAGGED_ENA,
635
492
ANA_PORT_DROP_CFG, port);
493
+
494
+
return ocelot_update_vlan_reclassify_rule(ocelot, port);
636
495
}
637
496
638
497
static struct ocelot_bridge_vlan *ocelot_bridge_vlan_find(struct ocelot *ocelot,
···
788
631
ANA_PORT_VLAN_CFG_VLAN_POP_CNT_M,
789
632
ANA_PORT_VLAN_CFG, port);
790
633
791
-
ocelot_port_set_pvid(ocelot, port, ocelot_port->pvid_vlan);
634
+
err = ocelot_port_set_pvid(ocelot, port, ocelot_port->pvid_vlan);
635
+
if (err)
636
+
return err;
637
+
792
638
ocelot_port_manage_port_tag(ocelot, port);
793
639
794
640
return 0;
···
844
684
return err;
845
685
846
686
/* Default ingress vlan classification */
847
-
if (pvid)
848
-
ocelot_port_set_pvid(ocelot, port,
849
-
ocelot_bridge_vlan_find(ocelot, vid));
687
+
if (pvid) {
688
+
err = ocelot_port_set_pvid(ocelot, port,
689
+
ocelot_bridge_vlan_find(ocelot, vid));
690
+
if (err)
691
+
return err;
692
+
}
850
693
851
694
/* Untagged egress vlan clasification */
852
695
ocelot_port_manage_port_tag(ocelot, port);
···
875
712
return err;
876
713
877
714
/* Ingress */
878
-
if (del_pvid)
879
-
ocelot_port_set_pvid(ocelot, port, NULL);
715
+
if (del_pvid) {
716
+
err = ocelot_port_set_pvid(ocelot, port, NULL);
717
+
if (err)
718
+
return err;
719
+
}
880
720
881
721
/* Egress */
882
722
ocelot_port_manage_port_tag(ocelot, port);
···
1265
1099
}
1266
1100
EXPORT_SYMBOL(ocelot_ptp_rx_timestamp);
1267
1101
1102
+
void ocelot_lock_inj_grp(struct ocelot *ocelot, int grp)
1103
+
__acquires(&ocelot->inj_lock)
1104
+
{
1105
+
spin_lock(&ocelot->inj_lock);
1106
+
}
1107
+
EXPORT_SYMBOL_GPL(ocelot_lock_inj_grp);
1108
+
1109
+
void ocelot_unlock_inj_grp(struct ocelot *ocelot, int grp)
1110
+
__releases(&ocelot->inj_lock)
1111
+
{
1112
+
spin_unlock(&ocelot->inj_lock);
1113
+
}
1114
+
EXPORT_SYMBOL_GPL(ocelot_unlock_inj_grp);
1115
+
1116
+
void ocelot_lock_xtr_grp(struct ocelot *ocelot, int grp)
1117
+
__acquires(&ocelot->inj_lock)
1118
+
{
1119
+
spin_lock(&ocelot->inj_lock);
1120
+
}
1121
+
EXPORT_SYMBOL_GPL(ocelot_lock_xtr_grp);
1122
+
1123
+
void ocelot_unlock_xtr_grp(struct ocelot *ocelot, int grp)
1124
+
__releases(&ocelot->inj_lock)
1125
+
{
1126
+
spin_unlock(&ocelot->inj_lock);
1127
+
}
1128
+
EXPORT_SYMBOL_GPL(ocelot_unlock_xtr_grp);
1129
+
1130
+
void ocelot_lock_xtr_grp_bh(struct ocelot *ocelot, int grp)
1131
+
__acquires(&ocelot->xtr_lock)
1132
+
{
1133
+
spin_lock_bh(&ocelot->xtr_lock);
1134
+
}
1135
+
EXPORT_SYMBOL_GPL(ocelot_lock_xtr_grp_bh);
1136
+
1137
+
void ocelot_unlock_xtr_grp_bh(struct ocelot *ocelot, int grp)
1138
+
__releases(&ocelot->xtr_lock)
1139
+
{
1140
+
spin_unlock_bh(&ocelot->xtr_lock);
1141
+
}
1142
+
EXPORT_SYMBOL_GPL(ocelot_unlock_xtr_grp_bh);
1143
+
1268
1144
int ocelot_xtr_poll_frame(struct ocelot *ocelot, int grp, struct sk_buff **nskb)
1269
1145
{
1270
1146
u64 timestamp, src_port, len;
···
1316
1108
int sz, buf_len;
1317
1109
u32 val, *buf;
1318
1110
int err;
1111
+
1112
+
lockdep_assert_held(&ocelot->xtr_lock);
1319
1113
1320
1114
err = ocelot_xtr_poll_xfh(ocelot, grp, xfh);
1321
1115
if (err)
···
1394
1184
{
1395
1185
u32 val = ocelot_read(ocelot, QS_INJ_STATUS);
1396
1186
1187
+
lockdep_assert_held(&ocelot->inj_lock);
1188
+
1397
1189
if (!(val & QS_INJ_STATUS_FIFO_RDY(BIT(grp))))
1398
1190
return false;
1399
1191
if (val & QS_INJ_STATUS_WMARK_REACHED(BIT(grp)))
···
1405
1193
}
1406
1194
EXPORT_SYMBOL(ocelot_can_inject);
1407
1195
1408
-
void ocelot_ifh_port_set(void *ifh, int port, u32 rew_op, u32 vlan_tag)
1196
+
/**
1197
+
* ocelot_ifh_set_basic - Set basic information in Injection Frame Header
1198
+
* @ifh: Pointer to Injection Frame Header memory
1199
+
* @ocelot: Switch private data structure
1200
+
* @port: Egress port number
1201
+
* @rew_op: Egress rewriter operation for PTP
1202
+
* @skb: Pointer to socket buffer (packet)
1203
+
*
1204
+
* Populate the Injection Frame Header with basic information for this skb: the
1205
+
* analyzer bypass bit, destination port, VLAN info, egress rewriter info.
1206
+
*/
1207
+
void ocelot_ifh_set_basic(void *ifh, struct ocelot *ocelot, int port,
1208
+
u32 rew_op, struct sk_buff *skb)
1409
1209
{
1210
+
struct ocelot_port *ocelot_port = ocelot->ports[port];
1211
+
struct net_device *dev = skb->dev;
1212
+
u64 vlan_tci, tag_type;
1213
+
int qos_class;
1214
+
1215
+
ocelot_xmit_get_vlan_info(skb, ocelot_port->bridge, &vlan_tci,
1216
+
&tag_type);
1217
+
1218
+
qos_class = netdev_get_num_tc(dev) ?
1219
+
netdev_get_prio_tc_map(dev, skb->priority) : skb->priority;
1220
+
1221
+
memset(ifh, 0, OCELOT_TAG_LEN);
1410
1222
ocelot_ifh_set_bypass(ifh, 1);
1223
+
ocelot_ifh_set_src(ifh, BIT_ULL(ocelot->num_phys_ports));
1411
1224
ocelot_ifh_set_dest(ifh, BIT_ULL(port));
1412
-
ocelot_ifh_set_tag_type(ifh, IFH_TAG_TYPE_C);
1413
-
if (vlan_tag)
1414
-
ocelot_ifh_set_vlan_tci(ifh, vlan_tag);
1225
+
ocelot_ifh_set_qos_class(ifh, qos_class);
1226
+
ocelot_ifh_set_tag_type(ifh, tag_type);
1227
+
ocelot_ifh_set_vlan_tci(ifh, vlan_tci);
1415
1228
if (rew_op)
1416
1229
ocelot_ifh_set_rew_op(ifh, rew_op);
1417
1230
}
1418
-
EXPORT_SYMBOL(ocelot_ifh_port_set);
1231
+
EXPORT_SYMBOL(ocelot_ifh_set_basic);
1419
1232
1420
1233
void ocelot_port_inject_frame(struct ocelot *ocelot, int port, int grp,
1421
1234
u32 rew_op, struct sk_buff *skb)
1422
1235
{
1423
-
u32 ifh[OCELOT_TAG_LEN / 4] = {0};
1236
+
u32 ifh[OCELOT_TAG_LEN / 4];
1424
1237
unsigned int i, count, last;
1238
+
1239
+
lockdep_assert_held(&ocelot->inj_lock);
1425
1240
1426
1241
ocelot_write_rix(ocelot, QS_INJ_CTRL_GAP_SIZE(1) |
1427
1242
QS_INJ_CTRL_SOF, QS_INJ_CTRL, grp);
1428
1243
1429
-
ocelot_ifh_port_set(ifh, port, rew_op, skb_vlan_tag_get(skb));
1244
+
ocelot_ifh_set_basic(ifh, ocelot, port, rew_op, skb);
1430
1245
1431
1246
for (i = 0; i < OCELOT_TAG_LEN / 4; i++)
1432
1247
ocelot_write_rix(ocelot, ifh[i], QS_INJ_WR, grp);
···
1486
1247
1487
1248
void ocelot_drain_cpu_queue(struct ocelot *ocelot, int grp)
1488
1249
{
1250
+
lockdep_assert_held(&ocelot->xtr_lock);
1251
+
1489
1252
while (ocelot_read(ocelot, QS_XTR_DATA_PRESENT) & BIT(grp))
1490
1253
ocelot_read_rix(ocelot, QS_XTR_RD, grp);
1491
1254
}
···
2773
2532
ANA_PORT_QOS_CFG,
2774
2533
port);
2775
2534
2776
-
return 0;
2535
+
return ocelot_update_vlan_reclassify_rule(ocelot, port);
2777
2536
}
2778
2537
EXPORT_SYMBOL_GPL(ocelot_port_set_default_prio);
2779
2538
···
3170
2929
mutex_init(&ocelot->fwd_domain_lock);
3171
2930
spin_lock_init(&ocelot->ptp_clock_lock);
3172
2931
spin_lock_init(&ocelot->ts_id_lock);
2932
+
spin_lock_init(&ocelot->inj_lock);
2933
+
spin_lock_init(&ocelot->xtr_lock);
3173
2934
3174
2935
ocelot->owq = alloc_ordered_workqueue("ocelot-owq", 0);
3175
2936
if (!ocelot->owq)
+1
-2
drivers/net/ethernet/mscc/ocelot_fdma.c
+1
-2
drivers/net/ethernet/mscc/ocelot_fdma.c
+1
drivers/net/ethernet/mscc/ocelot_vcap.c
+1
drivers/net/ethernet/mscc/ocelot_vcap.c
···
695
695
vcap_key_bit_set(vcap, &data, VCAP_IS1_HK_L2_MC, filter->dmac_mc);
696
696
vcap_key_bit_set(vcap, &data, VCAP_IS1_HK_L2_BC, filter->dmac_bc);
697
697
vcap_key_bit_set(vcap, &data, VCAP_IS1_HK_VLAN_TAGGED, tag->tagged);
698
+
vcap_key_bit_set(vcap, &data, VCAP_IS1_HK_TPID, tag->tpid);
698
699
vcap_key_set(vcap, &data, VCAP_IS1_HK_VID,
699
700
tag->vid.value, tag->vid.mask);
700
701
vcap_key_set(vcap, &data, VCAP_IS1_HK_PCP,
+4
drivers/net/ethernet/mscc/ocelot_vsc7514.c
+4
drivers/net/ethernet/mscc/ocelot_vsc7514.c
···
51
51
struct ocelot *ocelot = arg;
52
52
int grp = 0, err;
53
53
54
+
ocelot_lock_xtr_grp(ocelot, grp);
55
+
54
56
while (ocelot_read(ocelot, QS_XTR_DATA_PRESENT) & BIT(grp)) {
55
57
struct sk_buff *skb;
56
58
···
70
68
out:
71
69
if (err < 0)
72
70
ocelot_drain_cpu_queue(ocelot, 0);
71
+
72
+
ocelot_unlock_xtr_grp(ocelot, grp);
73
73
74
74
return IRQ_HANDLED;
75
75
}
+6
-2
drivers/net/ethernet/wangxun/ngbe/ngbe_mdio.c
+6
-2
drivers/net/ethernet/wangxun/ngbe/ngbe_mdio.c
···
124
124
MAC_SYM_PAUSE | MAC_ASYM_PAUSE;
125
125
config->mac_managed_pm = true;
126
126
127
-
phy_mode = PHY_INTERFACE_MODE_RGMII_ID;
128
-
__set_bit(PHY_INTERFACE_MODE_RGMII_ID, config->supported_interfaces);
127
+
/* The MAC only has add the Tx delay and it can not be modified.
128
+
* So just disable TX delay in PHY, and it is does not matter to
129
+
* internal phy.
130
+
*/
131
+
phy_mode = PHY_INTERFACE_MODE_RGMII_RXID;
132
+
__set_bit(PHY_INTERFACE_MODE_RGMII_RXID, config->supported_interfaces);
129
133
130
134
phylink = phylink_create(config, NULL, phy_mode, &ngbe_mac_ops);
131
135
if (IS_ERR(phylink))
+1
drivers/net/ethernet/xilinx/xilinx_axienet.h
+1
drivers/net/ethernet/xilinx/xilinx_axienet.h
···
172
172
#define XAE_UAW0_OFFSET 0x00000700 /* Unicast address word 0 */
173
173
#define XAE_UAW1_OFFSET 0x00000704 /* Unicast address word 1 */
174
174
#define XAE_FMI_OFFSET 0x00000708 /* Frame Filter Control */
175
+
#define XAE_FFE_OFFSET 0x0000070C /* Frame Filter Enable */
175
176
#define XAE_AF0_OFFSET 0x00000710 /* Address Filter 0 */
176
177
#define XAE_AF1_OFFSET 0x00000714 /* Address Filter 1 */
177
178
+13
-12
drivers/net/ethernet/xilinx/xilinx_axienet_main.c
+13
-12
drivers/net/ethernet/xilinx/xilinx_axienet_main.c
···
433
433
*/
434
434
static void axienet_set_multicast_list(struct net_device *ndev)
435
435
{
436
-
int i;
436
+
int i = 0;
437
437
u32 reg, af0reg, af1reg;
438
438
struct axienet_local *lp = netdev_priv(ndev);
439
439
···
451
451
} else if (!netdev_mc_empty(ndev)) {
452
452
struct netdev_hw_addr *ha;
453
453
454
-
i = 0;
454
+
reg = axienet_ior(lp, XAE_FMI_OFFSET);
455
+
reg &= ~XAE_FMI_PM_MASK;
456
+
axienet_iow(lp, XAE_FMI_OFFSET, reg);
457
+
455
458
netdev_for_each_mc_addr(ha, ndev) {
456
459
if (i >= XAE_MULTICAST_CAM_TABLE_NUM)
457
460
break;
···
473
470
axienet_iow(lp, XAE_FMI_OFFSET, reg);
474
471
axienet_iow(lp, XAE_AF0_OFFSET, af0reg);
475
472
axienet_iow(lp, XAE_AF1_OFFSET, af1reg);
473
+
axienet_iow(lp, XAE_FFE_OFFSET, 1);
476
474
i++;
477
475
}
478
476
} else {
···
481
477
reg &= ~XAE_FMI_PM_MASK;
482
478
483
479
axienet_iow(lp, XAE_FMI_OFFSET, reg);
484
-
485
-
for (i = 0; i < XAE_MULTICAST_CAM_TABLE_NUM; i++) {
486
-
reg = axienet_ior(lp, XAE_FMI_OFFSET) & 0xFFFFFF00;
487
-
reg |= i;
488
-
489
-
axienet_iow(lp, XAE_FMI_OFFSET, reg);
490
-
axienet_iow(lp, XAE_AF0_OFFSET, 0);
491
-
axienet_iow(lp, XAE_AF1_OFFSET, 0);
492
-
}
493
-
494
480
dev_info(&ndev->dev, "Promiscuous mode disabled.\n");
481
+
}
482
+
483
+
for (; i < XAE_MULTICAST_CAM_TABLE_NUM; i++) {
484
+
reg = axienet_ior(lp, XAE_FMI_OFFSET) & 0xFFFFFF00;
485
+
reg |= i;
486
+
axienet_iow(lp, XAE_FMI_OFFSET, reg);
487
+
axienet_iow(lp, XAE_FFE_OFFSET, 0);
495
488
}
496
489
}
497
490
+2
-1
drivers/net/phy/realtek.c
+2
-1
drivers/net/phy/realtek.c
···
555
555
unsigned long rules)
556
556
{
557
557
const u16 mask = RTL8211F_LEDCR_MASK << (RTL8211F_LEDCR_SHIFT * index);
558
-
u16 reg = RTL8211F_LEDCR_MODE; /* Mode B */
558
+
u16 reg = 0;
559
559
560
560
if (index >= RTL8211F_LED_COUNT)
561
561
return -EINVAL;
···
575
575
}
576
576
577
577
reg <<= RTL8211F_LEDCR_SHIFT * index;
578
+
reg |= RTL8211F_LEDCR_MODE; /* Mode B */
578
579
579
580
return phy_modify_paged(phydev, 0xd04, RTL8211F_LEDCR, mask, reg);
580
581
}
+1
-1
drivers/net/virtio_net.c
+1
-1
drivers/net/virtio_net.c
···
2867
2867
if (err < 0)
2868
2868
goto err_xdp_reg_mem_model;
2869
2869
2870
-
virtnet_napi_enable(vi->rq[qp_index].vq, &vi->rq[qp_index].napi);
2871
2870
netdev_tx_reset_queue(netdev_get_tx_queue(vi->dev, qp_index));
2871
+
virtnet_napi_enable(vi->rq[qp_index].vq, &vi->rq[qp_index].napi);
2872
2872
virtnet_napi_tx_enable(vi, vi->sq[qp_index].vq, &vi->sq[qp_index].napi);
2873
2873
2874
2874
return 0;
+1
-1
drivers/nvdimm/pmem.c
+1
-1
drivers/nvdimm/pmem.c
+11
-4
drivers/of/irq.c
+11
-4
drivers/of/irq.c
···
344
344
struct device_node *p;
345
345
const __be32 *addr;
346
346
u32 intsize;
347
-
int i, res;
347
+
int i, res, addr_len;
348
+
__be32 addr_buf[3] = { 0 };
348
349
349
350
pr_debug("of_irq_parse_one: dev=%pOF, index=%d\n", device, index);
350
351
···
354
353
return of_irq_parse_oldworld(device, index, out_irq);
355
354
356
355
/* Get the reg property (if any) */
357
-
addr = of_get_property(device, "reg", NULL);
356
+
addr = of_get_property(device, "reg", &addr_len);
357
+
358
+
/* Prevent out-of-bounds read in case of longer interrupt parent address size */
359
+
if (addr_len > (3 * sizeof(__be32)))
360
+
addr_len = 3 * sizeof(__be32);
361
+
if (addr)
362
+
memcpy(addr_buf, addr, addr_len);
358
363
359
364
/* Try the new-style interrupts-extended first */
360
365
res = of_parse_phandle_with_args(device, "interrupts-extended",
361
366
"#interrupt-cells", index, out_irq);
362
367
if (!res)
363
-
return of_irq_parse_raw(addr, out_irq);
368
+
return of_irq_parse_raw(addr_buf, out_irq);
364
369
365
370
/* Look for the interrupt parent. */
366
371
p = of_irq_find_parent(device);
···
396
389
397
390
398
391
/* Check if there are any interrupt-map translations to process */
399
-
res = of_irq_parse_raw(addr, out_irq);
392
+
res = of_irq_parse_raw(addr_buf, out_irq);
400
393
out:
401
394
of_node_put(p);
402
395
return res;
+2
-1
drivers/platform/surface/aggregator/controller.c
+2
-1
drivers/platform/surface/aggregator/controller.c
···
1354
1354
if (ctrl->state == SSAM_CONTROLLER_UNINITIALIZED)
1355
1355
return;
1356
1356
1357
-
WARN_ON(ctrl->state != SSAM_CONTROLLER_STOPPED);
1357
+
WARN_ON(ctrl->state != SSAM_CONTROLLER_STOPPED &&
1358
+
ctrl->state != SSAM_CONTROLLER_INITIALIZED);
1358
1359
1359
1360
/*
1360
1361
* Note: New events could still have been received after the previous
+52
-6
drivers/platform/surface/surface_aggregator_registry.c
+52
-6
drivers/platform/surface/surface_aggregator_registry.c
···
265
265
&ssam_node_root,
266
266
&ssam_node_bat_ac,
267
267
&ssam_node_bat_main,
268
-
&ssam_node_tmp_perf_profile,
268
+
&ssam_node_tmp_perf_profile_with_fan,
269
+
&ssam_node_tmp_sensors,
270
+
&ssam_node_fan_speed,
269
271
&ssam_node_hid_main_keyboard,
270
272
&ssam_node_hid_main_touchpad,
271
273
&ssam_node_hid_main_iid5,
···
275
273
NULL,
276
274
};
277
275
278
-
/* Devices for Surface Laptop Studio. */
279
-
static const struct software_node *ssam_node_group_sls[] = {
276
+
/* Devices for Surface Laptop 6. */
277
+
static const struct software_node *ssam_node_group_sl6[] = {
278
+
&ssam_node_root,
279
+
&ssam_node_bat_ac,
280
+
&ssam_node_bat_main,
281
+
&ssam_node_tmp_perf_profile_with_fan,
282
+
&ssam_node_tmp_sensors,
283
+
&ssam_node_fan_speed,
284
+
&ssam_node_hid_main_keyboard,
285
+
&ssam_node_hid_main_touchpad,
286
+
&ssam_node_hid_main_iid5,
287
+
&ssam_node_hid_sam_sensors,
288
+
&ssam_node_hid_sam_ucm_ucsi,
289
+
NULL,
290
+
};
291
+
292
+
/* Devices for Surface Laptop Studio 1. */
293
+
static const struct software_node *ssam_node_group_sls1[] = {
280
294
&ssam_node_root,
281
295
&ssam_node_bat_ac,
282
296
&ssam_node_bat_main,
···
304
286
&ssam_node_hid_sam_sensors,
305
287
&ssam_node_hid_sam_ucm_ucsi,
306
288
&ssam_node_hid_sam_sysctrl,
289
+
NULL,
290
+
};
291
+
292
+
/* Devices for Surface Laptop Studio 2. */
293
+
static const struct software_node *ssam_node_group_sls2[] = {
294
+
&ssam_node_root,
295
+
&ssam_node_bat_ac,
296
+
&ssam_node_bat_main,
297
+
&ssam_node_tmp_perf_profile_with_fan,
298
+
&ssam_node_tmp_sensors,
299
+
&ssam_node_fan_speed,
300
+
&ssam_node_pos_tablet_switch,
301
+
&ssam_node_hid_sam_keyboard,
302
+
&ssam_node_hid_sam_penstash,
303
+
&ssam_node_hid_sam_sensors,
304
+
&ssam_node_hid_sam_ucm_ucsi,
307
305
NULL,
308
306
};
309
307
···
358
324
NULL,
359
325
};
360
326
361
-
/* Devices for Surface Pro 9 */
327
+
/* Devices for Surface Pro 9 and 10 */
362
328
static const struct software_node *ssam_node_group_sp9[] = {
363
329
&ssam_node_root,
364
330
&ssam_node_hub_kip,
···
399
365
/* Surface Pro 9 */
400
366
{ "MSHW0343", (unsigned long)ssam_node_group_sp9 },
401
367
368
+
/* Surface Pro 10 */
369
+
{ "MSHW0510", (unsigned long)ssam_node_group_sp9 },
370
+
402
371
/* Surface Book 2 */
403
372
{ "MSHW0107", (unsigned long)ssam_node_group_gen5 },
404
373
···
426
389
/* Surface Laptop 5 */
427
390
{ "MSHW0350", (unsigned long)ssam_node_group_sl5 },
428
391
392
+
/* Surface Laptop 6 */
393
+
{ "MSHW0530", (unsigned long)ssam_node_group_sl6 },
394
+
429
395
/* Surface Laptop Go 1 */
430
396
{ "MSHW0118", (unsigned long)ssam_node_group_slg1 },
431
397
432
398
/* Surface Laptop Go 2 */
433
399
{ "MSHW0290", (unsigned long)ssam_node_group_slg1 },
434
400
435
-
/* Surface Laptop Studio */
436
-
{ "MSHW0123", (unsigned long)ssam_node_group_sls },
401
+
/* Surface Laptop Go 3 */
402
+
{ "MSHW0440", (unsigned long)ssam_node_group_slg1 },
403
+
404
+
/* Surface Laptop Studio 1 */
405
+
{ "MSHW0123", (unsigned long)ssam_node_group_sls1 },
406
+
407
+
/* Surface Laptop Studio 2 */
408
+
{ "MSHW0360", (unsigned long)ssam_node_group_sls2 },
437
409
438
410
{ },
439
411
};
+15
-1
drivers/platform/x86/asus-wmi.c
+15
-1
drivers/platform/x86/asus-wmi.c
···
146
146
147
147
static int throttle_thermal_policy_write(struct asus_wmi *);
148
148
149
+
static const struct dmi_system_id asus_ally_mcu_quirk[] = {
150
+
{
151
+
.matches = {
152
+
DMI_MATCH(DMI_BOARD_NAME, "RC71L"),
153
+
},
154
+
},
155
+
{
156
+
.matches = {
157
+
DMI_MATCH(DMI_BOARD_NAME, "RC72L"),
158
+
},
159
+
},
160
+
{ },
161
+
};
162
+
149
163
static bool ashs_present(void)
150
164
{
151
165
int i = 0;
···
4699
4685
asus->dgpu_disable_available = asus_wmi_dev_is_present(asus, ASUS_WMI_DEVID_DGPU);
4700
4686
asus->kbd_rgb_state_available = asus_wmi_dev_is_present(asus, ASUS_WMI_DEVID_TUF_RGB_STATE);
4701
4687
asus->ally_mcu_usb_switch = acpi_has_method(NULL, ASUS_USB0_PWR_EC0_CSEE)
4702
-
&& dmi_match(DMI_BOARD_NAME, "RC71L");
4688
+
&& dmi_check_system(asus_ally_mcu_quirk);
4703
4689
4704
4690
if (asus_wmi_dev_is_present(asus, ASUS_WMI_DEVID_MINI_LED_MODE))
4705
4691
asus->mini_led_dev_id = ASUS_WMI_DEVID_MINI_LED_MODE;
+1
-2
drivers/platform/x86/intel/speed_select_if/isst_tpmi_core.c
+1
-2
drivers/platform/x86/intel/speed_select_if/isst_tpmi_core.c
+23
-13
drivers/s390/block/dasd.c
+23
-13
drivers/s390/block/dasd.c
···
1601
1601
if (!sense)
1602
1602
return 0;
1603
1603
1604
-
return !!(sense[1] & SNS1_NO_REC_FOUND) ||
1605
-
!!(sense[1] & SNS1_FILE_PROTECTED) ||
1606
-
scsw_cstat(&irb->scsw) == SCHN_STAT_INCORR_LEN;
1604
+
if (sense[1] & SNS1_NO_REC_FOUND)
1605
+
return 1;
1606
+
1607
+
if ((sense[1] & SNS1_INV_TRACK_FORMAT) &&
1608
+
scsw_is_tm(&irb->scsw) &&
1609
+
!(sense[2] & SNS2_ENV_DATA_PRESENT))
1610
+
return 1;
1611
+
1612
+
return 0;
1607
1613
}
1608
1614
1609
1615
static int dasd_ese_oos_cond(u8 *sense)
···
1630
1624
struct dasd_device *device;
1631
1625
unsigned long now;
1632
1626
int nrf_suppressed = 0;
1633
-
int fp_suppressed = 0;
1627
+
int it_suppressed = 0;
1634
1628
struct request *req;
1635
1629
u8 *sense = NULL;
1636
1630
int expires;
···
1685
1679
*/
1686
1680
sense = dasd_get_sense(irb);
1687
1681
if (sense) {
1688
-
fp_suppressed = (sense[1] & SNS1_FILE_PROTECTED) &&
1689
-
test_bit(DASD_CQR_SUPPRESS_FP, &cqr->flags);
1682
+
it_suppressed = (sense[1] & SNS1_INV_TRACK_FORMAT) &&
1683
+
!(sense[2] & SNS2_ENV_DATA_PRESENT) &&
1684
+
test_bit(DASD_CQR_SUPPRESS_IT, &cqr->flags);
1690
1685
nrf_suppressed = (sense[1] & SNS1_NO_REC_FOUND) &&
1691
1686
test_bit(DASD_CQR_SUPPRESS_NRF, &cqr->flags);
1692
1687
···
1702
1695
return;
1703
1696
}
1704
1697
}
1705
-
if (!(fp_suppressed || nrf_suppressed))
1698
+
if (!(it_suppressed || nrf_suppressed))
1706
1699
device->discipline->dump_sense_dbf(device, irb, "int");
1707
1700
1708
1701
if (device->features & DASD_FEATURE_ERPLOG)
···
2466
2459
rc = 0;
2467
2460
list_for_each_entry_safe(cqr, n, ccw_queue, blocklist) {
2468
2461
/*
2469
-
* In some cases the 'File Protected' or 'Incorrect Length'
2470
-
* error might be expected and error recovery would be
2471
-
* unnecessary in these cases. Check if the according suppress
2472
-
* bit is set.
2462
+
* In some cases certain errors might be expected and
2463
+
* error recovery would be unnecessary in these cases.
2464
+
* Check if the according suppress bit is set.
2473
2465
*/
2474
2466
sense = dasd_get_sense(&cqr->irb);
2475
-
if (sense && sense[1] & SNS1_FILE_PROTECTED &&
2476
-
test_bit(DASD_CQR_SUPPRESS_FP, &cqr->flags))
2467
+
if (sense && (sense[1] & SNS1_INV_TRACK_FORMAT) &&
2468
+
!(sense[2] & SNS2_ENV_DATA_PRESENT) &&
2469
+
test_bit(DASD_CQR_SUPPRESS_IT, &cqr->flags))
2470
+
continue;
2471
+
if (sense && (sense[1] & SNS1_NO_REC_FOUND) &&
2472
+
test_bit(DASD_CQR_SUPPRESS_NRF, &cqr->flags))
2477
2473
continue;
2478
2474
if (scsw_cstat(&cqr->irb.scsw) == 0x40 &&
2479
2475
test_bit(DASD_CQR_SUPPRESS_IL, &cqr->flags))
+2
-8
drivers/s390/block/dasd_3990_erp.c
+2
-8
drivers/s390/block/dasd_3990_erp.c
···
1386
1386
1387
1387
struct dasd_device *device = erp->startdev;
1388
1388
1389
-
/*
1390
-
* In some cases the 'File Protected' error might be expected and
1391
-
* log messages shouldn't be written then.
1392
-
* Check if the according suppress bit is set.
1393
-
*/
1394
-
if (!test_bit(DASD_CQR_SUPPRESS_FP, &erp->flags))
1395
-
dev_err(&device->cdev->dev,
1396
-
"Accessing the DASD failed because of a hardware error\n");
1389
+
dev_err(&device->cdev->dev,
1390
+
"Accessing the DASD failed because of a hardware error\n");
1397
1391
1398
1392
return dasd_3990_erp_cleanup(erp, DASD_CQR_FAILED);
1399
1393
+25
-32
drivers/s390/block/dasd_eckd.c
+25
-32
drivers/s390/block/dasd_eckd.c
···
2275
2275
cqr->status = DASD_CQR_FILLED;
2276
2276
/* Set flags to suppress output for expected errors */
2277
2277
set_bit(DASD_CQR_SUPPRESS_NRF, &cqr->flags);
2278
+
set_bit(DASD_CQR_SUPPRESS_IT, &cqr->flags);
2278
2279
2279
2280
return cqr;
2280
2281
}
···
2557
2556
cqr->buildclk = get_tod_clock();
2558
2557
cqr->status = DASD_CQR_FILLED;
2559
2558
/* Set flags to suppress output for expected errors */
2560
-
set_bit(DASD_CQR_SUPPRESS_FP, &cqr->flags);
2561
2559
set_bit(DASD_CQR_SUPPRESS_IL, &cqr->flags);
2562
2560
2563
2561
return cqr;
···
4130
4130
4131
4131
/* Set flags to suppress output for expected errors */
4132
4132
if (dasd_eckd_is_ese(basedev)) {
4133
-
set_bit(DASD_CQR_SUPPRESS_FP, &cqr->flags);
4134
-
set_bit(DASD_CQR_SUPPRESS_IL, &cqr->flags);
4135
4133
set_bit(DASD_CQR_SUPPRESS_NRF, &cqr->flags);
4136
4134
}
4137
4135
···
4631
4633
4632
4634
/* Set flags to suppress output for expected errors */
4633
4635
if (dasd_eckd_is_ese(basedev)) {
4634
-
set_bit(DASD_CQR_SUPPRESS_FP, &cqr->flags);
4635
-
set_bit(DASD_CQR_SUPPRESS_IL, &cqr->flags);
4636
4636
set_bit(DASD_CQR_SUPPRESS_NRF, &cqr->flags);
4637
+
set_bit(DASD_CQR_SUPPRESS_IT, &cqr->flags);
4637
4638
}
4638
4639
4639
4640
return cqr;
···
5777
5780
{
5778
5781
u8 *sense = dasd_get_sense(irb);
5779
5782
5780
-
if (scsw_is_tm(&irb->scsw)) {
5781
-
/*
5782
-
* In some cases the 'File Protected' or 'Incorrect Length'
5783
-
* error might be expected and log messages shouldn't be written
5784
-
* then. Check if the according suppress bit is set.
5785
-
*/
5786
-
if (sense && (sense[1] & SNS1_FILE_PROTECTED) &&
5787
-
test_bit(DASD_CQR_SUPPRESS_FP, &req->flags))
5788
-
return;
5789
-
if (scsw_cstat(&irb->scsw) == 0x40 &&
5790
-
test_bit(DASD_CQR_SUPPRESS_IL, &req->flags))
5791
-
return;
5783
+
/*
5784
+
* In some cases certain errors might be expected and
5785
+
* log messages shouldn't be written then.
5786
+
* Check if the according suppress bit is set.
5787
+
*/
5788
+
if (sense && (sense[1] & SNS1_INV_TRACK_FORMAT) &&
5789
+
!(sense[2] & SNS2_ENV_DATA_PRESENT) &&
5790
+
test_bit(DASD_CQR_SUPPRESS_IT, &req->flags))
5791
+
return;
5792
5792
5793
+
if (sense && sense[0] & SNS0_CMD_REJECT &&
5794
+
test_bit(DASD_CQR_SUPPRESS_CR, &req->flags))
5795
+
return;
5796
+
5797
+
if (sense && sense[1] & SNS1_NO_REC_FOUND &&
5798
+
test_bit(DASD_CQR_SUPPRESS_NRF, &req->flags))
5799
+
return;
5800
+
5801
+
if (scsw_cstat(&irb->scsw) == 0x40 &&
5802
+
test_bit(DASD_CQR_SUPPRESS_IL, &req->flags))
5803
+
return;
5804
+
5805
+
if (scsw_is_tm(&irb->scsw))
5793
5806
dasd_eckd_dump_sense_tcw(device, req, irb);
5794
-
} else {
5795
-
/*
5796
-
* In some cases the 'Command Reject' or 'No Record Found'
5797
-
* error might be expected and log messages shouldn't be
5798
-
* written then. Check if the according suppress bit is set.
5799
-
*/
5800
-
if (sense && sense[0] & SNS0_CMD_REJECT &&
5801
-
test_bit(DASD_CQR_SUPPRESS_CR, &req->flags))
5802
-
return;
5803
-
5804
-
if (sense && sense[1] & SNS1_NO_REC_FOUND &&
5805
-
test_bit(DASD_CQR_SUPPRESS_NRF, &req->flags))
5806
-
return;
5807
-
5807
+
else
5808
5808
dasd_eckd_dump_sense_ccw(device, req, irb);
5809
-
}
5810
5809
}
5811
5810
5812
5811
static int dasd_eckd_reload_device(struct dasd_device *device)
-1
drivers/s390/block/dasd_genhd.c
-1
drivers/s390/block/dasd_genhd.c
+1
-1
drivers/s390/block/dasd_int.h
+1
-1
drivers/s390/block/dasd_int.h
···
196
196
* The following flags are used to suppress output of certain errors.
197
197
*/
198
198
#define DASD_CQR_SUPPRESS_NRF 4 /* Suppress 'No Record Found' error */
199
-
#define DASD_CQR_SUPPRESS_FP 5 /* Suppress 'File Protected' error*/
199
+
#define DASD_CQR_SUPPRESS_IT 5 /* Suppress 'Invalid Track' error*/
200
200
#define DASD_CQR_SUPPRESS_IL 6 /* Suppress 'Incorrect Length' error */
201
201
#define DASD_CQR_SUPPRESS_CR 7 /* Suppress 'Command Reject' error */
202
202
+8
-3
drivers/scsi/mpi3mr/mpi3mr_app.c
+8
-3
drivers/scsi/mpi3mr/mpi3mr_app.c
···
100
100
dprint_init(mrioc,
101
101
"trying to allocate trace diag buffer of size = %dKB\n",
102
102
trace_size / 1024);
103
-
if (mpi3mr_alloc_trace_buffer(mrioc, trace_size)) {
103
+
if (get_order(trace_size) > MAX_PAGE_ORDER ||
104
+
mpi3mr_alloc_trace_buffer(mrioc, trace_size)) {
104
105
retry = true;
105
106
trace_size -= trace_dec_size;
106
107
dprint_init(mrioc, "trace diag buffer allocation failed\n"
···
119
118
diag_buffer->type = MPI3_DIAG_BUFFER_TYPE_FW;
120
119
diag_buffer->status = MPI3MR_HDB_BUFSTATUS_NOT_ALLOCATED;
121
120
if ((mrioc->facts.diag_fw_sz < fw_size) && (fw_size >= fw_min_size)) {
122
-
diag_buffer->addr = dma_alloc_coherent(&mrioc->pdev->dev,
123
-
fw_size, &diag_buffer->dma_addr, GFP_KERNEL);
121
+
if (get_order(fw_size) <= MAX_PAGE_ORDER) {
122
+
diag_buffer->addr
123
+
= dma_alloc_coherent(&mrioc->pdev->dev, fw_size,
124
+
&diag_buffer->dma_addr,
125
+
GFP_KERNEL);
126
+
}
124
127
if (!retry)
125
128
dprint_init(mrioc,
126
129
"%s:trying to allocate firmware diag buffer of size = %dKB\n",
+1
drivers/scsi/mpi3mr/mpi3mr_os.c
+1
drivers/scsi/mpi3mr/mpi3mr_os.c
-5
drivers/soc/fsl/qbman/qman.c
-5
drivers/soc/fsl/qbman/qman.c
-1
drivers/staging/media/atomisp/include/linux/atomisp.h
-1
drivers/staging/media/atomisp/include/linux/atomisp.h
+66
-17
drivers/thermal/gov_bang_bang.c
+66
-17
drivers/thermal/gov_bang_bang.c
···
13
13
14
14
#include "thermal_core.h"
15
15
16
+
static void bang_bang_set_instance_target(struct thermal_instance *instance,
17
+
unsigned int target)
18
+
{
19
+
if (instance->target != 0 && instance->target != 1 &&
20
+
instance->target != THERMAL_NO_TARGET)
21
+
pr_debug("Unexpected state %ld of thermal instance %s in bang-bang\n",
22
+
instance->target, instance->name);
23
+
24
+
/*
25
+
* Enable the fan when the trip is crossed on the way up and disable it
26
+
* when the trip is crossed on the way down.
27
+
*/
28
+
instance->target = target;
29
+
instance->initialized = true;
30
+
31
+
dev_dbg(&instance->cdev->device, "target=%ld\n", instance->target);
32
+
33
+
mutex_lock(&instance->cdev->lock);
34
+
__thermal_cdev_update(instance->cdev);
35
+
mutex_unlock(&instance->cdev->lock);
36
+
}
37
+
16
38
/**
17
39
* bang_bang_control - controls devices associated with the given zone
18
40
* @tz: thermal_zone_device
···
76
54
tz->temperature, trip->hysteresis);
77
55
78
56
list_for_each_entry(instance, &tz->thermal_instances, tz_node) {
79
-
if (instance->trip != trip)
57
+
if (instance->trip == trip)
58
+
bang_bang_set_instance_target(instance, crossed_up);
59
+
}
60
+
}
61
+
62
+
static void bang_bang_manage(struct thermal_zone_device *tz)
63
+
{
64
+
const struct thermal_trip_desc *td;
65
+
struct thermal_instance *instance;
66
+
67
+
/* If the code below has run already, nothing needs to be done. */
68
+
if (tz->governor_data)
69
+
return;
70
+
71
+
for_each_trip_desc(tz, td) {
72
+
const struct thermal_trip *trip = &td->trip;
73
+
74
+
if (tz->temperature >= td->threshold ||
75
+
trip->temperature == THERMAL_TEMP_INVALID ||
76
+
trip->type == THERMAL_TRIP_CRITICAL ||
77
+
trip->type == THERMAL_TRIP_HOT)
80
78
continue;
81
79
82
-
if (instance->target != 0 && instance->target != 1 &&
83
-
instance->target != THERMAL_NO_TARGET)
84
-
pr_debug("Unexpected state %ld of thermal instance %s in bang-bang\n",
85
-
instance->target, instance->name);
86
-
87
80
/*
88
-
* Enable the fan when the trip is crossed on the way up and
89
-
* disable it when the trip is crossed on the way down.
81
+
* If the initial cooling device state is "on", but the zone
82
+
* temperature is not above the trip point, the core will not
83
+
* call bang_bang_control() until the zone temperature reaches
84
+
* the trip point temperature which may be never. In those
85
+
* cases, set the initial state of the cooling device to 0.
90
86
*/
91
-
instance->target = crossed_up;
92
-
93
-
dev_dbg(&instance->cdev->device, "target=%ld\n", instance->target);
94
-
95
-
mutex_lock(&instance->cdev->lock);
96
-
instance->cdev->updated = false; /* cdev needs update */
97
-
mutex_unlock(&instance->cdev->lock);
87
+
list_for_each_entry(instance, &tz->thermal_instances, tz_node) {
88
+
if (!instance->initialized && instance->trip == trip)
89
+
bang_bang_set_instance_target(instance, 0);
90
+
}
98
91
}
99
92
100
-
list_for_each_entry(instance, &tz->thermal_instances, tz_node)
101
-
thermal_cdev_update(instance->cdev);
93
+
tz->governor_data = (void *)true;
94
+
}
95
+
96
+
static void bang_bang_update_tz(struct thermal_zone_device *tz,
97
+
enum thermal_notify_event reason)
98
+
{
99
+
/*
100
+
* Let bang_bang_manage() know that it needs to walk trips after binding
101
+
* a new cdev and after system resume.
102
+
*/
103
+
if (reason == THERMAL_TZ_BIND_CDEV || reason == THERMAL_TZ_RESUME)
104
+
tz->governor_data = NULL;
102
105
}
103
106
104
107
static struct thermal_governor thermal_gov_bang_bang = {
105
108
.name = "bang_bang",
106
109
.trip_crossed = bang_bang_control,
110
+
.manage = bang_bang_manage,
111
+
.update_tz = bang_bang_update_tz,
107
112
};
108
113
THERMAL_GOVERNOR_DECLARE(thermal_gov_bang_bang);
+2
-1
drivers/thermal/thermal_core.c
+2
-1
drivers/thermal/thermal_core.c
···
1728
1728
1729
1729
thermal_debug_tz_resume(tz);
1730
1730
thermal_zone_device_init(tz);
1731
-
__thermal_zone_device_update(tz, THERMAL_EVENT_UNSPECIFIED);
1731
+
thermal_governor_update_tz(tz, THERMAL_TZ_RESUME);
1732
+
__thermal_zone_device_update(tz, THERMAL_TZ_RESUME);
1732
1733
1733
1734
complete(&tz->resume);
1734
1735
tz->resuming = false;
+6
-4
drivers/thunderbolt/debugfs.c
+6
-4
drivers/thunderbolt/debugfs.c
···
323
323
324
324
if (mutex_lock_interruptible(&tb->lock)) {
325
325
ret = -ERESTARTSYS;
326
-
goto out_rpm_put;
326
+
goto out;
327
327
}
328
328
329
329
ret = sb_regs_write(port, port_sb_regs, ARRAY_SIZE(port_sb_regs),
330
330
USB4_SB_TARGET_ROUTER, 0, buf, count, ppos);
331
331
332
332
mutex_unlock(&tb->lock);
333
-
out_rpm_put:
333
+
out:
334
334
pm_runtime_mark_last_busy(&sw->dev);
335
335
pm_runtime_put_autosuspend(&sw->dev);
336
+
free_page((unsigned long)buf);
336
337
337
338
return ret < 0 ? ret : count;
338
339
}
···
356
355
357
356
if (mutex_lock_interruptible(&tb->lock)) {
358
357
ret = -ERESTARTSYS;
359
-
goto out_rpm_put;
358
+
goto out;
360
359
}
361
360
362
361
ret = sb_regs_write(rt->port, retimer_sb_regs, ARRAY_SIZE(retimer_sb_regs),
363
362
USB4_SB_TARGET_RETIMER, rt->index, buf, count, ppos);
364
363
365
364
mutex_unlock(&tb->lock);
366
-
out_rpm_put:
365
+
out:
367
366
pm_runtime_mark_last_busy(&rt->dev);
368
367
pm_runtime_put_autosuspend(&rt->dev);
368
+
free_page((unsigned long)buf);
369
369
370
370
return ret < 0 ? ret : count;
371
371
}
+1
drivers/thunderbolt/switch.c
+1
drivers/thunderbolt/switch.c
+5
-28
drivers/tty/serial/8250/8250_omap.c
+5
-28
drivers/tty/serial/8250/8250_omap.c
···
27
27
#include <linux/pm_wakeirq.h>
28
28
#include <linux/dma-mapping.h>
29
29
#include <linux/sys_soc.h>
30
-
#include <linux/pm_domain.h>
31
30
32
31
#include "8250.h"
33
32
···
117
118
/* Timeout low and High */
118
119
#define UART_OMAP_TO_L 0x26
119
120
#define UART_OMAP_TO_H 0x27
120
-
121
-
/*
122
-
* Copy of the genpd flags for the console.
123
-
* Only used if console suspend is disabled
124
-
*/
125
-
static unsigned int genpd_flags_console;
126
121
127
122
struct omap8250_priv {
128
123
void __iomem *membase;
···
1648
1655
{
1649
1656
struct omap8250_priv *priv = dev_get_drvdata(dev);
1650
1657
struct uart_8250_port *up = serial8250_get_port(priv->line);
1651
-
struct generic_pm_domain *genpd = pd_to_genpd(dev->pm_domain);
1652
1658
int err = 0;
1653
1659
1654
1660
serial8250_suspend_port(priv->line);
···
1658
1666
if (!device_may_wakeup(dev))
1659
1667
priv->wer = 0;
1660
1668
serial_out(up, UART_OMAP_WER, priv->wer);
1661
-
if (uart_console(&up->port)) {
1662
-
if (console_suspend_enabled)
1663
-
err = pm_runtime_force_suspend(dev);
1664
-
else {
1665
-
/*
1666
-
* The pd shall not be powered-off (no console suspend).
1667
-
* Make copy of genpd flags before to set it always on.
1668
-
* The original value is restored during the resume.
1669
-
*/
1670
-
genpd_flags_console = genpd->flags;
1671
-
genpd->flags |= GENPD_FLAG_ALWAYS_ON;
1672
-
}
1673
-
}
1669
+
if (uart_console(&up->port) && console_suspend_enabled)
1670
+
err = pm_runtime_force_suspend(dev);
1674
1671
flush_work(&priv->qos_work);
1675
1672
1676
1673
return err;
···
1669
1688
{
1670
1689
struct omap8250_priv *priv = dev_get_drvdata(dev);
1671
1690
struct uart_8250_port *up = serial8250_get_port(priv->line);
1672
-
struct generic_pm_domain *genpd = pd_to_genpd(dev->pm_domain);
1673
1691
int err;
1674
1692
1675
1693
if (uart_console(&up->port) && console_suspend_enabled) {
1676
-
if (console_suspend_enabled) {
1677
-
err = pm_runtime_force_resume(dev);
1678
-
if (err)
1679
-
return err;
1680
-
} else
1681
-
genpd->flags = genpd_flags_console;
1694
+
err = pm_runtime_force_resume(dev);
1695
+
if (err)
1696
+
return err;
1682
1697
}
1683
1698
1684
1699
serial8250_resume_port(priv->line);
+1
-1
drivers/tty/serial/atmel_serial.c
+1
-1
drivers/tty/serial/atmel_serial.c
···
2514
2514
};
2515
2515
2516
2516
static const struct serial_rs485 atmel_rs485_supported = {
2517
-
.flags = SER_RS485_ENABLED | SER_RS485_RTS_AFTER_SEND | SER_RS485_RX_DURING_TX,
2517
+
.flags = SER_RS485_ENABLED | SER_RS485_RTS_ON_SEND | SER_RS485_RX_DURING_TX,
2518
2518
.delay_rts_before_send = 1,
2519
2519
.delay_rts_after_send = 1,
2520
2520
};
+1
drivers/tty/serial/fsl_lpuart.c
+1
drivers/tty/serial/fsl_lpuart.c
+2
-10
drivers/tty/vt/conmakehash.c
+2
-10
drivers/tty/vt/conmakehash.c
···
11
11
* Copyright (C) 1995-1997 H. Peter Anvin
12
12
*/
13
13
14
-
#include <libgen.h>
15
-
#include <linux/limits.h>
16
14
#include <stdio.h>
17
15
#include <stdlib.h>
18
16
#include <sysexits.h>
···
77
79
{
78
80
FILE *ctbl;
79
81
const char *tblname;
80
-
char base_tblname[PATH_MAX];
81
82
char buffer[65536];
82
83
int fontlen;
83
84
int i, nuni, nent;
···
242
245
for ( i = 0 ; i < fontlen ; i++ )
243
246
nuni += unicount[i];
244
247
245
-
strncpy(base_tblname, tblname, PATH_MAX);
246
-
base_tblname[PATH_MAX - 1] = 0;
247
248
printf("\
248
249
/*\n\
249
-
* Do not edit this file; it was automatically generated by\n\
250
-
*\n\
251
-
* conmakehash %s > [this file]\n\
252
-
*\n\
250
+
* Automatically generated file; Do not edit.\n\
253
251
*/\n\
254
252
\n\
255
253
#include <linux/types.h>\n\
256
254
\n\
257
255
u8 dfont_unicount[%d] = \n\
258
-
{\n\t", basename(base_tblname), fontlen);
256
+
{\n\t", fontlen);
259
257
260
258
for ( i = 0 ; i < fontlen ; i++ )
261
259
{
+1
-1
drivers/usb/host/xhci-mem.c
+1
-1
drivers/usb/host/xhci-mem.c
···
1872
1872
1873
1873
cancel_delayed_work_sync(&xhci->cmd_timer);
1874
1874
1875
-
for (i = 0; i < xhci->max_interrupters; i++) {
1875
+
for (i = 0; xhci->interrupters && i < xhci->max_interrupters; i++) {
1876
1876
if (xhci->interrupters[i]) {
1877
1877
xhci_remove_interrupter(xhci, xhci->interrupters[i]);
1878
1878
xhci_free_interrupter(xhci, xhci->interrupters[i]);
+1
drivers/usb/host/xhci-ring.c
+1
drivers/usb/host/xhci-ring.c
+5
-3
drivers/usb/host/xhci.c
+5
-3
drivers/usb/host/xhci.c
···
2837
2837
xhci->num_active_eps);
2838
2838
return -ENOMEM;
2839
2839
}
2840
-
if ((xhci->quirks & XHCI_SW_BW_CHECKING) &&
2840
+
if ((xhci->quirks & XHCI_SW_BW_CHECKING) && !ctx_change &&
2841
2841
xhci_reserve_bandwidth(xhci, virt_dev, command->in_ctx)) {
2842
2842
if ((xhci->quirks & XHCI_EP_LIMIT_QUIRK))
2843
2843
xhci_free_host_resources(xhci, ctrl_ctx);
···
4200
4200
mutex_unlock(&xhci->mutex);
4201
4201
ret = xhci_disable_slot(xhci, udev->slot_id);
4202
4202
xhci_free_virt_device(xhci, udev->slot_id);
4203
-
if (!ret)
4204
-
xhci_alloc_dev(hcd, udev);
4203
+
if (!ret) {
4204
+
if (xhci_alloc_dev(hcd, udev) == 1)
4205
+
xhci_setup_addressable_virt_dev(xhci, udev);
4206
+
}
4205
4207
kfree(command->completion);
4206
4208
kfree(command);
4207
4209
return -EPROTO;
+1
drivers/usb/misc/usb-ljca.c
+1
drivers/usb/misc/usb-ljca.c
-1
drivers/usb/typec/tcpm/tcpm.c
-1
drivers/usb/typec/tcpm/tcpm.c
+1
-1
drivers/usb/typec/ucsi/ucsi.c
+1
-1
drivers/usb/typec/ucsi/ucsi.c
+42
-35
fs/bcachefs/alloc_background.c
+42
-35
fs/bcachefs/alloc_background.c
···
196
196
return DIV_ROUND_UP(bytes, sizeof(u64));
197
197
}
198
198
199
-
int bch2_alloc_v1_invalid(struct bch_fs *c, struct bkey_s_c k,
200
-
enum bch_validate_flags flags,
201
-
struct printbuf *err)
199
+
int bch2_alloc_v1_validate(struct bch_fs *c, struct bkey_s_c k,
200
+
enum bch_validate_flags flags)
202
201
{
203
202
struct bkey_s_c_alloc a = bkey_s_c_to_alloc(k);
204
203
int ret = 0;
205
204
206
205
/* allow for unknown fields */
207
-
bkey_fsck_err_on(bkey_val_u64s(a.k) < bch_alloc_v1_val_u64s(a.v), c, err,
208
-
alloc_v1_val_size_bad,
206
+
bkey_fsck_err_on(bkey_val_u64s(a.k) < bch_alloc_v1_val_u64s(a.v),
207
+
c, alloc_v1_val_size_bad,
209
208
"incorrect value size (%zu < %u)",
210
209
bkey_val_u64s(a.k), bch_alloc_v1_val_u64s(a.v));
211
210
fsck_err:
212
211
return ret;
213
212
}
214
213
215
-
int bch2_alloc_v2_invalid(struct bch_fs *c, struct bkey_s_c k,
216
-
enum bch_validate_flags flags,
217
-
struct printbuf *err)
214
+
int bch2_alloc_v2_validate(struct bch_fs *c, struct bkey_s_c k,
215
+
enum bch_validate_flags flags)
218
216
{
219
217
struct bkey_alloc_unpacked u;
220
218
int ret = 0;
221
219
222
-
bkey_fsck_err_on(bch2_alloc_unpack_v2(&u, k), c, err,
223
-
alloc_v2_unpack_error,
220
+
bkey_fsck_err_on(bch2_alloc_unpack_v2(&u, k),
221
+
c, alloc_v2_unpack_error,
224
222
"unpack error");
225
223
fsck_err:
226
224
return ret;
227
225
}
228
226
229
-
int bch2_alloc_v3_invalid(struct bch_fs *c, struct bkey_s_c k,
230
-
enum bch_validate_flags flags,
231
-
struct printbuf *err)
227
+
int bch2_alloc_v3_validate(struct bch_fs *c, struct bkey_s_c k,
228
+
enum bch_validate_flags flags)
232
229
{
233
230
struct bkey_alloc_unpacked u;
234
231
int ret = 0;
235
232
236
-
bkey_fsck_err_on(bch2_alloc_unpack_v3(&u, k), c, err,
237
-
alloc_v2_unpack_error,
233
+
bkey_fsck_err_on(bch2_alloc_unpack_v3(&u, k),
234
+
c, alloc_v2_unpack_error,
238
235
"unpack error");
239
236
fsck_err:
240
237
return ret;
241
238
}
242
239
243
-
int bch2_alloc_v4_invalid(struct bch_fs *c, struct bkey_s_c k,
244
-
enum bch_validate_flags flags, struct printbuf *err)
240
+
int bch2_alloc_v4_validate(struct bch_fs *c, struct bkey_s_c k,
241
+
enum bch_validate_flags flags)
245
242
{
246
243
struct bkey_s_c_alloc_v4 a = bkey_s_c_to_alloc_v4(k);
247
244
int ret = 0;
248
245
249
-
bkey_fsck_err_on(alloc_v4_u64s_noerror(a.v) > bkey_val_u64s(k.k), c, err,
250
-
alloc_v4_val_size_bad,
246
+
bkey_fsck_err_on(alloc_v4_u64s_noerror(a.v) > bkey_val_u64s(k.k),
247
+
c, alloc_v4_val_size_bad,
251
248
"bad val size (%u > %zu)",
252
249
alloc_v4_u64s_noerror(a.v), bkey_val_u64s(k.k));
253
250
254
251
bkey_fsck_err_on(!BCH_ALLOC_V4_BACKPOINTERS_START(a.v) &&
255
-
BCH_ALLOC_V4_NR_BACKPOINTERS(a.v), c, err,
256
-
alloc_v4_backpointers_start_bad,
252
+
BCH_ALLOC_V4_NR_BACKPOINTERS(a.v),
253
+
c, alloc_v4_backpointers_start_bad,
257
254
"invalid backpointers_start");
258
255
259
-
bkey_fsck_err_on(alloc_data_type(*a.v, a.v->data_type) != a.v->data_type, c, err,
260
-
alloc_key_data_type_bad,
256
+
bkey_fsck_err_on(alloc_data_type(*a.v, a.v->data_type) != a.v->data_type,
257
+
c, alloc_key_data_type_bad,
261
258
"invalid data type (got %u should be %u)",
262
259
a.v->data_type, alloc_data_type(*a.v, a.v->data_type));
263
260
264
261
for (unsigned i = 0; i < 2; i++)
265
262
bkey_fsck_err_on(a.v->io_time[i] > LRU_TIME_MAX,
266
-
c, err,
267
-
alloc_key_io_time_bad,
263
+
c, alloc_key_io_time_bad,
268
264
"invalid io_time[%s]: %llu, max %llu",
269
265
i == READ ? "read" : "write",
270
266
a.v->io_time[i], LRU_TIME_MAX);
···
278
282
a.v->dirty_sectors ||
279
283
a.v->cached_sectors ||
280
284
a.v->stripe,
281
-
c, err, alloc_key_empty_but_have_data,
285
+
c, alloc_key_empty_but_have_data,
282
286
"empty data type free but have data %u.%u.%u %u",
283
287
stripe_sectors,
284
288
a.v->dirty_sectors,
···
292
296
case BCH_DATA_parity:
293
297
bkey_fsck_err_on(!a.v->dirty_sectors &&
294
298
!stripe_sectors,
295
-
c, err, alloc_key_dirty_sectors_0,
299
+
c, alloc_key_dirty_sectors_0,
296
300
"data_type %s but dirty_sectors==0",
297
301
bch2_data_type_str(a.v->data_type));
298
302
break;
···
301
305
a.v->dirty_sectors ||
302
306
stripe_sectors ||
303
307
a.v->stripe,
304
-
c, err, alloc_key_cached_inconsistency,
308
+
c, alloc_key_cached_inconsistency,
305
309
"data type inconsistency");
306
310
307
311
bkey_fsck_err_on(!a.v->io_time[READ] &&
308
312
c->curr_recovery_pass > BCH_RECOVERY_PASS_check_alloc_to_lru_refs,
309
-
c, err, alloc_key_cached_but_read_time_zero,
313
+
c, alloc_key_cached_but_read_time_zero,
310
314
"cached bucket with read_time == 0");
311
315
break;
312
316
case BCH_DATA_stripe:
···
509
513
: 0;
510
514
}
511
515
512
-
int bch2_bucket_gens_invalid(struct bch_fs *c, struct bkey_s_c k,
513
-
enum bch_validate_flags flags,
514
-
struct printbuf *err)
516
+
int bch2_bucket_gens_validate(struct bch_fs *c, struct bkey_s_c k,
517
+
enum bch_validate_flags flags)
515
518
{
516
519
int ret = 0;
517
520
518
-
bkey_fsck_err_on(bkey_val_bytes(k.k) != sizeof(struct bch_bucket_gens), c, err,
519
-
bucket_gens_val_size_bad,
521
+
bkey_fsck_err_on(bkey_val_bytes(k.k) != sizeof(struct bch_bucket_gens),
522
+
c, bucket_gens_val_size_bad,
520
523
"bad val size (%zu != %zu)",
521
524
bkey_val_bytes(k.k), sizeof(struct bch_bucket_gens));
522
525
fsck_err:
···
824
829
825
830
struct bch_alloc_v4 old_a_convert;
826
831
const struct bch_alloc_v4 *old_a = bch2_alloc_to_v4(old, &old_a_convert);
827
-
struct bch_alloc_v4 *new_a = bkey_s_to_alloc_v4(new).v;
832
+
833
+
struct bch_alloc_v4 *new_a;
834
+
if (likely(new.k->type == KEY_TYPE_alloc_v4)) {
835
+
new_a = bkey_s_to_alloc_v4(new).v;
836
+
} else {
837
+
BUG_ON(!(flags & BTREE_TRIGGER_gc));
838
+
839
+
struct bkey_i_alloc_v4 *new_ka = bch2_alloc_to_v4_mut_inlined(trans, new.s_c);
840
+
ret = PTR_ERR_OR_ZERO(new_ka);
841
+
if (unlikely(ret))
842
+
goto err;
843
+
new_a = &new_ka->v;
844
+
}
828
845
829
846
if (flags & BTREE_TRIGGER_transactional) {
830
847
alloc_data_type_set(new_a, new_a->data_type);
+14
-16
fs/bcachefs/alloc_background.h
+14
-16
fs/bcachefs/alloc_background.h
···
150
150
151
151
static inline u64 alloc_lru_idx_read(struct bch_alloc_v4 a)
152
152
{
153
-
return a.data_type == BCH_DATA_cached ? a.io_time[READ] : 0;
153
+
return a.data_type == BCH_DATA_cached
154
+
? a.io_time[READ] & LRU_TIME_MAX
155
+
: 0;
154
156
}
155
157
156
158
#define DATA_TYPES_MOVABLE \
···
242
240
243
241
int bch2_bucket_io_time_reset(struct btree_trans *, unsigned, size_t, int);
244
242
245
-
int bch2_alloc_v1_invalid(struct bch_fs *, struct bkey_s_c,
246
-
enum bch_validate_flags, struct printbuf *);
247
-
int bch2_alloc_v2_invalid(struct bch_fs *, struct bkey_s_c,
248
-
enum bch_validate_flags, struct printbuf *);
249
-
int bch2_alloc_v3_invalid(struct bch_fs *, struct bkey_s_c,
250
-
enum bch_validate_flags, struct printbuf *);
251
-
int bch2_alloc_v4_invalid(struct bch_fs *, struct bkey_s_c,
252
-
enum bch_validate_flags, struct printbuf *);
243
+
int bch2_alloc_v1_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
244
+
int bch2_alloc_v2_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
245
+
int bch2_alloc_v3_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
246
+
int bch2_alloc_v4_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
253
247
void bch2_alloc_v4_swab(struct bkey_s);
254
248
void bch2_alloc_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
255
249
256
250
#define bch2_bkey_ops_alloc ((struct bkey_ops) { \
257
-
.key_invalid = bch2_alloc_v1_invalid, \
251
+
.key_validate = bch2_alloc_v1_validate, \
258
252
.val_to_text = bch2_alloc_to_text, \
259
253
.trigger = bch2_trigger_alloc, \
260
254
.min_val_size = 8, \
261
255
})
262
256
263
257
#define bch2_bkey_ops_alloc_v2 ((struct bkey_ops) { \
264
-
.key_invalid = bch2_alloc_v2_invalid, \
258
+
.key_validate = bch2_alloc_v2_validate, \
265
259
.val_to_text = bch2_alloc_to_text, \
266
260
.trigger = bch2_trigger_alloc, \
267
261
.min_val_size = 8, \
268
262
})
269
263
270
264
#define bch2_bkey_ops_alloc_v3 ((struct bkey_ops) { \
271
-
.key_invalid = bch2_alloc_v3_invalid, \
265
+
.key_validate = bch2_alloc_v3_validate, \
272
266
.val_to_text = bch2_alloc_to_text, \
273
267
.trigger = bch2_trigger_alloc, \
274
268
.min_val_size = 16, \
275
269
})
276
270
277
271
#define bch2_bkey_ops_alloc_v4 ((struct bkey_ops) { \
278
-
.key_invalid = bch2_alloc_v4_invalid, \
272
+
.key_validate = bch2_alloc_v4_validate, \
279
273
.val_to_text = bch2_alloc_to_text, \
280
274
.swab = bch2_alloc_v4_swab, \
281
275
.trigger = bch2_trigger_alloc, \
282
276
.min_val_size = 48, \
283
277
})
284
278
285
-
int bch2_bucket_gens_invalid(struct bch_fs *, struct bkey_s_c,
286
-
enum bch_validate_flags, struct printbuf *);
279
+
int bch2_bucket_gens_validate(struct bch_fs *, struct bkey_s_c,
280
+
enum bch_validate_flags);
287
281
void bch2_bucket_gens_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
288
282
289
283
#define bch2_bkey_ops_bucket_gens ((struct bkey_ops) { \
290
-
.key_invalid = bch2_bucket_gens_invalid, \
284
+
.key_validate = bch2_bucket_gens_validate, \
291
285
.val_to_text = bch2_bucket_gens_to_text, \
292
286
})
293
287
+8
-15
fs/bcachefs/backpointers.c
+8
-15
fs/bcachefs/backpointers.c
···
47
47
return false;
48
48
}
49
49
50
-
int bch2_backpointer_invalid(struct bch_fs *c, struct bkey_s_c k,
51
-
enum bch_validate_flags flags,
52
-
struct printbuf *err)
50
+
int bch2_backpointer_validate(struct bch_fs *c, struct bkey_s_c k,
51
+
enum bch_validate_flags flags)
53
52
{
54
53
struct bkey_s_c_backpointer bp = bkey_s_c_to_backpointer(k);
55
54
···
67
68
68
69
bkey_fsck_err_on((bp.v->bucket_offset >> MAX_EXTENT_COMPRESS_RATIO_SHIFT) >= ca->mi.bucket_size ||
69
70
!bpos_eq(bp.k->p, bp_pos),
70
-
c, err,
71
-
backpointer_bucket_offset_wrong,
71
+
c, backpointer_bucket_offset_wrong,
72
72
"backpointer bucket_offset wrong");
73
73
fsck_err:
74
74
return ret;
···
761
763
btree < BTREE_ID_NR && !ret;
762
764
btree++) {
763
765
unsigned depth = (BIT_ULL(btree) & btree_leaf_mask) ? 0 : 1;
764
-
struct btree_iter iter;
765
-
struct btree *b;
766
766
767
767
if (!(BIT_ULL(btree) & btree_leaf_mask) &&
768
768
!(BIT_ULL(btree) & btree_interior_mask))
769
769
continue;
770
770
771
-
bch2_trans_begin(trans);
772
-
773
-
__for_each_btree_node(trans, iter, btree,
771
+
ret = __for_each_btree_node(trans, iter, btree,
774
772
btree == start.btree ? start.pos : POS_MIN,
775
-
0, depth, BTREE_ITER_prefetch, b, ret) {
773
+
0, depth, BTREE_ITER_prefetch, b, ({
776
774
mem_may_pin -= btree_buf_bytes(b);
777
775
if (mem_may_pin <= 0) {
778
776
c->btree_cache.pinned_nodes_end = *end =
779
777
BBPOS(btree, b->key.k.p);
780
-
bch2_trans_iter_exit(trans, &iter);
781
-
return 0;
778
+
break;
782
779
}
783
-
}
784
-
bch2_trans_iter_exit(trans, &iter);
780
+
0;
781
+
}));
785
782
}
786
783
787
784
return ret;
+2
-3
fs/bcachefs/backpointers.h
+2
-3
fs/bcachefs/backpointers.h
···
18
18
((x & 0xff00000000ULL) >> 32));
19
19
}
20
20
21
-
int bch2_backpointer_invalid(struct bch_fs *, struct bkey_s_c k,
22
-
enum bch_validate_flags, struct printbuf *);
21
+
int bch2_backpointer_validate(struct bch_fs *, struct bkey_s_c k, enum bch_validate_flags);
23
22
void bch2_backpointer_to_text(struct printbuf *, const struct bch_backpointer *);
24
23
void bch2_backpointer_k_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
25
24
void bch2_backpointer_swab(struct bkey_s);
26
25
27
26
#define bch2_bkey_ops_backpointer ((struct bkey_ops) { \
28
-
.key_invalid = bch2_backpointer_invalid, \
27
+
.key_validate = bch2_backpointer_validate, \
29
28
.val_to_text = bch2_backpointer_k_to_text, \
30
29
.swab = bch2_backpointer_swab, \
31
30
.min_val_size = 32, \
+1
fs/bcachefs/bcachefs.h
+1
fs/bcachefs/bcachefs.h
+2
-1
fs/bcachefs/bcachefs_format.h
+2
-1
fs/bcachefs/bcachefs_format.h
···
676
676
x(mi_btree_bitmap, BCH_VERSION(1, 7)) \
677
677
x(bucket_stripe_sectors, BCH_VERSION(1, 8)) \
678
678
x(disk_accounting_v2, BCH_VERSION(1, 9)) \
679
-
x(disk_accounting_v3, BCH_VERSION(1, 10))
679
+
x(disk_accounting_v3, BCH_VERSION(1, 10)) \
680
+
x(disk_accounting_inum, BCH_VERSION(1, 11))
680
681
681
682
enum bcachefs_metadata_version {
682
683
bcachefs_metadata_version_min = 9,
+4
-3
fs/bcachefs/bkey.h
+4
-3
fs/bcachefs/bkey.h
···
10
10
#include "vstructs.h"
11
11
12
12
enum bch_validate_flags {
13
-
BCH_VALIDATE_write = (1U << 0),
14
-
BCH_VALIDATE_commit = (1U << 1),
15
-
BCH_VALIDATE_journal = (1U << 2),
13
+
BCH_VALIDATE_write = BIT(0),
14
+
BCH_VALIDATE_commit = BIT(1),
15
+
BCH_VALIDATE_journal = BIT(2),
16
+
BCH_VALIDATE_silent = BIT(3),
16
17
};
17
18
18
19
#if 0
+53
-56
fs/bcachefs/bkey_methods.c
+53
-56
fs/bcachefs/bkey_methods.c
···
27
27
NULL
28
28
};
29
29
30
-
static int deleted_key_invalid(struct bch_fs *c, struct bkey_s_c k,
31
-
enum bch_validate_flags flags, struct printbuf *err)
30
+
static int deleted_key_validate(struct bch_fs *c, struct bkey_s_c k,
31
+
enum bch_validate_flags flags)
32
32
{
33
33
return 0;
34
34
}
35
35
36
36
#define bch2_bkey_ops_deleted ((struct bkey_ops) { \
37
-
.key_invalid = deleted_key_invalid, \
37
+
.key_validate = deleted_key_validate, \
38
38
})
39
39
40
40
#define bch2_bkey_ops_whiteout ((struct bkey_ops) { \
41
-
.key_invalid = deleted_key_invalid, \
41
+
.key_validate = deleted_key_validate, \
42
42
})
43
43
44
-
static int empty_val_key_invalid(struct bch_fs *c, struct bkey_s_c k,
45
-
enum bch_validate_flags flags, struct printbuf *err)
44
+
static int empty_val_key_validate(struct bch_fs *c, struct bkey_s_c k,
45
+
enum bch_validate_flags flags)
46
46
{
47
47
int ret = 0;
48
48
49
-
bkey_fsck_err_on(bkey_val_bytes(k.k), c, err,
50
-
bkey_val_size_nonzero,
49
+
bkey_fsck_err_on(bkey_val_bytes(k.k),
50
+
c, bkey_val_size_nonzero,
51
51
"incorrect value size (%zu != 0)",
52
52
bkey_val_bytes(k.k));
53
53
fsck_err:
···
55
55
}
56
56
57
57
#define bch2_bkey_ops_error ((struct bkey_ops) { \
58
-
.key_invalid = empty_val_key_invalid, \
58
+
.key_validate = empty_val_key_validate, \
59
59
})
60
60
61
-
static int key_type_cookie_invalid(struct bch_fs *c, struct bkey_s_c k,
62
-
enum bch_validate_flags flags, struct printbuf *err)
61
+
static int key_type_cookie_validate(struct bch_fs *c, struct bkey_s_c k,
62
+
enum bch_validate_flags flags)
63
63
{
64
64
return 0;
65
65
}
···
73
73
}
74
74
75
75
#define bch2_bkey_ops_cookie ((struct bkey_ops) { \
76
-
.key_invalid = key_type_cookie_invalid, \
76
+
.key_validate = key_type_cookie_validate, \
77
77
.val_to_text = key_type_cookie_to_text, \
78
78
.min_val_size = 8, \
79
79
})
80
80
81
81
#define bch2_bkey_ops_hash_whiteout ((struct bkey_ops) {\
82
-
.key_invalid = empty_val_key_invalid, \
82
+
.key_validate = empty_val_key_validate, \
83
83
})
84
84
85
-
static int key_type_inline_data_invalid(struct bch_fs *c, struct bkey_s_c k,
86
-
enum bch_validate_flags flags, struct printbuf *err)
85
+
static int key_type_inline_data_validate(struct bch_fs *c, struct bkey_s_c k,
86
+
enum bch_validate_flags flags)
87
87
{
88
88
return 0;
89
89
}
···
98
98
datalen, min(datalen, 32U), d.v->data);
99
99
}
100
100
101
-
#define bch2_bkey_ops_inline_data ((struct bkey_ops) { \
102
-
.key_invalid = key_type_inline_data_invalid, \
103
-
.val_to_text = key_type_inline_data_to_text, \
101
+
#define bch2_bkey_ops_inline_data ((struct bkey_ops) { \
102
+
.key_validate = key_type_inline_data_validate, \
103
+
.val_to_text = key_type_inline_data_to_text, \
104
104
})
105
105
106
106
static bool key_type_set_merge(struct bch_fs *c, struct bkey_s l, struct bkey_s_c r)
···
110
110
}
111
111
112
112
#define bch2_bkey_ops_set ((struct bkey_ops) { \
113
-
.key_invalid = empty_val_key_invalid, \
113
+
.key_validate = empty_val_key_validate, \
114
114
.key_merge = key_type_set_merge, \
115
115
})
116
116
···
123
123
const struct bkey_ops bch2_bkey_null_ops = {
124
124
};
125
125
126
-
int bch2_bkey_val_invalid(struct bch_fs *c, struct bkey_s_c k,
127
-
enum bch_validate_flags flags,
128
-
struct printbuf *err)
126
+
int bch2_bkey_val_validate(struct bch_fs *c, struct bkey_s_c k,
127
+
enum bch_validate_flags flags)
129
128
{
130
129
if (test_bit(BCH_FS_no_invalid_checks, &c->flags))
131
130
return 0;
···
132
133
const struct bkey_ops *ops = bch2_bkey_type_ops(k.k->type);
133
134
int ret = 0;
134
135
135
-
bkey_fsck_err_on(bkey_val_bytes(k.k) < ops->min_val_size, c, err,
136
-
bkey_val_size_too_small,
136
+
bkey_fsck_err_on(bkey_val_bytes(k.k) < ops->min_val_size,
137
+
c, bkey_val_size_too_small,
137
138
"bad val size (%zu < %u)",
138
139
bkey_val_bytes(k.k), ops->min_val_size);
139
140
140
-
if (!ops->key_invalid)
141
+
if (!ops->key_validate)
141
142
return 0;
142
143
143
-
ret = ops->key_invalid(c, k, flags, err);
144
+
ret = ops->key_validate(c, k, flags);
144
145
fsck_err:
145
146
return ret;
146
147
}
···
160
161
return type == BKEY_TYPE_btree ? "internal btree node" : bch2_btree_id_str(type - 1);
161
162
}
162
163
163
-
int __bch2_bkey_invalid(struct bch_fs *c, struct bkey_s_c k,
164
-
enum btree_node_type type,
165
-
enum bch_validate_flags flags,
166
-
struct printbuf *err)
164
+
int __bch2_bkey_validate(struct bch_fs *c, struct bkey_s_c k,
165
+
enum btree_node_type type,
166
+
enum bch_validate_flags flags)
167
167
{
168
168
if (test_bit(BCH_FS_no_invalid_checks, &c->flags))
169
169
return 0;
170
170
171
171
int ret = 0;
172
172
173
-
bkey_fsck_err_on(k.k->u64s < BKEY_U64s, c, err,
174
-
bkey_u64s_too_small,
173
+
bkey_fsck_err_on(k.k->u64s < BKEY_U64s,
174
+
c, bkey_u64s_too_small,
175
175
"u64s too small (%u < %zu)", k.k->u64s, BKEY_U64s);
176
176
177
177
if (type >= BKEY_TYPE_NR)
···
178
180
179
181
bkey_fsck_err_on(k.k->type < KEY_TYPE_MAX &&
180
182
(type == BKEY_TYPE_btree || (flags & BCH_VALIDATE_commit)) &&
181
-
!(bch2_key_types_allowed[type] & BIT_ULL(k.k->type)), c, err,
182
-
bkey_invalid_type_for_btree,
183
+
!(bch2_key_types_allowed[type] & BIT_ULL(k.k->type)),
184
+
c, bkey_invalid_type_for_btree,
183
185
"invalid key type for btree %s (%s)",
184
186
bch2_btree_node_type_str(type),
185
187
k.k->type < KEY_TYPE_MAX
···
187
189
: "(unknown)");
188
190
189
191
if (btree_node_type_is_extents(type) && !bkey_whiteout(k.k)) {
190
-
bkey_fsck_err_on(k.k->size == 0, c, err,
191
-
bkey_extent_size_zero,
192
+
bkey_fsck_err_on(k.k->size == 0,
193
+
c, bkey_extent_size_zero,
192
194
"size == 0");
193
195
194
-
bkey_fsck_err_on(k.k->size > k.k->p.offset, c, err,
195
-
bkey_extent_size_greater_than_offset,
196
+
bkey_fsck_err_on(k.k->size > k.k->p.offset,
197
+
c, bkey_extent_size_greater_than_offset,
196
198
"size greater than offset (%u > %llu)",
197
199
k.k->size, k.k->p.offset);
198
200
} else {
199
-
bkey_fsck_err_on(k.k->size, c, err,
200
-
bkey_size_nonzero,
201
+
bkey_fsck_err_on(k.k->size,
202
+
c, bkey_size_nonzero,
201
203
"size != 0");
202
204
}
203
205
···
205
207
enum btree_id btree = type - 1;
206
208
207
209
if (btree_type_has_snapshots(btree)) {
208
-
bkey_fsck_err_on(!k.k->p.snapshot, c, err,
209
-
bkey_snapshot_zero,
210
+
bkey_fsck_err_on(!k.k->p.snapshot,
211
+
c, bkey_snapshot_zero,
210
212
"snapshot == 0");
211
213
} else if (!btree_type_has_snapshot_field(btree)) {
212
-
bkey_fsck_err_on(k.k->p.snapshot, c, err,
213
-
bkey_snapshot_nonzero,
214
+
bkey_fsck_err_on(k.k->p.snapshot,
215
+
c, bkey_snapshot_nonzero,
214
216
"nonzero snapshot");
215
217
} else {
216
218
/*
···
219
221
*/
220
222
}
221
223
222
-
bkey_fsck_err_on(bkey_eq(k.k->p, POS_MAX), c, err,
223
-
bkey_at_pos_max,
224
+
bkey_fsck_err_on(bkey_eq(k.k->p, POS_MAX),
225
+
c, bkey_at_pos_max,
224
226
"key at POS_MAX");
225
227
}
226
228
fsck_err:
227
229
return ret;
228
230
}
229
231
230
-
int bch2_bkey_invalid(struct bch_fs *c, struct bkey_s_c k,
232
+
int bch2_bkey_validate(struct bch_fs *c, struct bkey_s_c k,
231
233
enum btree_node_type type,
232
-
enum bch_validate_flags flags,
233
-
struct printbuf *err)
234
+
enum bch_validate_flags flags)
234
235
{
235
-
return __bch2_bkey_invalid(c, k, type, flags, err) ?:
236
-
bch2_bkey_val_invalid(c, k, flags, err);
236
+
return __bch2_bkey_validate(c, k, type, flags) ?:
237
+
bch2_bkey_val_validate(c, k, flags);
237
238
}
238
239
239
240
int bch2_bkey_in_btree_node(struct bch_fs *c, struct btree *b,
240
-
struct bkey_s_c k, struct printbuf *err)
241
+
struct bkey_s_c k, enum bch_validate_flags flags)
241
242
{
242
243
int ret = 0;
243
244
244
-
bkey_fsck_err_on(bpos_lt(k.k->p, b->data->min_key), c, err,
245
-
bkey_before_start_of_btree_node,
245
+
bkey_fsck_err_on(bpos_lt(k.k->p, b->data->min_key),
246
+
c, bkey_before_start_of_btree_node,
246
247
"key before start of btree node");
247
248
248
-
bkey_fsck_err_on(bpos_gt(k.k->p, b->data->max_key), c, err,
249
-
bkey_after_end_of_btree_node,
249
+
bkey_fsck_err_on(bpos_gt(k.k->p, b->data->max_key),
250
+
c, bkey_after_end_of_btree_node,
250
251
"key past end of btree node");
251
252
fsck_err:
252
253
return ret;
+10
-11
fs/bcachefs/bkey_methods.h
+10
-11
fs/bcachefs/bkey_methods.h
···
14
14
extern const struct bkey_ops bch2_bkey_null_ops;
15
15
16
16
/*
17
-
* key_invalid: checks validity of @k, returns 0 if good or -EINVAL if bad. If
17
+
* key_validate: checks validity of @k, returns 0 if good or -EINVAL if bad. If
18
18
* invalid, entire key will be deleted.
19
19
*
20
20
* When invalid, error string is returned via @err. @rw indicates whether key is
21
21
* being read or written; more aggressive checks can be enabled when rw == WRITE.
22
22
*/
23
23
struct bkey_ops {
24
-
int (*key_invalid)(struct bch_fs *c, struct bkey_s_c k,
25
-
enum bch_validate_flags flags, struct printbuf *err);
24
+
int (*key_validate)(struct bch_fs *c, struct bkey_s_c k,
25
+
enum bch_validate_flags flags);
26
26
void (*val_to_text)(struct printbuf *, struct bch_fs *,
27
27
struct bkey_s_c);
28
28
void (*swab)(struct bkey_s);
···
48
48
: &bch2_bkey_null_ops;
49
49
}
50
50
51
-
int bch2_bkey_val_invalid(struct bch_fs *, struct bkey_s_c,
52
-
enum bch_validate_flags, struct printbuf *);
53
-
int __bch2_bkey_invalid(struct bch_fs *, struct bkey_s_c, enum btree_node_type,
54
-
enum bch_validate_flags, struct printbuf *);
55
-
int bch2_bkey_invalid(struct bch_fs *, struct bkey_s_c, enum btree_node_type,
56
-
enum bch_validate_flags, struct printbuf *);
57
-
int bch2_bkey_in_btree_node(struct bch_fs *, struct btree *,
58
-
struct bkey_s_c, struct printbuf *);
51
+
int bch2_bkey_val_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
52
+
int __bch2_bkey_validate(struct bch_fs *, struct bkey_s_c, enum btree_node_type,
53
+
enum bch_validate_flags);
54
+
int bch2_bkey_validate(struct bch_fs *, struct bkey_s_c, enum btree_node_type,
55
+
enum bch_validate_flags);
56
+
int bch2_bkey_in_btree_node(struct bch_fs *, struct btree *, struct bkey_s_c,
57
+
enum bch_validate_flags);
59
58
60
59
void bch2_bpos_to_text(struct printbuf *, struct bpos);
61
60
void bch2_bkey_to_text(struct printbuf *, const struct bkey *);
+1
-4
fs/bcachefs/btree_gc.c
+1
-4
fs/bcachefs/btree_gc.c
···
741
741
742
742
static int bch2_mark_superblocks(struct bch_fs *c)
743
743
{
744
-
mutex_lock(&c->sb_lock);
745
744
gc_pos_set(c, gc_phase(GC_PHASE_sb));
746
745
747
-
int ret = bch2_trans_mark_dev_sbs_flags(c, BTREE_TRIGGER_gc);
748
-
mutex_unlock(&c->sb_lock);
749
-
return ret;
746
+
return bch2_trans_mark_dev_sbs_flags(c, BTREE_TRIGGER_gc);
750
747
}
751
748
752
749
static void bch2_gc_free(struct bch_fs *c)
+22
-47
fs/bcachefs/btree_io.c
+22
-47
fs/bcachefs/btree_io.c
···
836
836
return ret;
837
837
}
838
838
839
-
static int bset_key_invalid(struct bch_fs *c, struct btree *b,
840
-
struct bkey_s_c k,
841
-
bool updated_range, int rw,
842
-
struct printbuf *err)
839
+
static int bset_key_validate(struct bch_fs *c, struct btree *b,
840
+
struct bkey_s_c k,
841
+
bool updated_range, int rw)
843
842
{
844
-
return __bch2_bkey_invalid(c, k, btree_node_type(b), READ, err) ?:
845
-
(!updated_range ? bch2_bkey_in_btree_node(c, b, k, err) : 0) ?:
846
-
(rw == WRITE ? bch2_bkey_val_invalid(c, k, READ, err) : 0);
843
+
return __bch2_bkey_validate(c, k, btree_node_type(b), 0) ?:
844
+
(!updated_range ? bch2_bkey_in_btree_node(c, b, k, 0) : 0) ?:
845
+
(rw == WRITE ? bch2_bkey_val_validate(c, k, 0) : 0);
847
846
}
848
847
849
848
static bool bkey_packed_valid(struct bch_fs *c, struct btree *b,
···
857
858
if (!bkeyp_u64s_valid(&b->format, k))
858
859
return false;
859
860
860
-
struct printbuf buf = PRINTBUF;
861
861
struct bkey tmp;
862
862
struct bkey_s u = __bkey_disassemble(b, k, &tmp);
863
-
bool ret = __bch2_bkey_invalid(c, u.s_c, btree_node_type(b), READ, &buf);
864
-
printbuf_exit(&buf);
865
-
return ret;
863
+
return !__bch2_bkey_validate(c, u.s_c, btree_node_type(b), BCH_VALIDATE_silent);
866
864
}
867
865
868
866
static int validate_bset_keys(struct bch_fs *c, struct btree *b,
···
911
915
912
916
u = __bkey_disassemble(b, k, &tmp);
913
917
914
-
printbuf_reset(&buf);
915
-
if (bset_key_invalid(c, b, u.s_c, updated_range, write, &buf)) {
916
-
printbuf_reset(&buf);
917
-
bset_key_invalid(c, b, u.s_c, updated_range, write, &buf);
918
-
prt_printf(&buf, "\n ");
919
-
bch2_bkey_val_to_text(&buf, c, u.s_c);
920
-
921
-
btree_err(-BCH_ERR_btree_node_read_err_fixable,
922
-
c, NULL, b, i, k,
923
-
btree_node_bad_bkey,
924
-
"invalid bkey: %s", buf.buf);
918
+
ret = bset_key_validate(c, b, u.s_c, updated_range, write);
919
+
if (ret == -BCH_ERR_fsck_delete_bkey)
925
920
goto drop_this_key;
926
-
}
921
+
if (ret)
922
+
goto fsck_err;
927
923
928
924
if (write)
929
925
bch2_bkey_compat(b->c.level, b->c.btree_id, version,
···
1216
1228
struct bkey tmp;
1217
1229
struct bkey_s u = __bkey_disassemble(b, k, &tmp);
1218
1230
1219
-
printbuf_reset(&buf);
1220
-
1221
-
if (bch2_bkey_val_invalid(c, u.s_c, READ, &buf) ||
1231
+
ret = bch2_bkey_val_validate(c, u.s_c, READ);
1232
+
if (ret == -BCH_ERR_fsck_delete_bkey ||
1222
1233
(bch2_inject_invalid_keys &&
1223
1234
!bversion_cmp(u.k->version, MAX_VERSION))) {
1224
-
printbuf_reset(&buf);
1225
-
1226
-
prt_printf(&buf, "invalid bkey: ");
1227
-
bch2_bkey_val_invalid(c, u.s_c, READ, &buf);
1228
-
prt_printf(&buf, "\n ");
1229
-
bch2_bkey_val_to_text(&buf, c, u.s_c);
1230
-
1231
-
btree_err(-BCH_ERR_btree_node_read_err_fixable,
1232
-
c, NULL, b, i, k,
1233
-
btree_node_bad_bkey,
1234
-
"%s", buf.buf);
1235
-
1236
1235
btree_keys_account_key_drop(&b->nr, 0, k);
1237
1236
1238
1237
i->u64s = cpu_to_le16(le16_to_cpu(i->u64s) - k->u64s);
···
1228
1253
set_btree_bset_end(b, b->set);
1229
1254
continue;
1230
1255
}
1256
+
if (ret)
1257
+
goto fsck_err;
1231
1258
1232
1259
if (u.k->type == KEY_TYPE_btree_ptr_v2) {
1233
1260
struct bkey_s_btree_ptr_v2 bp = bkey_s_to_btree_ptr_v2(u);
···
1744
1767
1745
1768
set_btree_node_read_in_flight(b);
1746
1769
1770
+
/* we can't pass the trans to read_done() for fsck errors, so it must be unlocked */
1771
+
bch2_trans_unlock(trans);
1747
1772
bch2_btree_node_read(trans, b, true);
1748
1773
1749
1774
if (btree_node_read_error(b)) {
···
1931
1952
static int validate_bset_for_write(struct bch_fs *c, struct btree *b,
1932
1953
struct bset *i, unsigned sectors)
1933
1954
{
1934
-
struct printbuf buf = PRINTBUF;
1935
1955
bool saw_error;
1936
-
int ret;
1937
1956
1938
-
ret = bch2_bkey_invalid(c, bkey_i_to_s_c(&b->key),
1939
-
BKEY_TYPE_btree, WRITE, &buf);
1940
-
1941
-
if (ret)
1942
-
bch2_fs_inconsistent(c, "invalid btree node key before write: %s", buf.buf);
1943
-
printbuf_exit(&buf);
1944
-
if (ret)
1957
+
int ret = bch2_bkey_validate(c, bkey_i_to_s_c(&b->key),
1958
+
BKEY_TYPE_btree, WRITE);
1959
+
if (ret) {
1960
+
bch2_fs_inconsistent(c, "invalid btree node key before write");
1945
1961
return ret;
1962
+
}
1946
1963
1947
1964
ret = validate_bset_keys(c, b, i, WRITE, false, &saw_error) ?:
1948
1965
validate_bset(c, NULL, b, i, b->written, sectors, WRITE, false, &saw_error);
+1
fs/bcachefs/btree_iter.c
+1
fs/bcachefs/btree_iter.c
+27
-15
fs/bcachefs/btree_iter.h
+27
-15
fs/bcachefs/btree_iter.h
···
600
600
601
601
u32 bch2_trans_begin(struct btree_trans *);
602
602
603
-
/*
604
-
* XXX
605
-
* this does not handle transaction restarts from bch2_btree_iter_next_node()
606
-
* correctly
607
-
*/
608
-
#define __for_each_btree_node(_trans, _iter, _btree_id, _start, \
609
-
_locks_want, _depth, _flags, _b, _ret) \
610
-
for (bch2_trans_node_iter_init((_trans), &(_iter), (_btree_id), \
611
-
_start, _locks_want, _depth, _flags); \
612
-
(_b) = bch2_btree_iter_peek_node_and_restart(&(_iter)), \
613
-
!((_ret) = PTR_ERR_OR_ZERO(_b)) && (_b); \
614
-
(_b) = bch2_btree_iter_next_node(&(_iter)))
603
+
#define __for_each_btree_node(_trans, _iter, _btree_id, _start, \
604
+
_locks_want, _depth, _flags, _b, _do) \
605
+
({ \
606
+
bch2_trans_begin((_trans)); \
607
+
\
608
+
struct btree_iter _iter; \
609
+
bch2_trans_node_iter_init((_trans), &_iter, (_btree_id), \
610
+
_start, _locks_want, _depth, _flags); \
611
+
int _ret3 = 0; \
612
+
do { \
613
+
_ret3 = lockrestart_do((_trans), ({ \
614
+
struct btree *_b = bch2_btree_iter_peek_node(&_iter); \
615
+
if (!_b) \
616
+
break; \
617
+
\
618
+
PTR_ERR_OR_ZERO(_b) ?: (_do); \
619
+
})) ?: \
620
+
lockrestart_do((_trans), \
621
+
PTR_ERR_OR_ZERO(bch2_btree_iter_next_node(&_iter))); \
622
+
} while (!_ret3); \
623
+
\
624
+
bch2_trans_iter_exit((_trans), &(_iter)); \
625
+
_ret3; \
626
+
})
615
627
616
628
#define for_each_btree_node(_trans, _iter, _btree_id, _start, \
617
-
_flags, _b, _ret) \
618
-
__for_each_btree_node(_trans, _iter, _btree_id, _start, \
619
-
0, 0, _flags, _b, _ret)
629
+
_flags, _b, _do) \
630
+
__for_each_btree_node(_trans, _iter, _btree_id, _start, \
631
+
0, 0, _flags, _b, _do)
620
632
621
633
static inline struct bkey_s_c bch2_btree_iter_peek_prev_type(struct btree_iter *iter,
622
634
unsigned flags)
-5
fs/bcachefs/btree_key_cache.c
-5
fs/bcachefs/btree_key_cache.c
+16
-2
fs/bcachefs/btree_key_cache.h
+16
-2
fs/bcachefs/btree_key_cache.h
···
11
11
return max_t(ssize_t, 0, nr_dirty - max_dirty);
12
12
}
13
13
14
-
static inline bool bch2_btree_key_cache_must_wait(struct bch_fs *c)
14
+
static inline ssize_t __bch2_btree_key_cache_must_wait(struct bch_fs *c)
15
15
{
16
16
size_t nr_dirty = atomic_long_read(&c->btree_key_cache.nr_dirty);
17
17
size_t nr_keys = atomic_long_read(&c->btree_key_cache.nr_keys);
18
18
size_t max_dirty = 4096 + (nr_keys * 3) / 4;
19
19
20
-
return nr_dirty > max_dirty;
20
+
return nr_dirty - max_dirty;
21
+
}
22
+
23
+
static inline bool bch2_btree_key_cache_must_wait(struct bch_fs *c)
24
+
{
25
+
return __bch2_btree_key_cache_must_wait(c) > 0;
26
+
}
27
+
28
+
static inline bool bch2_btree_key_cache_wait_done(struct bch_fs *c)
29
+
{
30
+
size_t nr_dirty = atomic_long_read(&c->btree_key_cache.nr_dirty);
31
+
size_t nr_keys = atomic_long_read(&c->btree_key_cache.nr_keys);
32
+
size_t max_dirty = 2048 + (nr_keys * 5) / 8;
33
+
34
+
return nr_dirty <= max_dirty;
21
35
}
22
36
23
37
int bch2_btree_key_cache_journal_flush(struct journal *,
+1
-1
fs/bcachefs/btree_node_scan.c
+1
-1
fs/bcachefs/btree_node_scan.c
···
530
530
bch_verbose(c, "%s(): recovering %s", __func__, buf.buf);
531
531
printbuf_exit(&buf);
532
532
533
-
BUG_ON(bch2_bkey_invalid(c, bkey_i_to_s_c(&tmp.k), BKEY_TYPE_btree, 0, NULL));
533
+
BUG_ON(bch2_bkey_validate(c, bkey_i_to_s_c(&tmp.k), BKEY_TYPE_btree, 0));
534
534
535
535
ret = bch2_journal_key_insert(c, btree, level + 1, &tmp.k);
536
536
if (ret)
+21
-61
fs/bcachefs/btree_trans_commit.c
+21
-61
fs/bcachefs/btree_trans_commit.c
···
712
712
a->k.version = journal_pos_to_bversion(&trans->journal_res,
713
713
(u64 *) entry - (u64 *) trans->journal_entries);
714
714
BUG_ON(bversion_zero(a->k.version));
715
-
ret = bch2_accounting_mem_mod_locked(trans, accounting_i_to_s_c(a), false);
715
+
ret = bch2_accounting_mem_mod_locked(trans, accounting_i_to_s_c(a), false, false);
716
716
if (ret)
717
717
goto revert_fs_usage;
718
718
}
···
798
798
struct bkey_s_accounting a = bkey_i_to_s_accounting(entry2->start);
799
799
800
800
bch2_accounting_neg(a);
801
-
bch2_accounting_mem_mod_locked(trans, a.c, false);
801
+
bch2_accounting_mem_mod_locked(trans, a.c, false, false);
802
802
bch2_accounting_neg(a);
803
803
}
804
804
percpu_up_read(&c->mark_lock);
···
816
816
trans_for_each_update(trans, i)
817
817
if (i->k->k.type != KEY_TYPE_accounting)
818
818
bch2_journal_key_overwritten(trans->c, i->btree_id, i->level, i->k->k.p);
819
-
}
820
-
821
-
static noinline int bch2_trans_commit_bkey_invalid(struct btree_trans *trans,
822
-
enum bch_validate_flags flags,
823
-
struct btree_insert_entry *i,
824
-
struct printbuf *err)
825
-
{
826
-
struct bch_fs *c = trans->c;
827
-
828
-
printbuf_reset(err);
829
-
prt_printf(err, "invalid bkey on insert from %s -> %ps\n",
830
-
trans->fn, (void *) i->ip_allocated);
831
-
printbuf_indent_add(err, 2);
832
-
833
-
bch2_bkey_val_to_text(err, c, bkey_i_to_s_c(i->k));
834
-
prt_newline(err);
835
-
836
-
bch2_bkey_invalid(c, bkey_i_to_s_c(i->k), i->bkey_type, flags, err);
837
-
bch2_print_string_as_lines(KERN_ERR, err->buf);
838
-
839
-
bch2_inconsistent_error(c);
840
-
bch2_dump_trans_updates(trans);
841
-
842
-
return -EINVAL;
843
-
}
844
-
845
-
static noinline int bch2_trans_commit_journal_entry_invalid(struct btree_trans *trans,
846
-
struct jset_entry *i)
847
-
{
848
-
struct bch_fs *c = trans->c;
849
-
struct printbuf buf = PRINTBUF;
850
-
851
-
prt_printf(&buf, "invalid bkey on insert from %s\n", trans->fn);
852
-
printbuf_indent_add(&buf, 2);
853
-
854
-
bch2_journal_entry_to_text(&buf, c, i);
855
-
prt_newline(&buf);
856
-
857
-
bch2_print_string_as_lines(KERN_ERR, buf.buf);
858
-
859
-
bch2_inconsistent_error(c);
860
-
bch2_dump_trans_updates(trans);
861
-
862
-
return -EINVAL;
863
819
}
864
820
865
821
static int bch2_trans_commit_journal_pin_flush(struct journal *j,
···
883
927
static int journal_reclaim_wait_done(struct bch_fs *c)
884
928
{
885
929
int ret = bch2_journal_error(&c->journal) ?:
886
-
!bch2_btree_key_cache_must_wait(c);
930
+
bch2_btree_key_cache_wait_done(c);
887
931
888
932
if (!ret)
889
933
journal_reclaim_kick(&c->journal);
···
929
973
bch2_trans_unlock(trans);
930
974
931
975
trace_and_count(c, trans_blocked_journal_reclaim, trans, trace_ip);
976
+
track_event_change(&c->times[BCH_TIME_blocked_key_cache_flush], true);
932
977
933
978
wait_event_freezable(c->journal.reclaim_wait,
934
979
(ret = journal_reclaim_wait_done(c)));
980
+
981
+
track_event_change(&c->times[BCH_TIME_blocked_key_cache_flush], false);
982
+
935
983
if (ret < 0)
936
984
break;
937
985
···
1020
1060
goto out_reset;
1021
1061
1022
1062
trans_for_each_update(trans, i) {
1023
-
struct printbuf buf = PRINTBUF;
1024
1063
enum bch_validate_flags invalid_flags = 0;
1025
1064
1026
1065
if (!(flags & BCH_TRANS_COMMIT_no_journal_res))
1027
1066
invalid_flags |= BCH_VALIDATE_write|BCH_VALIDATE_commit;
1028
1067
1029
-
if (unlikely(bch2_bkey_invalid(c, bkey_i_to_s_c(i->k),
1030
-
i->bkey_type, invalid_flags, &buf)))
1031
-
ret = bch2_trans_commit_bkey_invalid(trans, invalid_flags, i, &buf);
1032
-
btree_insert_entry_checks(trans, i);
1033
-
printbuf_exit(&buf);
1034
-
1035
-
if (ret)
1068
+
ret = bch2_bkey_validate(c, bkey_i_to_s_c(i->k),
1069
+
i->bkey_type, invalid_flags);
1070
+
if (unlikely(ret)){
1071
+
bch2_trans_inconsistent(trans, "invalid bkey on insert from %s -> %ps\n",
1072
+
trans->fn, (void *) i->ip_allocated);
1036
1073
return ret;
1074
+
}
1075
+
btree_insert_entry_checks(trans, i);
1037
1076
}
1038
1077
1039
1078
for (struct jset_entry *i = trans->journal_entries;
···
1043
1084
if (!(flags & BCH_TRANS_COMMIT_no_journal_res))
1044
1085
invalid_flags |= BCH_VALIDATE_write|BCH_VALIDATE_commit;
1045
1086
1046
-
if (unlikely(bch2_journal_entry_validate(c, NULL, i,
1047
-
bcachefs_metadata_version_current,
1048
-
CPU_BIG_ENDIAN, invalid_flags)))
1049
-
ret = bch2_trans_commit_journal_entry_invalid(trans, i);
1050
-
1051
-
if (ret)
1087
+
ret = bch2_journal_entry_validate(c, NULL, i,
1088
+
bcachefs_metadata_version_current,
1089
+
CPU_BIG_ENDIAN, invalid_flags);
1090
+
if (unlikely(ret)) {
1091
+
bch2_trans_inconsistent(trans, "invalid journal entry on insert from %s\n",
1092
+
trans->fn);
1052
1093
return ret;
1094
+
}
1053
1095
}
1054
1096
1055
1097
if (unlikely(!test_bit(BCH_FS_may_go_rw, &c->flags))) {
+4
-12
fs/bcachefs/btree_update_interior.c
+4
-12
fs/bcachefs/btree_update_interior.c
···
1364
1364
if (unlikely(!test_bit(JOURNAL_replay_done, &c->journal.flags)))
1365
1365
bch2_journal_key_overwritten(c, b->c.btree_id, b->c.level, insert->k.p);
1366
1366
1367
-
if (bch2_bkey_invalid(c, bkey_i_to_s_c(insert),
1368
-
btree_node_type(b), WRITE, &buf) ?:
1369
-
bch2_bkey_in_btree_node(c, b, bkey_i_to_s_c(insert), &buf)) {
1370
-
printbuf_reset(&buf);
1371
-
prt_printf(&buf, "inserting invalid bkey\n ");
1372
-
bch2_bkey_val_to_text(&buf, c, bkey_i_to_s_c(insert));
1373
-
prt_printf(&buf, "\n ");
1374
-
bch2_bkey_invalid(c, bkey_i_to_s_c(insert),
1375
-
btree_node_type(b), WRITE, &buf);
1376
-
bch2_bkey_in_btree_node(c, b, bkey_i_to_s_c(insert), &buf);
1377
-
1378
-
bch2_fs_inconsistent(c, "%s", buf.buf);
1367
+
if (bch2_bkey_validate(c, bkey_i_to_s_c(insert),
1368
+
btree_node_type(b), BCH_VALIDATE_write) ?:
1369
+
bch2_bkey_in_btree_node(c, b, bkey_i_to_s_c(insert), BCH_VALIDATE_write)) {
1370
+
bch2_fs_inconsistent(c, "%s: inserting invalid bkey", __func__);
1379
1371
dump_stack();
1380
1372
}
1381
1373
+24
-6
fs/bcachefs/buckets.c
+24
-6
fs/bcachefs/buckets.c
···
810
810
ret = bch2_disk_accounting_mod(trans, &acc_btree_key, &replicas_sectors, 1, gc);
811
811
if (ret)
812
812
return ret;
813
+
} else {
814
+
bool insert = !(flags & BTREE_TRIGGER_overwrite);
815
+
struct disk_accounting_pos acc_inum_key = {
816
+
.type = BCH_DISK_ACCOUNTING_inum,
817
+
.inum.inum = k.k->p.inode,
818
+
};
819
+
s64 v[3] = {
820
+
insert ? 1 : -1,
821
+
insert ? k.k->size : -((s64) k.k->size),
822
+
replicas_sectors,
823
+
};
824
+
ret = bch2_disk_accounting_mod(trans, &acc_inum_key, v, ARRAY_SIZE(v), gc);
825
+
if (ret)
826
+
return ret;
813
827
}
814
828
815
829
if (bch2_bkey_rebalance_opts(k)) {
···
915
901
enum bch_data_type type,
916
902
unsigned sectors)
917
903
{
918
-
struct bch_fs *c = trans->c;
919
904
struct btree_iter iter;
920
905
int ret = 0;
921
906
···
924
911
return PTR_ERR(a);
925
912
926
913
if (a->v.data_type && type && a->v.data_type != type) {
927
-
bch2_fsck_err(c, FSCK_CAN_IGNORE|FSCK_NEED_FSCK,
914
+
bch2_fsck_err(trans, FSCK_CAN_IGNORE|FSCK_NEED_FSCK,
928
915
bucket_metadata_type_mismatch,
929
916
"bucket %llu:%llu gen %u different types of data in same bucket: %s, %s\n"
930
917
"while marking %s",
···
1045
1032
static int __bch2_trans_mark_dev_sb(struct btree_trans *trans, struct bch_dev *ca,
1046
1033
enum btree_iter_update_trigger_flags flags)
1047
1034
{
1048
-
struct bch_sb_layout *layout = &ca->disk_sb.sb->layout;
1035
+
struct bch_fs *c = trans->c;
1036
+
1037
+
mutex_lock(&c->sb_lock);
1038
+
struct bch_sb_layout layout = ca->disk_sb.sb->layout;
1039
+
mutex_unlock(&c->sb_lock);
1040
+
1049
1041
u64 bucket = 0;
1050
1042
unsigned i, bucket_sectors = 0;
1051
1043
int ret;
1052
1044
1053
-
for (i = 0; i < layout->nr_superblocks; i++) {
1054
-
u64 offset = le64_to_cpu(layout->sb_offset[i]);
1045
+
for (i = 0; i < layout.nr_superblocks; i++) {
1046
+
u64 offset = le64_to_cpu(layout.sb_offset[i]);
1055
1047
1056
1048
if (offset == BCH_SB_SECTOR) {
1057
1049
ret = bch2_trans_mark_metadata_sectors(trans, ca,
···
1067
1049
}
1068
1050
1069
1051
ret = bch2_trans_mark_metadata_sectors(trans, ca, offset,
1070
-
offset + (1 << layout->sb_max_size_bits),
1052
+
offset + (1 << layout.sb_max_size_bits),
1071
1053
BCH_DATA_sb, &bucket, &bucket_sectors, flags);
1072
1054
if (ret)
1073
1055
return ret;
+9
-2
fs/bcachefs/buckets_waiting_for_journal.c
+9
-2
fs/bcachefs/buckets_waiting_for_journal.c
···
93
93
.dev_bucket = (u64) dev << 56 | bucket,
94
94
.journal_seq = journal_seq,
95
95
};
96
-
size_t i, size, new_bits, nr_elements = 1, nr_rehashes = 0;
96
+
size_t i, size, new_bits, nr_elements = 1, nr_rehashes = 0, nr_rehashes_this_size = 0;
97
97
int ret = 0;
98
98
99
99
mutex_lock(&b->lock);
···
106
106
for (i = 0; i < size; i++)
107
107
nr_elements += t->d[i].journal_seq > flushed_seq;
108
108
109
-
new_bits = t->bits + (nr_elements * 3 > size);
109
+
new_bits = ilog2(roundup_pow_of_two(nr_elements * 3));
110
110
111
111
n = kvmalloc(sizeof(*n) + (sizeof(n->d[0]) << new_bits), GFP_KERNEL);
112
112
if (!n) {
···
115
115
}
116
116
117
117
retry_rehash:
118
+
if (nr_rehashes_this_size == 3) {
119
+
new_bits++;
120
+
nr_rehashes_this_size = 0;
121
+
}
122
+
118
123
nr_rehashes++;
124
+
nr_rehashes_this_size++;
125
+
119
126
bucket_table_init(n, new_bits);
120
127
121
128
tmp = new;
+2
-4
fs/bcachefs/data_update.c
+2
-4
fs/bcachefs/data_update.c
···
250
250
* it's been hard to reproduce, so this should give us some more
251
251
* information when it does occur:
252
252
*/
253
-
struct printbuf err = PRINTBUF;
254
-
int invalid = bch2_bkey_invalid(c, bkey_i_to_s_c(insert), __btree_node_type(0, m->btree_id), 0, &err);
255
-
printbuf_exit(&err);
256
-
253
+
int invalid = bch2_bkey_validate(c, bkey_i_to_s_c(insert), __btree_node_type(0, m->btree_id),
254
+
BCH_VALIDATE_commit);
257
255
if (invalid) {
258
256
struct printbuf buf = PRINTBUF;
259
257
+9
-29
fs/bcachefs/debug.c
+9
-29
fs/bcachefs/debug.c
···
397
397
size_t size, loff_t *ppos)
398
398
{
399
399
struct dump_iter *i = file->private_data;
400
-
struct btree_trans *trans;
401
-
struct btree_iter iter;
402
-
struct btree *b;
403
-
ssize_t ret;
404
400
405
401
i->ubuf = buf;
406
402
i->size = size;
407
403
i->ret = 0;
408
404
409
-
ret = flush_buf(i);
405
+
ssize_t ret = flush_buf(i);
410
406
if (ret)
411
407
return ret;
412
408
413
409
if (bpos_eq(SPOS_MAX, i->from))
414
410
return i->ret;
415
411
416
-
trans = bch2_trans_get(i->c);
417
-
retry:
418
-
bch2_trans_begin(trans);
412
+
return bch2_trans_run(i->c,
413
+
for_each_btree_node(trans, iter, i->id, i->from, 0, b, ({
414
+
bch2_btree_node_to_text(&i->buf, i->c, b);
415
+
i->from = !bpos_eq(SPOS_MAX, b->key.k.p)
416
+
? bpos_successor(b->key.k.p)
417
+
: b->key.k.p;
419
418
420
-
for_each_btree_node(trans, iter, i->id, i->from, 0, b, ret) {
421
-
bch2_btree_node_to_text(&i->buf, i->c, b);
422
-
i->from = !bpos_eq(SPOS_MAX, b->key.k.p)
423
-
? bpos_successor(b->key.k.p)
424
-
: b->key.k.p;
425
-
426
-
ret = drop_locks_do(trans, flush_buf(i));
427
-
if (ret)
428
-
break;
429
-
}
430
-
bch2_trans_iter_exit(trans, &iter);
431
-
432
-
if (bch2_err_matches(ret, BCH_ERR_transaction_restart))
433
-
goto retry;
434
-
435
-
bch2_trans_put(trans);
436
-
437
-
if (!ret)
438
-
ret = flush_buf(i);
439
-
440
-
return ret ?: i->ret;
419
+
drop_locks_do(trans, flush_buf(i));
420
+
}))) ?: i->ret;
441
421
}
442
422
443
423
static const struct file_operations btree_format_debug_ops = {
+16
-17
fs/bcachefs/dirent.c
+16
-17
fs/bcachefs/dirent.c
···
100
100
.is_visible = dirent_is_visible,
101
101
};
102
102
103
-
int bch2_dirent_invalid(struct bch_fs *c, struct bkey_s_c k,
104
-
enum bch_validate_flags flags,
105
-
struct printbuf *err)
103
+
int bch2_dirent_validate(struct bch_fs *c, struct bkey_s_c k,
104
+
enum bch_validate_flags flags)
106
105
{
107
106
struct bkey_s_c_dirent d = bkey_s_c_to_dirent(k);
108
107
struct qstr d_name = bch2_dirent_get_name(d);
109
108
int ret = 0;
110
109
111
-
bkey_fsck_err_on(!d_name.len, c, err,
112
-
dirent_empty_name,
110
+
bkey_fsck_err_on(!d_name.len,
111
+
c, dirent_empty_name,
113
112
"empty name");
114
113
115
-
bkey_fsck_err_on(bkey_val_u64s(k.k) > dirent_val_u64s(d_name.len), c, err,
116
-
dirent_val_too_big,
114
+
bkey_fsck_err_on(bkey_val_u64s(k.k) > dirent_val_u64s(d_name.len),
115
+
c, dirent_val_too_big,
117
116
"value too big (%zu > %u)",
118
117
bkey_val_u64s(k.k), dirent_val_u64s(d_name.len));
119
118
···
120
121
* Check new keys don't exceed the max length
121
122
* (older keys may be larger.)
122
123
*/
123
-
bkey_fsck_err_on((flags & BCH_VALIDATE_commit) && d_name.len > BCH_NAME_MAX, c, err,
124
-
dirent_name_too_long,
124
+
bkey_fsck_err_on((flags & BCH_VALIDATE_commit) && d_name.len > BCH_NAME_MAX,
125
+
c, dirent_name_too_long,
125
126
"dirent name too big (%u > %u)",
126
127
d_name.len, BCH_NAME_MAX);
127
128
128
-
bkey_fsck_err_on(d_name.len != strnlen(d_name.name, d_name.len), c, err,
129
-
dirent_name_embedded_nul,
129
+
bkey_fsck_err_on(d_name.len != strnlen(d_name.name, d_name.len),
130
+
c, dirent_name_embedded_nul,
130
131
"dirent has stray data after name's NUL");
131
132
132
133
bkey_fsck_err_on((d_name.len == 1 && !memcmp(d_name.name, ".", 1)) ||
133
-
(d_name.len == 2 && !memcmp(d_name.name, "..", 2)), c, err,
134
-
dirent_name_dot_or_dotdot,
134
+
(d_name.len == 2 && !memcmp(d_name.name, "..", 2)),
135
+
c, dirent_name_dot_or_dotdot,
135
136
"invalid name");
136
137
137
-
bkey_fsck_err_on(memchr(d_name.name, '/', d_name.len), c, err,
138
-
dirent_name_has_slash,
138
+
bkey_fsck_err_on(memchr(d_name.name, '/', d_name.len),
139
+
c, dirent_name_has_slash,
139
140
"name with /");
140
141
141
142
bkey_fsck_err_on(d.v->d_type != DT_SUBVOL &&
142
-
le64_to_cpu(d.v->d_inum) == d.k->p.inode, c, err,
143
-
dirent_to_itself,
143
+
le64_to_cpu(d.v->d_inum) == d.k->p.inode,
144
+
c, dirent_to_itself,
144
145
"dirent points to own directory");
145
146
fsck_err:
146
147
return ret;
+2
-3
fs/bcachefs/dirent.h
+2
-3
fs/bcachefs/dirent.h
···
7
7
enum bch_validate_flags;
8
8
extern const struct bch_hash_desc bch2_dirent_hash_desc;
9
9
10
-
int bch2_dirent_invalid(struct bch_fs *, struct bkey_s_c,
11
-
enum bch_validate_flags, struct printbuf *);
10
+
int bch2_dirent_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
12
11
void bch2_dirent_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
13
12
14
13
#define bch2_bkey_ops_dirent ((struct bkey_ops) { \
15
-
.key_invalid = bch2_dirent_invalid, \
14
+
.key_validate = bch2_dirent_validate, \
16
15
.val_to_text = bch2_dirent_to_text, \
17
16
.min_val_size = 16, \
18
17
})
+21
-13
fs/bcachefs/disk_accounting.c
+21
-13
fs/bcachefs/disk_accounting.c
···
126
126
127
127
#define field_end(p, member) (((void *) (&p.member)) + sizeof(p.member))
128
128
129
-
int bch2_accounting_invalid(struct bch_fs *c, struct bkey_s_c k,
130
-
enum bch_validate_flags flags,
131
-
struct printbuf *err)
129
+
int bch2_accounting_validate(struct bch_fs *c, struct bkey_s_c k,
130
+
enum bch_validate_flags flags)
132
131
{
133
132
struct disk_accounting_pos acc_k;
134
133
bpos_to_disk_accounting_pos(&acc_k, k.k->p);
···
143
144
break;
144
145
case BCH_DISK_ACCOUNTING_replicas:
145
146
bkey_fsck_err_on(!acc_k.replicas.nr_devs,
146
-
c, err, accounting_key_replicas_nr_devs_0,
147
+
c, accounting_key_replicas_nr_devs_0,
147
148
"accounting key replicas entry with nr_devs=0");
148
149
149
150
bkey_fsck_err_on(acc_k.replicas.nr_required > acc_k.replicas.nr_devs ||
150
151
(acc_k.replicas.nr_required > 1 &&
151
152
acc_k.replicas.nr_required == acc_k.replicas.nr_devs),
152
-
c, err, accounting_key_replicas_nr_required_bad,
153
+
c, accounting_key_replicas_nr_required_bad,
153
154
"accounting key replicas entry with bad nr_required");
154
155
155
156
for (unsigned i = 0; i + 1 < acc_k.replicas.nr_devs; i++)
156
-
bkey_fsck_err_on(acc_k.replicas.devs[i] > acc_k.replicas.devs[i + 1],
157
-
c, err, accounting_key_replicas_devs_unsorted,
157
+
bkey_fsck_err_on(acc_k.replicas.devs[i] >= acc_k.replicas.devs[i + 1],
158
+
c, accounting_key_replicas_devs_unsorted,
158
159
"accounting key replicas entry with unsorted devs");
159
160
160
161
end = (void *) &acc_k.replicas + replicas_entry_bytes(&acc_k.replicas);
···
177
178
}
178
179
179
180
bkey_fsck_err_on(!is_zero(end, (void *) (&acc_k + 1)),
180
-
c, err, accounting_key_junk_at_end,
181
+
c, accounting_key_junk_at_end,
181
182
"junk at end of accounting key");
182
183
fsck_err:
183
184
return ret;
···
527
528
struct disk_accounting_pos acc_k;
528
529
bpos_to_disk_accounting_pos(&acc_k, e->pos);
529
530
531
+
if (acc_k.type >= BCH_DISK_ACCOUNTING_TYPE_NR)
532
+
continue;
533
+
530
534
u64 src_v[BCH_ACCOUNTING_MAX_COUNTERS];
531
535
u64 dst_v[BCH_ACCOUNTING_MAX_COUNTERS];
532
536
···
566
564
struct { __BKEY_PADDED(k, BCH_ACCOUNTING_MAX_COUNTERS); } k_i;
567
565
568
566
accounting_key_init(&k_i.k, &acc_k, src_v, nr);
569
-
bch2_accounting_mem_mod_locked(trans, bkey_i_to_s_c_accounting(&k_i.k), false);
567
+
bch2_accounting_mem_mod_locked(trans, bkey_i_to_s_c_accounting(&k_i.k), false, false);
570
568
571
569
preempt_disable();
572
570
struct bch_fs_usage_base *dst = this_cpu_ptr(c->usage);
···
595
593
return 0;
596
594
597
595
percpu_down_read(&c->mark_lock);
598
-
int ret = __bch2_accounting_mem_mod(c, bkey_s_c_to_accounting(k), false);
596
+
int ret = bch2_accounting_mem_mod_locked(trans, bkey_s_c_to_accounting(k), false, true);
599
597
percpu_up_read(&c->mark_lock);
600
598
601
599
if (bch2_accounting_key_is_zero(bkey_s_c_to_accounting(k)) &&
···
762
760
struct bkey_s_c_accounting a = bkey_s_c_to_accounting(k);
763
761
unsigned nr = bch2_accounting_counters(k.k);
764
762
763
+
struct disk_accounting_pos acc_k;
764
+
bpos_to_disk_accounting_pos(&acc_k, k.k->p);
765
+
766
+
if (acc_k.type >= BCH_DISK_ACCOUNTING_TYPE_NR)
767
+
continue;
768
+
769
+
if (acc_k.type == BCH_DISK_ACCOUNTING_inum)
770
+
continue;
771
+
765
772
bch2_accounting_mem_read(c, k.k->p, v, nr);
766
773
767
774
if (memcmp(a.v->d, v, nr * sizeof(u64))) {
···
785
774
printbuf_exit(&buf);
786
775
mismatch = true;
787
776
}
788
-
789
-
struct disk_accounting_pos acc_k;
790
-
bpos_to_disk_accounting_pos(&acc_k, a.k->p);
791
777
792
778
switch (acc_k.type) {
793
779
case BCH_DISK_ACCOUNTING_persistent_reserved:
+36
-40
fs/bcachefs/disk_accounting.h
+36
-40
fs/bcachefs/disk_accounting.h
···
82
82
s64 *, unsigned, bool);
83
83
int bch2_mod_dev_cached_sectors(struct btree_trans *, unsigned, s64, bool);
84
84
85
-
int bch2_accounting_invalid(struct bch_fs *, struct bkey_s_c,
86
-
enum bch_validate_flags, struct printbuf *);
85
+
int bch2_accounting_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
87
86
void bch2_accounting_key_to_text(struct printbuf *, struct disk_accounting_pos *);
88
87
void bch2_accounting_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
89
88
void bch2_accounting_swab(struct bkey_s);
90
89
91
90
#define bch2_bkey_ops_accounting ((struct bkey_ops) { \
92
-
.key_invalid = bch2_accounting_invalid, \
91
+
.key_validate = bch2_accounting_validate, \
93
92
.val_to_text = bch2_accounting_to_text, \
94
93
.swab = bch2_accounting_swab, \
95
94
.min_val_size = 8, \
···
106
107
int bch2_accounting_mem_insert(struct bch_fs *, struct bkey_s_c_accounting, bool);
107
108
void bch2_accounting_mem_gc(struct bch_fs *);
108
109
109
-
static inline int __bch2_accounting_mem_mod(struct bch_fs *c, struct bkey_s_c_accounting a, bool gc)
110
+
/*
111
+
* Update in memory counters so they match the btree update we're doing; called
112
+
* from transaction commit path
113
+
*/
114
+
static inline int bch2_accounting_mem_mod_locked(struct btree_trans *trans, struct bkey_s_c_accounting a, bool gc, bool read)
110
115
{
116
+
struct bch_fs *c = trans->c;
117
+
struct disk_accounting_pos acc_k;
118
+
bpos_to_disk_accounting_pos(&acc_k, a.k->p);
119
+
120
+
if (acc_k.type == BCH_DISK_ACCOUNTING_inum)
121
+
return 0;
122
+
123
+
if (!gc && !read) {
124
+
switch (acc_k.type) {
125
+
case BCH_DISK_ACCOUNTING_persistent_reserved:
126
+
trans->fs_usage_delta.reserved += acc_k.persistent_reserved.nr_replicas * a.v->d[0];
127
+
break;
128
+
case BCH_DISK_ACCOUNTING_replicas:
129
+
fs_usage_data_type_to_base(&trans->fs_usage_delta, acc_k.replicas.data_type, a.v->d[0]);
130
+
break;
131
+
case BCH_DISK_ACCOUNTING_dev_data_type:
132
+
rcu_read_lock();
133
+
struct bch_dev *ca = bch2_dev_rcu(c, acc_k.dev_data_type.dev);
134
+
if (ca) {
135
+
this_cpu_add(ca->usage->d[acc_k.dev_data_type.data_type].buckets, a.v->d[0]);
136
+
this_cpu_add(ca->usage->d[acc_k.dev_data_type.data_type].sectors, a.v->d[1]);
137
+
this_cpu_add(ca->usage->d[acc_k.dev_data_type.data_type].fragmented, a.v->d[2]);
138
+
}
139
+
rcu_read_unlock();
140
+
break;
141
+
}
142
+
}
143
+
111
144
struct bch_accounting_mem *acc = &c->accounting;
112
145
unsigned idx;
113
146
···
161
130
return 0;
162
131
}
163
132
164
-
/*
165
-
* Update in memory counters so they match the btree update we're doing; called
166
-
* from transaction commit path
167
-
*/
168
-
static inline int bch2_accounting_mem_mod_locked(struct btree_trans *trans, struct bkey_s_c_accounting a, bool gc)
169
-
{
170
-
struct bch_fs *c = trans->c;
171
-
172
-
if (!gc) {
173
-
struct disk_accounting_pos acc_k;
174
-
bpos_to_disk_accounting_pos(&acc_k, a.k->p);
175
-
176
-
switch (acc_k.type) {
177
-
case BCH_DISK_ACCOUNTING_persistent_reserved:
178
-
trans->fs_usage_delta.reserved += acc_k.persistent_reserved.nr_replicas * a.v->d[0];
179
-
break;
180
-
case BCH_DISK_ACCOUNTING_replicas:
181
-
fs_usage_data_type_to_base(&trans->fs_usage_delta, acc_k.replicas.data_type, a.v->d[0]);
182
-
break;
183
-
case BCH_DISK_ACCOUNTING_dev_data_type:
184
-
rcu_read_lock();
185
-
struct bch_dev *ca = bch2_dev_rcu(c, acc_k.dev_data_type.dev);
186
-
if (ca) {
187
-
this_cpu_add(ca->usage->d[acc_k.dev_data_type.data_type].buckets, a.v->d[0]);
188
-
this_cpu_add(ca->usage->d[acc_k.dev_data_type.data_type].sectors, a.v->d[1]);
189
-
this_cpu_add(ca->usage->d[acc_k.dev_data_type.data_type].fragmented, a.v->d[2]);
190
-
}
191
-
rcu_read_unlock();
192
-
break;
193
-
}
194
-
}
195
-
196
-
return __bch2_accounting_mem_mod(c, a, gc);
197
-
}
198
-
199
133
static inline int bch2_accounting_mem_add(struct btree_trans *trans, struct bkey_s_c_accounting a, bool gc)
200
134
{
201
135
percpu_down_read(&trans->c->mark_lock);
202
-
int ret = bch2_accounting_mem_mod_locked(trans, a, gc);
136
+
int ret = bch2_accounting_mem_mod_locked(trans, a, gc, false);
203
137
percpu_up_read(&trans->c->mark_lock);
204
138
return ret;
205
139
}
+7
-1
fs/bcachefs/disk_accounting_format.h
+7
-1
fs/bcachefs/disk_accounting_format.h
···
103
103
x(compression, 4) \
104
104
x(snapshot, 5) \
105
105
x(btree, 6) \
106
-
x(rebalance_work, 7)
106
+
x(rebalance_work, 7) \
107
+
x(inum, 8)
107
108
108
109
enum disk_accounting_type {
109
110
#define x(f, nr) BCH_DISK_ACCOUNTING_##f = nr,
···
137
136
__u32 id;
138
137
} __packed;
139
138
139
+
struct bch_acct_inum {
140
+
__u64 inum;
141
+
} __packed;
142
+
140
143
struct bch_acct_rebalance_work {
141
144
};
142
145
···
157
152
struct bch_acct_snapshot snapshot;
158
153
struct bch_acct_btree btree;
159
154
struct bch_acct_rebalance_work rebalance_work;
155
+
struct bch_acct_inum inum;
160
156
} __packed;
161
157
} __packed;
162
158
struct bpos _pad;
+7
-8
fs/bcachefs/ec.c
+7
-8
fs/bcachefs/ec.c
···
107
107
108
108
/* Stripes btree keys: */
109
109
110
-
int bch2_stripe_invalid(struct bch_fs *c, struct bkey_s_c k,
111
-
enum bch_validate_flags flags,
112
-
struct printbuf *err)
110
+
int bch2_stripe_validate(struct bch_fs *c, struct bkey_s_c k,
111
+
enum bch_validate_flags flags)
113
112
{
114
113
const struct bch_stripe *s = bkey_s_c_to_stripe(k).v;
115
114
int ret = 0;
116
115
117
116
bkey_fsck_err_on(bkey_eq(k.k->p, POS_MIN) ||
118
-
bpos_gt(k.k->p, POS(0, U32_MAX)), c, err,
119
-
stripe_pos_bad,
117
+
bpos_gt(k.k->p, POS(0, U32_MAX)),
118
+
c, stripe_pos_bad,
120
119
"stripe at bad pos");
121
120
122
-
bkey_fsck_err_on(bkey_val_u64s(k.k) < stripe_val_u64s(s), c, err,
123
-
stripe_val_size_bad,
121
+
bkey_fsck_err_on(bkey_val_u64s(k.k) < stripe_val_u64s(s),
122
+
c, stripe_val_size_bad,
124
123
"incorrect value size (%zu < %u)",
125
124
bkey_val_u64s(k.k), stripe_val_u64s(s));
126
125
127
-
ret = bch2_bkey_ptrs_invalid(c, k, flags, err);
126
+
ret = bch2_bkey_ptrs_validate(c, k, flags);
128
127
fsck_err:
129
128
return ret;
130
129
}
+2
-3
fs/bcachefs/ec.h
+2
-3
fs/bcachefs/ec.h
···
8
8
9
9
enum bch_validate_flags;
10
10
11
-
int bch2_stripe_invalid(struct bch_fs *, struct bkey_s_c,
12
-
enum bch_validate_flags, struct printbuf *);
11
+
int bch2_stripe_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
13
12
void bch2_stripe_to_text(struct printbuf *, struct bch_fs *,
14
13
struct bkey_s_c);
15
14
int bch2_trigger_stripe(struct btree_trans *, enum btree_id, unsigned,
···
16
17
enum btree_iter_update_trigger_flags);
17
18
18
19
#define bch2_bkey_ops_stripe ((struct bkey_ops) { \
19
-
.key_invalid = bch2_stripe_invalid, \
20
+
.key_validate = bch2_stripe_validate, \
20
21
.val_to_text = bch2_stripe_to_text, \
21
22
.swab = bch2_ptr_swab, \
22
23
.trigger = bch2_trigger_stripe, \
+1
fs/bcachefs/errcode.h
+1
fs/bcachefs/errcode.h
+22
fs/bcachefs/error.c
+22
fs/bcachefs/error.c
···
416
416
return ret;
417
417
}
418
418
419
+
int __bch2_bkey_fsck_err(struct bch_fs *c,
420
+
struct bkey_s_c k,
421
+
enum bch_fsck_flags flags,
422
+
enum bch_sb_error_id err,
423
+
const char *fmt, ...)
424
+
{
425
+
struct printbuf buf = PRINTBUF;
426
+
va_list args;
427
+
428
+
prt_str(&buf, "invalid bkey ");
429
+
bch2_bkey_val_to_text(&buf, c, k);
430
+
prt_str(&buf, "\n ");
431
+
va_start(args, fmt);
432
+
prt_vprintf(&buf, fmt, args);
433
+
va_end(args);
434
+
prt_str(&buf, ": delete?");
435
+
436
+
int ret = __bch2_fsck_err(c, NULL, flags, err, "%s", buf.buf);
437
+
printbuf_exit(&buf);
438
+
return ret;
439
+
}
440
+
419
441
void bch2_flush_fsck_errs(struct bch_fs *c)
420
442
{
421
443
struct fsck_err_state *s, *n;
+23
-16
fs/bcachefs/error.h
+23
-16
fs/bcachefs/error.h
···
4
4
5
5
#include <linux/list.h>
6
6
#include <linux/printk.h>
7
+
#include "bkey_types.h"
7
8
#include "sb-errors.h"
8
9
9
10
struct bch_dev;
···
167
166
#define fsck_err_on(cond, c, _err_type, ...) \
168
167
__fsck_err_on(cond, c, FSCK_CAN_FIX|FSCK_CAN_IGNORE, _err_type, __VA_ARGS__)
169
168
170
-
__printf(4, 0)
171
-
static inline void bch2_bkey_fsck_err(struct bch_fs *c,
172
-
struct printbuf *err_msg,
173
-
enum bch_sb_error_id err_type,
174
-
const char *fmt, ...)
175
-
{
176
-
va_list args;
169
+
__printf(5, 6)
170
+
int __bch2_bkey_fsck_err(struct bch_fs *,
171
+
struct bkey_s_c,
172
+
enum bch_fsck_flags,
173
+
enum bch_sb_error_id,
174
+
const char *, ...);
177
175
178
-
va_start(args, fmt);
179
-
prt_vprintf(err_msg, fmt, args);
180
-
va_end(args);
181
-
}
182
-
183
-
#define bkey_fsck_err(c, _err_msg, _err_type, ...) \
176
+
/*
177
+
* for now, bkey fsck errors are always handled by deleting the entire key -
178
+
* this will change at some point
179
+
*/
180
+
#define bkey_fsck_err(c, _err_type, _err_msg, ...) \
184
181
do { \
185
-
prt_printf(_err_msg, __VA_ARGS__); \
186
-
bch2_sb_error_count(c, BCH_FSCK_ERR_##_err_type); \
187
-
ret = -BCH_ERR_invalid_bkey; \
182
+
if ((flags & BCH_VALIDATE_silent)) { \
183
+
ret = -BCH_ERR_fsck_delete_bkey; \
184
+
goto fsck_err; \
185
+
} \
186
+
int _ret = __bch2_bkey_fsck_err(c, k, FSCK_CAN_FIX, \
187
+
BCH_FSCK_ERR_##_err_type, \
188
+
_err_msg, ##__VA_ARGS__); \
189
+
if (_ret != -BCH_ERR_fsck_fix && \
190
+
_ret != -BCH_ERR_fsck_ignore) \
191
+
ret = _ret; \
192
+
ret = -BCH_ERR_fsck_delete_bkey; \
188
193
goto fsck_err; \
189
194
} while (0)
190
195
+72
-72
fs/bcachefs/extents.c
+72
-72
fs/bcachefs/extents.c
···
171
171
172
172
/* KEY_TYPE_btree_ptr: */
173
173
174
-
int bch2_btree_ptr_invalid(struct bch_fs *c, struct bkey_s_c k,
175
-
enum bch_validate_flags flags,
176
-
struct printbuf *err)
174
+
int bch2_btree_ptr_validate(struct bch_fs *c, struct bkey_s_c k,
175
+
enum bch_validate_flags flags)
177
176
{
178
177
int ret = 0;
179
178
180
-
bkey_fsck_err_on(bkey_val_u64s(k.k) > BCH_REPLICAS_MAX, c, err,
181
-
btree_ptr_val_too_big,
179
+
bkey_fsck_err_on(bkey_val_u64s(k.k) > BCH_REPLICAS_MAX,
180
+
c, btree_ptr_val_too_big,
182
181
"value too big (%zu > %u)", bkey_val_u64s(k.k), BCH_REPLICAS_MAX);
183
182
184
-
ret = bch2_bkey_ptrs_invalid(c, k, flags, err);
183
+
ret = bch2_bkey_ptrs_validate(c, k, flags);
185
184
fsck_err:
186
185
return ret;
187
186
}
···
191
192
bch2_bkey_ptrs_to_text(out, c, k);
192
193
}
193
194
194
-
int bch2_btree_ptr_v2_invalid(struct bch_fs *c, struct bkey_s_c k,
195
-
enum bch_validate_flags flags,
196
-
struct printbuf *err)
195
+
int bch2_btree_ptr_v2_validate(struct bch_fs *c, struct bkey_s_c k,
196
+
enum bch_validate_flags flags)
197
197
{
198
198
struct bkey_s_c_btree_ptr_v2 bp = bkey_s_c_to_btree_ptr_v2(k);
199
199
int ret = 0;
200
200
201
201
bkey_fsck_err_on(bkey_val_u64s(k.k) > BKEY_BTREE_PTR_VAL_U64s_MAX,
202
-
c, err, btree_ptr_v2_val_too_big,
202
+
c, btree_ptr_v2_val_too_big,
203
203
"value too big (%zu > %zu)",
204
204
bkey_val_u64s(k.k), BKEY_BTREE_PTR_VAL_U64s_MAX);
205
205
206
206
bkey_fsck_err_on(bpos_ge(bp.v->min_key, bp.k->p),
207
-
c, err, btree_ptr_v2_min_key_bad,
207
+
c, btree_ptr_v2_min_key_bad,
208
208
"min_key > key");
209
209
210
210
if (flags & BCH_VALIDATE_write)
211
211
bkey_fsck_err_on(!bp.v->sectors_written,
212
-
c, err, btree_ptr_v2_written_0,
212
+
c, btree_ptr_v2_written_0,
213
213
"sectors_written == 0");
214
214
215
-
ret = bch2_bkey_ptrs_invalid(c, k, flags, err);
215
+
ret = bch2_bkey_ptrs_validate(c, k, flags);
216
216
fsck_err:
217
217
return ret;
218
218
}
···
397
399
398
400
/* KEY_TYPE_reservation: */
399
401
400
-
int bch2_reservation_invalid(struct bch_fs *c, struct bkey_s_c k,
401
-
enum bch_validate_flags flags,
402
-
struct printbuf *err)
402
+
int bch2_reservation_validate(struct bch_fs *c, struct bkey_s_c k,
403
+
enum bch_validate_flags flags)
403
404
{
404
405
struct bkey_s_c_reservation r = bkey_s_c_to_reservation(k);
405
406
int ret = 0;
406
407
407
-
bkey_fsck_err_on(!r.v->nr_replicas || r.v->nr_replicas > BCH_REPLICAS_MAX, c, err,
408
-
reservation_key_nr_replicas_invalid,
408
+
bkey_fsck_err_on(!r.v->nr_replicas || r.v->nr_replicas > BCH_REPLICAS_MAX,
409
+
c, reservation_key_nr_replicas_invalid,
409
410
"invalid nr_replicas (%u)", r.v->nr_replicas);
410
411
fsck_err:
411
412
return ret;
···
1099
1102
}
1100
1103
}
1101
1104
1102
-
1103
-
static int extent_ptr_invalid(struct bch_fs *c,
1104
-
struct bkey_s_c k,
1105
-
enum bch_validate_flags flags,
1106
-
const struct bch_extent_ptr *ptr,
1107
-
unsigned size_ondisk,
1108
-
bool metadata,
1109
-
struct printbuf *err)
1105
+
static int extent_ptr_validate(struct bch_fs *c,
1106
+
struct bkey_s_c k,
1107
+
enum bch_validate_flags flags,
1108
+
const struct bch_extent_ptr *ptr,
1109
+
unsigned size_ondisk,
1110
+
bool metadata)
1110
1111
{
1111
1112
int ret = 0;
1112
1113
···
1123
1128
1124
1129
struct bkey_ptrs_c ptrs = bch2_bkey_ptrs_c(k);
1125
1130
bkey_for_each_ptr(ptrs, ptr2)
1126
-
bkey_fsck_err_on(ptr != ptr2 && ptr->dev == ptr2->dev, c, err,
1127
-
ptr_to_duplicate_device,
1131
+
bkey_fsck_err_on(ptr != ptr2 && ptr->dev == ptr2->dev,
1132
+
c, ptr_to_duplicate_device,
1128
1133
"multiple pointers to same device (%u)", ptr->dev);
1129
1134
1130
1135
1131
-
bkey_fsck_err_on(bucket >= nbuckets, c, err,
1132
-
ptr_after_last_bucket,
1136
+
bkey_fsck_err_on(bucket >= nbuckets,
1137
+
c, ptr_after_last_bucket,
1133
1138
"pointer past last bucket (%llu > %llu)", bucket, nbuckets);
1134
-
bkey_fsck_err_on(bucket < first_bucket, c, err,
1135
-
ptr_before_first_bucket,
1139
+
bkey_fsck_err_on(bucket < first_bucket,
1140
+
c, ptr_before_first_bucket,
1136
1141
"pointer before first bucket (%llu < %u)", bucket, first_bucket);
1137
-
bkey_fsck_err_on(bucket_offset + size_ondisk > bucket_size, c, err,
1138
-
ptr_spans_multiple_buckets,
1142
+
bkey_fsck_err_on(bucket_offset + size_ondisk > bucket_size,
1143
+
c, ptr_spans_multiple_buckets,
1139
1144
"pointer spans multiple buckets (%u + %u > %u)",
1140
1145
bucket_offset, size_ondisk, bucket_size);
1141
1146
fsck_err:
1142
1147
return ret;
1143
1148
}
1144
1149
1145
-
int bch2_bkey_ptrs_invalid(struct bch_fs *c, struct bkey_s_c k,
1146
-
enum bch_validate_flags flags,
1147
-
struct printbuf *err)
1150
+
int bch2_bkey_ptrs_validate(struct bch_fs *c, struct bkey_s_c k,
1151
+
enum bch_validate_flags flags)
1148
1152
{
1149
1153
struct bkey_ptrs_c ptrs = bch2_bkey_ptrs_c(k);
1150
1154
const union bch_extent_entry *entry;
···
1158
1164
size_ondisk = btree_sectors(c);
1159
1165
1160
1166
bkey_extent_entry_for_each(ptrs, entry) {
1161
-
bkey_fsck_err_on(__extent_entry_type(entry) >= BCH_EXTENT_ENTRY_MAX, c, err,
1162
-
extent_ptrs_invalid_entry,
1163
-
"invalid extent entry type (got %u, max %u)",
1164
-
__extent_entry_type(entry), BCH_EXTENT_ENTRY_MAX);
1167
+
bkey_fsck_err_on(__extent_entry_type(entry) >= BCH_EXTENT_ENTRY_MAX,
1168
+
c, extent_ptrs_invalid_entry,
1169
+
"invalid extent entry type (got %u, max %u)",
1170
+
__extent_entry_type(entry), BCH_EXTENT_ENTRY_MAX);
1165
1171
1166
1172
bkey_fsck_err_on(bkey_is_btree_ptr(k.k) &&
1167
-
!extent_entry_is_ptr(entry), c, err,
1168
-
btree_ptr_has_non_ptr,
1173
+
!extent_entry_is_ptr(entry),
1174
+
c, btree_ptr_has_non_ptr,
1169
1175
"has non ptr field");
1170
1176
1171
1177
switch (extent_entry_type(entry)) {
1172
1178
case BCH_EXTENT_ENTRY_ptr:
1173
-
ret = extent_ptr_invalid(c, k, flags, &entry->ptr,
1174
-
size_ondisk, false, err);
1179
+
ret = extent_ptr_validate(c, k, flags, &entry->ptr, size_ondisk, false);
1175
1180
if (ret)
1176
1181
return ret;
1177
1182
1178
-
bkey_fsck_err_on(entry->ptr.cached && have_ec, c, err,
1179
-
ptr_cached_and_erasure_coded,
1183
+
bkey_fsck_err_on(entry->ptr.cached && have_ec,
1184
+
c, ptr_cached_and_erasure_coded,
1180
1185
"cached, erasure coded ptr");
1181
1186
1182
1187
if (!entry->ptr.unwritten)
···
1192
1199
case BCH_EXTENT_ENTRY_crc128:
1193
1200
crc = bch2_extent_crc_unpack(k.k, entry_to_crc(entry));
1194
1201
1195
-
bkey_fsck_err_on(crc.offset + crc.live_size > crc.uncompressed_size, c, err,
1196
-
ptr_crc_uncompressed_size_too_small,
1202
+
bkey_fsck_err_on(crc.offset + crc.live_size > crc.uncompressed_size,
1203
+
c, ptr_crc_uncompressed_size_too_small,
1197
1204
"checksum offset + key size > uncompressed size");
1198
-
bkey_fsck_err_on(!bch2_checksum_type_valid(c, crc.csum_type), c, err,
1199
-
ptr_crc_csum_type_unknown,
1205
+
bkey_fsck_err_on(!bch2_checksum_type_valid(c, crc.csum_type),
1206
+
c, ptr_crc_csum_type_unknown,
1200
1207
"invalid checksum type");
1201
-
bkey_fsck_err_on(crc.compression_type >= BCH_COMPRESSION_TYPE_NR, c, err,
1202
-
ptr_crc_compression_type_unknown,
1208
+
bkey_fsck_err_on(crc.compression_type >= BCH_COMPRESSION_TYPE_NR,
1209
+
c, ptr_crc_compression_type_unknown,
1203
1210
"invalid compression type");
1204
1211
1205
1212
if (bch2_csum_type_is_encryption(crc.csum_type)) {
1206
1213
if (nonce == UINT_MAX)
1207
1214
nonce = crc.offset + crc.nonce;
1208
1215
else if (nonce != crc.offset + crc.nonce)
1209
-
bkey_fsck_err(c, err, ptr_crc_nonce_mismatch,
1216
+
bkey_fsck_err(c, ptr_crc_nonce_mismatch,
1210
1217
"incorrect nonce");
1211
1218
}
1212
1219
1213
-
bkey_fsck_err_on(crc_since_last_ptr, c, err,
1214
-
ptr_crc_redundant,
1220
+
bkey_fsck_err_on(crc_since_last_ptr,
1221
+
c, ptr_crc_redundant,
1215
1222
"redundant crc entry");
1216
1223
crc_since_last_ptr = true;
1217
1224
1218
1225
bkey_fsck_err_on(crc_is_encoded(crc) &&
1219
1226
(crc.uncompressed_size > c->opts.encoded_extent_max >> 9) &&
1220
-
(flags & (BCH_VALIDATE_write|BCH_VALIDATE_commit)), c, err,
1221
-
ptr_crc_uncompressed_size_too_big,
1227
+
(flags & (BCH_VALIDATE_write|BCH_VALIDATE_commit)),
1228
+
c, ptr_crc_uncompressed_size_too_big,
1222
1229
"too large encoded extent");
1223
1230
1224
1231
size_ondisk = crc.compressed_size;
1225
1232
break;
1226
1233
case BCH_EXTENT_ENTRY_stripe_ptr:
1227
-
bkey_fsck_err_on(have_ec, c, err,
1228
-
ptr_stripe_redundant,
1234
+
bkey_fsck_err_on(have_ec,
1235
+
c, ptr_stripe_redundant,
1229
1236
"redundant stripe entry");
1230
1237
have_ec = true;
1231
1238
break;
1232
1239
case BCH_EXTENT_ENTRY_rebalance: {
1240
+
/*
1241
+
* this shouldn't be a fsck error, for forward
1242
+
* compatibility; the rebalance code should just refetch
1243
+
* the compression opt if it's unknown
1244
+
*/
1245
+
#if 0
1233
1246
const struct bch_extent_rebalance *r = &entry->rebalance;
1234
1247
1235
1248
if (!bch2_compression_opt_valid(r->compression)) {
···
1244
1245
opt.type, opt.level);
1245
1246
return -BCH_ERR_invalid_bkey;
1246
1247
}
1248
+
#endif
1247
1249
break;
1248
1250
}
1249
1251
}
1250
1252
}
1251
1253
1252
-
bkey_fsck_err_on(!nr_ptrs, c, err,
1253
-
extent_ptrs_no_ptrs,
1254
+
bkey_fsck_err_on(!nr_ptrs,
1255
+
c, extent_ptrs_no_ptrs,
1254
1256
"no ptrs");
1255
-
bkey_fsck_err_on(nr_ptrs > BCH_BKEY_PTRS_MAX, c, err,
1256
-
extent_ptrs_too_many_ptrs,
1257
+
bkey_fsck_err_on(nr_ptrs > BCH_BKEY_PTRS_MAX,
1258
+
c, extent_ptrs_too_many_ptrs,
1257
1259
"too many ptrs: %u > %u", nr_ptrs, BCH_BKEY_PTRS_MAX);
1258
-
bkey_fsck_err_on(have_written && have_unwritten, c, err,
1259
-
extent_ptrs_written_and_unwritten,
1260
+
bkey_fsck_err_on(have_written && have_unwritten,
1261
+
c, extent_ptrs_written_and_unwritten,
1260
1262
"extent with unwritten and written ptrs");
1261
-
bkey_fsck_err_on(k.k->type != KEY_TYPE_extent && have_unwritten, c, err,
1262
-
extent_ptrs_unwritten,
1263
+
bkey_fsck_err_on(k.k->type != KEY_TYPE_extent && have_unwritten,
1264
+
c, extent_ptrs_unwritten,
1263
1265
"has unwritten ptrs");
1264
-
bkey_fsck_err_on(crc_since_last_ptr, c, err,
1265
-
extent_ptrs_redundant_crc,
1266
+
bkey_fsck_err_on(crc_since_last_ptr,
1267
+
c, extent_ptrs_redundant_crc,
1266
1268
"redundant crc entry");
1267
-
bkey_fsck_err_on(have_ec, c, err,
1268
-
extent_ptrs_redundant_stripe,
1269
+
bkey_fsck_err_on(have_ec,
1270
+
c, extent_ptrs_redundant_stripe,
1269
1271
"redundant stripe entry");
1270
1272
fsck_err:
1271
1273
return ret;
+12
-12
fs/bcachefs/extents.h
+12
-12
fs/bcachefs/extents.h
···
409
409
410
410
/* KEY_TYPE_btree_ptr: */
411
411
412
-
int bch2_btree_ptr_invalid(struct bch_fs *, struct bkey_s_c,
413
-
enum bch_validate_flags, struct printbuf *);
412
+
int bch2_btree_ptr_validate(struct bch_fs *, struct bkey_s_c,
413
+
enum bch_validate_flags);
414
414
void bch2_btree_ptr_to_text(struct printbuf *, struct bch_fs *,
415
415
struct bkey_s_c);
416
416
417
-
int bch2_btree_ptr_v2_invalid(struct bch_fs *, struct bkey_s_c,
418
-
enum bch_validate_flags, struct printbuf *);
417
+
int bch2_btree_ptr_v2_validate(struct bch_fs *, struct bkey_s_c,
418
+
enum bch_validate_flags);
419
419
void bch2_btree_ptr_v2_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
420
420
void bch2_btree_ptr_v2_compat(enum btree_id, unsigned, unsigned,
421
421
int, struct bkey_s);
422
422
423
423
#define bch2_bkey_ops_btree_ptr ((struct bkey_ops) { \
424
-
.key_invalid = bch2_btree_ptr_invalid, \
424
+
.key_validate = bch2_btree_ptr_validate, \
425
425
.val_to_text = bch2_btree_ptr_to_text, \
426
426
.swab = bch2_ptr_swab, \
427
427
.trigger = bch2_trigger_extent, \
428
428
})
429
429
430
430
#define bch2_bkey_ops_btree_ptr_v2 ((struct bkey_ops) { \
431
-
.key_invalid = bch2_btree_ptr_v2_invalid, \
431
+
.key_validate = bch2_btree_ptr_v2_validate, \
432
432
.val_to_text = bch2_btree_ptr_v2_to_text, \
433
433
.swab = bch2_ptr_swab, \
434
434
.compat = bch2_btree_ptr_v2_compat, \
···
441
441
bool bch2_extent_merge(struct bch_fs *, struct bkey_s, struct bkey_s_c);
442
442
443
443
#define bch2_bkey_ops_extent ((struct bkey_ops) { \
444
-
.key_invalid = bch2_bkey_ptrs_invalid, \
444
+
.key_validate = bch2_bkey_ptrs_validate, \
445
445
.val_to_text = bch2_bkey_ptrs_to_text, \
446
446
.swab = bch2_ptr_swab, \
447
447
.key_normalize = bch2_extent_normalize, \
···
451
451
452
452
/* KEY_TYPE_reservation: */
453
453
454
-
int bch2_reservation_invalid(struct bch_fs *, struct bkey_s_c,
455
-
enum bch_validate_flags, struct printbuf *);
454
+
int bch2_reservation_validate(struct bch_fs *, struct bkey_s_c,
455
+
enum bch_validate_flags);
456
456
void bch2_reservation_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
457
457
bool bch2_reservation_merge(struct bch_fs *, struct bkey_s, struct bkey_s_c);
458
458
459
459
#define bch2_bkey_ops_reservation ((struct bkey_ops) { \
460
-
.key_invalid = bch2_reservation_invalid, \
460
+
.key_validate = bch2_reservation_validate, \
461
461
.val_to_text = bch2_reservation_to_text, \
462
462
.key_merge = bch2_reservation_merge, \
463
463
.trigger = bch2_trigger_reservation, \
···
683
683
void bch2_extent_ptr_to_text(struct printbuf *out, struct bch_fs *, const struct bch_extent_ptr *);
684
684
void bch2_bkey_ptrs_to_text(struct printbuf *, struct bch_fs *,
685
685
struct bkey_s_c);
686
-
int bch2_bkey_ptrs_invalid(struct bch_fs *, struct bkey_s_c,
687
-
enum bch_validate_flags, struct printbuf *);
686
+
int bch2_bkey_ptrs_validate(struct bch_fs *, struct bkey_s_c,
687
+
enum bch_validate_flags);
688
688
689
689
void bch2_ptr_swab(struct bkey_s);
690
690
+1
-1
fs/bcachefs/fs.c
+1
-1
fs/bcachefs/fs.c
···
193
193
* only insert fully created inodes in the inode hash table. But
194
194
* discard_new_inode() expects it to be set...
195
195
*/
196
-
inode->v.i_flags |= I_NEW;
196
+
inode->v.i_state |= I_NEW;
197
197
/*
198
198
* We don't want bch2_evict_inode() to delete the inode on disk,
199
199
* we just raced and had another inode in cache. Normally new
+37
-40
fs/bcachefs/inode.c
+37
-40
fs/bcachefs/inode.c
···
434
434
return &inode_p->inode.k_i;
435
435
}
436
436
437
-
static int __bch2_inode_invalid(struct bch_fs *c, struct bkey_s_c k, struct printbuf *err)
437
+
static int __bch2_inode_validate(struct bch_fs *c, struct bkey_s_c k,
438
+
enum bch_validate_flags flags)
438
439
{
439
440
struct bch_inode_unpacked unpacked;
440
441
int ret = 0;
441
442
442
-
bkey_fsck_err_on(k.k->p.inode, c, err,
443
-
inode_pos_inode_nonzero,
443
+
bkey_fsck_err_on(k.k->p.inode,
444
+
c, inode_pos_inode_nonzero,
444
445
"nonzero k.p.inode");
445
446
446
-
bkey_fsck_err_on(k.k->p.offset < BLOCKDEV_INODE_MAX, c, err,
447
-
inode_pos_blockdev_range,
447
+
bkey_fsck_err_on(k.k->p.offset < BLOCKDEV_INODE_MAX,
448
+
c, inode_pos_blockdev_range,
448
449
"fs inode in blockdev range");
449
450
450
-
bkey_fsck_err_on(bch2_inode_unpack(k, &unpacked), c, err,
451
-
inode_unpack_error,
451
+
bkey_fsck_err_on(bch2_inode_unpack(k, &unpacked),
452
+
c, inode_unpack_error,
452
453
"invalid variable length fields");
453
454
454
-
bkey_fsck_err_on(unpacked.bi_data_checksum >= BCH_CSUM_OPT_NR + 1, c, err,
455
-
inode_checksum_type_invalid,
455
+
bkey_fsck_err_on(unpacked.bi_data_checksum >= BCH_CSUM_OPT_NR + 1,
456
+
c, inode_checksum_type_invalid,
456
457
"invalid data checksum type (%u >= %u",
457
458
unpacked.bi_data_checksum, BCH_CSUM_OPT_NR + 1);
458
459
459
460
bkey_fsck_err_on(unpacked.bi_compression &&
460
-
!bch2_compression_opt_valid(unpacked.bi_compression - 1), c, err,
461
-
inode_compression_type_invalid,
461
+
!bch2_compression_opt_valid(unpacked.bi_compression - 1),
462
+
c, inode_compression_type_invalid,
462
463
"invalid compression opt %u", unpacked.bi_compression - 1);
463
464
464
465
bkey_fsck_err_on((unpacked.bi_flags & BCH_INODE_unlinked) &&
465
-
unpacked.bi_nlink != 0, c, err,
466
-
inode_unlinked_but_nlink_nonzero,
466
+
unpacked.bi_nlink != 0,
467
+
c, inode_unlinked_but_nlink_nonzero,
467
468
"flagged as unlinked but bi_nlink != 0");
468
469
469
-
bkey_fsck_err_on(unpacked.bi_subvol && !S_ISDIR(unpacked.bi_mode), c, err,
470
-
inode_subvol_root_but_not_dir,
470
+
bkey_fsck_err_on(unpacked.bi_subvol && !S_ISDIR(unpacked.bi_mode),
471
+
c, inode_subvol_root_but_not_dir,
471
472
"subvolume root but not a directory");
472
473
fsck_err:
473
474
return ret;
474
475
}
475
476
476
-
int bch2_inode_invalid(struct bch_fs *c, struct bkey_s_c k,
477
-
enum bch_validate_flags flags,
478
-
struct printbuf *err)
477
+
int bch2_inode_validate(struct bch_fs *c, struct bkey_s_c k,
478
+
enum bch_validate_flags flags)
479
479
{
480
480
struct bkey_s_c_inode inode = bkey_s_c_to_inode(k);
481
481
int ret = 0;
482
482
483
-
bkey_fsck_err_on(INODE_STR_HASH(inode.v) >= BCH_STR_HASH_NR, c, err,
484
-
inode_str_hash_invalid,
483
+
bkey_fsck_err_on(INODE_STR_HASH(inode.v) >= BCH_STR_HASH_NR,
484
+
c, inode_str_hash_invalid,
485
485
"invalid str hash type (%llu >= %u)",
486
486
INODE_STR_HASH(inode.v), BCH_STR_HASH_NR);
487
487
488
-
ret = __bch2_inode_invalid(c, k, err);
488
+
ret = __bch2_inode_validate(c, k, flags);
489
489
fsck_err:
490
490
return ret;
491
491
}
492
492
493
-
int bch2_inode_v2_invalid(struct bch_fs *c, struct bkey_s_c k,
494
-
enum bch_validate_flags flags,
495
-
struct printbuf *err)
493
+
int bch2_inode_v2_validate(struct bch_fs *c, struct bkey_s_c k,
494
+
enum bch_validate_flags flags)
496
495
{
497
496
struct bkey_s_c_inode_v2 inode = bkey_s_c_to_inode_v2(k);
498
497
int ret = 0;
499
498
500
-
bkey_fsck_err_on(INODEv2_STR_HASH(inode.v) >= BCH_STR_HASH_NR, c, err,
501
-
inode_str_hash_invalid,
499
+
bkey_fsck_err_on(INODEv2_STR_HASH(inode.v) >= BCH_STR_HASH_NR,
500
+
c, inode_str_hash_invalid,
502
501
"invalid str hash type (%llu >= %u)",
503
502
INODEv2_STR_HASH(inode.v), BCH_STR_HASH_NR);
504
503
505
-
ret = __bch2_inode_invalid(c, k, err);
504
+
ret = __bch2_inode_validate(c, k, flags);
506
505
fsck_err:
507
506
return ret;
508
507
}
509
508
510
-
int bch2_inode_v3_invalid(struct bch_fs *c, struct bkey_s_c k,
511
-
enum bch_validate_flags flags,
512
-
struct printbuf *err)
509
+
int bch2_inode_v3_validate(struct bch_fs *c, struct bkey_s_c k,
510
+
enum bch_validate_flags flags)
513
511
{
514
512
struct bkey_s_c_inode_v3 inode = bkey_s_c_to_inode_v3(k);
515
513
int ret = 0;
516
514
517
515
bkey_fsck_err_on(INODEv3_FIELDS_START(inode.v) < INODEv3_FIELDS_START_INITIAL ||
518
-
INODEv3_FIELDS_START(inode.v) > bkey_val_u64s(inode.k), c, err,
519
-
inode_v3_fields_start_bad,
516
+
INODEv3_FIELDS_START(inode.v) > bkey_val_u64s(inode.k),
517
+
c, inode_v3_fields_start_bad,
520
518
"invalid fields_start (got %llu, min %u max %zu)",
521
519
INODEv3_FIELDS_START(inode.v),
522
520
INODEv3_FIELDS_START_INITIAL,
523
521
bkey_val_u64s(inode.k));
524
522
525
-
bkey_fsck_err_on(INODEv3_STR_HASH(inode.v) >= BCH_STR_HASH_NR, c, err,
526
-
inode_str_hash_invalid,
523
+
bkey_fsck_err_on(INODEv3_STR_HASH(inode.v) >= BCH_STR_HASH_NR,
524
+
c, inode_str_hash_invalid,
527
525
"invalid str hash type (%llu >= %u)",
528
526
INODEv3_STR_HASH(inode.v), BCH_STR_HASH_NR);
529
527
530
-
ret = __bch2_inode_invalid(c, k, err);
528
+
ret = __bch2_inode_validate(c, k, flags);
531
529
fsck_err:
532
530
return ret;
533
531
}
···
623
625
return 0;
624
626
}
625
627
626
-
int bch2_inode_generation_invalid(struct bch_fs *c, struct bkey_s_c k,
627
-
enum bch_validate_flags flags,
628
-
struct printbuf *err)
628
+
int bch2_inode_generation_validate(struct bch_fs *c, struct bkey_s_c k,
629
+
enum bch_validate_flags flags)
629
630
{
630
631
int ret = 0;
631
632
632
-
bkey_fsck_err_on(k.k->p.inode, c, err,
633
-
inode_pos_inode_nonzero,
633
+
bkey_fsck_err_on(k.k->p.inode,
634
+
c, inode_pos_inode_nonzero,
634
635
"nonzero k.p.inode");
635
636
fsck_err:
636
637
return ret;
+12
-12
fs/bcachefs/inode.h
+12
-12
fs/bcachefs/inode.h
···
9
9
enum bch_validate_flags;
10
10
extern const char * const bch2_inode_opts[];
11
11
12
-
int bch2_inode_invalid(struct bch_fs *, struct bkey_s_c,
13
-
enum bch_validate_flags, struct printbuf *);
14
-
int bch2_inode_v2_invalid(struct bch_fs *, struct bkey_s_c,
15
-
enum bch_validate_flags, struct printbuf *);
16
-
int bch2_inode_v3_invalid(struct bch_fs *, struct bkey_s_c,
17
-
enum bch_validate_flags, struct printbuf *);
12
+
int bch2_inode_validate(struct bch_fs *, struct bkey_s_c,
13
+
enum bch_validate_flags);
14
+
int bch2_inode_v2_validate(struct bch_fs *, struct bkey_s_c,
15
+
enum bch_validate_flags);
16
+
int bch2_inode_v3_validate(struct bch_fs *, struct bkey_s_c,
17
+
enum bch_validate_flags);
18
18
void bch2_inode_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
19
19
20
20
int bch2_trigger_inode(struct btree_trans *, enum btree_id, unsigned,
···
22
22
enum btree_iter_update_trigger_flags);
23
23
24
24
#define bch2_bkey_ops_inode ((struct bkey_ops) { \
25
-
.key_invalid = bch2_inode_invalid, \
25
+
.key_validate = bch2_inode_validate, \
26
26
.val_to_text = bch2_inode_to_text, \
27
27
.trigger = bch2_trigger_inode, \
28
28
.min_val_size = 16, \
29
29
})
30
30
31
31
#define bch2_bkey_ops_inode_v2 ((struct bkey_ops) { \
32
-
.key_invalid = bch2_inode_v2_invalid, \
32
+
.key_validate = bch2_inode_v2_validate, \
33
33
.val_to_text = bch2_inode_to_text, \
34
34
.trigger = bch2_trigger_inode, \
35
35
.min_val_size = 32, \
36
36
})
37
37
38
38
#define bch2_bkey_ops_inode_v3 ((struct bkey_ops) { \
39
-
.key_invalid = bch2_inode_v3_invalid, \
39
+
.key_validate = bch2_inode_v3_validate, \
40
40
.val_to_text = bch2_inode_to_text, \
41
41
.trigger = bch2_trigger_inode, \
42
42
.min_val_size = 48, \
···
49
49
k->type == KEY_TYPE_inode_v3;
50
50
}
51
51
52
-
int bch2_inode_generation_invalid(struct bch_fs *, struct bkey_s_c,
53
-
enum bch_validate_flags, struct printbuf *);
52
+
int bch2_inode_generation_validate(struct bch_fs *, struct bkey_s_c,
53
+
enum bch_validate_flags);
54
54
void bch2_inode_generation_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
55
55
56
56
#define bch2_bkey_ops_inode_generation ((struct bkey_ops) { \
57
-
.key_invalid = bch2_inode_generation_invalid, \
57
+
.key_validate = bch2_inode_generation_validate, \
58
58
.val_to_text = bch2_inode_generation_to_text, \
59
59
.min_val_size = 8, \
60
60
})
+5
-19
fs/bcachefs/journal_io.c
+5
-19
fs/bcachefs/journal_io.c
···
332
332
{
333
333
int write = flags & BCH_VALIDATE_write;
334
334
void *next = vstruct_next(entry);
335
-
struct printbuf buf = PRINTBUF;
336
335
int ret = 0;
337
336
338
337
if (journal_entry_err_on(!k->k.u64s,
···
367
368
bch2_bkey_compat(level, btree_id, version, big_endian,
368
369
write, NULL, bkey_to_packed(k));
369
370
370
-
if (bch2_bkey_invalid(c, bkey_i_to_s_c(k),
371
-
__btree_node_type(level, btree_id), write, &buf)) {
372
-
printbuf_reset(&buf);
373
-
journal_entry_err_msg(&buf, version, jset, entry);
374
-
prt_newline(&buf);
375
-
printbuf_indent_add(&buf, 2);
376
-
377
-
bch2_bkey_val_to_text(&buf, c, bkey_i_to_s_c(k));
378
-
prt_newline(&buf);
379
-
bch2_bkey_invalid(c, bkey_i_to_s_c(k),
380
-
__btree_node_type(level, btree_id), write, &buf);
381
-
382
-
mustfix_fsck_err(c, journal_entry_bkey_invalid,
383
-
"%s", buf.buf);
384
-
371
+
ret = bch2_bkey_validate(c, bkey_i_to_s_c(k),
372
+
__btree_node_type(level, btree_id), write);
373
+
if (ret == -BCH_ERR_fsck_delete_bkey) {
385
374
le16_add_cpu(&entry->u64s, -((u16) k->k.u64s));
386
375
memmove(k, bkey_next(k), next - (void *) bkey_next(k));
387
376
journal_entry_null_range(vstruct_next(entry), next);
388
-
389
-
printbuf_exit(&buf);
390
377
return FSCK_DELETED_KEY;
391
378
}
379
+
if (ret)
380
+
goto fsck_err;
392
381
393
382
if (write)
394
383
bch2_bkey_compat(level, btree_id, version, big_endian,
395
384
write, NULL, bkey_to_packed(k));
396
385
fsck_err:
397
-
printbuf_exit(&buf);
398
386
return ret;
399
387
}
400
388
+4
-5
fs/bcachefs/lru.c
+4
-5
fs/bcachefs/lru.c
···
10
10
#include "recovery.h"
11
11
12
12
/* KEY_TYPE_lru is obsolete: */
13
-
int bch2_lru_invalid(struct bch_fs *c, struct bkey_s_c k,
14
-
enum bch_validate_flags flags,
15
-
struct printbuf *err)
13
+
int bch2_lru_validate(struct bch_fs *c, struct bkey_s_c k,
14
+
enum bch_validate_flags flags)
16
15
{
17
16
int ret = 0;
18
17
19
-
bkey_fsck_err_on(!lru_pos_time(k.k->p), c, err,
20
-
lru_entry_at_time_0,
18
+
bkey_fsck_err_on(!lru_pos_time(k.k->p),
19
+
c, lru_entry_at_time_0,
21
20
"lru entry at time=0");
22
21
fsck_err:
23
22
return ret;
+2
-3
fs/bcachefs/lru.h
+2
-3
fs/bcachefs/lru.h
···
33
33
return BCH_LRU_read;
34
34
}
35
35
36
-
int bch2_lru_invalid(struct bch_fs *, struct bkey_s_c,
37
-
enum bch_validate_flags, struct printbuf *);
36
+
int bch2_lru_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
38
37
void bch2_lru_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
39
38
40
39
void bch2_lru_pos_to_text(struct printbuf *, struct bpos);
41
40
42
41
#define bch2_bkey_ops_lru ((struct bkey_ops) { \
43
-
.key_invalid = bch2_lru_invalid, \
42
+
.key_validate = bch2_lru_validate, \
44
43
.val_to_text = bch2_lru_to_text, \
45
44
.min_val_size = 8, \
46
45
})
+4
-4
fs/bcachefs/quota.c
+4
-4
fs/bcachefs/quota.c
···
59
59
.to_text = bch2_sb_quota_to_text,
60
60
};
61
61
62
-
int bch2_quota_invalid(struct bch_fs *c, struct bkey_s_c k,
63
-
enum bch_validate_flags flags, struct printbuf *err)
62
+
int bch2_quota_validate(struct bch_fs *c, struct bkey_s_c k,
63
+
enum bch_validate_flags flags)
64
64
{
65
65
int ret = 0;
66
66
67
-
bkey_fsck_err_on(k.k->p.inode >= QTYP_NR, c, err,
68
-
quota_type_invalid,
67
+
bkey_fsck_err_on(k.k->p.inode >= QTYP_NR,
68
+
c, quota_type_invalid,
69
69
"invalid quota type (%llu >= %u)",
70
70
k.k->p.inode, QTYP_NR);
71
71
fsck_err:
+2
-3
fs/bcachefs/quota.h
+2
-3
fs/bcachefs/quota.h
···
8
8
enum bch_validate_flags;
9
9
extern const struct bch_sb_field_ops bch_sb_field_ops_quota;
10
10
11
-
int bch2_quota_invalid(struct bch_fs *, struct bkey_s_c,
12
-
enum bch_validate_flags, struct printbuf *);
11
+
int bch2_quota_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
13
12
void bch2_quota_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
14
13
15
14
#define bch2_bkey_ops_quota ((struct bkey_ops) { \
16
-
.key_invalid = bch2_quota_invalid, \
15
+
.key_validate = bch2_quota_validate, \
17
16
.val_to_text = bch2_quota_to_text, \
18
17
.min_val_size = 32, \
19
18
})
+8
-11
fs/bcachefs/reflink.c
+8
-11
fs/bcachefs/reflink.c
···
29
29
30
30
/* reflink pointers */
31
31
32
-
int bch2_reflink_p_invalid(struct bch_fs *c, struct bkey_s_c k,
33
-
enum bch_validate_flags flags,
34
-
struct printbuf *err)
32
+
int bch2_reflink_p_validate(struct bch_fs *c, struct bkey_s_c k,
33
+
enum bch_validate_flags flags)
35
34
{
36
35
struct bkey_s_c_reflink_p p = bkey_s_c_to_reflink_p(k);
37
36
int ret = 0;
38
37
39
38
bkey_fsck_err_on(le64_to_cpu(p.v->idx) < le32_to_cpu(p.v->front_pad),
40
-
c, err, reflink_p_front_pad_bad,
39
+
c, reflink_p_front_pad_bad,
41
40
"idx < front_pad (%llu < %u)",
42
41
le64_to_cpu(p.v->idx), le32_to_cpu(p.v->front_pad));
43
42
fsck_err:
···
255
256
256
257
/* indirect extents */
257
258
258
-
int bch2_reflink_v_invalid(struct bch_fs *c, struct bkey_s_c k,
259
-
enum bch_validate_flags flags,
260
-
struct printbuf *err)
259
+
int bch2_reflink_v_validate(struct bch_fs *c, struct bkey_s_c k,
260
+
enum bch_validate_flags flags)
261
261
{
262
-
return bch2_bkey_ptrs_invalid(c, k, flags, err);
262
+
return bch2_bkey_ptrs_validate(c, k, flags);
263
263
}
264
264
265
265
void bch2_reflink_v_to_text(struct printbuf *out, struct bch_fs *c,
···
309
311
310
312
/* indirect inline data */
311
313
312
-
int bch2_indirect_inline_data_invalid(struct bch_fs *c, struct bkey_s_c k,
313
-
enum bch_validate_flags flags,
314
-
struct printbuf *err)
314
+
int bch2_indirect_inline_data_validate(struct bch_fs *c, struct bkey_s_c k,
315
+
enum bch_validate_flags flags)
315
316
{
316
317
return 0;
317
318
}
+9
-13
fs/bcachefs/reflink.h
+9
-13
fs/bcachefs/reflink.h
···
4
4
5
5
enum bch_validate_flags;
6
6
7
-
int bch2_reflink_p_invalid(struct bch_fs *, struct bkey_s_c,
8
-
enum bch_validate_flags, struct printbuf *);
9
-
void bch2_reflink_p_to_text(struct printbuf *, struct bch_fs *,
10
-
struct bkey_s_c);
7
+
int bch2_reflink_p_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
8
+
void bch2_reflink_p_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
11
9
bool bch2_reflink_p_merge(struct bch_fs *, struct bkey_s, struct bkey_s_c);
12
10
int bch2_trigger_reflink_p(struct btree_trans *, enum btree_id, unsigned,
13
11
struct bkey_s_c, struct bkey_s,
14
12
enum btree_iter_update_trigger_flags);
15
13
16
14
#define bch2_bkey_ops_reflink_p ((struct bkey_ops) { \
17
-
.key_invalid = bch2_reflink_p_invalid, \
15
+
.key_validate = bch2_reflink_p_validate, \
18
16
.val_to_text = bch2_reflink_p_to_text, \
19
17
.key_merge = bch2_reflink_p_merge, \
20
18
.trigger = bch2_trigger_reflink_p, \
21
19
.min_val_size = 16, \
22
20
})
23
21
24
-
int bch2_reflink_v_invalid(struct bch_fs *, struct bkey_s_c,
25
-
enum bch_validate_flags, struct printbuf *);
26
-
void bch2_reflink_v_to_text(struct printbuf *, struct bch_fs *,
27
-
struct bkey_s_c);
22
+
int bch2_reflink_v_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
23
+
void bch2_reflink_v_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
28
24
int bch2_trigger_reflink_v(struct btree_trans *, enum btree_id, unsigned,
29
25
struct bkey_s_c, struct bkey_s,
30
26
enum btree_iter_update_trigger_flags);
31
27
32
28
#define bch2_bkey_ops_reflink_v ((struct bkey_ops) { \
33
-
.key_invalid = bch2_reflink_v_invalid, \
29
+
.key_validate = bch2_reflink_v_validate, \
34
30
.val_to_text = bch2_reflink_v_to_text, \
35
31
.swab = bch2_ptr_swab, \
36
32
.trigger = bch2_trigger_reflink_v, \
37
33
.min_val_size = 8, \
38
34
})
39
35
40
-
int bch2_indirect_inline_data_invalid(struct bch_fs *, struct bkey_s_c,
41
-
enum bch_validate_flags, struct printbuf *);
36
+
int bch2_indirect_inline_data_validate(struct bch_fs *, struct bkey_s_c,
37
+
enum bch_validate_flags);
42
38
void bch2_indirect_inline_data_to_text(struct printbuf *,
43
39
struct bch_fs *, struct bkey_s_c);
44
40
int bch2_trigger_indirect_inline_data(struct btree_trans *,
···
43
47
enum btree_iter_update_trigger_flags);
44
48
45
49
#define bch2_bkey_ops_indirect_inline_data ((struct bkey_ops) { \
46
-
.key_invalid = bch2_indirect_inline_data_invalid, \
50
+
.key_validate = bch2_indirect_inline_data_validate, \
47
51
.val_to_text = bch2_indirect_inline_data_to_text, \
48
52
.trigger = bch2_trigger_indirect_inline_data, \
49
53
.min_val_size = 8, \
+5
-1
fs/bcachefs/sb-downgrade.c
+5
-1
fs/bcachefs/sb-downgrade.c
···
72
72
BCH_FSCK_ERR_accounting_key_replicas_nr_devs_0, \
73
73
BCH_FSCK_ERR_accounting_key_replicas_nr_required_bad, \
74
74
BCH_FSCK_ERR_accounting_key_replicas_devs_unsorted, \
75
-
BCH_FSCK_ERR_accounting_key_junk_at_end)
75
+
BCH_FSCK_ERR_accounting_key_junk_at_end) \
76
+
x(disk_accounting_inum, \
77
+
BIT_ULL(BCH_RECOVERY_PASS_check_allocations), \
78
+
BCH_FSCK_ERR_accounting_mismatch)
76
79
77
80
#define DOWNGRADE_TABLE() \
78
81
x(bucket_stripe_sectors, \
···
107
104
BCH_FSCK_ERR_fs_usage_nr_inodes_wrong, \
108
105
BCH_FSCK_ERR_fs_usage_persistent_reserved_wrong, \
109
106
BCH_FSCK_ERR_fs_usage_replicas_wrong, \
107
+
BCH_FSCK_ERR_accounting_replicas_not_marked, \
110
108
BCH_FSCK_ERR_bkey_version_in_future)
111
109
112
110
struct upgrade_downgrade_entry {
+20
-22
fs/bcachefs/snapshot.c
+20
-22
fs/bcachefs/snapshot.c
···
31
31
le32_to_cpu(t.v->root_snapshot));
32
32
}
33
33
34
-
int bch2_snapshot_tree_invalid(struct bch_fs *c, struct bkey_s_c k,
35
-
enum bch_validate_flags flags,
36
-
struct printbuf *err)
34
+
int bch2_snapshot_tree_validate(struct bch_fs *c, struct bkey_s_c k,
35
+
enum bch_validate_flags flags)
37
36
{
38
37
int ret = 0;
39
38
40
39
bkey_fsck_err_on(bkey_gt(k.k->p, POS(0, U32_MAX)) ||
41
-
bkey_lt(k.k->p, POS(0, 1)), c, err,
42
-
snapshot_tree_pos_bad,
40
+
bkey_lt(k.k->p, POS(0, 1)),
41
+
c, snapshot_tree_pos_bad,
43
42
"bad pos");
44
43
fsck_err:
45
44
return ret;
···
224
225
le32_to_cpu(s.v->skip[2]));
225
226
}
226
227
227
-
int bch2_snapshot_invalid(struct bch_fs *c, struct bkey_s_c k,
228
-
enum bch_validate_flags flags,
229
-
struct printbuf *err)
228
+
int bch2_snapshot_validate(struct bch_fs *c, struct bkey_s_c k,
229
+
enum bch_validate_flags flags)
230
230
{
231
231
struct bkey_s_c_snapshot s;
232
232
u32 i, id;
233
233
int ret = 0;
234
234
235
235
bkey_fsck_err_on(bkey_gt(k.k->p, POS(0, U32_MAX)) ||
236
-
bkey_lt(k.k->p, POS(0, 1)), c, err,
237
-
snapshot_pos_bad,
236
+
bkey_lt(k.k->p, POS(0, 1)),
237
+
c, snapshot_pos_bad,
238
238
"bad pos");
239
239
240
240
s = bkey_s_c_to_snapshot(k);
241
241
242
242
id = le32_to_cpu(s.v->parent);
243
-
bkey_fsck_err_on(id && id <= k.k->p.offset, c, err,
244
-
snapshot_parent_bad,
243
+
bkey_fsck_err_on(id && id <= k.k->p.offset,
244
+
c, snapshot_parent_bad,
245
245
"bad parent node (%u <= %llu)",
246
246
id, k.k->p.offset);
247
247
248
-
bkey_fsck_err_on(le32_to_cpu(s.v->children[0]) < le32_to_cpu(s.v->children[1]), c, err,
249
-
snapshot_children_not_normalized,
248
+
bkey_fsck_err_on(le32_to_cpu(s.v->children[0]) < le32_to_cpu(s.v->children[1]),
249
+
c, snapshot_children_not_normalized,
250
250
"children not normalized");
251
251
252
-
bkey_fsck_err_on(s.v->children[0] && s.v->children[0] == s.v->children[1], c, err,
253
-
snapshot_child_duplicate,
252
+
bkey_fsck_err_on(s.v->children[0] && s.v->children[0] == s.v->children[1],
253
+
c, snapshot_child_duplicate,
254
254
"duplicate child nodes");
255
255
256
256
for (i = 0; i < 2; i++) {
257
257
id = le32_to_cpu(s.v->children[i]);
258
258
259
-
bkey_fsck_err_on(id >= k.k->p.offset, c, err,
260
-
snapshot_child_bad,
259
+
bkey_fsck_err_on(id >= k.k->p.offset,
260
+
c, snapshot_child_bad,
261
261
"bad child node (%u >= %llu)",
262
262
id, k.k->p.offset);
263
263
}
264
264
265
265
if (bkey_val_bytes(k.k) > offsetof(struct bch_snapshot, skip)) {
266
266
bkey_fsck_err_on(le32_to_cpu(s.v->skip[0]) > le32_to_cpu(s.v->skip[1]) ||
267
-
le32_to_cpu(s.v->skip[1]) > le32_to_cpu(s.v->skip[2]), c, err,
268
-
snapshot_skiplist_not_normalized,
267
+
le32_to_cpu(s.v->skip[1]) > le32_to_cpu(s.v->skip[2]),
268
+
c, snapshot_skiplist_not_normalized,
269
269
"skiplist not normalized");
270
270
271
271
for (i = 0; i < ARRAY_SIZE(s.v->skip); i++) {
272
272
id = le32_to_cpu(s.v->skip[i]);
273
273
274
-
bkey_fsck_err_on(id && id < le32_to_cpu(s.v->parent), c, err,
275
-
snapshot_skiplist_bad,
274
+
bkey_fsck_err_on(id && id < le32_to_cpu(s.v->parent),
275
+
c, snapshot_skiplist_bad,
276
276
"bad skiplist node %u", id);
277
277
}
278
278
}
+5
-6
fs/bcachefs/snapshot.h
+5
-6
fs/bcachefs/snapshot.h
···
5
5
enum bch_validate_flags;
6
6
7
7
void bch2_snapshot_tree_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
8
-
int bch2_snapshot_tree_invalid(struct bch_fs *, struct bkey_s_c,
9
-
enum bch_validate_flags, struct printbuf *);
8
+
int bch2_snapshot_tree_validate(struct bch_fs *, struct bkey_s_c,
9
+
enum bch_validate_flags);
10
10
11
11
#define bch2_bkey_ops_snapshot_tree ((struct bkey_ops) { \
12
-
.key_invalid = bch2_snapshot_tree_invalid, \
12
+
.key_validate = bch2_snapshot_tree_validate, \
13
13
.val_to_text = bch2_snapshot_tree_to_text, \
14
14
.min_val_size = 8, \
15
15
})
···
19
19
int bch2_snapshot_tree_lookup(struct btree_trans *, u32, struct bch_snapshot_tree *);
20
20
21
21
void bch2_snapshot_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
22
-
int bch2_snapshot_invalid(struct bch_fs *, struct bkey_s_c,
23
-
enum bch_validate_flags, struct printbuf *);
22
+
int bch2_snapshot_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
24
23
int bch2_mark_snapshot(struct btree_trans *, enum btree_id, unsigned,
25
24
struct bkey_s_c, struct bkey_s,
26
25
enum btree_iter_update_trigger_flags);
27
26
28
27
#define bch2_bkey_ops_snapshot ((struct bkey_ops) { \
29
-
.key_invalid = bch2_snapshot_invalid, \
28
+
.key_validate = bch2_snapshot_validate, \
30
29
.val_to_text = bch2_snapshot_to_text, \
31
30
.trigger = bch2_mark_snapshot, \
32
31
.min_val_size = 24, \
+8
-8
fs/bcachefs/subvolume.c
+8
-8
fs/bcachefs/subvolume.c
···
207
207
208
208
/* Subvolumes: */
209
209
210
-
int bch2_subvolume_invalid(struct bch_fs *c, struct bkey_s_c k,
211
-
enum bch_validate_flags flags, struct printbuf *err)
210
+
int bch2_subvolume_validate(struct bch_fs *c, struct bkey_s_c k,
211
+
enum bch_validate_flags flags)
212
212
{
213
213
struct bkey_s_c_subvolume subvol = bkey_s_c_to_subvolume(k);
214
214
int ret = 0;
215
215
216
216
bkey_fsck_err_on(bkey_lt(k.k->p, SUBVOL_POS_MIN) ||
217
-
bkey_gt(k.k->p, SUBVOL_POS_MAX), c, err,
218
-
subvol_pos_bad,
217
+
bkey_gt(k.k->p, SUBVOL_POS_MAX),
218
+
c, subvol_pos_bad,
219
219
"invalid pos");
220
220
221
-
bkey_fsck_err_on(!subvol.v->snapshot, c, err,
222
-
subvol_snapshot_bad,
221
+
bkey_fsck_err_on(!subvol.v->snapshot,
222
+
c, subvol_snapshot_bad,
223
223
"invalid snapshot");
224
224
225
-
bkey_fsck_err_on(!subvol.v->inode, c, err,
226
-
subvol_inode_bad,
225
+
bkey_fsck_err_on(!subvol.v->inode,
226
+
c, subvol_inode_bad,
227
227
"invalid inode");
228
228
fsck_err:
229
229
return ret;
+2
-3
fs/bcachefs/subvolume.h
+2
-3
fs/bcachefs/subvolume.h
···
10
10
int bch2_check_subvols(struct bch_fs *);
11
11
int bch2_check_subvol_children(struct bch_fs *);
12
12
13
-
int bch2_subvolume_invalid(struct bch_fs *, struct bkey_s_c,
14
-
enum bch_validate_flags, struct printbuf *);
13
+
int bch2_subvolume_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
15
14
void bch2_subvolume_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
16
15
int bch2_subvolume_trigger(struct btree_trans *, enum btree_id, unsigned,
17
16
struct bkey_s_c, struct bkey_s,
18
17
enum btree_iter_update_trigger_flags);
19
18
20
19
#define bch2_bkey_ops_subvolume ((struct bkey_ops) { \
21
-
.key_invalid = bch2_subvolume_invalid, \
20
+
.key_validate = bch2_subvolume_validate, \
22
21
.val_to_text = bch2_subvolume_to_text, \
23
22
.trigger = bch2_subvolume_trigger, \
24
23
.min_val_size = 16, \
+1
fs/bcachefs/trace.c
+1
fs/bcachefs/trace.c
+25
-2
fs/bcachefs/trace.h
+25
-2
fs/bcachefs/trace.h
···
988
988
__entry->u64s_remaining)
989
989
);
990
990
991
-
DEFINE_EVENT(transaction_event, trans_blocked_journal_reclaim,
991
+
TRACE_EVENT(trans_blocked_journal_reclaim,
992
992
TP_PROTO(struct btree_trans *trans,
993
993
unsigned long caller_ip),
994
-
TP_ARGS(trans, caller_ip)
994
+
TP_ARGS(trans, caller_ip),
995
+
996
+
TP_STRUCT__entry(
997
+
__array(char, trans_fn, 32 )
998
+
__field(unsigned long, caller_ip )
999
+
1000
+
__field(unsigned long, key_cache_nr_keys )
1001
+
__field(unsigned long, key_cache_nr_dirty )
1002
+
__field(long, must_wait )
1003
+
),
1004
+
1005
+
TP_fast_assign(
1006
+
strscpy(__entry->trans_fn, trans->fn, sizeof(__entry->trans_fn));
1007
+
__entry->caller_ip = caller_ip;
1008
+
__entry->key_cache_nr_keys = atomic_long_read(&trans->c->btree_key_cache.nr_keys);
1009
+
__entry->key_cache_nr_dirty = atomic_long_read(&trans->c->btree_key_cache.nr_dirty);
1010
+
__entry->must_wait = __bch2_btree_key_cache_must_wait(trans->c);
1011
+
),
1012
+
1013
+
TP_printk("%s %pS key cache keys %lu dirty %lu must_wait %li",
1014
+
__entry->trans_fn, (void *) __entry->caller_ip,
1015
+
__entry->key_cache_nr_keys,
1016
+
__entry->key_cache_nr_dirty,
1017
+
__entry->must_wait)
995
1018
);
996
1019
997
1020
TRACE_EVENT(trans_restart_journal_preres_get,
+10
-11
fs/bcachefs/xattr.c
+10
-11
fs/bcachefs/xattr.c
···
70
70
.cmp_bkey = xattr_cmp_bkey,
71
71
};
72
72
73
-
int bch2_xattr_invalid(struct bch_fs *c, struct bkey_s_c k,
74
-
enum bch_validate_flags flags,
75
-
struct printbuf *err)
73
+
int bch2_xattr_validate(struct bch_fs *c, struct bkey_s_c k,
74
+
enum bch_validate_flags flags)
76
75
{
77
76
struct bkey_s_c_xattr xattr = bkey_s_c_to_xattr(k);
78
77
unsigned val_u64s = xattr_val_u64s(xattr.v->x_name_len,
79
78
le16_to_cpu(xattr.v->x_val_len));
80
79
int ret = 0;
81
80
82
-
bkey_fsck_err_on(bkey_val_u64s(k.k) < val_u64s, c, err,
83
-
xattr_val_size_too_small,
81
+
bkey_fsck_err_on(bkey_val_u64s(k.k) < val_u64s,
82
+
c, xattr_val_size_too_small,
84
83
"value too small (%zu < %u)",
85
84
bkey_val_u64s(k.k), val_u64s);
86
85
···
87
88
val_u64s = xattr_val_u64s(xattr.v->x_name_len,
88
89
le16_to_cpu(xattr.v->x_val_len) + 4);
89
90
90
-
bkey_fsck_err_on(bkey_val_u64s(k.k) > val_u64s, c, err,
91
-
xattr_val_size_too_big,
91
+
bkey_fsck_err_on(bkey_val_u64s(k.k) > val_u64s,
92
+
c, xattr_val_size_too_big,
92
93
"value too big (%zu > %u)",
93
94
bkey_val_u64s(k.k), val_u64s);
94
95
95
-
bkey_fsck_err_on(!bch2_xattr_type_to_handler(xattr.v->x_type), c, err,
96
-
xattr_invalid_type,
96
+
bkey_fsck_err_on(!bch2_xattr_type_to_handler(xattr.v->x_type),
97
+
c, xattr_invalid_type,
97
98
"invalid type (%u)", xattr.v->x_type);
98
99
99
-
bkey_fsck_err_on(memchr(xattr.v->x_name, '\0', xattr.v->x_name_len), c, err,
100
-
xattr_name_invalid_chars,
100
+
bkey_fsck_err_on(memchr(xattr.v->x_name, '\0', xattr.v->x_name_len),
101
+
c, xattr_name_invalid_chars,
101
102
"xattr name has invalid characters");
102
103
fsck_err:
103
104
return ret;
+2
-3
fs/bcachefs/xattr.h
+2
-3
fs/bcachefs/xattr.h
···
6
6
7
7
extern const struct bch_hash_desc bch2_xattr_hash_desc;
8
8
9
-
int bch2_xattr_invalid(struct bch_fs *, struct bkey_s_c,
10
-
enum bch_validate_flags, struct printbuf *);
9
+
int bch2_xattr_validate(struct bch_fs *, struct bkey_s_c, enum bch_validate_flags);
11
10
void bch2_xattr_to_text(struct printbuf *, struct bch_fs *, struct bkey_s_c);
12
11
13
12
#define bch2_bkey_ops_xattr ((struct bkey_ops) { \
14
-
.key_invalid = bch2_xattr_invalid, \
13
+
.key_validate = bch2_xattr_validate, \
15
14
.val_to_text = bch2_xattr_to_text, \
16
15
.min_val_size = 8, \
17
16
})
+8
-6
fs/btrfs/free-space-cache.c
+8
-6
fs/btrfs/free-space-cache.c
···
2697
2697
u64 offset = bytenr - block_group->start;
2698
2698
u64 to_free, to_unusable;
2699
2699
int bg_reclaim_threshold = 0;
2700
-
bool initial = ((size == block_group->length) && (block_group->alloc_offset == 0));
2700
+
bool initial;
2701
2701
u64 reclaimable_unusable;
2702
2702
2703
-
WARN_ON(!initial && offset + size > block_group->zone_capacity);
2703
+
spin_lock(&block_group->lock);
2704
2704
2705
+
initial = ((size == block_group->length) && (block_group->alloc_offset == 0));
2706
+
WARN_ON(!initial && offset + size > block_group->zone_capacity);
2705
2707
if (!initial)
2706
2708
bg_reclaim_threshold = READ_ONCE(sinfo->bg_reclaim_threshold);
2707
2709
2708
-
spin_lock(&ctl->tree_lock);
2709
2710
if (!used)
2710
2711
to_free = size;
2711
2712
else if (initial)
···
2719
2718
to_free = offset + size - block_group->alloc_offset;
2720
2719
to_unusable = size - to_free;
2721
2720
2721
+
spin_lock(&ctl->tree_lock);
2722
2722
ctl->free_space += to_free;
2723
+
spin_unlock(&ctl->tree_lock);
2723
2724
/*
2724
2725
* If the block group is read-only, we should account freed space into
2725
2726
* bytes_readonly.
···
2730
2727
block_group->zone_unusable += to_unusable;
2731
2728
WARN_ON(block_group->zone_unusable > block_group->length);
2732
2729
}
2733
-
spin_unlock(&ctl->tree_lock);
2734
2730
if (!used) {
2735
-
spin_lock(&block_group->lock);
2736
2731
block_group->alloc_offset -= size;
2737
-
spin_unlock(&block_group->lock);
2738
2732
}
2739
2733
2740
2734
reclaimable_unusable = block_group->zone_unusable -
···
2744
2744
mult_perc(block_group->zone_capacity, bg_reclaim_threshold)) {
2745
2745
btrfs_mark_bg_to_reclaim(block_group);
2746
2746
}
2747
+
2748
+
spin_unlock(&block_group->lock);
2747
2749
2748
2750
return 0;
2749
2751
}
+1
fs/btrfs/inode.c
+1
fs/btrfs/inode.c
···
4195
4195
4196
4196
btrfs_i_size_write(dir, dir->vfs_inode.i_size - name->len * 2);
4197
4197
inode_inc_iversion(&inode->vfs_inode);
4198
+
inode_set_ctime_current(&inode->vfs_inode);
4198
4199
inode_inc_iversion(&dir->vfs_inode);
4199
4200
inode_set_mtime_to_ts(&dir->vfs_inode, inode_set_ctime_current(&dir->vfs_inode));
4200
4201
ret = btrfs_update_inode(trans, dir);
+1
-1
fs/btrfs/send.c
+1
-1
fs/btrfs/send.c
+7
-1
fs/btrfs/super.c
+7
-1
fs/btrfs/super.c
···
2402
2402
2403
2403
trace_btrfs_extent_map_shrinker_count(fs_info, nr);
2404
2404
2405
-
return nr;
2405
+
/*
2406
+
* Only report the real number for DEBUG builds, as there are reports of
2407
+
* serious performance degradation caused by too frequent shrinks.
2408
+
*/
2409
+
if (IS_ENABLED(CONFIG_BTRFS_DEBUG))
2410
+
return nr;
2411
+
return 0;
2406
2412
}
2407
2413
2408
2414
static long btrfs_free_cached_objects(struct super_block *sb, struct shrink_control *sc)
+69
fs/btrfs/tree-checker.c
+69
fs/btrfs/tree-checker.c
···
1764
1764
return 0;
1765
1765
}
1766
1766
1767
+
static int check_dev_extent_item(const struct extent_buffer *leaf,
1768
+
const struct btrfs_key *key,
1769
+
int slot,
1770
+
struct btrfs_key *prev_key)
1771
+
{
1772
+
struct btrfs_dev_extent *de;
1773
+
const u32 sectorsize = leaf->fs_info->sectorsize;
1774
+
1775
+
de = btrfs_item_ptr(leaf, slot, struct btrfs_dev_extent);
1776
+
/* Basic fixed member checks. */
1777
+
if (unlikely(btrfs_dev_extent_chunk_tree(leaf, de) !=
1778
+
BTRFS_CHUNK_TREE_OBJECTID)) {
1779
+
generic_err(leaf, slot,
1780
+
"invalid dev extent chunk tree id, has %llu expect %llu",
1781
+
btrfs_dev_extent_chunk_tree(leaf, de),
1782
+
BTRFS_CHUNK_TREE_OBJECTID);
1783
+
return -EUCLEAN;
1784
+
}
1785
+
if (unlikely(btrfs_dev_extent_chunk_objectid(leaf, de) !=
1786
+
BTRFS_FIRST_CHUNK_TREE_OBJECTID)) {
1787
+
generic_err(leaf, slot,
1788
+
"invalid dev extent chunk objectid, has %llu expect %llu",
1789
+
btrfs_dev_extent_chunk_objectid(leaf, de),
1790
+
BTRFS_FIRST_CHUNK_TREE_OBJECTID);
1791
+
return -EUCLEAN;
1792
+
}
1793
+
/* Alignment check. */
1794
+
if (unlikely(!IS_ALIGNED(key->offset, sectorsize))) {
1795
+
generic_err(leaf, slot,
1796
+
"invalid dev extent key.offset, has %llu not aligned to %u",
1797
+
key->offset, sectorsize);
1798
+
return -EUCLEAN;
1799
+
}
1800
+
if (unlikely(!IS_ALIGNED(btrfs_dev_extent_chunk_offset(leaf, de),
1801
+
sectorsize))) {
1802
+
generic_err(leaf, slot,
1803
+
"invalid dev extent chunk offset, has %llu not aligned to %u",
1804
+
btrfs_dev_extent_chunk_objectid(leaf, de),
1805
+
sectorsize);
1806
+
return -EUCLEAN;
1807
+
}
1808
+
if (unlikely(!IS_ALIGNED(btrfs_dev_extent_length(leaf, de),
1809
+
sectorsize))) {
1810
+
generic_err(leaf, slot,
1811
+
"invalid dev extent length, has %llu not aligned to %u",
1812
+
btrfs_dev_extent_length(leaf, de), sectorsize);
1813
+
return -EUCLEAN;
1814
+
}
1815
+
/* Overlap check with previous dev extent. */
1816
+
if (slot && prev_key->objectid == key->objectid &&
1817
+
prev_key->type == key->type) {
1818
+
struct btrfs_dev_extent *prev_de;
1819
+
u64 prev_len;
1820
+
1821
+
prev_de = btrfs_item_ptr(leaf, slot - 1, struct btrfs_dev_extent);
1822
+
prev_len = btrfs_dev_extent_length(leaf, prev_de);
1823
+
if (unlikely(prev_key->offset + prev_len > key->offset)) {
1824
+
generic_err(leaf, slot,
1825
+
"dev extent overlap, prev offset %llu len %llu current offset %llu",
1826
+
prev_key->objectid, prev_len, key->offset);
1827
+
return -EUCLEAN;
1828
+
}
1829
+
}
1830
+
return 0;
1831
+
}
1832
+
1767
1833
/*
1768
1834
* Common point to switch the item-specific validation.
1769
1835
*/
···
1865
1799
break;
1866
1800
case BTRFS_DEV_ITEM_KEY:
1867
1801
ret = check_dev_item(leaf, key, slot);
1802
+
break;
1803
+
case BTRFS_DEV_EXTENT_KEY:
1804
+
ret = check_dev_extent_item(leaf, key, slot, prev_key);
1868
1805
break;
1869
1806
case BTRFS_INODE_ITEM_KEY:
1870
1807
ret = check_inode_item(leaf, key, slot);
+12
-23
fs/erofs/dir.c
+12
-23
fs/erofs/dir.c
···
8
8
9
9
static int erofs_fill_dentries(struct inode *dir, struct dir_context *ctx,
10
10
void *dentry_blk, struct erofs_dirent *de,
11
-
unsigned int nameoff, unsigned int maxsize)
11
+
unsigned int nameoff0, unsigned int maxsize)
12
12
{
13
-
const struct erofs_dirent *end = dentry_blk + nameoff;
13
+
const struct erofs_dirent *end = dentry_blk + nameoff0;
14
14
15
15
while (de < end) {
16
-
const char *de_name;
16
+
unsigned char d_type = fs_ftype_to_dtype(de->file_type);
17
+
unsigned int nameoff = le16_to_cpu(de->nameoff);
18
+
const char *de_name = (char *)dentry_blk + nameoff;
17
19
unsigned int de_namelen;
18
-
unsigned char d_type;
19
-
20
-
d_type = fs_ftype_to_dtype(de->file_type);
21
-
22
-
nameoff = le16_to_cpu(de->nameoff);
23
-
de_name = (char *)dentry_blk + nameoff;
24
20
25
21
/* the last dirent in the block? */
26
22
if (de + 1 >= end)
···
48
52
struct erofs_buf buf = __EROFS_BUF_INITIALIZER;
49
53
struct super_block *sb = dir->i_sb;
50
54
unsigned long bsz = sb->s_blocksize;
51
-
const size_t dirsize = i_size_read(dir);
52
-
unsigned int i = erofs_blknr(sb, ctx->pos);
53
55
unsigned int ofs = erofs_blkoff(sb, ctx->pos);
54
56
int err = 0;
55
57
bool initial = true;
56
58
57
59
buf.mapping = dir->i_mapping;
58
-
while (ctx->pos < dirsize) {
60
+
while (ctx->pos < dir->i_size) {
61
+
erofs_off_t dbstart = ctx->pos - ofs;
59
62
struct erofs_dirent *de;
60
63
unsigned int nameoff, maxsize;
61
64
62
-
de = erofs_bread(&buf, erofs_pos(sb, i), EROFS_KMAP);
65
+
de = erofs_bread(&buf, dbstart, EROFS_KMAP);
63
66
if (IS_ERR(de)) {
64
67
erofs_err(sb, "fail to readdir of logical block %u of nid %llu",
65
-
i, EROFS_I(dir)->nid);
68
+
erofs_blknr(sb, dbstart), EROFS_I(dir)->nid);
66
69
err = PTR_ERR(de);
67
70
break;
68
71
}
···
74
79
break;
75
80
}
76
81
77
-
maxsize = min_t(unsigned int, dirsize - ctx->pos + ofs, bsz);
78
-
82
+
maxsize = min_t(unsigned int, dir->i_size - dbstart, bsz);
79
83
/* search dirents at the arbitrary position */
80
84
if (initial) {
81
85
initial = false;
82
-
83
86
ofs = roundup(ofs, sizeof(struct erofs_dirent));
84
-
ctx->pos = erofs_pos(sb, i) + ofs;
85
-
if (ofs >= nameoff)
86
-
goto skip_this;
87
+
ctx->pos = dbstart + ofs;
87
88
}
88
89
89
90
err = erofs_fill_dentries(dir, ctx, de, (void *)de + ofs,
90
91
nameoff, maxsize);
91
92
if (err)
92
93
break;
93
-
skip_this:
94
-
ctx->pos = erofs_pos(sb, i) + maxsize;
95
-
++i;
94
+
ctx->pos = dbstart + maxsize;
96
95
ofs = 0;
97
96
}
98
97
erofs_put_metabuf(&buf);
+9
-11
fs/erofs/inode.c
+9
-11
fs/erofs/inode.c
···
257
257
goto out_unlock;
258
258
}
259
259
260
+
mapping_set_large_folios(inode->i_mapping);
260
261
if (erofs_inode_is_data_compressed(vi->datalayout)) {
261
262
#ifdef CONFIG_EROFS_FS_ZIP
262
263
DO_ONCE_LITE_IF(inode->i_blkbits != PAGE_SHIFT,
263
264
erofs_info, inode->i_sb,
264
265
"EXPERIMENTAL EROFS subpage compressed block support in use. Use at your own risk!");
265
266
inode->i_mapping->a_ops = &z_erofs_aops;
266
-
err = 0;
267
-
goto out_unlock;
268
-
#endif
267
+
#else
269
268
err = -EOPNOTSUPP;
270
-
goto out_unlock;
271
-
}
272
-
inode->i_mapping->a_ops = &erofs_raw_access_aops;
273
-
mapping_set_large_folios(inode->i_mapping);
274
-
#ifdef CONFIG_EROFS_FS_ONDEMAND
275
-
if (erofs_is_fscache_mode(inode->i_sb))
276
-
inode->i_mapping->a_ops = &erofs_fscache_access_aops;
277
269
#endif
278
-
270
+
} else {
271
+
inode->i_mapping->a_ops = &erofs_raw_access_aops;
272
+
#ifdef CONFIG_EROFS_FS_ONDEMAND
273
+
if (erofs_is_fscache_mode(inode->i_sb))
274
+
inode->i_mapping->a_ops = &erofs_fscache_access_aops;
275
+
#endif
276
+
}
279
277
out_unlock:
280
278
erofs_put_metabuf(&buf);
281
279
return err;
+1
-1
fs/erofs/internal.h
+1
-1
fs/erofs/internal.h
···
220
220
};
221
221
#define __EROFS_BUF_INITIALIZER ((struct erofs_buf){ .page = NULL })
222
222
223
-
#define erofs_blknr(sb, addr) ((addr) >> (sb)->s_blocksize_bits)
223
+
#define erofs_blknr(sb, addr) ((erofs_blk_t)((addr) >> (sb)->s_blocksize_bits))
224
224
#define erofs_blkoff(sb, addr) ((addr) & ((sb)->s_blocksize - 1))
225
225
#define erofs_pos(sb, blk) ((erofs_off_t)(blk) << (sb)->s_blocksize_bits)
226
226
#define erofs_iblks(i) (round_up((i)->i_size, i_blocksize(i)) >> (i)->i_blkbits)
+6
-20
fs/erofs/super.c
+6
-20
fs/erofs/super.c
···
108
108
kmem_cache_free(erofs_inode_cachep, vi);
109
109
}
110
110
111
-
static bool check_layout_compatibility(struct super_block *sb,
112
-
struct erofs_super_block *dsb)
113
-
{
114
-
const unsigned int feature = le32_to_cpu(dsb->feature_incompat);
115
-
116
-
EROFS_SB(sb)->feature_incompat = feature;
117
-
118
-
/* check if current kernel meets all mandatory requirements */
119
-
if (feature & (~EROFS_ALL_FEATURE_INCOMPAT)) {
120
-
erofs_err(sb, "unidentified incompatible feature %x, please upgrade kernel",
121
-
feature & ~EROFS_ALL_FEATURE_INCOMPAT);
122
-
return false;
123
-
}
124
-
return true;
125
-
}
126
-
127
111
/* read variable-sized metadata, offset will be aligned by 4-byte */
128
112
void *erofs_read_metadata(struct super_block *sb, struct erofs_buf *buf,
129
113
erofs_off_t *offset, int *lengthp)
···
263
279
264
280
static int erofs_read_superblock(struct super_block *sb)
265
281
{
266
-
struct erofs_sb_info *sbi;
282
+
struct erofs_sb_info *sbi = EROFS_SB(sb);
267
283
struct erofs_buf buf = __EROFS_BUF_INITIALIZER;
268
284
struct erofs_super_block *dsb;
269
285
void *data;
···
275
291
return PTR_ERR(data);
276
292
}
277
293
278
-
sbi = EROFS_SB(sb);
279
294
dsb = (struct erofs_super_block *)(data + EROFS_SUPER_OFFSET);
280
-
281
295
ret = -EINVAL;
282
296
if (le32_to_cpu(dsb->magic) != EROFS_SUPER_MAGIC_V1) {
283
297
erofs_err(sb, "cannot find valid erofs superblock");
···
300
318
}
301
319
302
320
ret = -EINVAL;
303
-
if (!check_layout_compatibility(sb, dsb))
321
+
sbi->feature_incompat = le32_to_cpu(dsb->feature_incompat);
322
+
if (sbi->feature_incompat & ~EROFS_ALL_FEATURE_INCOMPAT) {
323
+
erofs_err(sb, "unidentified incompatible feature %x, please upgrade kernel",
324
+
sbi->feature_incompat & ~EROFS_ALL_FEATURE_INCOMPAT);
304
325
goto out;
326
+
}
305
327
306
328
sbi->sb_size = 128 + dsb->sb_extslots * EROFS_SB_EXTSLOT_SIZE;
307
329
if (sbi->sb_size > PAGE_SIZE - EROFS_SUPER_OFFSET) {
+2
-1
fs/erofs/zutil.c
+2
-1
fs/erofs/zutil.c
···
111
111
out:
112
112
if (i < z_erofs_gbuf_count && tmp_pages) {
113
113
for (j = 0; j < nrpages; ++j)
114
-
if (tmp_pages[j] && tmp_pages[j] != gbuf->pages[j])
114
+
if (tmp_pages[j] && (j >= gbuf->nrpages ||
115
+
tmp_pages[j] != gbuf->pages[j]))
115
116
__free_page(tmp_pages[j]);
116
117
kfree(tmp_pages);
117
118
}
+4
-2
fs/fuse/dev.c
+4
-2
fs/fuse/dev.c
···
1618
1618
1619
1619
this_num = min_t(unsigned, num, PAGE_SIZE - offset);
1620
1620
err = fuse_copy_page(cs, &page, offset, this_num, 0);
1621
-
if (!err && offset == 0 &&
1622
-
(this_num == PAGE_SIZE || file_size == end))
1621
+
if (!PageUptodate(page) && !err && offset == 0 &&
1622
+
(this_num == PAGE_SIZE || file_size == end)) {
1623
+
zero_user_segment(page, this_num, PAGE_SIZE);
1623
1624
SetPageUptodate(page);
1625
+
}
1624
1626
unlock_page(page);
1625
1627
put_page(page);
1626
1628
+15
-6
fs/smb/client/file.c
+15
-6
fs/smb/client/file.c
···
316
316
#endif
317
317
}
318
318
319
-
if (rdata->credits.value != 0)
319
+
if (rdata->credits.value != 0) {
320
320
trace_smb3_rw_credits(rdata->rreq->debug_id,
321
321
rdata->subreq.debug_index,
322
322
rdata->credits.value,
···
324
324
rdata->server ? rdata->server->in_flight : 0,
325
325
-rdata->credits.value,
326
326
cifs_trace_rw_credits_free_subreq);
327
+
if (rdata->server)
328
+
add_credits_and_wake_if(rdata->server, &rdata->credits, 0);
329
+
else
330
+
rdata->credits.value = 0;
331
+
}
327
332
328
-
add_credits_and_wake_if(rdata->server, &rdata->credits, 0);
329
333
if (rdata->have_xid)
330
334
free_xid(rdata->xid);
331
335
}
···
2754
2750
struct inode *inode = file->f_mapping->host;
2755
2751
struct cifsInodeInfo *cinode = CIFS_I(inode);
2756
2752
struct TCP_Server_Info *server = tlink_tcon(cfile->tlink)->ses->server;
2753
+
struct cifs_sb_info *cifs_sb = CIFS_SB(inode->i_sb);
2757
2754
ssize_t rc;
2758
2755
2759
2756
rc = netfs_start_io_write(inode);
···
2771
2766
if (rc <= 0)
2772
2767
goto out;
2773
2768
2774
-
if (!cifs_find_lock_conflict(cfile, iocb->ki_pos, iov_iter_count(from),
2769
+
if ((cifs_sb->mnt_cifs_flags & CIFS_MOUNT_NOPOSIXBRL) &&
2770
+
(cifs_find_lock_conflict(cfile, iocb->ki_pos, iov_iter_count(from),
2775
2771
server->vals->exclusive_lock_type, 0,
2776
-
NULL, CIFS_WRITE_OP))
2777
-
rc = netfs_buffered_write_iter_locked(iocb, from, NULL);
2778
-
else
2772
+
NULL, CIFS_WRITE_OP))) {
2779
2773
rc = -EACCES;
2774
+
goto out;
2775
+
}
2776
+
2777
+
rc = netfs_buffered_write_iter_locked(iocb, from, NULL);
2778
+
2780
2779
out:
2781
2780
up_read(&cinode->lock_sem);
2782
2781
netfs_end_io_write(inode);
+2
fs/smb/common/smb2pdu.h
+2
fs/smb/common/smb2pdu.h
+33
-1
fs/smb/server/connection.c
+33
-1
fs/smb/server/connection.c
···
165
165
up_read(&conn_list_lock);
166
166
}
167
167
168
-
void ksmbd_conn_wait_idle(struct ksmbd_conn *conn, u64 sess_id)
168
+
void ksmbd_conn_wait_idle(struct ksmbd_conn *conn)
169
169
{
170
170
wait_event(conn->req_running_q, atomic_read(&conn->req_running) < 2);
171
+
}
172
+
173
+
int ksmbd_conn_wait_idle_sess_id(struct ksmbd_conn *curr_conn, u64 sess_id)
174
+
{
175
+
struct ksmbd_conn *conn;
176
+
int rc, retry_count = 0, max_timeout = 120;
177
+
int rcount = 1;
178
+
179
+
retry_idle:
180
+
if (retry_count >= max_timeout)
181
+
return -EIO;
182
+
183
+
down_read(&conn_list_lock);
184
+
list_for_each_entry(conn, &conn_list, conns_list) {
185
+
if (conn->binding || xa_load(&conn->sessions, sess_id)) {
186
+
if (conn == curr_conn)
187
+
rcount = 2;
188
+
if (atomic_read(&conn->req_running) >= rcount) {
189
+
rc = wait_event_timeout(conn->req_running_q,
190
+
atomic_read(&conn->req_running) < rcount,
191
+
HZ);
192
+
if (!rc) {
193
+
up_read(&conn_list_lock);
194
+
retry_count++;
195
+
goto retry_idle;
196
+
}
197
+
}
198
+
}
199
+
}
200
+
up_read(&conn_list_lock);
201
+
202
+
return 0;
171
203
}
172
204
173
205
int ksmbd_conn_write(struct ksmbd_work *work)
+2
-1
fs/smb/server/connection.h
+2
-1
fs/smb/server/connection.h
···
145
145
extern struct rw_semaphore conn_list_lock;
146
146
147
147
bool ksmbd_conn_alive(struct ksmbd_conn *conn);
148
-
void ksmbd_conn_wait_idle(struct ksmbd_conn *conn, u64 sess_id);
148
+
void ksmbd_conn_wait_idle(struct ksmbd_conn *conn);
149
+
int ksmbd_conn_wait_idle_sess_id(struct ksmbd_conn *curr_conn, u64 sess_id);
149
150
struct ksmbd_conn *ksmbd_conn_alloc(void);
150
151
void ksmbd_conn_free(struct ksmbd_conn *conn);
151
152
bool ksmbd_conn_lookup_dialect(struct ksmbd_conn *c);
+9
fs/smb/server/mgmt/user_session.c
+9
fs/smb/server/mgmt/user_session.c
···
311
311
{
312
312
struct ksmbd_session *prev_sess;
313
313
struct ksmbd_user *prev_user;
314
+
int err;
314
315
315
316
down_write(&sessions_table_lock);
316
317
down_write(&conn->session_lock);
···
326
325
memcmp(user->passkey, prev_user->passkey, user->passkey_sz))
327
326
goto out;
328
327
328
+
ksmbd_all_conn_set_status(id, KSMBD_SESS_NEED_RECONNECT);
329
+
err = ksmbd_conn_wait_idle_sess_id(conn, id);
330
+
if (err) {
331
+
ksmbd_all_conn_set_status(id, KSMBD_SESS_NEED_NEGOTIATE);
332
+
goto out;
333
+
}
334
+
329
335
ksmbd_destroy_file_table(&prev_sess->file_table);
330
336
prev_sess->state = SMB2_SESSION_EXPIRED;
337
+
ksmbd_all_conn_set_status(id, KSMBD_SESS_NEED_NEGOTIATE);
331
338
ksmbd_launch_ksmbd_durable_scavenger();
332
339
out:
333
340
up_write(&conn->session_lock);
+8
-5
fs/smb/server/smb2pdu.c
+8
-5
fs/smb/server/smb2pdu.c
···
1370
1370
}
1371
1371
1372
1372
sz = le16_to_cpu(rsp->SecurityBufferOffset);
1373
-
memcpy((char *)&rsp->hdr.ProtocolId + sz, spnego_blob, spnego_blob_len);
1373
+
unsafe_memcpy((char *)&rsp->hdr.ProtocolId + sz, spnego_blob, spnego_blob_len,
1374
+
/* alloc is larger than blob, see smb2_allocate_rsp_buf() */);
1374
1375
rsp->SecurityBufferLength = cpu_to_le16(spnego_blob_len);
1375
1376
1376
1377
out:
···
1454
1453
return -ENOMEM;
1455
1454
1456
1455
sz = le16_to_cpu(rsp->SecurityBufferOffset);
1457
-
memcpy((char *)&rsp->hdr.ProtocolId + sz, spnego_blob, spnego_blob_len);
1456
+
unsafe_memcpy((char *)&rsp->hdr.ProtocolId + sz, spnego_blob,
1457
+
spnego_blob_len,
1458
+
/* alloc is larger than blob, see smb2_allocate_rsp_buf() */);
1458
1459
rsp->SecurityBufferLength = cpu_to_le16(spnego_blob_len);
1459
1460
kfree(spnego_blob);
1460
1461
}
···
2213
2210
ksmbd_conn_unlock(conn);
2214
2211
2215
2212
ksmbd_close_session_fds(work);
2216
-
ksmbd_conn_wait_idle(conn, sess_id);
2213
+
ksmbd_conn_wait_idle(conn);
2217
2214
2218
2215
/*
2219
2216
* Re-lookup session to validate if session is deleted
···
5360
5357
"NTFS", PATH_MAX, conn->local_nls, 0);
5361
5358
len = len * 2;
5362
5359
info->FileSystemNameLen = cpu_to_le32(len);
5363
-
sz = sizeof(struct filesystem_attribute_info) - 2 + len;
5360
+
sz = sizeof(struct filesystem_attribute_info) + len;
5364
5361
rsp->OutputBufferLength = cpu_to_le32(sz);
5365
5362
break;
5366
5363
}
···
5386
5383
len = len * 2;
5387
5384
info->VolumeLabelSize = cpu_to_le32(len);
5388
5385
info->Reserved = 0;
5389
-
sz = sizeof(struct filesystem_vol_info) - 2 + len;
5386
+
sz = sizeof(struct filesystem_vol_info) + len;
5390
5387
rsp->OutputBufferLength = cpu_to_le32(sz);
5391
5388
break;
5392
5389
}
+2
-2
fs/smb/server/smb_common.h
+2
-2
fs/smb/server/smb_common.h
···
213
213
__le32 Attributes;
214
214
__le32 MaxPathNameComponentLength;
215
215
__le32 FileSystemNameLen;
216
-
__le16 FileSystemName[1]; /* do not have to save this - get subset? */
216
+
__le16 FileSystemName[]; /* do not have to save this - get subset? */
217
217
} __packed;
218
218
219
219
struct filesystem_device_info {
···
226
226
__le32 SerialNumber;
227
227
__le32 VolumeLabelSize;
228
228
__le16 Reserved;
229
-
__le16 VolumeLabel[1];
229
+
__le16 VolumeLabel[];
230
230
} __packed;
231
231
232
232
struct filesystem_info {
+7
-1
fs/xfs/scrub/bmap.c
+7
-1
fs/xfs/scrub/bmap.c
···
938
938
}
939
939
break;
940
940
case XFS_ATTR_FORK:
941
-
if (!xfs_has_attr(mp) && !xfs_has_attr2(mp))
941
+
/*
942
+
* "attr" means that an attr fork was created at some point in
943
+
* the life of this filesystem. "attr2" means that inodes have
944
+
* variable-sized data/attr fork areas. Hence we only check
945
+
* attr here.
946
+
*/
947
+
if (!xfs_has_attr(mp))
942
948
xchk_ino_set_corrupt(sc, sc->ip->i_ino);
943
949
break;
944
950
default:
+11
fs/xfs/xfs_ioctl.c
+11
fs/xfs/xfs_ioctl.c
···
483
483
/* Can't change realtime flag if any extents are allocated. */
484
484
if (ip->i_df.if_nextents || ip->i_delayed_blks)
485
485
return -EINVAL;
486
+
487
+
/*
488
+
* If S_DAX is enabled on this file, we can only switch the
489
+
* device if both support fsdax. We can't update S_DAX because
490
+
* there might be other threads walking down the access paths.
491
+
*/
492
+
if (IS_DAX(VFS_I(ip)) &&
493
+
(mp->m_ddev_targp->bt_daxdev == NULL ||
494
+
(mp->m_rtdev_targp &&
495
+
mp->m_rtdev_targp->bt_daxdev == NULL)))
496
+
return -EINVAL;
486
497
}
487
498
488
499
if (rtflag) {
+6
-1
fs/xfs/xfs_trans_ail.c
+6
-1
fs/xfs/xfs_trans_ail.c
···
644
644
set_freezable();
645
645
646
646
while (1) {
647
-
if (tout)
647
+
/*
648
+
* Long waits of 50ms or more occur when we've run out of items
649
+
* to push, so we only want uninterruptible state if we're
650
+
* actually blocked on something.
651
+
*/
652
+
if (tout && tout <= 20)
648
653
set_current_state(TASK_KILLABLE|TASK_FREEZABLE);
649
654
else
650
655
set_current_state(TASK_INTERRUPTIBLE|TASK_FREEZABLE);
+1
-4
include/acpi/acpixf.h
+1
-4
include/acpi/acpixf.h
···
660
660
void *context))
661
661
ACPI_EXTERNAL_RETURN_STATUS(acpi_status
662
662
acpi_execute_reg_methods(acpi_handle device,
663
+
u32 nax_depth,
663
664
acpi_adr_space_type
664
665
space_id))
665
-
ACPI_EXTERNAL_RETURN_STATUS(acpi_status
666
-
acpi_execute_orphan_reg_method(acpi_handle device,
667
-
acpi_adr_space_type
668
-
space_id))
669
666
ACPI_EXTERNAL_RETURN_STATUS(acpi_status
670
667
acpi_remove_address_space_handler(acpi_handle
671
668
device,
+47
include/linux/dsa/ocelot.h
+47
include/linux/dsa/ocelot.h
···
5
5
#ifndef _NET_DSA_TAG_OCELOT_H
6
6
#define _NET_DSA_TAG_OCELOT_H
7
7
8
+
#include <linux/if_bridge.h>
9
+
#include <linux/if_vlan.h>
8
10
#include <linux/kthread.h>
9
11
#include <linux/packing.h>
10
12
#include <linux/skbuff.h>
···
273
271
}
274
272
275
273
return rew_op;
274
+
}
275
+
276
+
/**
277
+
* ocelot_xmit_get_vlan_info: Determine VLAN_TCI and TAG_TYPE for injected frame
278
+
* @skb: Pointer to socket buffer
279
+
* @br: Pointer to bridge device that the port is under, if any
280
+
* @vlan_tci:
281
+
* @tag_type:
282
+
*
283
+
* If the port is under a VLAN-aware bridge, remove the VLAN header from the
284
+
* payload and move it into the DSA tag, which will make the switch classify
285
+
* the packet to the bridge VLAN. Otherwise, leave the classified VLAN at zero,
286
+
* which is the pvid of standalone ports (OCELOT_STANDALONE_PVID), although not
287
+
* of VLAN-unaware bridge ports (that would be ocelot_vlan_unaware_pvid()).
288
+
* Anyway, VID 0 is fine because it is stripped on egress for these port modes,
289
+
* and source address learning is not performed for packets injected from the
290
+
* CPU anyway, so it doesn't matter that the VID is "wrong".
291
+
*/
292
+
static inline void ocelot_xmit_get_vlan_info(struct sk_buff *skb,
293
+
struct net_device *br,
294
+
u64 *vlan_tci, u64 *tag_type)
295
+
{
296
+
struct vlan_ethhdr *hdr;
297
+
u16 proto, tci;
298
+
299
+
if (!br || !br_vlan_enabled(br)) {
300
+
*vlan_tci = 0;
301
+
*tag_type = IFH_TAG_TYPE_C;
302
+
return;
303
+
}
304
+
305
+
hdr = (struct vlan_ethhdr *)skb_mac_header(skb);
306
+
br_vlan_get_proto(br, &proto);
307
+
308
+
if (ntohs(hdr->h_vlan_proto) == proto) {
309
+
vlan_remove_tag(skb, &tci);
310
+
*vlan_tci = tci;
311
+
} else {
312
+
rcu_read_lock();
313
+
br_vlan_get_pvid_rcu(br, &tci);
314
+
rcu_read_unlock();
315
+
*vlan_tci = tci;
316
+
}
317
+
318
+
*tag_type = (proto != ETH_P_8021Q) ? IFH_TAG_TYPE_S : IFH_TAG_TYPE_C;
276
319
}
277
320
278
321
#endif
+30
-3
include/linux/hugetlb.h
+30
-3
include/linux/hugetlb.h
···
944
944
static inline spinlock_t *huge_pte_lockptr(struct hstate *h,
945
945
struct mm_struct *mm, pte_t *pte)
946
946
{
947
-
if (huge_page_size(h) == PMD_SIZE)
947
+
const unsigned long size = huge_page_size(h);
948
+
949
+
VM_WARN_ON(size == PAGE_SIZE);
950
+
951
+
/*
952
+
* hugetlb must use the exact same PT locks as core-mm page table
953
+
* walkers would. When modifying a PTE table, hugetlb must take the
954
+
* PTE PT lock, when modifying a PMD table, hugetlb must take the PMD
955
+
* PT lock etc.
956
+
*
957
+
* The expectation is that any hugetlb folio smaller than a PMD is
958
+
* always mapped into a single PTE table and that any hugetlb folio
959
+
* smaller than a PUD (but at least as big as a PMD) is always mapped
960
+
* into a single PMD table.
961
+
*
962
+
* If that does not hold for an architecture, then that architecture
963
+
* must disable split PT locks such that all *_lockptr() functions
964
+
* will give us the same result: the per-MM PT lock.
965
+
*
966
+
* Note that with e.g., CONFIG_PGTABLE_LEVELS=2 where
967
+
* PGDIR_SIZE==P4D_SIZE==PUD_SIZE==PMD_SIZE, we'd use pud_lockptr()
968
+
* and core-mm would use pmd_lockptr(). However, in such configurations
969
+
* split PMD locks are disabled -- they don't make sense on a single
970
+
* PGDIR page table -- and the end result is the same.
971
+
*/
972
+
if (size >= PUD_SIZE)
973
+
return pud_lockptr(mm, (pud_t *) pte);
974
+
else if (size >= PMD_SIZE || IS_ENABLED(CONFIG_HIGHPTE))
948
975
return pmd_lockptr(mm, (pmd_t *) pte);
949
-
VM_BUG_ON(huge_page_size(h) == PAGE_SIZE);
950
-
return &mm->page_table_lock;
976
+
/* pte_alloc_huge() only applies with !CONFIG_HIGHPTE */
977
+
return ptep_lockptr(mm, pte);
951
978
}
952
979
953
980
#ifndef hugepages_supported
+1
-1
include/linux/i2c.h
+1
-1
include/linux/i2c.h
···
1066
1066
struct acpi_resource;
1067
1067
struct acpi_resource_i2c_serialbus;
1068
1068
1069
-
#if IS_ENABLED(CONFIG_ACPI) && IS_ENABLED(CONFIG_I2C)
1069
+
#if IS_REACHABLE(CONFIG_ACPI) && IS_REACHABLE(CONFIG_I2C)
1070
1070
bool i2c_acpi_get_i2c_resource(struct acpi_resource *ares,
1071
1071
struct acpi_resource_i2c_serialbus **i2c);
1072
1072
int i2c_acpi_client_count(struct acpi_device *adev);
-2
include/linux/iommu.h
-2
include/linux/iommu.h
···
795
795
struct device *dev);
796
796
extern void iommu_detach_device(struct iommu_domain *domain,
797
797
struct device *dev);
798
-
extern int iommu_sva_unbind_gpasid(struct iommu_domain *domain,
799
-
struct device *dev, ioasid_t pasid);
800
798
extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev);
801
799
extern struct iommu_domain *iommu_get_dma_domain(struct device *dev);
802
800
extern int iommu_map(struct iommu_domain *domain, unsigned long iova,
+11
include/linux/mm.h
+11
include/linux/mm.h
···
2920
2920
return ptlock_ptr(page_ptdesc(pmd_page(*pmd)));
2921
2921
}
2922
2922
2923
+
static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte)
2924
+
{
2925
+
BUILD_BUG_ON(IS_ENABLED(CONFIG_HIGHPTE));
2926
+
BUILD_BUG_ON(MAX_PTRS_PER_PTE * sizeof(pte_t) > PAGE_SIZE);
2927
+
return ptlock_ptr(virt_to_ptdesc(pte));
2928
+
}
2929
+
2923
2930
static inline bool ptlock_init(struct ptdesc *ptdesc)
2924
2931
{
2925
2932
/*
···
2948
2941
* We use mm->page_table_lock to guard all pagetable pages of the mm.
2949
2942
*/
2950
2943
static inline spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd)
2944
+
{
2945
+
return &mm->page_table_lock;
2946
+
}
2947
+
static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte)
2951
2948
{
2952
2949
return &mm->page_table_lock;
2953
2950
}
-2
include/linux/mmzone.h
-2
include/linux/mmzone.h
+1
include/linux/panic.h
+1
include/linux/panic.h
+13
include/linux/pgalloc_tag.h
+13
include/linux/pgalloc_tag.h
···
43
43
page_ext_put(page_ext_from_codetag_ref(ref));
44
44
}
45
45
46
+
static inline void clear_page_tag_ref(struct page *page)
47
+
{
48
+
if (mem_alloc_profiling_enabled()) {
49
+
union codetag_ref *ref = get_page_tag_ref(page);
50
+
51
+
if (ref) {
52
+
set_codetag_empty(ref);
53
+
put_page_tag_ref(ref);
54
+
}
55
+
}
56
+
}
57
+
46
58
static inline void pgalloc_tag_add(struct page *page, struct task_struct *task,
47
59
unsigned int nr)
48
60
{
···
138
126
139
127
static inline union codetag_ref *get_page_tag_ref(struct page *page) { return NULL; }
140
128
static inline void put_page_tag_ref(union codetag_ref *ref) {}
129
+
static inline void clear_page_tag_ref(struct page *page) {}
141
130
static inline void pgalloc_tag_add(struct page *page, struct task_struct *task,
142
131
unsigned int nr) {}
143
132
static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) {}
+2
-2
include/linux/refcount.h
+2
-2
include/linux/refcount.h
···
266
266
if (oldp)
267
267
*oldp = old;
268
268
269
-
if (old == i) {
269
+
if (old > 0 && old == i) {
270
270
smp_acquire__after_ctrl_dep();
271
271
return true;
272
272
}
273
273
274
-
if (unlikely(old < 0 || old - i < 0))
274
+
if (unlikely(old <= 0 || old - i < 0))
275
275
refcount_warn_saturate(r, REFCOUNT_SUB_UAF);
276
276
277
277
return false;
+18
-1
include/linux/spi/spi.h
+18
-1
include/linux/spi/spi.h
···
902
902
struct spi_controller *ctlr);
903
903
extern void spi_unregister_controller(struct spi_controller *ctlr);
904
904
905
-
#if IS_ENABLED(CONFIG_ACPI)
905
+
#if IS_ENABLED(CONFIG_ACPI) && IS_ENABLED(CONFIG_SPI_MASTER)
906
906
extern struct spi_controller *acpi_spi_find_controller_by_adev(struct acpi_device *adev);
907
907
extern struct spi_device *acpi_spi_device_alloc(struct spi_controller *ctlr,
908
908
struct acpi_device *adev,
909
909
int index);
910
910
int acpi_spi_count_resources(struct acpi_device *adev);
911
+
#else
912
+
static inline struct spi_controller *acpi_spi_find_controller_by_adev(struct acpi_device *adev)
913
+
{
914
+
return NULL;
915
+
}
916
+
917
+
static inline struct spi_device *acpi_spi_device_alloc(struct spi_controller *ctlr,
918
+
struct acpi_device *adev,
919
+
int index)
920
+
{
921
+
return ERR_PTR(-ENODEV);
922
+
}
923
+
924
+
static inline int acpi_spi_count_resources(struct acpi_device *adev)
925
+
{
926
+
return 0;
927
+
}
911
928
#endif
912
929
913
930
/*
+1
include/linux/thermal.h
+1
include/linux/thermal.h
···
55
55
THERMAL_TZ_BIND_CDEV, /* Cooling dev is bind to the thermal zone */
56
56
THERMAL_TZ_UNBIND_CDEV, /* Cooling dev is unbind from the thermal zone */
57
57
THERMAL_INSTANCE_WEIGHT_CHANGED, /* Thermal instance weight changed */
58
+
THERMAL_TZ_RESUME, /* Thermal zone is resuming after system sleep */
58
59
};
59
60
60
61
/**
+8
-14
include/linux/vmstat.h
+8
-14
include/linux/vmstat.h
···
34
34
unsigned nr_lazyfree_fail;
35
35
};
36
36
37
-
enum writeback_stat_item {
37
+
/* Stat data for system wide items */
38
+
enum vm_stat_item {
38
39
NR_DIRTY_THRESHOLD,
39
40
NR_DIRTY_BG_THRESHOLD,
40
-
NR_VM_WRITEBACK_STAT_ITEMS,
41
+
NR_MEMMAP_PAGES, /* page metadata allocated through buddy allocator */
42
+
NR_MEMMAP_BOOT_PAGES, /* page metadata allocated through boot allocator */
43
+
NR_VM_STAT_ITEMS,
41
44
};
42
45
43
46
#ifdef CONFIG_VM_EVENT_COUNTERS
···
517
514
return node_stat_name(NR_LRU_BASE + lru) + 3; // skip "nr_"
518
515
}
519
516
520
-
static inline const char *writeback_stat_name(enum writeback_stat_item item)
521
-
{
522
-
return vmstat_text[NR_VM_ZONE_STAT_ITEMS +
523
-
NR_VM_NUMA_EVENT_ITEMS +
524
-
NR_VM_NODE_STAT_ITEMS +
525
-
item];
526
-
}
527
-
528
517
#if defined(CONFIG_VM_EVENT_COUNTERS) || defined(CONFIG_MEMCG)
529
518
static inline const char *vm_event_name(enum vm_event_item item)
530
519
{
531
520
return vmstat_text[NR_VM_ZONE_STAT_ITEMS +
532
521
NR_VM_NUMA_EVENT_ITEMS +
533
522
NR_VM_NODE_STAT_ITEMS +
534
-
NR_VM_WRITEBACK_STAT_ITEMS +
523
+
NR_VM_STAT_ITEMS +
535
524
item];
536
525
}
537
526
#endif /* CONFIG_VM_EVENT_COUNTERS || CONFIG_MEMCG */
···
620
625
lruvec_stat_mod_folio(folio, idx, -folio_nr_pages(folio));
621
626
}
622
627
623
-
void __meminit mod_node_early_perpage_metadata(int nid, long delta);
624
-
void __meminit store_early_perpage_metadata(void);
625
-
628
+
void memmap_boot_pages_add(long delta);
629
+
void memmap_pages_add(long delta);
626
630
#endif /* _LINUX_VMSTAT_H */
+10
-7
include/net/bluetooth/hci.h
+10
-7
include/net/bluetooth/hci.h
···
206
206
*/
207
207
HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED,
208
208
209
-
/* When this quirk is set, the controller has validated that
210
-
* LE states reported through the HCI_LE_READ_SUPPORTED_STATES are
211
-
* valid. This mechanism is necessary as many controllers have
212
-
* been seen has having trouble initiating a connectable
213
-
* advertisement despite the state combination being reported as
214
-
* supported.
209
+
/* When this quirk is set, the LE states reported through the
210
+
* HCI_LE_READ_SUPPORTED_STATES are invalid/broken.
211
+
*
212
+
* This mechanism is necessary as many controllers have been seen has
213
+
* having trouble initiating a connectable advertisement despite the
214
+
* state combination being reported as supported.
215
+
*
216
+
* This quirk can be set before hci_register_dev is called or
217
+
* during the hdev->setup vendor callback.
215
218
*/
216
-
HCI_QUIRK_VALID_LE_STATES,
219
+
HCI_QUIRK_BROKEN_LE_STATES,
217
220
218
221
/* When this quirk is set, then erroneous data reporting
219
222
* is ignored. This is mainly due to the fact that the HCI
+1
-1
include/net/bluetooth/hci_core.h
+1
-1
include/net/bluetooth/hci_core.h
···
825
825
} while (0)
826
826
827
827
#define hci_dev_le_state_simultaneous(hdev) \
828
-
(test_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks) && \
828
+
(!test_bit(HCI_QUIRK_BROKEN_LE_STATES, &hdev->quirks) && \
829
829
(hdev->le_states[4] & 0x08) && /* Central */ \
830
830
(hdev->le_states[4] & 0x40) && /* Peripheral */ \
831
831
(hdev->le_states[3] & 0x10)) /* Simultaneous */
+10
-6
include/net/dsa.h
+10
-6
include/net/dsa.h
···
403
403
*/
404
404
u32 configure_vlan_while_not_filtering:1;
405
405
406
-
/* If the switch driver always programs the CPU port as egress tagged
407
-
* despite the VLAN configuration indicating otherwise, then setting
408
-
* @untag_bridge_pvid will force the DSA receive path to pop the
409
-
* bridge's default_pvid VLAN tagged frames to offer a consistent
410
-
* behavior between a vlan_filtering=0 and vlan_filtering=1 bridge
411
-
* device.
406
+
/* Pop the default_pvid of VLAN-unaware bridge ports from tagged frames.
407
+
* DEPRECATED: Do NOT set this field in new drivers. Instead look at
408
+
* the dsa_software_vlan_untag() comments.
412
409
*/
413
410
u32 untag_bridge_pvid:1;
411
+
/* Pop the default_pvid of VLAN-aware bridge ports from tagged frames.
412
+
* Useful if the switch cannot preserve the VLAN tag as seen on the
413
+
* wire for user port ingress, and chooses to send all frames as
414
+
* VLAN-tagged to the CPU, including those which were originally
415
+
* untagged.
416
+
*/
417
+
u32 untag_vlan_aware_bridge_pvid:1;
414
418
415
419
/* Let DSA manage the FDB entries towards the
416
420
* CPU, based on the software bridge database.
+1
include/net/kcm.h
+1
include/net/kcm.h
+11
-1
include/soc/mscc/ocelot.h
+11
-1
include/soc/mscc/ocelot.h
···
813
813
const u32 *const *map;
814
814
struct list_head stats_regions;
815
815
816
+
spinlock_t inj_lock;
817
+
spinlock_t xtr_lock;
818
+
816
819
u32 pool_size[OCELOT_SB_NUM][OCELOT_SB_POOL_NUM];
817
820
int packet_buffer_size;
818
821
int num_frame_refs;
···
969
966
u32 val, u32 reg, u32 offset);
970
967
971
968
/* Packet I/O */
969
+
void ocelot_lock_inj_grp(struct ocelot *ocelot, int grp);
970
+
void ocelot_unlock_inj_grp(struct ocelot *ocelot, int grp);
971
+
void ocelot_lock_xtr_grp(struct ocelot *ocelot, int grp);
972
+
void ocelot_unlock_xtr_grp(struct ocelot *ocelot, int grp);
973
+
void ocelot_lock_xtr_grp_bh(struct ocelot *ocelot, int grp);
974
+
void ocelot_unlock_xtr_grp_bh(struct ocelot *ocelot, int grp);
972
975
bool ocelot_can_inject(struct ocelot *ocelot, int grp);
973
976
void ocelot_port_inject_frame(struct ocelot *ocelot, int port, int grp,
974
977
u32 rew_op, struct sk_buff *skb);
975
-
void ocelot_ifh_port_set(void *ifh, int port, u32 rew_op, u32 vlan_tag);
978
+
void ocelot_ifh_set_basic(void *ifh, struct ocelot *ocelot, int port,
979
+
u32 rew_op, struct sk_buff *skb);
976
980
int ocelot_xtr_poll_frame(struct ocelot *ocelot, int grp, struct sk_buff **skb);
977
981
void ocelot_drain_cpu_queue(struct ocelot *ocelot, int grp);
978
982
void ocelot_ptp_rx_timestamp(struct ocelot *ocelot, struct sk_buff *skb,
+2
include/soc/mscc/ocelot_vcap.h
+2
include/soc/mscc/ocelot_vcap.h
···
13
13
*/
14
14
#define OCELOT_VCAP_ES0_TAG_8021Q_RXVLAN(ocelot, port, upstream) ((upstream) << 16 | (port))
15
15
#define OCELOT_VCAP_IS1_TAG_8021Q_TXVLAN(ocelot, port) (port)
16
+
#define OCELOT_VCAP_IS1_VLAN_RECLASSIFY(ocelot, port) ((ocelot)->num_phys_ports + (port))
16
17
#define OCELOT_VCAP_IS2_TAG_8021Q_TXVLAN(ocelot, port) (port)
17
18
#define OCELOT_VCAP_IS2_MRP_REDIRECT(ocelot, port) ((ocelot)->num_phys_ports + (port))
18
19
#define OCELOT_VCAP_IS2_MRP_TRAP(ocelot) ((ocelot)->num_phys_ports * 2)
···
500
499
struct ocelot_vcap_u8 pcp; /* PCP (3 bit) */
501
500
enum ocelot_vcap_bit dei; /* DEI */
502
501
enum ocelot_vcap_bit tagged; /* Tagged/untagged frame */
502
+
enum ocelot_vcap_bit tpid;
503
503
};
504
504
505
505
struct ocelot_vcap_key_etype {
+1
-1
include/uapi/linux/io_uring.h
+1
-1
include/uapi/linux/io_uring.h
···
421
421
* IO completion data structure (Completion Queue Entry)
422
422
*/
423
423
struct io_uring_cqe {
424
-
__u64 user_data; /* sqe->data submission passed back */
424
+
__u64 user_data; /* sqe->user_data value passed back */
425
425
__s32 res; /* result code for this event */
426
426
__u32 flags;
427
427
-3
include/uapi/misc/fastrpc.h
-3
include/uapi/misc/fastrpc.h
···
8
8
#define FASTRPC_IOCTL_ALLOC_DMA_BUFF _IOWR('R', 1, struct fastrpc_alloc_dma_buf)
9
9
#define FASTRPC_IOCTL_FREE_DMA_BUFF _IOWR('R', 2, __u32)
10
10
#define FASTRPC_IOCTL_INVOKE _IOWR('R', 3, struct fastrpc_invoke)
11
-
/* This ioctl is only supported with secure device nodes */
12
11
#define FASTRPC_IOCTL_INIT_ATTACH _IO('R', 4)
13
12
#define FASTRPC_IOCTL_INIT_CREATE _IOWR('R', 5, struct fastrpc_init_create)
14
13
#define FASTRPC_IOCTL_MMAP _IOWR('R', 6, struct fastrpc_req_mmap)
15
14
#define FASTRPC_IOCTL_MUNMAP _IOWR('R', 7, struct fastrpc_req_munmap)
16
-
/* This ioctl is only supported with secure device nodes */
17
15
#define FASTRPC_IOCTL_INIT_ATTACH_SNS _IO('R', 8)
18
-
/* This ioctl is only supported with secure device nodes */
19
16
#define FASTRPC_IOCTL_INIT_CREATE_STATIC _IOWR('R', 9, struct fastrpc_init_create_static)
20
17
#define FASTRPC_IOCTL_MEM_MAP _IOWR('R', 10, struct fastrpc_mem_map)
21
18
#define FASTRPC_IOCTL_MEM_UNMAP _IOWR('R', 11, struct fastrpc_mem_unmap)
+2
-2
init/Kconfig
+2
-2
init/Kconfig
···
1920
1920
config RUSTC_VERSION_TEXT
1921
1921
string
1922
1922
depends on RUST
1923
-
default $(shell,command -v $(RUSTC) >/dev/null 2>&1 && $(RUSTC) --version || echo n)
1923
+
default "$(shell,$(RUSTC) --version 2>/dev/null)"
1924
1924
1925
1925
config BINDGEN_VERSION_TEXT
1926
1926
string
···
1928
1928
# The dummy parameter `workaround-for-0.69.0` is required to support 0.69.0
1929
1929
# (https://github.com/rust-lang/rust-bindgen/pull/2678). It can be removed when
1930
1930
# the minimum version is upgraded past that (0.69.1 already fixed the issue).
1931
-
default $(shell,command -v $(BINDGEN) >/dev/null 2>&1 && $(BINDGEN) --version workaround-for-0.69.0 || echo n)
1931
+
default "$(shell,$(BINDGEN) --version workaround-for-0.69.0 2>/dev/null)"
1932
1932
1933
1933
#
1934
1934
# Place an empty function call at each tracepoint site. Can be
+1
-2
io_uring/napi.c
+1
-2
io_uring/napi.c
···
26
26
hlist_for_each_entry_rcu(e, hash_list, node) {
27
27
if (e->napi_id != napi_id)
28
28
continue;
29
-
e->timeout = jiffies + NAPI_TIMEOUT;
30
29
return e;
31
30
}
32
31
···
301
302
{
302
303
iowq->napi_prefer_busy_poll = READ_ONCE(ctx->napi_prefer_busy_poll);
303
304
304
-
if (!(ctx->flags & IORING_SETUP_SQPOLL) && ctx->napi_enabled)
305
+
if (!(ctx->flags & IORING_SETUP_SQPOLL))
305
306
io_napi_blocking_busy_loop(ctx, iowq);
306
307
}
307
308
+1
-1
io_uring/napi.h
+1
-1
io_uring/napi.h
+1
-1
io_uring/sqpoll.c
+1
-1
io_uring/sqpoll.c
+11
-1
kernel/cpu.c
+11
-1
kernel/cpu.c
···
2689
2689
return ret;
2690
2690
}
2691
2691
2692
+
/**
2693
+
* Check if the core a CPU belongs to is online
2694
+
*/
2695
+
#if !defined(topology_is_core_online)
2696
+
static inline bool topology_is_core_online(unsigned int cpu)
2697
+
{
2698
+
return true;
2699
+
}
2700
+
#endif
2701
+
2692
2702
int cpuhp_smt_enable(void)
2693
2703
{
2694
2704
int cpu, ret = 0;
···
2709
2699
/* Skip online CPUs and CPUs on offline nodes */
2710
2700
if (cpu_online(cpu) || !node_online(cpu_to_node(cpu)))
2711
2701
continue;
2712
-
if (!cpu_smt_thread_allowed(cpu))
2702
+
if (!cpu_smt_thread_allowed(cpu) || !topology_is_core_online(cpu))
2713
2703
continue;
2714
2704
ret = _cpu_up(cpu, 0, CPUHP_ONLINE);
2715
2705
if (ret)
+2
-1
kernel/crash_reserve.c
+2
-1
kernel/crash_reserve.c
···
423
423
if (high && search_end == CRASH_ADDR_HIGH_MAX) {
424
424
search_end = CRASH_ADDR_LOW_MAX;
425
425
search_base = 0;
426
-
goto retry;
426
+
if (search_end != CRASH_ADDR_HIGH_MAX)
427
+
goto retry;
427
428
}
428
429
pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
429
430
crash_size);
+6
-49
kernel/kallsyms.c
+6
-49
kernel/kallsyms.c
···
160
160
return kallsyms_relative_base - 1 - kallsyms_offsets[idx];
161
161
}
162
162
163
-
static void cleanup_symbol_name(char *s)
164
-
{
165
-
char *res;
166
-
167
-
if (!IS_ENABLED(CONFIG_LTO_CLANG))
168
-
return;
169
-
170
-
/*
171
-
* LLVM appends various suffixes for local functions and variables that
172
-
* must be promoted to global scope as part of LTO. This can break
173
-
* hooking of static functions with kprobes. '.' is not a valid
174
-
* character in an identifier in C. Suffixes only in LLVM LTO observed:
175
-
* - foo.llvm.[0-9a-f]+
176
-
*/
177
-
res = strstr(s, ".llvm.");
178
-
if (res)
179
-
*res = '\0';
180
-
181
-
return;
182
-
}
183
-
184
-
static int compare_symbol_name(const char *name, char *namebuf)
185
-
{
186
-
/* The kallsyms_seqs_of_names is sorted based on names after
187
-
* cleanup_symbol_name() (see scripts/kallsyms.c) if clang lto is enabled.
188
-
* To ensure correct bisection in kallsyms_lookup_names(), do
189
-
* cleanup_symbol_name(namebuf) before comparing name and namebuf.
190
-
*/
191
-
cleanup_symbol_name(namebuf);
192
-
return strcmp(name, namebuf);
193
-
}
194
-
195
163
static unsigned int get_symbol_seq(int index)
196
164
{
197
165
unsigned int i, seq = 0;
···
187
219
seq = get_symbol_seq(mid);
188
220
off = get_symbol_offset(seq);
189
221
kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf));
190
-
ret = compare_symbol_name(name, namebuf);
222
+
ret = strcmp(name, namebuf);
191
223
if (ret > 0)
192
224
low = mid + 1;
193
225
else if (ret < 0)
···
204
236
seq = get_symbol_seq(low - 1);
205
237
off = get_symbol_offset(seq);
206
238
kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf));
207
-
if (compare_symbol_name(name, namebuf))
239
+
if (strcmp(name, namebuf))
208
240
break;
209
241
low--;
210
242
}
···
216
248
seq = get_symbol_seq(high + 1);
217
249
off = get_symbol_offset(seq);
218
250
kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf));
219
-
if (compare_symbol_name(name, namebuf))
251
+
if (strcmp(name, namebuf))
220
252
break;
221
253
high++;
222
254
}
···
375
407
if (modbuildid)
376
408
*modbuildid = NULL;
377
409
378
-
ret = strlen(namebuf);
379
-
goto found;
410
+
return strlen(namebuf);
380
411
}
381
412
382
413
/* See if it's in a module or a BPF JITed image. */
···
389
422
ret = ftrace_mod_address_lookup(addr, symbolsize,
390
423
offset, modname, namebuf);
391
424
392
-
found:
393
-
cleanup_symbol_name(namebuf);
394
425
return ret;
395
426
}
396
427
···
415
450
416
451
int lookup_symbol_name(unsigned long addr, char *symname)
417
452
{
418
-
int res;
419
-
420
453
symname[0] = '\0';
421
454
symname[KSYM_NAME_LEN - 1] = '\0';
422
455
···
425
462
/* Grab name */
426
463
kallsyms_expand_symbol(get_symbol_offset(pos),
427
464
symname, KSYM_NAME_LEN);
428
-
goto found;
465
+
return 0;
429
466
}
430
467
/* See if it's in a module. */
431
-
res = lookup_module_symbol_name(addr, symname);
432
-
if (res)
433
-
return res;
434
-
435
-
found:
436
-
cleanup_symbol_name(symname);
437
-
return 0;
468
+
return lookup_module_symbol_name(addr, symname);
438
469
}
439
470
440
471
/* Look up a kernel symbol and return it in a text buffer. */
+1
-21
kernel/kallsyms_selftest.c
+1
-21
kernel/kallsyms_selftest.c
···
187
187
stat.min, stat.max, div_u64(stat.sum, stat.real_cnt));
188
188
}
189
189
190
-
static bool match_cleanup_name(const char *s, const char *name)
191
-
{
192
-
char *p;
193
-
int len;
194
-
195
-
if (!IS_ENABLED(CONFIG_LTO_CLANG))
196
-
return false;
197
-
198
-
p = strstr(s, ".llvm.");
199
-
if (!p)
200
-
return false;
201
-
202
-
len = strlen(name);
203
-
if (p - s != len)
204
-
return false;
205
-
206
-
return !strncmp(s, name, len);
207
-
}
208
-
209
190
static int find_symbol(void *data, const char *name, unsigned long addr)
210
191
{
211
192
struct test_stat *stat = (struct test_stat *)data;
212
193
213
-
if (strcmp(name, stat->name) == 0 ||
214
-
(!stat->perf && match_cleanup_name(name, stat->name))) {
194
+
if (!strcmp(name, stat->name)) {
215
195
stat->real_cnt++;
216
196
stat->addr = addr;
217
197
+7
-1
kernel/panic.c
+7
-1
kernel/panic.c
···
64
64
bool panic_on_taint_nousertaint = false;
65
65
static unsigned int warn_limit __read_mostly;
66
66
67
+
bool panic_triggering_all_cpu_backtrace;
68
+
67
69
int panic_timeout = CONFIG_PANIC_TIMEOUT;
68
70
EXPORT_SYMBOL_GPL(panic_timeout);
69
71
···
255
253
*/
256
254
static void panic_other_cpus_shutdown(bool crash_kexec)
257
255
{
258
-
if (panic_print & PANIC_PRINT_ALL_CPU_BT)
256
+
if (panic_print & PANIC_PRINT_ALL_CPU_BT) {
257
+
/* Temporary allow non-panic CPUs to write their backtraces. */
258
+
panic_triggering_all_cpu_backtrace = true;
259
259
trigger_all_cpu_backtrace();
260
+
panic_triggering_all_cpu_backtrace = false;
261
+
}
260
262
261
263
/*
262
264
* Note that smp_send_stop() is the usual SMP shutdown function,
+1
-1
kernel/printk/printk.c
+1
-1
kernel/printk/printk.c
+1
-1
kernel/trace/trace.c
+1
-1
kernel/trace/trace.c
+2
lib/generic-radix-tree.c
+2
lib/generic-radix-tree.c
+1
-2
lib/overflow_kunit.c
+1
-2
lib/overflow_kunit.c
···
668
668
669
669
static void overflow_allocation_test(struct kunit *test)
670
670
{
671
-
const char device_name[] = "overflow-test";
672
671
struct device *dev;
673
672
int count = 0;
674
673
···
677
678
} while (0)
678
679
679
680
/* Create dummy device for devm_kmalloc()-family tests. */
680
-
dev = kunit_device_register(test, device_name);
681
+
dev = kunit_device_register(test, "overflow-test");
681
682
KUNIT_ASSERT_FALSE_MSG(test, IS_ERR(dev),
682
683
"Cannot register test device\n");
683
684
+13
-16
mm/huge_memory.c
+13
-16
mm/huge_memory.c
···
1685
1685
vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
1686
1686
if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) {
1687
1687
spin_unlock(vmf->ptl);
1688
-
goto out;
1688
+
return 0;
1689
1689
}
1690
1690
1691
1691
pmd = pmd_modify(oldpmd, vma->vm_page_prot);
···
1728
1728
if (!migrate_misplaced_folio(folio, vma, target_nid)) {
1729
1729
flags |= TNF_MIGRATED;
1730
1730
nid = target_nid;
1731
-
} else {
1732
-
flags |= TNF_MIGRATE_FAIL;
1733
-
vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
1734
-
if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) {
1735
-
spin_unlock(vmf->ptl);
1736
-
goto out;
1737
-
}
1738
-
goto out_map;
1731
+
task_numa_fault(last_cpupid, nid, HPAGE_PMD_NR, flags);
1732
+
return 0;
1739
1733
}
1740
1734
1741
-
out:
1742
-
if (nid != NUMA_NO_NODE)
1743
-
task_numa_fault(last_cpupid, nid, HPAGE_PMD_NR, flags);
1744
-
1745
-
return 0;
1746
-
1735
+
flags |= TNF_MIGRATE_FAIL;
1736
+
vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
1737
+
if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) {
1738
+
spin_unlock(vmf->ptl);
1739
+
return 0;
1740
+
}
1747
1741
out_map:
1748
1742
/* Restore the PMD */
1749
1743
pmd = pmd_modify(oldpmd, vma->vm_page_prot);
···
1747
1753
set_pmd_at(vma->vm_mm, haddr, vmf->pmd, pmd);
1748
1754
update_mmu_cache_pmd(vma, vmf->address, vmf->pmd);
1749
1755
spin_unlock(vmf->ptl);
1750
-
goto out;
1756
+
1757
+
if (nid != NUMA_NO_NODE)
1758
+
task_numa_fault(last_cpupid, nid, HPAGE_PMD_NR, flags);
1759
+
return 0;
1751
1760
}
1752
1761
1753
1762
/*
+5
-8
mm/hugetlb_vmemmap.c
+5
-8
mm/hugetlb_vmemmap.c
···
185
185
static inline void free_vmemmap_page(struct page *page)
186
186
{
187
187
if (PageReserved(page)) {
188
+
memmap_boot_pages_add(-1);
188
189
free_bootmem_page(page);
189
-
mod_node_page_state(page_pgdat(page), NR_MEMMAP_BOOT, -1);
190
190
} else {
191
+
memmap_pages_add(-1);
191
192
__free_page(page);
192
-
mod_node_page_state(page_pgdat(page), NR_MEMMAP, -1);
193
193
}
194
194
}
195
195
···
341
341
copy_page(page_to_virt(walk.reuse_page),
342
342
(void *)walk.reuse_addr);
343
343
list_add(&walk.reuse_page->lru, vmemmap_pages);
344
-
mod_node_page_state(NODE_DATA(nid), NR_MEMMAP, 1);
344
+
memmap_pages_add(1);
345
345
}
346
346
347
347
/*
···
392
392
393
393
for (i = 0; i < nr_pages; i++) {
394
394
page = alloc_pages_node(nid, gfp_mask, 0);
395
-
if (!page) {
396
-
mod_node_page_state(NODE_DATA(nid), NR_MEMMAP, i);
395
+
if (!page)
397
396
goto out;
398
-
}
399
397
list_add(&page->lru, list);
400
398
}
401
-
402
-
mod_node_page_state(NODE_DATA(nid), NR_MEMMAP, nr_pages);
399
+
memmap_pages_add(nr_pages);
403
400
404
401
return 0;
405
402
out:
+5
-2
mm/memcontrol-v1.c
+5
-2
mm/memcontrol-v1.c
···
1842
1842
buf = endp + 1;
1843
1843
1844
1844
cfd = simple_strtoul(buf, &endp, 10);
1845
-
if ((*endp != ' ') && (*endp != '\0'))
1845
+
if (*endp == '\0')
1846
+
buf = endp;
1847
+
else if (*endp == ' ')
1848
+
buf = endp + 1;
1849
+
else
1846
1850
return -EINVAL;
1847
-
buf = endp + 1;
1848
1851
1849
1852
event = kzalloc(sizeof(*event), GFP_KERNEL);
1850
1853
if (!event)
+11
-9
mm/memory-failure.c
+11
-9
mm/memory-failure.c
···
2417
2417
struct memory_failure_cpu {
2418
2418
DECLARE_KFIFO(fifo, struct memory_failure_entry,
2419
2419
MEMORY_FAILURE_FIFO_SIZE);
2420
-
spinlock_t lock;
2420
+
raw_spinlock_t lock;
2421
2421
struct work_struct work;
2422
2422
};
2423
2423
···
2443
2443
{
2444
2444
struct memory_failure_cpu *mf_cpu;
2445
2445
unsigned long proc_flags;
2446
+
bool buffer_overflow;
2446
2447
struct memory_failure_entry entry = {
2447
2448
.pfn = pfn,
2448
2449
.flags = flags,
2449
2450
};
2450
2451
2451
2452
mf_cpu = &get_cpu_var(memory_failure_cpu);
2452
-
spin_lock_irqsave(&mf_cpu->lock, proc_flags);
2453
-
if (kfifo_put(&mf_cpu->fifo, entry))
2453
+
raw_spin_lock_irqsave(&mf_cpu->lock, proc_flags);
2454
+
buffer_overflow = !kfifo_put(&mf_cpu->fifo, entry);
2455
+
if (!buffer_overflow)
2454
2456
schedule_work_on(smp_processor_id(), &mf_cpu->work);
2455
-
else
2457
+
raw_spin_unlock_irqrestore(&mf_cpu->lock, proc_flags);
2458
+
put_cpu_var(memory_failure_cpu);
2459
+
if (buffer_overflow)
2456
2460
pr_err("buffer overflow when queuing memory failure at %#lx\n",
2457
2461
pfn);
2458
-
spin_unlock_irqrestore(&mf_cpu->lock, proc_flags);
2459
-
put_cpu_var(memory_failure_cpu);
2460
2462
}
2461
2463
EXPORT_SYMBOL_GPL(memory_failure_queue);
2462
2464
···
2471
2469
2472
2470
mf_cpu = container_of(work, struct memory_failure_cpu, work);
2473
2471
for (;;) {
2474
-
spin_lock_irqsave(&mf_cpu->lock, proc_flags);
2472
+
raw_spin_lock_irqsave(&mf_cpu->lock, proc_flags);
2475
2473
gotten = kfifo_get(&mf_cpu->fifo, &entry);
2476
-
spin_unlock_irqrestore(&mf_cpu->lock, proc_flags);
2474
+
raw_spin_unlock_irqrestore(&mf_cpu->lock, proc_flags);
2477
2475
if (!gotten)
2478
2476
break;
2479
2477
if (entry.flags & MF_SOFT_OFFLINE)
···
2503
2501
2504
2502
for_each_possible_cpu(cpu) {
2505
2503
mf_cpu = &per_cpu(memory_failure_cpu, cpu);
2506
-
spin_lock_init(&mf_cpu->lock);
2504
+
raw_spin_lock_init(&mf_cpu->lock);
2507
2505
INIT_KFIFO(mf_cpu->fifo);
2508
2506
INIT_WORK(&mf_cpu->work, memory_failure_work_func);
2509
2507
}
+16
-17
mm/memory.c
+16
-17
mm/memory.c
···
5295
5295
5296
5296
if (unlikely(!pte_same(old_pte, vmf->orig_pte))) {
5297
5297
pte_unmap_unlock(vmf->pte, vmf->ptl);
5298
-
goto out;
5298
+
return 0;
5299
5299
}
5300
5300
5301
5301
pte = pte_modify(old_pte, vma->vm_page_prot);
···
5358
5358
if (!migrate_misplaced_folio(folio, vma, target_nid)) {
5359
5359
nid = target_nid;
5360
5360
flags |= TNF_MIGRATED;
5361
-
} else {
5362
-
flags |= TNF_MIGRATE_FAIL;
5363
-
vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
5364
-
vmf->address, &vmf->ptl);
5365
-
if (unlikely(!vmf->pte))
5366
-
goto out;
5367
-
if (unlikely(!pte_same(ptep_get(vmf->pte), vmf->orig_pte))) {
5368
-
pte_unmap_unlock(vmf->pte, vmf->ptl);
5369
-
goto out;
5370
-
}
5371
-
goto out_map;
5361
+
task_numa_fault(last_cpupid, nid, nr_pages, flags);
5362
+
return 0;
5372
5363
}
5373
5364
5374
-
out:
5375
-
if (nid != NUMA_NO_NODE)
5376
-
task_numa_fault(last_cpupid, nid, nr_pages, flags);
5377
-
return 0;
5365
+
flags |= TNF_MIGRATE_FAIL;
5366
+
vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
5367
+
vmf->address, &vmf->ptl);
5368
+
if (unlikely(!vmf->pte))
5369
+
return 0;
5370
+
if (unlikely(!pte_same(ptep_get(vmf->pte), vmf->orig_pte))) {
5371
+
pte_unmap_unlock(vmf->pte, vmf->ptl);
5372
+
return 0;
5373
+
}
5378
5374
out_map:
5379
5375
/*
5380
5376
* Make it present again, depending on how arch implements
···
5383
5387
numa_rebuild_single_mapping(vmf, vma, vmf->address, vmf->pte,
5384
5388
writable);
5385
5389
pte_unmap_unlock(vmf->pte, vmf->ptl);
5386
-
goto out;
5390
+
5391
+
if (nid != NUMA_NO_NODE)
5392
+
task_numa_fault(last_cpupid, nid, nr_pages, flags);
5393
+
return 0;
5387
5394
}
5388
5395
5389
5396
static inline vm_fault_t create_huge_pmd(struct vm_fault *vmf)
+11
-5
mm/migrate.c
+11
-5
mm/migrate.c
···
1479
1479
return rc;
1480
1480
}
1481
1481
1482
-
static inline int try_split_folio(struct folio *folio, struct list_head *split_folios)
1482
+
static inline int try_split_folio(struct folio *folio, struct list_head *split_folios,
1483
+
enum migrate_mode mode)
1483
1484
{
1484
1485
int rc;
1485
1486
1486
-
folio_lock(folio);
1487
+
if (mode == MIGRATE_ASYNC) {
1488
+
if (!folio_trylock(folio))
1489
+
return -EAGAIN;
1490
+
} else {
1491
+
folio_lock(folio);
1492
+
}
1487
1493
rc = split_folio_to_list(folio, split_folios);
1488
1494
folio_unlock(folio);
1489
1495
if (!rc)
···
1683
1677
*/
1684
1678
if (nr_pages > 2 &&
1685
1679
!list_empty(&folio->_deferred_list)) {
1686
-
if (try_split_folio(folio, split_folios) == 0) {
1680
+
if (!try_split_folio(folio, split_folios, mode)) {
1687
1681
nr_failed++;
1688
1682
stats->nr_thp_failed += is_thp;
1689
1683
stats->nr_thp_split += is_thp;
···
1705
1699
if (!thp_migration_supported() && is_thp) {
1706
1700
nr_failed++;
1707
1701
stats->nr_thp_failed++;
1708
-
if (!try_split_folio(folio, split_folios)) {
1702
+
if (!try_split_folio(folio, split_folios, mode)) {
1709
1703
stats->nr_thp_split++;
1710
1704
stats->nr_split++;
1711
1705
continue;
···
1737
1731
stats->nr_thp_failed += is_thp;
1738
1732
/* Large folio NUMA faulting doesn't split to retry. */
1739
1733
if (is_large && !nosplit) {
1740
-
int ret = try_split_folio(folio, split_folios);
1734
+
int ret = try_split_folio(folio, split_folios, mode);
1741
1735
1742
1736
if (!ret) {
1743
1737
stats->nr_thp_split += is_thp;
+4
-11
mm/mm_init.c
+4
-11
mm/mm_init.c
···
1623
1623
panic("Failed to allocate %ld bytes for node %d memory map\n",
1624
1624
size, pgdat->node_id);
1625
1625
pgdat->node_mem_map = map + offset;
1626
-
mod_node_early_perpage_metadata(pgdat->node_id,
1627
-
DIV_ROUND_UP(size, PAGE_SIZE));
1626
+
memmap_boot_pages_add(DIV_ROUND_UP(size, PAGE_SIZE));
1628
1627
pr_debug("%s: node %d, pgdat %08lx, node_mem_map %08lx\n",
1629
1628
__func__, pgdat->node_id, (unsigned long)pgdat,
1630
1629
(unsigned long)pgdat->node_mem_map);
···
2244
2245
2245
2246
set_pageblock_migratetype(page, MIGRATE_CMA);
2246
2247
set_page_refcounted(page);
2248
+
/* pages were reserved and not allocated */
2249
+
clear_page_tag_ref(page);
2247
2250
__free_pages(page, pageblock_order);
2248
2251
2249
2252
adjust_managed_page_count(page, pageblock_nr_pages);
···
2461
2460
}
2462
2461
2463
2462
/* pages were reserved and not allocated */
2464
-
if (mem_alloc_profiling_enabled()) {
2465
-
union codetag_ref *ref = get_page_tag_ref(page);
2466
-
2467
-
if (ref) {
2468
-
set_codetag_empty(ref);
2469
-
put_page_tag_ref(ref);
2470
-
}
2471
-
}
2472
-
2463
+
clear_page_tag_ref(page);
2473
2464
__free_pages_core(page, order, MEMINIT_EARLY);
2474
2465
}
2475
2466
+11
-3
mm/mseal.c
+11
-3
mm/mseal.c
···
40
40
41
41
static bool is_madv_discard(int behavior)
42
42
{
43
-
return behavior &
44
-
(MADV_FREE | MADV_DONTNEED | MADV_DONTNEED_LOCKED |
45
-
MADV_REMOVE | MADV_DONTFORK | MADV_WIPEONFORK);
43
+
switch (behavior) {
44
+
case MADV_FREE:
45
+
case MADV_DONTNEED:
46
+
case MADV_DONTNEED_LOCKED:
47
+
case MADV_REMOVE:
48
+
case MADV_DONTFORK:
49
+
case MADV_WIPEONFORK:
50
+
return true;
51
+
}
52
+
53
+
return false;
46
54
}
47
55
48
56
static bool is_ro_anon(struct vm_area_struct *vma)
+21
-31
mm/page_alloc.c
+21
-31
mm/page_alloc.c
···
287
287
288
288
static bool page_contains_unaccepted(struct page *page, unsigned int order);
289
289
static void accept_page(struct page *page, unsigned int order);
290
-
static bool try_to_accept_memory(struct zone *zone, unsigned int order);
290
+
static bool cond_accept_memory(struct zone *zone, unsigned int order);
291
291
static inline bool has_unaccepted_memory(void);
292
292
static bool __free_unaccepted(struct page *page);
293
293
···
3072
3072
if (!(alloc_flags & ALLOC_CMA))
3073
3073
unusable_free += zone_page_state(z, NR_FREE_CMA_PAGES);
3074
3074
#endif
3075
-
#ifdef CONFIG_UNACCEPTED_MEMORY
3076
-
unusable_free += zone_page_state(z, NR_UNACCEPTED);
3077
-
#endif
3078
3075
3079
3076
return unusable_free;
3080
3077
}
···
3365
3368
}
3366
3369
}
3367
3370
3371
+
cond_accept_memory(zone, order);
3372
+
3368
3373
/*
3369
3374
* Detect whether the number of free pages is below high
3370
3375
* watermark. If so, we will decrease pcp->high and free
···
3392
3393
gfp_mask)) {
3393
3394
int ret;
3394
3395
3395
-
if (has_unaccepted_memory()) {
3396
-
if (try_to_accept_memory(zone, order))
3397
-
goto try_this_zone;
3398
-
}
3396
+
if (cond_accept_memory(zone, order))
3397
+
goto try_this_zone;
3399
3398
3400
3399
#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
3401
3400
/*
···
3447
3450
3448
3451
return page;
3449
3452
} else {
3450
-
if (has_unaccepted_memory()) {
3451
-
if (try_to_accept_memory(zone, order))
3452
-
goto try_this_zone;
3453
-
}
3453
+
if (cond_accept_memory(zone, order))
3454
+
goto try_this_zone;
3454
3455
3455
3456
#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
3456
3457
/* Try again if zone has deferred pages */
···
5750
5755
for_each_online_pgdat(pgdat)
5751
5756
pgdat->per_cpu_nodestats =
5752
5757
alloc_percpu(struct per_cpu_nodestat);
5753
-
store_early_perpage_metadata();
5754
5758
}
5755
5759
5756
5760
__meminit void zone_pcp_init(struct zone *zone)
···
5815
5821
5816
5822
void free_reserved_page(struct page *page)
5817
5823
{
5818
-
if (mem_alloc_profiling_enabled()) {
5819
-
union codetag_ref *ref = get_page_tag_ref(page);
5820
-
5821
-
if (ref) {
5822
-
set_codetag_empty(ref);
5823
-
put_page_tag_ref(ref);
5824
-
}
5825
-
}
5824
+
clear_page_tag_ref(page);
5826
5825
ClearPageReserved(page);
5827
5826
init_page_count(page);
5828
5827
__free_page(page);
···
6938
6951
struct page *page;
6939
6952
bool last;
6940
6953
6941
-
if (list_empty(&zone->unaccepted_pages))
6942
-
return false;
6943
-
6944
6954
spin_lock_irqsave(&zone->lock, flags);
6945
6955
page = list_first_entry_or_null(&zone->unaccepted_pages,
6946
6956
struct page, lru);
···
6963
6979
return true;
6964
6980
}
6965
6981
6966
-
static bool try_to_accept_memory(struct zone *zone, unsigned int order)
6982
+
static bool cond_accept_memory(struct zone *zone, unsigned int order)
6967
6983
{
6968
6984
long to_accept;
6969
-
int ret = false;
6985
+
bool ret = false;
6986
+
6987
+
if (!has_unaccepted_memory())
6988
+
return false;
6989
+
6990
+
if (list_empty(&zone->unaccepted_pages))
6991
+
return false;
6970
6992
6971
6993
/* How much to accept to get to high watermark? */
6972
6994
to_accept = high_wmark_pages(zone) -
6973
6995
(zone_page_state(zone, NR_FREE_PAGES) -
6974
-
__zone_watermark_unusable_free(zone, order, 0));
6996
+
__zone_watermark_unusable_free(zone, order, 0) -
6997
+
zone_page_state(zone, NR_UNACCEPTED));
6975
6998
6976
-
/* Accept at least one page */
6977
-
do {
6999
+
while (to_accept > 0) {
6978
7000
if (!try_to_accept_memory_one(zone))
6979
7001
break;
6980
7002
ret = true;
6981
7003
to_accept -= MAX_ORDER_NR_PAGES;
6982
-
} while (to_accept > 0);
7004
+
}
6983
7005
6984
7006
return ret;
6985
7007
}
···
7028
7038
{
7029
7039
}
7030
7040
7031
-
static bool try_to_accept_memory(struct zone *zone, unsigned int order)
7041
+
static bool cond_accept_memory(struct zone *zone, unsigned int order)
7032
7042
{
7033
7043
return false;
7034
7044
}
+4
-14
mm/page_ext.c
+4
-14
mm/page_ext.c
···
214
214
return -ENOMEM;
215
215
NODE_DATA(nid)->node_page_ext = base;
216
216
total_usage += table_size;
217
-
mod_node_page_state(NODE_DATA(nid), NR_MEMMAP_BOOT,
218
-
DIV_ROUND_UP(table_size, PAGE_SIZE));
217
+
memmap_boot_pages_add(DIV_ROUND_UP(table_size, PAGE_SIZE));
219
218
return 0;
220
219
}
221
220
···
274
275
else
275
276
addr = vzalloc_node(size, nid);
276
277
277
-
if (addr) {
278
-
mod_node_page_state(NODE_DATA(nid), NR_MEMMAP,
279
-
DIV_ROUND_UP(size, PAGE_SIZE));
280
-
}
278
+
if (addr)
279
+
memmap_pages_add(DIV_ROUND_UP(size, PAGE_SIZE));
281
280
282
281
return addr;
283
282
}
···
320
323
{
321
324
size_t table_size;
322
325
struct page *page;
323
-
struct pglist_data *pgdat;
324
326
325
327
table_size = page_ext_size * PAGES_PER_SECTION;
328
+
memmap_pages_add(-1L * (DIV_ROUND_UP(table_size, PAGE_SIZE)));
326
329
327
330
if (is_vmalloc_addr(addr)) {
328
-
page = vmalloc_to_page(addr);
329
-
pgdat = page_pgdat(page);
330
331
vfree(addr);
331
332
} else {
332
333
page = virt_to_page(addr);
333
-
pgdat = page_pgdat(page);
334
334
BUG_ON(PageReserved(page));
335
335
kmemleak_free(addr);
336
336
free_pages_exact(addr, table_size);
337
337
}
338
-
339
-
mod_node_page_state(pgdat, NR_MEMMAP,
340
-
-1L * (DIV_ROUND_UP(table_size, PAGE_SIZE)));
341
-
342
338
}
343
339
344
340
static void __free_page_ext(unsigned long pfn)
+4
-7
mm/sparse-vmemmap.c
+4
-7
mm/sparse-vmemmap.c
···
469
469
if (r < 0)
470
470
return NULL;
471
471
472
-
if (system_state == SYSTEM_BOOTING) {
473
-
mod_node_early_perpage_metadata(nid, DIV_ROUND_UP(end - start,
474
-
PAGE_SIZE));
475
-
} else {
476
-
mod_node_page_state(NODE_DATA(nid), NR_MEMMAP,
477
-
DIV_ROUND_UP(end - start, PAGE_SIZE));
478
-
}
472
+
if (system_state == SYSTEM_BOOTING)
473
+
memmap_boot_pages_add(DIV_ROUND_UP(end - start, PAGE_SIZE));
474
+
else
475
+
memmap_pages_add(DIV_ROUND_UP(end - start, PAGE_SIZE));
479
476
480
477
return pfn_to_page(pfn);
481
478
}
+2
-3
mm/sparse.c
+2
-3
mm/sparse.c
···
463
463
sparsemap_buf = memmap_alloc(size, section_map_size(), addr, nid, true);
464
464
sparsemap_buf_end = sparsemap_buf + size;
465
465
#ifndef CONFIG_SPARSEMEM_VMEMMAP
466
-
mod_node_early_perpage_metadata(nid, DIV_ROUND_UP(size, PAGE_SIZE));
466
+
memmap_boot_pages_add(DIV_ROUND_UP(size, PAGE_SIZE));
467
467
#endif
468
468
}
469
469
···
643
643
unsigned long start = (unsigned long) pfn_to_page(pfn);
644
644
unsigned long end = start + nr_pages * sizeof(struct page);
645
645
646
-
mod_node_page_state(page_pgdat(pfn_to_page(pfn)), NR_MEMMAP,
647
-
-1L * (DIV_ROUND_UP(end - start, PAGE_SIZE)));
646
+
memmap_pages_add(-1L * (DIV_ROUND_UP(end - start, PAGE_SIZE)));
648
647
vmemmap_free(start, end, altmap);
649
648
}
650
649
static void free_map_bootmem(struct page *memmap)
+2
-9
mm/vmalloc.c
+2
-9
mm/vmalloc.c
···
3584
3584
page = alloc_pages_noprof(alloc_gfp, order);
3585
3585
else
3586
3586
page = alloc_pages_node_noprof(nid, alloc_gfp, order);
3587
-
if (unlikely(!page)) {
3588
-
if (!nofail)
3589
-
break;
3590
-
3591
-
/* fall back to the zero order allocations */
3592
-
alloc_gfp |= __GFP_NOFAIL;
3593
-
order = 0;
3594
-
continue;
3595
-
}
3587
+
if (unlikely(!page))
3588
+
break;
3596
3589
3597
3590
/*
3598
3591
* Higher order allocations must be able to be treated as
+25
-27
mm/vmstat.c
+25
-27
mm/vmstat.c
···
1033
1033
}
1034
1034
#endif
1035
1035
1036
+
/*
1037
+
* Count number of pages "struct page" and "struct page_ext" consume.
1038
+
* nr_memmap_boot_pages: # of pages allocated by boot allocator
1039
+
* nr_memmap_pages: # of pages that were allocated by buddy allocator
1040
+
*/
1041
+
static atomic_long_t nr_memmap_boot_pages = ATOMIC_LONG_INIT(0);
1042
+
static atomic_long_t nr_memmap_pages = ATOMIC_LONG_INIT(0);
1043
+
1044
+
void memmap_boot_pages_add(long delta)
1045
+
{
1046
+
atomic_long_add(delta, &nr_memmap_boot_pages);
1047
+
}
1048
+
1049
+
void memmap_pages_add(long delta)
1050
+
{
1051
+
atomic_long_add(delta, &nr_memmap_pages);
1052
+
}
1053
+
1036
1054
#ifdef CONFIG_COMPACTION
1037
1055
1038
1056
struct contig_page_info {
···
1273
1255
"pgdemote_kswapd",
1274
1256
"pgdemote_direct",
1275
1257
"pgdemote_khugepaged",
1276
-
"nr_memmap",
1277
-
"nr_memmap_boot",
1278
-
/* enum writeback_stat_item counters */
1258
+
/* system-wide enum vm_stat_item counters */
1279
1259
"nr_dirty_threshold",
1280
1260
"nr_dirty_background_threshold",
1261
+
"nr_memmap_pages",
1262
+
"nr_memmap_boot_pages",
1281
1263
1282
1264
#if defined(CONFIG_VM_EVENT_COUNTERS) || defined(CONFIG_MEMCG)
1283
1265
/* enum vm_event_item counters */
···
1808
1790
#define NR_VMSTAT_ITEMS (NR_VM_ZONE_STAT_ITEMS + \
1809
1791
NR_VM_NUMA_EVENT_ITEMS + \
1810
1792
NR_VM_NODE_STAT_ITEMS + \
1811
-
NR_VM_WRITEBACK_STAT_ITEMS + \
1793
+
NR_VM_STAT_ITEMS + \
1812
1794
(IS_ENABLED(CONFIG_VM_EVENT_COUNTERS) ? \
1813
1795
NR_VM_EVENT_ITEMS : 0))
1814
1796
···
1845
1827
1846
1828
global_dirty_limits(v + NR_DIRTY_BG_THRESHOLD,
1847
1829
v + NR_DIRTY_THRESHOLD);
1848
-
v += NR_VM_WRITEBACK_STAT_ITEMS;
1830
+
v[NR_MEMMAP_PAGES] = atomic_long_read(&nr_memmap_pages);
1831
+
v[NR_MEMMAP_BOOT_PAGES] = atomic_long_read(&nr_memmap_boot_pages);
1832
+
v += NR_VM_STAT_ITEMS;
1849
1833
1850
1834
#ifdef CONFIG_VM_EVENT_COUNTERS
1851
1835
all_vm_events(v);
···
2305
2285
module_init(extfrag_debug_init);
2306
2286
2307
2287
#endif
2308
-
2309
-
/*
2310
-
* Page metadata size (struct page and page_ext) in pages
2311
-
*/
2312
-
static unsigned long early_perpage_metadata[MAX_NUMNODES] __meminitdata;
2313
-
2314
-
void __meminit mod_node_early_perpage_metadata(int nid, long delta)
2315
-
{
2316
-
early_perpage_metadata[nid] += delta;
2317
-
}
2318
-
2319
-
void __meminit store_early_perpage_metadata(void)
2320
-
{
2321
-
int nid;
2322
-
struct pglist_data *pgdat;
2323
-
2324
-
for_each_online_pgdat(pgdat) {
2325
-
nid = pgdat->node_id;
2326
-
mod_node_page_state(NODE_DATA(nid), NR_MEMMAP_BOOT,
2327
-
early_perpage_metadata[nid]);
2328
-
}
2329
-
}
+7
-12
net/bluetooth/hci_core.c
+7
-12
net/bluetooth/hci_core.c
···
3664
3664
{
3665
3665
struct hci_chan *chan;
3666
3666
struct sk_buff *skb;
3667
-
int quote, cnt, tmp;
3667
+
int quote, *cnt, tmp;
3668
3668
3669
3669
BT_DBG("%s", hdev->name);
3670
3670
3671
3671
if (!hci_conn_num(hdev, LE_LINK))
3672
3672
return;
3673
3673
3674
-
cnt = hdev->le_pkts ? hdev->le_cnt : hdev->acl_cnt;
3674
+
cnt = hdev->le_pkts ? &hdev->le_cnt : &hdev->acl_cnt;
3675
3675
3676
-
__check_timeout(hdev, cnt, LE_LINK);
3676
+
__check_timeout(hdev, *cnt, LE_LINK);
3677
3677
3678
-
tmp = cnt;
3679
-
while (cnt && (chan = hci_chan_sent(hdev, LE_LINK, "e))) {
3678
+
tmp = *cnt;
3679
+
while (*cnt && (chan = hci_chan_sent(hdev, LE_LINK, "e))) {
3680
3680
u32 priority = (skb_peek(&chan->data_q))->priority;
3681
3681
while (quote-- && (skb = skb_peek(&chan->data_q))) {
3682
3682
BT_DBG("chan %p skb %p len %d priority %u", chan, skb,
···
3691
3691
hci_send_frame(hdev, skb);
3692
3692
hdev->le_last_tx = jiffies;
3693
3693
3694
-
cnt--;
3694
+
(*cnt)--;
3695
3695
chan->sent++;
3696
3696
chan->conn->sent++;
3697
3697
···
3701
3701
}
3702
3702
}
3703
3703
3704
-
if (hdev->le_pkts)
3705
-
hdev->le_cnt = cnt;
3706
-
else
3707
-
hdev->acl_cnt = cnt;
3708
-
3709
-
if (cnt != tmp)
3704
+
if (*cnt != tmp)
3710
3705
hci_prio_recalculate(hdev, LE_LINK);
3711
3706
}
3712
3707
+1
-1
net/bluetooth/hci_event.c
+1
-1
net/bluetooth/hci_event.c
···
5920
5920
* while we have an existing one in peripheral role.
5921
5921
*/
5922
5922
if (hdev->conn_hash.le_num_peripheral > 0 &&
5923
-
(!test_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks) ||
5923
+
(test_bit(HCI_QUIRK_BROKEN_LE_STATES, &hdev->quirks) ||
5924
5924
!(hdev->le_states[3] & 0x10)))
5925
5925
return NULL;
5926
5926
+4
net/bluetooth/mgmt.c
+4
net/bluetooth/mgmt.c
···
3456
3456
* will be kept and this function does nothing.
3457
3457
*/
3458
3458
p = hci_conn_params_add(hdev, &cp->addr.bdaddr, addr_type);
3459
+
if (!p) {
3460
+
err = -EIO;
3461
+
goto unlock;
3462
+
}
3459
3463
3460
3464
if (p->auto_connect == HCI_AUTO_CONN_EXPLICIT)
3461
3465
p->auto_connect = HCI_AUTO_CONN_DISABLED;
+73
-73
net/bluetooth/smp.c
+73
-73
net/bluetooth/smp.c
···
914
914
* Confirms and the responder Enters the passkey.
915
915
*/
916
916
if (smp->method == OVERLAP) {
917
-
if (hcon->role == HCI_ROLE_MASTER)
917
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags))
918
918
smp->method = CFM_PASSKEY;
919
919
else
920
920
smp->method = REQ_PASSKEY;
···
964
964
965
965
smp_send_cmd(smp->conn, SMP_CMD_PAIRING_CONFIRM, sizeof(cp), &cp);
966
966
967
-
if (conn->hcon->out)
967
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags))
968
968
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_CONFIRM);
969
969
else
970
970
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_RANDOM);
···
980
980
int ret;
981
981
982
982
bt_dev_dbg(conn->hcon->hdev, "conn %p %s", conn,
983
-
conn->hcon->out ? "initiator" : "responder");
983
+
test_bit(SMP_FLAG_INITIATOR, &smp->flags) ? "initiator" :
984
+
"responder");
984
985
985
986
ret = smp_c1(smp->tk, smp->rrnd, smp->preq, smp->prsp,
986
987
hcon->init_addr_type, &hcon->init_addr,
···
995
994
return SMP_CONFIRM_FAILED;
996
995
}
997
996
998
-
if (hcon->out) {
997
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
999
998
u8 stk[16];
1000
999
__le64 rand = 0;
1001
1000
__le16 ediv = 0;
···
1257
1256
rsp = (void *) &smp->prsp[1];
1258
1257
1259
1258
/* The responder sends its keys first */
1260
-
if (hcon->out && (smp->remote_key_dist & KEY_DIST_MASK)) {
1259
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags) &&
1260
+
(smp->remote_key_dist & KEY_DIST_MASK)) {
1261
1261
smp_allow_key_dist(smp);
1262
1262
return;
1263
1263
}
1264
1264
1265
1265
req = (void *) &smp->preq[1];
1266
1266
1267
-
if (hcon->out) {
1267
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
1268
1268
keydist = &rsp->init_key_dist;
1269
1269
*keydist &= req->init_key_dist;
1270
1270
} else {
···
1434
1432
struct hci_conn *hcon = smp->conn->hcon;
1435
1433
u8 *na, *nb, a[7], b[7];
1436
1434
1437
-
if (hcon->out) {
1435
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
1438
1436
na = smp->prnd;
1439
1437
nb = smp->rrnd;
1440
1438
} else {
···
1462
1460
a[6] = hcon->init_addr_type;
1463
1461
b[6] = hcon->resp_addr_type;
1464
1462
1465
-
if (hcon->out) {
1463
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
1466
1464
local_addr = a;
1467
1465
remote_addr = b;
1468
1466
memcpy(io_cap, &smp->preq[1], 3);
···
1541
1539
/* The round is only complete when the initiator
1542
1540
* receives pairing random.
1543
1541
*/
1544
-
if (!hcon->out) {
1542
+
if (!test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
1545
1543
smp_send_cmd(conn, SMP_CMD_PAIRING_RANDOM,
1546
1544
sizeof(smp->prnd), smp->prnd);
1547
1545
if (smp->passkey_round == 20)
···
1569
1567
1570
1568
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_RANDOM);
1571
1569
1572
-
if (hcon->out) {
1570
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
1573
1571
smp_send_cmd(conn, SMP_CMD_PAIRING_RANDOM,
1574
1572
sizeof(smp->prnd), smp->prnd);
1575
1573
return 0;
···
1580
1578
case SMP_CMD_PUBLIC_KEY:
1581
1579
default:
1582
1580
/* Initiating device starts the round */
1583
-
if (!hcon->out)
1581
+
if (!test_bit(SMP_FLAG_INITIATOR, &smp->flags))
1584
1582
return 0;
1585
1583
1586
1584
bt_dev_dbg(hdev, "Starting passkey round %u",
···
1625
1623
}
1626
1624
1627
1625
/* Initiator sends DHKey check first */
1628
-
if (hcon->out) {
1626
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
1629
1627
sc_dhkey_check(smp);
1630
1628
SMP_ALLOW_CMD(smp, SMP_CMD_DHKEY_CHECK);
1631
1629
} else if (test_and_clear_bit(SMP_FLAG_DHKEY_PENDING, &smp->flags)) {
···
1748
1746
struct smp_cmd_pairing rsp, *req = (void *) skb->data;
1749
1747
struct l2cap_chan *chan = conn->smp;
1750
1748
struct hci_dev *hdev = conn->hcon->hdev;
1751
-
struct smp_chan *smp;
1749
+
struct smp_chan *smp = chan->data;
1752
1750
u8 key_size, auth, sec_level;
1753
1751
int ret;
1754
1752
···
1757
1755
if (skb->len < sizeof(*req))
1758
1756
return SMP_INVALID_PARAMS;
1759
1757
1760
-
if (conn->hcon->role != HCI_ROLE_SLAVE)
1758
+
if (smp && test_bit(SMP_FLAG_INITIATOR, &smp->flags))
1761
1759
return SMP_CMD_NOTSUPP;
1762
1760
1763
-
if (!chan->data)
1761
+
if (!smp) {
1764
1762
smp = smp_chan_create(conn);
1765
-
else
1766
-
smp = chan->data;
1767
-
1768
-
if (!smp)
1769
-
return SMP_UNSPECIFIED;
1763
+
if (!smp)
1764
+
return SMP_UNSPECIFIED;
1765
+
}
1770
1766
1771
1767
/* We didn't start the pairing, so match remote */
1772
1768
auth = req->auth_req & AUTH_REQ_MASK(hdev);
···
1946
1946
if (skb->len < sizeof(*rsp))
1947
1947
return SMP_INVALID_PARAMS;
1948
1948
1949
-
if (conn->hcon->role != HCI_ROLE_MASTER)
1949
+
if (!test_bit(SMP_FLAG_INITIATOR, &smp->flags))
1950
1950
return SMP_CMD_NOTSUPP;
1951
1951
1952
1952
skb_pull(skb, sizeof(*rsp));
···
2041
2041
if (smp->method == REQ_PASSKEY || smp->method == DSP_PASSKEY)
2042
2042
return sc_passkey_round(smp, SMP_CMD_PAIRING_CONFIRM);
2043
2043
2044
-
if (conn->hcon->out) {
2044
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2045
2045
smp_send_cmd(conn, SMP_CMD_PAIRING_RANDOM, sizeof(smp->prnd),
2046
2046
smp->prnd);
2047
2047
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_RANDOM);
···
2063
2063
u8 auth;
2064
2064
2065
2065
/* The issue is only observed when we're in responder role */
2066
-
if (hcon->out)
2066
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags))
2067
2067
return SMP_UNSPECIFIED;
2068
2068
2069
2069
if (hci_dev_test_flag(hdev, HCI_SC_ONLY)) {
···
2099
2099
struct hci_dev *hdev = hcon->hdev;
2100
2100
2101
2101
bt_dev_dbg(hdev, "conn %p %s", conn,
2102
-
hcon->out ? "initiator" : "responder");
2102
+
test_bit(SMP_FLAG_INITIATOR, &smp->flags) ? "initiator" :
2103
+
"responder");
2103
2104
2104
2105
if (skb->len < sizeof(smp->pcnf))
2105
2106
return SMP_INVALID_PARAMS;
···
2122
2121
return ret;
2123
2122
}
2124
2123
2125
-
if (conn->hcon->out) {
2124
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2126
2125
smp_send_cmd(conn, SMP_CMD_PAIRING_RANDOM, sizeof(smp->prnd),
2127
2126
smp->prnd);
2128
2127
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_RANDOM);
···
2157
2156
if (!test_bit(SMP_FLAG_SC, &smp->flags))
2158
2157
return smp_random(smp);
2159
2158
2160
-
if (hcon->out) {
2159
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2161
2160
pkax = smp->local_pk;
2162
2161
pkbx = smp->remote_pk;
2163
2162
na = smp->prnd;
···
2170
2169
}
2171
2170
2172
2171
if (smp->method == REQ_OOB) {
2173
-
if (!hcon->out)
2172
+
if (!test_bit(SMP_FLAG_INITIATOR, &smp->flags))
2174
2173
smp_send_cmd(conn, SMP_CMD_PAIRING_RANDOM,
2175
2174
sizeof(smp->prnd), smp->prnd);
2176
2175
SMP_ALLOW_CMD(smp, SMP_CMD_DHKEY_CHECK);
···
2181
2180
if (smp->method == REQ_PASSKEY || smp->method == DSP_PASSKEY)
2182
2181
return sc_passkey_round(smp, SMP_CMD_PAIRING_RANDOM);
2183
2182
2184
-
if (hcon->out) {
2183
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2185
2184
u8 cfm[16];
2186
2185
2187
2186
err = smp_f4(smp->tfm_cmac, smp->remote_pk, smp->local_pk,
···
2222
2221
return SMP_UNSPECIFIED;
2223
2222
2224
2223
if (smp->method == REQ_OOB) {
2225
-
if (hcon->out) {
2224
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2226
2225
sc_dhkey_check(smp);
2227
2226
SMP_ALLOW_CMD(smp, SMP_CMD_DHKEY_CHECK);
2228
2227
}
···
2296
2295
return false;
2297
2296
}
2298
2297
2298
+
static void smp_send_pairing_req(struct smp_chan *smp, __u8 auth)
2299
+
{
2300
+
struct smp_cmd_pairing cp;
2301
+
2302
+
if (smp->conn->hcon->type == ACL_LINK)
2303
+
build_bredr_pairing_cmd(smp, &cp, NULL);
2304
+
else
2305
+
build_pairing_cmd(smp->conn, &cp, NULL, auth);
2306
+
2307
+
smp->preq[0] = SMP_CMD_PAIRING_REQ;
2308
+
memcpy(&smp->preq[1], &cp, sizeof(cp));
2309
+
2310
+
smp_send_cmd(smp->conn, SMP_CMD_PAIRING_REQ, sizeof(cp), &cp);
2311
+
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_RSP);
2312
+
2313
+
set_bit(SMP_FLAG_INITIATOR, &smp->flags);
2314
+
}
2315
+
2299
2316
static u8 smp_cmd_security_req(struct l2cap_conn *conn, struct sk_buff *skb)
2300
2317
{
2301
2318
struct smp_cmd_security_req *rp = (void *) skb->data;
2302
-
struct smp_cmd_pairing cp;
2303
2319
struct hci_conn *hcon = conn->hcon;
2304
2320
struct hci_dev *hdev = hcon->hdev;
2305
2321
struct smp_chan *smp;
···
2365
2347
2366
2348
skb_pull(skb, sizeof(*rp));
2367
2349
2368
-
memset(&cp, 0, sizeof(cp));
2369
-
build_pairing_cmd(conn, &cp, NULL, auth);
2370
-
2371
-
smp->preq[0] = SMP_CMD_PAIRING_REQ;
2372
-
memcpy(&smp->preq[1], &cp, sizeof(cp));
2373
-
2374
-
smp_send_cmd(conn, SMP_CMD_PAIRING_REQ, sizeof(cp), &cp);
2375
-
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_RSP);
2350
+
smp_send_pairing_req(smp, auth);
2376
2351
2377
2352
return 0;
2353
+
}
2354
+
2355
+
static void smp_send_security_req(struct smp_chan *smp, __u8 auth)
2356
+
{
2357
+
struct smp_cmd_security_req cp;
2358
+
2359
+
cp.auth_req = auth;
2360
+
smp_send_cmd(smp->conn, SMP_CMD_SECURITY_REQ, sizeof(cp), &cp);
2361
+
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_REQ);
2362
+
2363
+
clear_bit(SMP_FLAG_INITIATOR, &smp->flags);
2378
2364
}
2379
2365
2380
2366
int smp_conn_security(struct hci_conn *hcon, __u8 sec_level)
···
2449
2427
authreq |= SMP_AUTH_MITM;
2450
2428
}
2451
2429
2452
-
if (hcon->role == HCI_ROLE_MASTER) {
2453
-
struct smp_cmd_pairing cp;
2430
+
if (hcon->role == HCI_ROLE_MASTER)
2431
+
smp_send_pairing_req(smp, authreq);
2432
+
else
2433
+
smp_send_security_req(smp, authreq);
2454
2434
2455
-
build_pairing_cmd(conn, &cp, NULL, authreq);
2456
-
smp->preq[0] = SMP_CMD_PAIRING_REQ;
2457
-
memcpy(&smp->preq[1], &cp, sizeof(cp));
2458
-
2459
-
smp_send_cmd(conn, SMP_CMD_PAIRING_REQ, sizeof(cp), &cp);
2460
-
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_RSP);
2461
-
} else {
2462
-
struct smp_cmd_security_req cp;
2463
-
cp.auth_req = authreq;
2464
-
smp_send_cmd(conn, SMP_CMD_SECURITY_REQ, sizeof(cp), &cp);
2465
-
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_REQ);
2466
-
}
2467
-
2468
-
set_bit(SMP_FLAG_INITIATOR, &smp->flags);
2469
2435
ret = 0;
2470
2436
2471
2437
unlock:
···
2704
2694
2705
2695
static u8 sc_select_method(struct smp_chan *smp)
2706
2696
{
2707
-
struct l2cap_conn *conn = smp->conn;
2708
-
struct hci_conn *hcon = conn->hcon;
2709
2697
struct smp_cmd_pairing *local, *remote;
2710
2698
u8 local_mitm, remote_mitm, local_io, remote_io, method;
2711
2699
···
2716
2708
* the "struct smp_cmd_pairing" from them we need to skip the
2717
2709
* first byte which contains the opcode.
2718
2710
*/
2719
-
if (hcon->out) {
2711
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2720
2712
local = (void *) &smp->preq[1];
2721
2713
remote = (void *) &smp->prsp[1];
2722
2714
} else {
···
2785
2777
/* Non-initiating device sends its public key after receiving
2786
2778
* the key from the initiating device.
2787
2779
*/
2788
-
if (!hcon->out) {
2780
+
if (!test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2789
2781
err = sc_send_public_key(smp);
2790
2782
if (err)
2791
2783
return err;
···
2847
2839
}
2848
2840
2849
2841
if (smp->method == REQ_OOB) {
2850
-
if (hcon->out)
2842
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags))
2851
2843
smp_send_cmd(conn, SMP_CMD_PAIRING_RANDOM,
2852
2844
sizeof(smp->prnd), smp->prnd);
2853
2845
···
2856
2848
return 0;
2857
2849
}
2858
2850
2859
-
if (hcon->out)
2851
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags))
2860
2852
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_CONFIRM);
2861
2853
2862
2854
if (smp->method == REQ_PASSKEY) {
···
2871
2863
/* The Initiating device waits for the non-initiating device to
2872
2864
* send the confirm value.
2873
2865
*/
2874
-
if (conn->hcon->out)
2866
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags))
2875
2867
return 0;
2876
2868
2877
2869
err = smp_f4(smp->tfm_cmac, smp->local_pk, smp->remote_pk, smp->prnd,
···
2905
2897
a[6] = hcon->init_addr_type;
2906
2898
b[6] = hcon->resp_addr_type;
2907
2899
2908
-
if (hcon->out) {
2900
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2909
2901
local_addr = a;
2910
2902
remote_addr = b;
2911
2903
memcpy(io_cap, &smp->prsp[1], 3);
···
2930
2922
if (crypto_memneq(check->e, e, 16))
2931
2923
return SMP_DHKEY_CHECK_FAILED;
2932
2924
2933
-
if (!hcon->out) {
2925
+
if (!test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2934
2926
if (test_bit(SMP_FLAG_WAIT_USER, &smp->flags)) {
2935
2927
set_bit(SMP_FLAG_DHKEY_PENDING, &smp->flags);
2936
2928
return 0;
···
2942
2934
2943
2935
sc_add_ltk(smp);
2944
2936
2945
-
if (hcon->out) {
2937
+
if (test_bit(SMP_FLAG_INITIATOR, &smp->flags)) {
2946
2938
hci_le_start_enc(hcon, 0, 0, smp->tk, smp->enc_key_size);
2947
2939
hcon->enc_key_size = smp->enc_key_size;
2948
2940
}
···
3091
3083
struct l2cap_conn *conn = chan->conn;
3092
3084
struct hci_conn *hcon = conn->hcon;
3093
3085
struct hci_dev *hdev = hcon->hdev;
3094
-
struct smp_cmd_pairing req;
3095
3086
struct smp_chan *smp;
3096
3087
3097
3088
bt_dev_dbg(hdev, "chan %p", chan);
···
3142
3135
3143
3136
bt_dev_dbg(hdev, "starting SMP over BR/EDR");
3144
3137
3145
-
/* Prepare and send the BR/EDR SMP Pairing Request */
3146
-
build_bredr_pairing_cmd(smp, &req, NULL);
3147
-
3148
-
smp->preq[0] = SMP_CMD_PAIRING_REQ;
3149
-
memcpy(&smp->preq[1], &req, sizeof(req));
3150
-
3151
-
smp_send_cmd(conn, SMP_CMD_PAIRING_REQ, sizeof(req), &req);
3152
-
SMP_ALLOW_CMD(smp, SMP_CMD_PAIRING_RSP);
3138
+
smp_send_pairing_req(smp, 0x00);
3153
3139
}
3154
3140
3155
3141
static void smp_resume_cb(struct l2cap_chan *chan)
-2
net/core/netpoll.c
-2
net/core/netpoll.c
···
228
228
down(&ni->dev_lock);
229
229
srcu_read_unlock(&netpoll_srcu, idx);
230
230
}
231
-
EXPORT_SYMBOL(netpoll_poll_disable);
232
231
233
232
void netpoll_poll_enable(struct net_device *dev)
234
233
{
···
238
239
up(&ni->dev_lock);
239
240
rcu_read_unlock();
240
241
}
241
-
EXPORT_SYMBOL(netpoll_poll_enable);
242
242
243
243
static void refill_skbs(void)
244
244
{
+3
-2
net/dsa/tag.c
+3
-2
net/dsa/tag.c
···
105
105
106
106
p = netdev_priv(skb->dev);
107
107
108
-
if (unlikely(cpu_dp->ds->untag_bridge_pvid)) {
109
-
nskb = dsa_untag_bridge_pvid(skb);
108
+
if (unlikely(cpu_dp->ds->untag_bridge_pvid ||
109
+
cpu_dp->ds->untag_vlan_aware_bridge_pvid)) {
110
+
nskb = dsa_software_vlan_untag(skb);
110
111
if (!nskb) {
111
112
kfree_skb(skb);
112
113
return 0;
+108
-33
net/dsa/tag.h
+108
-33
net/dsa/tag.h
···
44
44
return NULL;
45
45
}
46
46
47
-
/* If under a bridge with vlan_filtering=0, make sure to send pvid-tagged
48
-
* frames as untagged, since the bridge will not untag them.
47
+
/**
48
+
* dsa_software_untag_vlan_aware_bridge: Software untagging for VLAN-aware bridge
49
+
* @skb: Pointer to received socket buffer (packet)
50
+
* @br: Pointer to bridge upper interface of ingress port
51
+
* @vid: Parsed VID from packet
52
+
*
53
+
* The bridge can process tagged packets. Software like STP/PTP may not. The
54
+
* bridge can also process untagged packets, to the same effect as if they were
55
+
* tagged with the PVID of the ingress port. So packets tagged with the PVID of
56
+
* the bridge port must be software-untagged, to support both use cases.
49
57
*/
50
-
static inline struct sk_buff *dsa_untag_bridge_pvid(struct sk_buff *skb)
58
+
static inline void dsa_software_untag_vlan_aware_bridge(struct sk_buff *skb,
59
+
struct net_device *br,
60
+
u16 vid)
51
61
{
52
-
struct dsa_port *dp = dsa_user_to_port(skb->dev);
53
-
struct net_device *br = dsa_port_bridge_dev_get(dp);
54
-
struct net_device *dev = skb->dev;
55
-
struct net_device *upper_dev;
56
-
u16 vid, pvid, proto;
62
+
u16 pvid, proto;
57
63
int err;
58
-
59
-
if (!br || br_vlan_enabled(br))
60
-
return skb;
61
64
62
65
err = br_vlan_get_proto(br, &proto);
63
66
if (err)
64
-
return skb;
67
+
return;
65
68
66
-
/* Move VLAN tag from data to hwaccel */
67
-
if (!skb_vlan_tag_present(skb) && skb->protocol == htons(proto)) {
68
-
skb = skb_vlan_untag(skb);
69
-
if (!skb)
70
-
return NULL;
71
-
}
72
-
73
-
if (!skb_vlan_tag_present(skb))
74
-
return skb;
75
-
76
-
vid = skb_vlan_tag_get_id(skb);
77
-
78
-
/* We already run under an RCU read-side critical section since
79
-
* we are called from netif_receive_skb_list_internal().
80
-
*/
81
-
err = br_vlan_get_pvid_rcu(dev, &pvid);
69
+
err = br_vlan_get_pvid_rcu(skb->dev, &pvid);
82
70
if (err)
83
-
return skb;
71
+
return;
84
72
85
-
if (vid != pvid)
86
-
return skb;
73
+
if (vid == pvid && skb->vlan_proto == htons(proto))
74
+
__vlan_hwaccel_clear_tag(skb);
75
+
}
76
+
77
+
/**
78
+
* dsa_software_untag_vlan_unaware_bridge: Software untagging for VLAN-unaware bridge
79
+
* @skb: Pointer to received socket buffer (packet)
80
+
* @br: Pointer to bridge upper interface of ingress port
81
+
* @vid: Parsed VID from packet
82
+
*
83
+
* The bridge ignores all VLAN tags. Software like STP/PTP may not (it may run
84
+
* on the plain port, or on a VLAN upper interface). Maybe packets are coming
85
+
* to software as tagged with a driver-defined VID which is NOT equal to the
86
+
* PVID of the bridge port (since the bridge is VLAN-unaware, its configuration
87
+
* should NOT be committed to hardware). DSA needs a method for this private
88
+
* VID to be communicated by software to it, and if packets are tagged with it,
89
+
* software-untag them. Note: the private VID may be different per bridge, to
90
+
* support the FDB isolation use case.
91
+
*
92
+
* FIXME: this is currently implemented based on the broken assumption that
93
+
* the "private VID" used by the driver in VLAN-unaware mode is equal to the
94
+
* bridge PVID. It should not be, except for a coincidence; the bridge PVID is
95
+
* irrelevant to the data path in the VLAN-unaware mode. Thus, the VID that
96
+
* this function removes is wrong.
97
+
*
98
+
* All users of ds->untag_bridge_pvid should fix their drivers, if necessary,
99
+
* to make the two independent. Only then, if there still remains a need to
100
+
* strip the private VID from packets, then a new ds->ops->get_private_vid()
101
+
* API shall be introduced to communicate to DSA what this VID is, which needs
102
+
* to be stripped here.
103
+
*/
104
+
static inline void dsa_software_untag_vlan_unaware_bridge(struct sk_buff *skb,
105
+
struct net_device *br,
106
+
u16 vid)
107
+
{
108
+
struct net_device *upper_dev;
109
+
u16 pvid, proto;
110
+
int err;
111
+
112
+
err = br_vlan_get_proto(br, &proto);
113
+
if (err)
114
+
return;
115
+
116
+
err = br_vlan_get_pvid_rcu(skb->dev, &pvid);
117
+
if (err)
118
+
return;
119
+
120
+
if (vid != pvid || skb->vlan_proto != htons(proto))
121
+
return;
87
122
88
123
/* The sad part about attempting to untag from DSA is that we
89
124
* don't know, unless we check, if the skb will end up in
···
130
95
* definitely keep the tag, to make sure it keeps working.
131
96
*/
132
97
upper_dev = __vlan_find_dev_deep_rcu(br, htons(proto), vid);
133
-
if (upper_dev)
98
+
if (!upper_dev)
99
+
__vlan_hwaccel_clear_tag(skb);
100
+
}
101
+
102
+
/**
103
+
* dsa_software_vlan_untag: Software VLAN untagging in DSA receive path
104
+
* @skb: Pointer to socket buffer (packet)
105
+
*
106
+
* Receive path method for switches which cannot avoid tagging all packets
107
+
* towards the CPU port. Called when ds->untag_bridge_pvid (legacy) or
108
+
* ds->untag_vlan_aware_bridge_pvid is set to true.
109
+
*
110
+
* As a side effect of this method, any VLAN tag from the skb head is moved
111
+
* to hwaccel.
112
+
*/
113
+
static inline struct sk_buff *dsa_software_vlan_untag(struct sk_buff *skb)
114
+
{
115
+
struct dsa_port *dp = dsa_user_to_port(skb->dev);
116
+
struct net_device *br = dsa_port_bridge_dev_get(dp);
117
+
u16 vid;
118
+
119
+
/* software untagging for standalone ports not yet necessary */
120
+
if (!br)
134
121
return skb;
135
122
136
-
__vlan_hwaccel_clear_tag(skb);
123
+
/* Move VLAN tag from data to hwaccel */
124
+
if (!skb_vlan_tag_present(skb)) {
125
+
skb = skb_vlan_untag(skb);
126
+
if (!skb)
127
+
return NULL;
128
+
}
129
+
130
+
if (!skb_vlan_tag_present(skb))
131
+
return skb;
132
+
133
+
vid = skb_vlan_tag_get_id(skb);
134
+
135
+
if (br_vlan_enabled(br)) {
136
+
if (dp->ds->untag_vlan_aware_bridge_pvid)
137
+
dsa_software_untag_vlan_aware_bridge(skb, br, vid);
138
+
} else {
139
+
if (dp->ds->untag_bridge_pvid)
140
+
dsa_software_untag_vlan_unaware_bridge(skb, br, vid);
141
+
}
137
142
138
143
return skb;
139
144
}
+2
-35
net/dsa/tag_ocelot.c
+2
-35
net/dsa/tag_ocelot.c
···
8
8
#define OCELOT_NAME "ocelot"
9
9
#define SEVILLE_NAME "seville"
10
10
11
-
/* If the port is under a VLAN-aware bridge, remove the VLAN header from the
12
-
* payload and move it into the DSA tag, which will make the switch classify
13
-
* the packet to the bridge VLAN. Otherwise, leave the classified VLAN at zero,
14
-
* which is the pvid of standalone and VLAN-unaware bridge ports.
15
-
*/
16
-
static void ocelot_xmit_get_vlan_info(struct sk_buff *skb, struct dsa_port *dp,
17
-
u64 *vlan_tci, u64 *tag_type)
18
-
{
19
-
struct net_device *br = dsa_port_bridge_dev_get(dp);
20
-
struct vlan_ethhdr *hdr;
21
-
u16 proto, tci;
22
-
23
-
if (!br || !br_vlan_enabled(br)) {
24
-
*vlan_tci = 0;
25
-
*tag_type = IFH_TAG_TYPE_C;
26
-
return;
27
-
}
28
-
29
-
hdr = skb_vlan_eth_hdr(skb);
30
-
br_vlan_get_proto(br, &proto);
31
-
32
-
if (ntohs(hdr->h_vlan_proto) == proto) {
33
-
vlan_remove_tag(skb, &tci);
34
-
*vlan_tci = tci;
35
-
} else {
36
-
rcu_read_lock();
37
-
br_vlan_get_pvid_rcu(br, &tci);
38
-
rcu_read_unlock();
39
-
*vlan_tci = tci;
40
-
}
41
-
42
-
*tag_type = (proto != ETH_P_8021Q) ? IFH_TAG_TYPE_S : IFH_TAG_TYPE_C;
43
-
}
44
-
45
11
static void ocelot_xmit_common(struct sk_buff *skb, struct net_device *netdev,
46
12
__be32 ifh_prefix, void **ifh)
47
13
{
···
19
53
u32 rew_op = 0;
20
54
u64 qos_class;
21
55
22
-
ocelot_xmit_get_vlan_info(skb, dp, &vlan_tci, &tag_type);
56
+
ocelot_xmit_get_vlan_info(skb, dsa_port_bridge_dev_get(dp), &vlan_tci,
57
+
&tag_type);
23
58
24
59
qos_class = netdev_get_num_tc(netdev) ?
25
60
netdev_get_prio_tc_map(netdev, skb->priority) : skb->priority;
+14
net/ipv4/tcp_ipv4.c
+14
net/ipv4/tcp_ipv4.c
···
97
97
.bh_lock = INIT_LOCAL_LOCK(bh_lock),
98
98
};
99
99
100
+
static DEFINE_MUTEX(tcp_exit_batch_mutex);
101
+
100
102
static u32 tcp_v4_init_seq(const struct sk_buff *skb)
101
103
{
102
104
return secure_tcp_seq(ip_hdr(skb)->daddr,
···
3516
3514
{
3517
3515
struct net *net;
3518
3516
3517
+
/* make sure concurrent calls to tcp_sk_exit_batch from net_cleanup_work
3518
+
* and failed setup_net error unwinding path are serialized.
3519
+
*
3520
+
* tcp_twsk_purge() handles twsk in any dead netns, not just those in
3521
+
* net_exit_list, the thread that dismantles a particular twsk must
3522
+
* do so without other thread progressing to refcount_dec_and_test() of
3523
+
* tcp_death_row.tw_refcount.
3524
+
*/
3525
+
mutex_lock(&tcp_exit_batch_mutex);
3526
+
3519
3527
tcp_twsk_purge(net_exit_list);
3520
3528
3521
3529
list_for_each_entry(net, net_exit_list, exit_list) {
···
3533
3521
WARN_ON_ONCE(!refcount_dec_and_test(&net->ipv4.tcp_death_row.tw_refcount));
3534
3522
tcp_fastopen_ctx_destroy(net);
3535
3523
}
3524
+
3525
+
mutex_unlock(&tcp_exit_batch_mutex);
3536
3526
}
3537
3527
3538
3528
static struct pernet_operations __net_initdata tcp_sk_ops = {
+2
-1
net/ipv4/udp_offload.c
+2
-1
net/ipv4/udp_offload.c
···
279
279
return ERR_PTR(-EINVAL);
280
280
281
281
if (unlikely(skb_checksum_start(gso_skb) !=
282
-
skb_transport_header(gso_skb)))
282
+
skb_transport_header(gso_skb) &&
283
+
!(skb_shinfo(gso_skb)->gso_type & SKB_GSO_FRAGLIST)))
283
284
return ERR_PTR(-EINVAL);
284
285
285
286
/* We don't know if egress device can segment and checksum the packet
+10
net/ipv6/ip6_output.c
+10
net/ipv6/ip6_output.c
···
70
70
71
71
/* Be paranoid, rather than too clever. */
72
72
if (unlikely(hh_len > skb_headroom(skb)) && dev->header_ops) {
73
+
/* Make sure idev stays alive */
74
+
rcu_read_lock();
73
75
skb = skb_expand_head(skb, hh_len);
74
76
if (!skb) {
75
77
IP6_INC_STATS(net, idev, IPSTATS_MIB_OUTDISCARDS);
78
+
rcu_read_unlock();
76
79
return -ENOMEM;
77
80
}
81
+
rcu_read_unlock();
78
82
}
79
83
80
84
hdr = ipv6_hdr(skb);
···
287
283
head_room += opt->opt_nflen + opt->opt_flen;
288
284
289
285
if (unlikely(head_room > skb_headroom(skb))) {
286
+
/* Make sure idev stays alive */
287
+
rcu_read_lock();
290
288
skb = skb_expand_head(skb, head_room);
291
289
if (!skb) {
292
290
IP6_INC_STATS(net, idev, IPSTATS_MIB_OUTDISCARDS);
291
+
rcu_read_unlock();
293
292
return -ENOBUFS;
294
293
}
294
+
rcu_read_unlock();
295
295
}
296
296
297
297
if (opt) {
···
1964
1956
struct rt6_info *rt = dst_rt6_info(skb_dst(skb));
1965
1957
int err;
1966
1958
1959
+
rcu_read_lock();
1967
1960
err = ip6_local_out(net, skb->sk, skb);
1968
1961
if (err) {
1969
1962
if (err > 0)
···
1974
1965
IPSTATS_MIB_OUTDISCARDS);
1975
1966
}
1976
1967
1968
+
rcu_read_unlock();
1977
1969
return err;
1978
1970
}
1979
1971
+7
-5
net/ipv6/ip6_tunnel.c
+7
-5
net/ipv6/ip6_tunnel.c
···
1507
1507
tdev = __dev_get_by_index(t->net, p->link);
1508
1508
1509
1509
if (tdev) {
1510
-
dev->hard_header_len = tdev->hard_header_len + t_hlen;
1510
+
dev->needed_headroom = tdev->hard_header_len +
1511
+
tdev->needed_headroom + t_hlen;
1511
1512
mtu = min_t(unsigned int, tdev->mtu, IP6_MAX_MTU);
1512
1513
1513
1514
mtu = mtu - t_hlen;
···
1732
1731
int ip6_tnl_change_mtu(struct net_device *dev, int new_mtu)
1733
1732
{
1734
1733
struct ip6_tnl *tnl = netdev_priv(dev);
1734
+
int t_hlen;
1735
1735
1736
+
t_hlen = tnl->hlen + sizeof(struct ipv6hdr);
1736
1737
if (tnl->parms.proto == IPPROTO_IPV6) {
1737
1738
if (new_mtu < IPV6_MIN_MTU)
1738
1739
return -EINVAL;
···
1743
1740
return -EINVAL;
1744
1741
}
1745
1742
if (tnl->parms.proto == IPPROTO_IPV6 || tnl->parms.proto == 0) {
1746
-
if (new_mtu > IP6_MAX_MTU - dev->hard_header_len)
1743
+
if (new_mtu > IP6_MAX_MTU - dev->hard_header_len - t_hlen)
1747
1744
return -EINVAL;
1748
1745
} else {
1749
-
if (new_mtu > IP_MAX_MTU - dev->hard_header_len)
1746
+
if (new_mtu > IP_MAX_MTU - dev->hard_header_len - t_hlen)
1750
1747
return -EINVAL;
1751
1748
}
1752
1749
WRITE_ONCE(dev->mtu, new_mtu);
···
1890
1887
t_hlen = t->hlen + sizeof(struct ipv6hdr);
1891
1888
1892
1889
dev->type = ARPHRD_TUNNEL6;
1893
-
dev->hard_header_len = LL_MAX_HEADER + t_hlen;
1894
1890
dev->mtu = ETH_DATA_LEN - t_hlen;
1895
1891
if (!(t->parms.flags & IP6_TNL_F_IGN_ENCAP_LIMIT))
1896
1892
dev->mtu -= 8;
1897
1893
dev->min_mtu = ETH_MIN_MTU;
1898
-
dev->max_mtu = IP6_MAX_MTU - dev->hard_header_len;
1894
+
dev->max_mtu = IP6_MAX_MTU - dev->hard_header_len - t_hlen;
1899
1895
1900
1896
netdev_hold(dev, &t->dev_tracker, GFP_KERNEL);
1901
1897
netdev_lockdep_set_classes(dev);
+3
-1
net/iucv/iucv.c
+3
-1
net/iucv/iucv.c
···
86
86
{
87
87
struct device *dev;
88
88
va_list vargs;
89
+
char buf[20];
89
90
int rc;
90
91
91
92
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
92
93
if (!dev)
93
94
goto out_error;
94
95
va_start(vargs, fmt);
95
-
rc = dev_set_name(dev, fmt, vargs);
96
+
vsnprintf(buf, sizeof(buf), fmt, vargs);
97
+
rc = dev_set_name(dev, "%s", buf);
96
98
va_end(vargs);
97
99
if (rc)
98
100
goto out_error;
+4
net/kcm/kcmsock.c
+4
net/kcm/kcmsock.c
···
755
755
!(msg->msg_flags & MSG_MORE) : !!(msg->msg_flags & MSG_EOR);
756
756
int err = -EPIPE;
757
757
758
+
mutex_lock(&kcm->tx_mutex);
758
759
lock_sock(sk);
759
760
760
761
/* Per tcp_sendmsg this should be in poll */
···
927
926
KCM_STATS_ADD(kcm->stats.tx_bytes, copied);
928
927
929
928
release_sock(sk);
929
+
mutex_unlock(&kcm->tx_mutex);
930
930
return copied;
931
931
932
932
out_error:
···
953
951
sk->sk_write_space(sk);
954
952
955
953
release_sock(sk);
954
+
mutex_unlock(&kcm->tx_mutex);
956
955
return err;
957
956
}
958
957
···
1207
1204
spin_unlock_bh(&mux->lock);
1208
1205
1209
1206
INIT_WORK(&kcm->tx_work, kcm_tx_work);
1207
+
mutex_init(&kcm->tx_mutex);
1210
1208
1211
1209
spin_lock_bh(&mux->rx_lock);
1212
1210
kcm_rcv_ready(kcm);
+1
-1
net/mctp/test/route-test.c
+1
-1
net/mctp/test/route-test.c
-13
net/mptcp/pm.c
-13
net/mptcp/pm.c
···
60
60
return 0;
61
61
}
62
62
63
-
int mptcp_pm_remove_subflow(struct mptcp_sock *msk, const struct mptcp_rm_list *rm_list)
64
-
{
65
-
pr_debug("msk=%p, rm_list_nr=%d", msk, rm_list->nr);
66
-
67
-
spin_lock_bh(&msk->pm.lock);
68
-
mptcp_pm_nl_rm_subflow_received(msk, rm_list);
69
-
spin_unlock_bh(&msk->pm.lock);
70
-
return 0;
71
-
}
72
-
73
63
/* path manager event handlers */
74
64
75
65
void mptcp_pm_new_connection(struct mptcp_sock *msk, const struct sock *ssk, int server_side)
···
433
443
{
434
444
*flags = 0;
435
445
*ifindex = 0;
436
-
437
-
if (!id)
438
-
return 0;
439
446
440
447
if (mptcp_pm_is_userspace(msk))
441
448
return mptcp_userspace_pm_get_flags_and_ifindex_by_id(msk, id, flags, ifindex);
+94
-48
net/mptcp/pm_netlink.c
+94
-48
net/mptcp/pm_netlink.c
···
143
143
return false;
144
144
}
145
145
146
-
static struct mptcp_pm_addr_entry *
146
+
static bool
147
147
select_local_address(const struct pm_nl_pernet *pernet,
148
-
const struct mptcp_sock *msk)
148
+
const struct mptcp_sock *msk,
149
+
struct mptcp_pm_addr_entry *new_entry)
149
150
{
150
-
struct mptcp_pm_addr_entry *entry, *ret = NULL;
151
+
struct mptcp_pm_addr_entry *entry;
152
+
bool found = false;
151
153
152
154
msk_owned_by_me(msk);
153
155
···
161
159
if (!test_bit(entry->addr.id, msk->pm.id_avail_bitmap))
162
160
continue;
163
161
164
-
ret = entry;
162
+
*new_entry = *entry;
163
+
found = true;
165
164
break;
166
165
}
167
166
rcu_read_unlock();
168
-
return ret;
167
+
168
+
return found;
169
169
}
170
170
171
-
static struct mptcp_pm_addr_entry *
172
-
select_signal_address(struct pm_nl_pernet *pernet, const struct mptcp_sock *msk)
171
+
static bool
172
+
select_signal_address(struct pm_nl_pernet *pernet, const struct mptcp_sock *msk,
173
+
struct mptcp_pm_addr_entry *new_entry)
173
174
{
174
-
struct mptcp_pm_addr_entry *entry, *ret = NULL;
175
+
struct mptcp_pm_addr_entry *entry;
176
+
bool found = false;
175
177
176
178
rcu_read_lock();
177
179
/* do not keep any additional per socket state, just signal
···
190
184
if (!(entry->flags & MPTCP_PM_ADDR_FLAG_SIGNAL))
191
185
continue;
192
186
193
-
ret = entry;
187
+
*new_entry = *entry;
188
+
found = true;
194
189
break;
195
190
}
196
191
rcu_read_unlock();
197
-
return ret;
192
+
193
+
return found;
198
194
}
199
195
200
196
unsigned int mptcp_pm_get_add_addr_signal_max(const struct mptcp_sock *msk)
···
520
512
521
513
static void mptcp_pm_create_subflow_or_signal_addr(struct mptcp_sock *msk)
522
514
{
523
-
struct mptcp_pm_addr_entry *local, *signal_and_subflow = NULL;
524
515
struct sock *sk = (struct sock *)msk;
516
+
struct mptcp_pm_addr_entry local;
525
517
unsigned int add_addr_signal_max;
518
+
bool signal_and_subflow = false;
526
519
unsigned int local_addr_max;
527
520
struct pm_nl_pernet *pernet;
528
521
unsigned int subflows_max;
···
574
565
if (msk->pm.addr_signal & BIT(MPTCP_ADD_ADDR_SIGNAL))
575
566
return;
576
567
577
-
local = select_signal_address(pernet, msk);
578
-
if (!local)
568
+
if (!select_signal_address(pernet, msk, &local))
579
569
goto subflow;
580
570
581
571
/* If the alloc fails, we are on memory pressure, not worth
582
572
* continuing, and trying to create subflows.
583
573
*/
584
-
if (!mptcp_pm_alloc_anno_list(msk, &local->addr))
574
+
if (!mptcp_pm_alloc_anno_list(msk, &local.addr))
585
575
return;
586
576
587
-
__clear_bit(local->addr.id, msk->pm.id_avail_bitmap);
577
+
__clear_bit(local.addr.id, msk->pm.id_avail_bitmap);
588
578
msk->pm.add_addr_signaled++;
589
-
mptcp_pm_announce_addr(msk, &local->addr, false);
579
+
mptcp_pm_announce_addr(msk, &local.addr, false);
590
580
mptcp_pm_nl_addr_send_ack(msk);
591
581
592
-
if (local->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW)
593
-
signal_and_subflow = local;
582
+
if (local.flags & MPTCP_PM_ADDR_FLAG_SUBFLOW)
583
+
signal_and_subflow = true;
594
584
}
595
585
596
586
subflow:
···
600
592
bool fullmesh;
601
593
int i, nr;
602
594
603
-
if (signal_and_subflow) {
604
-
local = signal_and_subflow;
605
-
signal_and_subflow = NULL;
606
-
} else {
607
-
local = select_local_address(pernet, msk);
608
-
if (!local)
609
-
break;
610
-
}
595
+
if (signal_and_subflow)
596
+
signal_and_subflow = false;
597
+
else if (!select_local_address(pernet, msk, &local))
598
+
break;
611
599
612
-
fullmesh = !!(local->flags & MPTCP_PM_ADDR_FLAG_FULLMESH);
600
+
fullmesh = !!(local.flags & MPTCP_PM_ADDR_FLAG_FULLMESH);
613
601
614
602
msk->pm.local_addr_used++;
615
-
__clear_bit(local->addr.id, msk->pm.id_avail_bitmap);
616
-
nr = fill_remote_addresses_vec(msk, &local->addr, fullmesh, addrs);
603
+
__clear_bit(local.addr.id, msk->pm.id_avail_bitmap);
604
+
nr = fill_remote_addresses_vec(msk, &local.addr, fullmesh, addrs);
617
605
if (nr == 0)
618
606
continue;
619
607
620
608
spin_unlock_bh(&msk->pm.lock);
621
609
for (i = 0; i < nr; i++)
622
-
__mptcp_subflow_connect(sk, &local->addr, &addrs[i]);
610
+
__mptcp_subflow_connect(sk, &local.addr, &addrs[i]);
623
611
spin_lock_bh(&msk->pm.lock);
624
612
}
625
613
mptcp_pm_nl_check_work_pending(msk);
···
640
636
{
641
637
struct sock *sk = (struct sock *)msk;
642
638
struct mptcp_pm_addr_entry *entry;
639
+
struct mptcp_addr_info mpc_addr;
643
640
struct pm_nl_pernet *pernet;
644
641
unsigned int subflows_max;
645
642
int i = 0;
646
643
647
644
pernet = pm_nl_get_pernet_from_msk(msk);
648
645
subflows_max = mptcp_pm_get_subflows_max(msk);
646
+
647
+
mptcp_local_address((struct sock_common *)msk, &mpc_addr);
649
648
650
649
rcu_read_lock();
651
650
list_for_each_entry_rcu(entry, &pernet->local_addr_list, list) {
···
660
653
661
654
if (msk->pm.subflows < subflows_max) {
662
655
msk->pm.subflows++;
663
-
addrs[i++] = entry->addr;
656
+
addrs[i] = entry->addr;
657
+
658
+
/* Special case for ID0: set the correct ID */
659
+
if (mptcp_addresses_equal(&entry->addr, &mpc_addr, entry->addr.port))
660
+
addrs[i].id = 0;
661
+
662
+
i++;
664
663
}
665
664
}
666
665
rcu_read_unlock();
···
842
829
mptcp_close_ssk(sk, ssk, subflow);
843
830
spin_lock_bh(&msk->pm.lock);
844
831
845
-
removed = true;
832
+
removed |= subflow->request_join;
846
833
if (rm_type == MPTCP_MIB_RMSUBFLOW)
847
834
__MPTCP_INC_STATS(sock_net(sk), rm_type);
848
835
}
849
-
if (rm_type == MPTCP_MIB_RMSUBFLOW)
850
-
__set_bit(rm_id ? rm_id : msk->mpc_endpoint_id, msk->pm.id_avail_bitmap);
851
-
else if (rm_type == MPTCP_MIB_RMADDR)
836
+
837
+
if (rm_type == MPTCP_MIB_RMADDR)
852
838
__MPTCP_INC_STATS(sock_net(sk), rm_type);
839
+
853
840
if (!removed)
854
841
continue;
855
842
856
843
if (!mptcp_pm_is_kernel(msk))
857
844
continue;
858
845
859
-
if (rm_type == MPTCP_MIB_RMADDR) {
860
-
msk->pm.add_addr_accepted--;
861
-
WRITE_ONCE(msk->pm.accept_addr, true);
862
-
} else if (rm_type == MPTCP_MIB_RMSUBFLOW) {
863
-
msk->pm.local_addr_used--;
846
+
if (rm_type == MPTCP_MIB_RMADDR && rm_id &&
847
+
!WARN_ON_ONCE(msk->pm.add_addr_accepted == 0)) {
848
+
/* Note: if the subflow has been closed before, this
849
+
* add_addr_accepted counter will not be decremented.
850
+
*/
851
+
if (--msk->pm.add_addr_accepted < mptcp_pm_get_add_addr_accept_max(msk))
852
+
WRITE_ONCE(msk->pm.accept_addr, true);
864
853
}
865
854
}
866
855
}
···
872
857
mptcp_pm_nl_rm_addr_or_subflow(msk, &msk->pm.rm_list_rx, MPTCP_MIB_RMADDR);
873
858
}
874
859
875
-
void mptcp_pm_nl_rm_subflow_received(struct mptcp_sock *msk,
876
-
const struct mptcp_rm_list *rm_list)
860
+
static void mptcp_pm_nl_rm_subflow_received(struct mptcp_sock *msk,
861
+
const struct mptcp_rm_list *rm_list)
877
862
{
878
863
mptcp_pm_nl_rm_addr_or_subflow(msk, rm_list, MPTCP_MIB_RMSUBFLOW);
879
864
}
···
1408
1393
struct sock *sk = (struct sock *)msk;
1409
1394
struct net *net = sock_net(sk);
1410
1395
1396
+
/* No entries with ID 0 */
1397
+
if (id == 0)
1398
+
return 0;
1399
+
1411
1400
rcu_read_lock();
1412
1401
entry = __lookup_addr_by_id(pm_nl_get_pernet(net), id);
1413
1402
if (entry) {
···
1450
1431
ret = remove_anno_list_by_saddr(msk, addr);
1451
1432
if (ret || force) {
1452
1433
spin_lock_bh(&msk->pm.lock);
1453
-
msk->pm.add_addr_signaled -= ret;
1434
+
if (ret) {
1435
+
__set_bit(addr->id, msk->pm.id_avail_bitmap);
1436
+
msk->pm.add_addr_signaled--;
1437
+
}
1454
1438
mptcp_pm_remove_addr(msk, &list);
1455
1439
spin_unlock_bh(&msk->pm.lock);
1456
1440
}
1457
1441
return ret;
1442
+
}
1443
+
1444
+
static void __mark_subflow_endp_available(struct mptcp_sock *msk, u8 id)
1445
+
{
1446
+
/* If it was marked as used, and not ID 0, decrement local_addr_used */
1447
+
if (!__test_and_set_bit(id ? : msk->mpc_endpoint_id, msk->pm.id_avail_bitmap) &&
1448
+
id && !WARN_ON_ONCE(msk->pm.local_addr_used == 0))
1449
+
msk->pm.local_addr_used--;
1458
1450
}
1459
1451
1460
1452
static int mptcp_nl_remove_subflow_and_signal_addr(struct net *net,
···
1496
1466
remove_subflow = lookup_subflow_by_saddr(&msk->conn_list, addr);
1497
1467
mptcp_pm_remove_anno_addr(msk, addr, remove_subflow &&
1498
1468
!(entry->flags & MPTCP_PM_ADDR_FLAG_IMPLICIT));
1499
-
if (remove_subflow)
1500
-
mptcp_pm_remove_subflow(msk, &list);
1469
+
1470
+
if (remove_subflow) {
1471
+
spin_lock_bh(&msk->pm.lock);
1472
+
mptcp_pm_nl_rm_subflow_received(msk, &list);
1473
+
spin_unlock_bh(&msk->pm.lock);
1474
+
}
1475
+
1476
+
if (entry->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW) {
1477
+
spin_lock_bh(&msk->pm.lock);
1478
+
__mark_subflow_endp_available(msk, list.ids[0]);
1479
+
spin_unlock_bh(&msk->pm.lock);
1480
+
}
1481
+
1501
1482
release_sock(sk);
1502
1483
1503
1484
next:
···
1543
1502
spin_lock_bh(&msk->pm.lock);
1544
1503
mptcp_pm_remove_addr(msk, &list);
1545
1504
mptcp_pm_nl_rm_subflow_received(msk, &list);
1505
+
__mark_subflow_endp_available(msk, 0);
1546
1506
spin_unlock_bh(&msk->pm.lock);
1547
1507
release_sock(sk);
1548
1508
···
1647
1605
alist.ids[alist.nr++] = entry->addr.id;
1648
1606
}
1649
1607
1608
+
spin_lock_bh(&msk->pm.lock);
1650
1609
if (alist.nr) {
1651
-
spin_lock_bh(&msk->pm.lock);
1652
1610
msk->pm.add_addr_signaled -= alist.nr;
1653
1611
mptcp_pm_remove_addr(msk, &alist);
1654
-
spin_unlock_bh(&msk->pm.lock);
1655
1612
}
1656
1613
if (slist.nr)
1657
-
mptcp_pm_remove_subflow(msk, &slist);
1614
+
mptcp_pm_nl_rm_subflow_received(msk, &slist);
1615
+
/* Reset counters: maybe some subflows have been removed before */
1616
+
bitmap_fill(msk->pm.id_avail_bitmap, MPTCP_PM_MAX_ADDR_ID + 1);
1617
+
msk->pm.local_addr_used = 0;
1618
+
spin_unlock_bh(&msk->pm.lock);
1658
1619
}
1659
1620
1660
1621
static void mptcp_nl_remove_addrs_list(struct net *net,
···
1945
1900
1946
1901
spin_lock_bh(&msk->pm.lock);
1947
1902
mptcp_pm_nl_rm_subflow_received(msk, &list);
1903
+
__mark_subflow_endp_available(msk, list.ids[0]);
1948
1904
mptcp_pm_create_subflow_or_signal_addr(msk);
1949
1905
spin_unlock_bh(&msk->pm.lock);
1950
1906
}
-3
net/mptcp/protocol.h
-3
net/mptcp/protocol.h
···
1026
1026
const struct mptcp_addr_info *addr,
1027
1027
bool echo);
1028
1028
int mptcp_pm_remove_addr(struct mptcp_sock *msk, const struct mptcp_rm_list *rm_list);
1029
-
int mptcp_pm_remove_subflow(struct mptcp_sock *msk, const struct mptcp_rm_list *rm_list);
1030
1029
void mptcp_pm_remove_addrs(struct mptcp_sock *msk, struct list_head *rm_list);
1031
1030
1032
1031
void mptcp_free_local_addr_list(struct mptcp_sock *msk);
···
1132
1133
1133
1134
void __init mptcp_pm_nl_init(void);
1134
1135
void mptcp_pm_nl_work(struct mptcp_sock *msk);
1135
-
void mptcp_pm_nl_rm_subflow_received(struct mptcp_sock *msk,
1136
-
const struct mptcp_rm_list *rm_list);
1137
1136
unsigned int mptcp_pm_get_add_addr_signal_max(const struct mptcp_sock *msk);
1138
1137
unsigned int mptcp_pm_get_add_addr_accept_max(const struct mptcp_sock *msk);
1139
1138
unsigned int mptcp_pm_get_subflows_max(const struct mptcp_sock *msk);
+3
net/netfilter/nf_flow_table_inet.c
+3
net/netfilter/nf_flow_table_inet.c
+3
net/netfilter/nf_flow_table_ip.c
+3
net/netfilter/nf_flow_table_ip.c
···
281
281
282
282
switch (skb->protocol) {
283
283
case htons(ETH_P_8021Q):
284
+
if (!pskb_may_pull(skb, skb_mac_offset(skb) + sizeof(*veth)))
285
+
return false;
286
+
284
287
veth = (struct vlan_ethhdr *)skb_mac_header(skb);
285
288
if (veth->h_vlan_encapsulated_proto == proto) {
286
289
*offset += VLAN_HLEN;
+7
-2
net/netfilter/nft_counter.c
+7
-2
net/netfilter/nft_counter.c
···
107
107
struct nft_counter *total)
108
108
{
109
109
struct nft_counter *this_cpu;
110
+
seqcount_t *myseq;
110
111
111
112
local_bh_disable();
112
113
this_cpu = this_cpu_ptr(priv->counter);
114
+
myseq = this_cpu_ptr(&nft_counter_seq);
115
+
116
+
write_seqcount_begin(myseq);
113
117
this_cpu->packets -= total->packets;
114
118
this_cpu->bytes -= total->bytes;
119
+
write_seqcount_end(myseq);
115
120
local_bh_enable();
116
121
}
117
122
···
270
265
struct nft_counter *this_cpu;
271
266
seqcount_t *myseq;
272
267
273
-
preempt_disable();
268
+
local_bh_disable();
274
269
this_cpu = this_cpu_ptr(priv->counter);
275
270
myseq = this_cpu_ptr(&nft_counter_seq);
276
271
···
278
273
this_cpu->packets += stats->pkts;
279
274
this_cpu->bytes += stats->bytes;
280
275
write_seqcount_end(myseq);
281
-
preempt_enable();
276
+
local_bh_enable();
282
277
}
283
278
284
279
void nft_counter_init_seqcount(void)
+1
-1
net/openvswitch/datapath.c
+1
-1
net/openvswitch/datapath.c
+29
-18
net/sched/sch_netem.c
+29
-18
net/sched/sch_netem.c
···
446
446
struct netem_sched_data *q = qdisc_priv(sch);
447
447
/* We don't fill cb now as skb_unshare() may invalidate it */
448
448
struct netem_skb_cb *cb;
449
-
struct sk_buff *skb2;
449
+
struct sk_buff *skb2 = NULL;
450
450
struct sk_buff *segs = NULL;
451
451
unsigned int prev_len = qdisc_pkt_len(skb);
452
452
int count = 1;
453
-
int rc = NET_XMIT_SUCCESS;
454
-
int rc_drop = NET_XMIT_DROP;
455
453
456
454
/* Do not fool qdisc_drop_all() */
457
455
skb->prev = NULL;
···
478
480
skb_orphan_partial(skb);
479
481
480
482
/*
481
-
* If we need to duplicate packet, then re-insert at top of the
482
-
* qdisc tree, since parent queuer expects that only one
483
-
* skb will be queued.
483
+
* If we need to duplicate packet, then clone it before
484
+
* original is modified.
484
485
*/
485
-
if (count > 1 && (skb2 = skb_clone(skb, GFP_ATOMIC)) != NULL) {
486
-
struct Qdisc *rootq = qdisc_root_bh(sch);
487
-
u32 dupsave = q->duplicate; /* prevent duplicating a dup... */
488
-
489
-
q->duplicate = 0;
490
-
rootq->enqueue(skb2, rootq, to_free);
491
-
q->duplicate = dupsave;
492
-
rc_drop = NET_XMIT_SUCCESS;
493
-
}
486
+
if (count > 1)
487
+
skb2 = skb_clone(skb, GFP_ATOMIC);
494
488
495
489
/*
496
490
* Randomized packet corruption.
···
494
504
if (skb_is_gso(skb)) {
495
505
skb = netem_segment(skb, sch, to_free);
496
506
if (!skb)
497
-
return rc_drop;
507
+
goto finish_segs;
508
+
498
509
segs = skb->next;
499
510
skb_mark_not_on_list(skb);
500
511
qdisc_skb_cb(skb)->pkt_len = skb->len;
···
521
530
/* re-link segs, so that qdisc_drop_all() frees them all */
522
531
skb->next = segs;
523
532
qdisc_drop_all(skb, sch, to_free);
524
-
return rc_drop;
533
+
if (skb2)
534
+
__qdisc_drop(skb2, to_free);
535
+
return NET_XMIT_DROP;
536
+
}
537
+
538
+
/*
539
+
* If doing duplication then re-insert at top of the
540
+
* qdisc tree, since parent queuer expects that only one
541
+
* skb will be queued.
542
+
*/
543
+
if (skb2) {
544
+
struct Qdisc *rootq = qdisc_root_bh(sch);
545
+
u32 dupsave = q->duplicate; /* prevent duplicating a dup... */
546
+
547
+
q->duplicate = 0;
548
+
rootq->enqueue(skb2, rootq, to_free);
549
+
q->duplicate = dupsave;
550
+
skb2 = NULL;
525
551
}
526
552
527
553
qdisc_qstats_backlog_inc(sch, skb);
···
609
601
}
610
602
611
603
finish_segs:
604
+
if (skb2)
605
+
__qdisc_drop(skb2, to_free);
606
+
612
607
if (segs) {
613
608
unsigned int len, last_len;
614
-
int nb;
609
+
int rc, nb;
615
610
616
611
len = skb ? skb->len : 0;
617
612
nb = skb ? 1 : 0;
+4
-4
rust/Makefile
+4
-4
rust/Makefile
···
227
227
-fno-reorder-blocks -fno-allow-store-data-races -fasan-shadow-offset=% \
228
228
-fzero-call-used-regs=% -fno-stack-clash-protection \
229
229
-fno-inline-functions-called-once -fsanitize=bounds-strict \
230
-
-fstrict-flex-arrays=% \
230
+
-fstrict-flex-arrays=% -fmin-function-alignment=% \
231
231
--param=% --param asan-%
232
232
233
233
# Derived from `scripts/Makefile.clang`.
···
350
350
$(Q)$(srctree)/scripts/generate_rust_analyzer.py \
351
351
--cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \
352
352
$(realpath $(srctree)) $(realpath $(objtree)) \
353
-
$(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \
353
+
$(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \
354
354
$(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json
355
355
356
356
redirect-intrinsics = \
357
-
__addsf3 __eqsf2 __gesf2 __lesf2 __ltsf2 __mulsf3 __nesf2 __unordsf2 \
358
-
__adddf3 __ledf2 __ltdf2 __muldf3 __unorddf2 \
357
+
__addsf3 __eqsf2 __extendsfdf2 __gesf2 __lesf2 __ltsf2 __mulsf3 __nesf2 __truncdfsf2 __unordsf2 \
358
+
__adddf3 __eqdf2 __ledf2 __ltdf2 __muldf3 __unorddf2 \
359
359
__muloti4 __multi3 \
360
360
__udivmodti4 __udivti3 __umodti3
361
361
+3
rust/compiler_builtins.rs
+3
rust/compiler_builtins.rs
···
40
40
define_panicking_intrinsics!("`f32` should not be used", {
41
41
__addsf3,
42
42
__eqsf2,
43
+
__extendsfdf2,
43
44
__gesf2,
44
45
__lesf2,
45
46
__ltsf2,
46
47
__mulsf3,
47
48
__nesf2,
49
+
__truncdfsf2,
48
50
__unordsf2,
49
51
});
50
52
51
53
define_panicking_intrinsics!("`f64` should not be used", {
52
54
__adddf3,
55
+
__eqdf2,
53
56
__ledf2,
54
57
__ltdf2,
55
58
__muldf3,
+1
-1
rust/macros/lib.rs
+1
-1
rust/macros/lib.rs
···
94
94
/// - `license`: ASCII string literal of the license of the kernel module (required).
95
95
/// - `alias`: array of ASCII string literals of the alias names of the kernel module.
96
96
/// - `firmware`: array of ASCII string literals of the firmware files of
97
-
/// the kernel module.
97
+
/// the kernel module.
98
98
#[proc_macro]
99
99
pub fn module(ts: TokenStream) -> TokenStream {
100
100
module::module(ts)
-1
samples/trace_events/trace_custom_sched.c
-1
samples/trace_events/trace_custom_sched.c
+1
-1
scripts/Makefile.build
+1
-1
scripts/Makefile.build
···
487
487
need-modorder=$(if $(filter $@/modules.order, $(subdir-modorder)),1) \
488
488
$(filter $@/%, $(single-subdir-goals))
489
489
490
-
# Add FORCE to the prequisites of a target to force it to be always rebuilt.
490
+
# Add FORCE to the prerequisites of a target to force it to be always rebuilt.
491
491
# ---------------------------------------------------------------------------
492
492
493
493
PHONY += FORCE
+10
-18
scripts/Makefile.lib
+10
-18
scripts/Makefile.lib
···
400
400
$(obj)/%.dtbo.S: $(obj)/%.dtbo FORCE
401
401
$(call if_changed,wrap_S_dtb)
402
402
403
-
quiet_cmd_dtc = DTC $@
403
+
quiet_dtb_check_tag = $(if $(dtb-check-enabled),[C], )
404
+
cmd_dtb_check = $(if $(dtb-check-enabled),; $(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@ || true)
405
+
406
+
quiet_cmd_dtc = DTC $(quiet_dtb_check_tag) $@
404
407
cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \
405
408
$(DTC) -o $@ -b 0 \
406
409
$(addprefix -i,$(dir $<) $(DTC_INCLUDE)) $(DTC_FLAGS) \
407
410
-d $(depfile).dtc.tmp $(dtc-tmp) ; \
408
-
cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
409
-
410
-
DT_CHECK_CMD = $(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA)
411
+
cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile) \
412
+
$(cmd_dtb_check)
411
413
412
414
# NOTE:
413
415
# Do not replace $(filter %.dtb %.dtbo, $^) with $(real-prereqs). When a single
414
416
# DTB is turned into a multi-blob DTB, $^ will contain header file dependencies
415
417
# recorded in the .*.cmd file.
416
-
ifneq ($(CHECK_DTBS),)
417
-
quiet_cmd_fdtoverlay = DTOVLCH $@
418
-
cmd_fdtoverlay = $(objtree)/scripts/dtc/fdtoverlay -o $@ -i $(filter %.dtb %.dtbo, $^) ; $(DT_CHECK_CMD) $@ || true
419
-
else
420
-
quiet_cmd_fdtoverlay = DTOVL $@
421
-
cmd_fdtoverlay = $(objtree)/scripts/dtc/fdtoverlay -o $@ -i $(filter %.dtb %.dtbo, $^)
422
-
endif
418
+
quiet_cmd_fdtoverlay = OVL $(quiet_dtb_check_tag) $@
419
+
cmd_fdtoverlay = $(objtree)/scripts/dtc/fdtoverlay -o $@ -i $(filter %.dtb %.dtbo, $^) $(cmd_dtb_check)
423
420
424
421
$(multi-dtb-y): FORCE
425
422
$(call if_changed,fdtoverlay)
···
427
430
DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
428
431
DT_BINDING_DIR := Documentation/devicetree/bindings
429
432
DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
430
-
431
-
quiet_cmd_dtb = DTC_CHK $@
432
-
cmd_dtb = $(cmd_dtc) ; $(DT_CHECK_CMD) $@ || true
433
-
else
434
-
quiet_cmd_dtb = $(quiet_cmd_dtc)
435
-
cmd_dtb = $(cmd_dtc)
433
+
dtb-check-enabled = $(if $(filter %.dtb, $@),y)
436
434
endif
437
435
438
436
$(obj)/%.dtb: $(obj)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
439
-
$(call if_changed_dep,dtb)
437
+
$(call if_changed_dep,dtc)
440
438
441
439
$(obj)/%.dtbo: $(src)/%.dtso $(DTC) FORCE
442
440
$(call if_changed_dep,dtc)
+1
-1
scripts/Makefile.modfinal
+1
-1
scripts/Makefile.modfinal
···
62
62
63
63
targets += $(modules:%.o=%.ko) $(modules:%.o=%.mod.o)
64
64
65
-
# Add FORCE to the prequisites of a target to force it to be always rebuilt.
65
+
# Add FORCE to the prerequisites of a target to force it to be always rebuilt.
66
66
# ---------------------------------------------------------------------------
67
67
68
68
PHONY += FORCE
+1
-1
scripts/Makefile.vmlinux
+1
-1
scripts/Makefile.vmlinux
···
33
33
vmlinux: scripts/link-vmlinux.sh vmlinux.o $(KBUILD_LDS) FORCE
34
34
+$(call if_changed_dep,link_vmlinux)
35
35
36
-
# Add FORCE to the prequisites of a target to force it to be always rebuilt.
36
+
# Add FORCE to the prerequisites of a target to force it to be always rebuilt.
37
37
# ---------------------------------------------------------------------------
38
38
39
39
PHONY += FORCE
+1
-1
scripts/Makefile.vmlinux_o
+1
-1
scripts/Makefile.vmlinux_o
···
87
87
modules.builtin: modules.builtin.modinfo FORCE
88
88
$(call if_changed,modules_builtin)
89
89
90
-
# Add FORCE to the prequisites of a target to force it to be always rebuilt.
90
+
# Add FORCE to the prerequisites of a target to force it to be always rebuilt.
91
91
# ---------------------------------------------------------------------------
92
92
93
93
PHONY += FORCE
-4
scripts/gcc-plugins/randomize_layout_plugin.c
-4
scripts/gcc-plugins/randomize_layout_plugin.c
···
19
19
#include "gcc-common.h"
20
20
#include "randomize_layout_seed.h"
21
21
22
-
#if BUILDING_GCC_MAJOR < 4 || (BUILDING_GCC_MAJOR == 4 && BUILDING_GCC_MINOR < 7)
23
-
#error "The RANDSTRUCT plugin requires GCC 4.7 or newer."
24
-
#endif
25
-
26
22
#define ORIG_TYPE_NAME(node) \
27
23
(TYPE_NAME(TYPE_MAIN_VARIANT(node)) != NULL_TREE ? ((const unsigned char *)IDENTIFIER_POINTER(TYPE_NAME(TYPE_MAIN_VARIANT(node)))) : (const unsigned char *)"anonymous")
28
24
+5
-1
scripts/generate_rust_analyzer.py
+5
-1
scripts/generate_rust_analyzer.py
···
145
145
parser.add_argument('--cfgs', action='append', default=[])
146
146
parser.add_argument("srctree", type=pathlib.Path)
147
147
parser.add_argument("objtree", type=pathlib.Path)
148
+
parser.add_argument("sysroot", type=pathlib.Path)
148
149
parser.add_argument("sysroot_src", type=pathlib.Path)
149
150
parser.add_argument("exttree", type=pathlib.Path, nargs="?")
150
151
args = parser.parse_args()
···
155
154
level=logging.INFO if args.verbose else logging.WARNING
156
155
)
157
156
157
+
# Making sure that the `sysroot` and `sysroot_src` belong to the same toolchain.
158
+
assert args.sysroot in args.sysroot_src.parents
159
+
158
160
rust_project = {
159
161
"crates": generate_crates(args.srctree, args.objtree, args.sysroot_src, args.exttree, args.cfgs),
160
-
"sysroot_src": str(args.sysroot_src),
162
+
"sysroot": str(args.sysroot),
161
163
}
162
164
163
165
json.dump(rust_project, sys.stdout, sort_keys=True, indent=4)
+2
-2
scripts/generate_rust_target.rs
+2
-2
scripts/generate_rust_target.rs
···
162
162
"data-layout",
163
163
"e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
164
164
);
165
-
let mut features = "-3dnow,-3dnowa,-mmx,+soft-float".to_string();
165
+
let mut features = "-mmx,+soft-float".to_string();
166
166
if cfg.has("MITIGATION_RETPOLINE") {
167
167
features += ",+retpoline-external-thunk";
168
168
}
···
179
179
"data-layout",
180
180
"e-m:e-p:32:32-p270:32:32-p271:32:32-p272:64:64-i128:128-f64:32:64-f80:32-n8:16:32-S128",
181
181
);
182
-
let mut features = "-3dnow,-3dnowa,-mmx,+soft-float".to_string();
182
+
let mut features = "-mmx,+soft-float".to_string();
183
183
if cfg.has("MITIGATION_RETPOLINE") {
184
184
features += ",+retpoline-external-thunk";
185
185
}
+2
-29
scripts/kallsyms.c
+2
-29
scripts/kallsyms.c
···
5
5
* This software may be used and distributed according to the terms
6
6
* of the GNU General Public License, incorporated herein by reference.
7
7
*
8
-
* Usage: kallsyms [--all-symbols] [--absolute-percpu]
9
-
* [--lto-clang] in.map > out.S
8
+
* Usage: kallsyms [--all-symbols] [--absolute-percpu] in.map > out.S
10
9
*
11
10
* Table compression uses all the unused char codes on the symbols and
12
11
* maps these to the most used substrings (tokens). For instance, it might
···
61
62
static unsigned int table_size, table_cnt;
62
63
static int all_symbols;
63
64
static int absolute_percpu;
64
-
static int lto_clang;
65
65
66
66
static int token_profit[0x10000];
67
67
···
71
73
72
74
static void usage(void)
73
75
{
74
-
fprintf(stderr, "Usage: kallsyms [--all-symbols] [--absolute-percpu] "
75
-
"[--lto-clang] in.map > out.S\n");
76
+
fprintf(stderr, "Usage: kallsyms [--all-symbols] [--absolute-percpu] in.map > out.S\n");
76
77
exit(1);
77
78
}
78
79
···
341
344
return s->percpu_absolute;
342
345
}
343
346
344
-
static void cleanup_symbol_name(char *s)
345
-
{
346
-
char *p;
347
-
348
-
/*
349
-
* ASCII[.] = 2e
350
-
* ASCII[0-9] = 30,39
351
-
* ASCII[A-Z] = 41,5a
352
-
* ASCII[_] = 5f
353
-
* ASCII[a-z] = 61,7a
354
-
*
355
-
* As above, replacing the first '.' in ".llvm." with '\0' does not
356
-
* affect the main sorting, but it helps us with subsorting.
357
-
*/
358
-
p = strstr(s, ".llvm.");
359
-
if (p)
360
-
*p = '\0';
361
-
}
362
-
363
347
static int compare_names(const void *a, const void *b)
364
348
{
365
349
int ret;
···
503
525
output_label("kallsyms_relative_base");
504
526
output_address(relative_base);
505
527
printf("\n");
506
-
507
-
if (lto_clang)
508
-
for (i = 0; i < table_cnt; i++)
509
-
cleanup_symbol_name((char *)table[i]->sym);
510
528
511
529
sort_symbols_by_name();
512
530
output_label("kallsyms_seqs_of_names");
···
781
807
static const struct option long_options[] = {
782
808
{"all-symbols", no_argument, &all_symbols, 1},
783
809
{"absolute-percpu", no_argument, &absolute_percpu, 1},
784
-
{"lto-clang", no_argument, <o_clang, 1},
785
810
{},
786
811
};
787
812
+2
scripts/kconfig/merge_config.sh
+2
scripts/kconfig/merge_config.sh
+2
-5
scripts/link-vmlinux.sh
+2
-5
scripts/link-vmlinux.sh
···
156
156
kallsymopt="${kallsymopt} --absolute-percpu"
157
157
fi
158
158
159
-
if is_enabled CONFIG_LTO_CLANG; then
160
-
kallsymopt="${kallsymopt} --lto-clang"
161
-
fi
162
-
163
159
info KSYMS "${2}.S"
164
160
scripts/kallsyms ${kallsymopt} "${1}" > "${2}.S"
165
161
···
215
219
strip_debug=
216
220
217
221
if is_enabled CONFIG_KALLSYMS; then
218
-
kallsyms /dev/null .tmp_vmlinux0.kallsyms
222
+
truncate -s0 .tmp_vmlinux.kallsyms0.syms
223
+
kallsyms .tmp_vmlinux.kallsyms0.syms .tmp_vmlinux0.kallsyms
219
224
fi
220
225
221
226
if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
+22
-13
security/keys/trusted-keys/trusted_dcp.c
+22
-13
security/keys/trusted-keys/trusted_dcp.c
···
186
186
return ret;
187
187
}
188
188
189
-
static int decrypt_blob_key(u8 *key)
189
+
static int decrypt_blob_key(u8 *encrypted_key, u8 *plain_key)
190
190
{
191
-
return do_dcp_crypto(key, key, false);
191
+
return do_dcp_crypto(encrypted_key, plain_key, false);
192
192
}
193
193
194
-
static int encrypt_blob_key(u8 *key)
194
+
static int encrypt_blob_key(u8 *plain_key, u8 *encrypted_key)
195
195
{
196
-
return do_dcp_crypto(key, key, true);
196
+
return do_dcp_crypto(plain_key, encrypted_key, true);
197
197
}
198
198
199
199
static int trusted_dcp_seal(struct trusted_key_payload *p, char *datablob)
200
200
{
201
201
struct dcp_blob_fmt *b = (struct dcp_blob_fmt *)p->blob;
202
202
int blen, ret;
203
+
u8 plain_blob_key[AES_KEYSIZE_128];
203
204
204
205
blen = calc_blob_len(p->key_len);
205
206
if (blen > MAX_BLOB_SIZE)
···
208
207
209
208
b->fmt_version = DCP_BLOB_VERSION;
210
209
get_random_bytes(b->nonce, AES_KEYSIZE_128);
211
-
get_random_bytes(b->blob_key, AES_KEYSIZE_128);
210
+
get_random_bytes(plain_blob_key, AES_KEYSIZE_128);
212
211
213
-
ret = do_aead_crypto(p->key, b->payload, p->key_len, b->blob_key,
212
+
ret = do_aead_crypto(p->key, b->payload, p->key_len, plain_blob_key,
214
213
b->nonce, true);
215
214
if (ret) {
216
215
pr_err("Unable to encrypt blob payload: %i\n", ret);
217
-
return ret;
216
+
goto out;
218
217
}
219
218
220
-
ret = encrypt_blob_key(b->blob_key);
219
+
ret = encrypt_blob_key(plain_blob_key, b->blob_key);
221
220
if (ret) {
222
221
pr_err("Unable to encrypt blob key: %i\n", ret);
223
-
return ret;
222
+
goto out;
224
223
}
225
224
226
-
b->payload_len = get_unaligned_le32(&p->key_len);
225
+
put_unaligned_le32(p->key_len, &b->payload_len);
227
226
p->blob_len = blen;
228
-
return 0;
227
+
ret = 0;
228
+
229
+
out:
230
+
memzero_explicit(plain_blob_key, sizeof(plain_blob_key));
231
+
232
+
return ret;
229
233
}
230
234
231
235
static int trusted_dcp_unseal(struct trusted_key_payload *p, char *datablob)
232
236
{
233
237
struct dcp_blob_fmt *b = (struct dcp_blob_fmt *)p->blob;
234
238
int blen, ret;
239
+
u8 plain_blob_key[AES_KEYSIZE_128];
235
240
236
241
if (b->fmt_version != DCP_BLOB_VERSION) {
237
242
pr_err("DCP blob has bad version: %i, expected %i\n",
···
255
248
goto out;
256
249
}
257
250
258
-
ret = decrypt_blob_key(b->blob_key);
251
+
ret = decrypt_blob_key(b->blob_key, plain_blob_key);
259
252
if (ret) {
260
253
pr_err("Unable to decrypt blob key: %i\n", ret);
261
254
goto out;
262
255
}
263
256
264
257
ret = do_aead_crypto(b->payload, p->key, p->key_len + DCP_BLOB_AUTHLEN,
265
-
b->blob_key, b->nonce, false);
258
+
plain_blob_key, b->nonce, false);
266
259
if (ret) {
267
260
pr_err("Unwrap of DCP payload failed: %i\n", ret);
268
261
goto out;
···
270
263
271
264
ret = 0;
272
265
out:
266
+
memzero_explicit(plain_blob_key, sizeof(plain_blob_key));
267
+
273
268
return ret;
274
269
}
275
270
+1
-1
sound/core/timer.c
+1
-1
sound/core/timer.c
···
547
547
/* check the actual time for the start tick;
548
548
* bail out as error if it's way too low (< 100us)
549
549
*/
550
-
if (start) {
550
+
if (start && !(timer->hw.flags & SNDRV_TIMER_HW_SLAVE)) {
551
551
if ((u64)snd_timer_hw_resolution(timer) * ticks < 100000)
552
552
return -EINVAL;
553
553
}
+1
-1
sound/pci/hda/cs35l41_hda.c
+1
-1
sound/pci/hda/cs35l41_hda.c
···
134
134
};
135
135
136
136
static const struct cs_dsp_client_ops client_ops = {
137
-
.control_remove = hda_cs_dsp_control_remove,
137
+
/* cs_dsp requires the client to provide this even if it is empty */
138
138
};
139
139
140
140
static int cs35l41_request_tuning_param_file(struct cs35l41_hda *cs35l41, char *tuning_filename,
+1
-1
sound/pci/hda/cs35l56_hda.c
+1
-1
sound/pci/hda/cs35l56_hda.c
···
413
413
}
414
414
415
415
static const struct cs_dsp_client_ops cs35l56_hda_client_ops = {
416
-
.control_remove = hda_cs_dsp_control_remove,
416
+
/* cs_dsp requires the client to provide this even if it is empty */
417
417
};
418
418
419
419
static int cs35l56_hda_request_firmware_file(struct cs35l56_hda *cs35l56,
+99
-1
sound/pci/hda/patch_realtek.c
+99
-1
sound/pci/hda/patch_realtek.c
···
11
11
*/
12
12
13
13
#include <linux/acpi.h>
14
+
#include <linux/cleanup.h>
14
15
#include <linux/init.h>
15
16
#include <linux/delay.h>
16
17
#include <linux/slab.h>
17
18
#include <linux/pci.h>
18
19
#include <linux/dmi.h>
19
20
#include <linux/module.h>
21
+
#include <linux/i2c.h>
20
22
#include <linux/input.h>
21
23
#include <linux/leds.h>
22
24
#include <linux/ctype.h>
25
+
#include <linux/spi/spi.h>
23
26
#include <sound/core.h>
24
27
#include <sound/jack.h>
25
28
#include <sound/hda_codec.h>
···
586
583
switch (codec->core.vendor_id) {
587
584
case 0x10ec0236:
588
585
case 0x10ec0256:
589
-
case 0x10ec0257:
590
586
case 0x19e58326:
591
587
case 0x10ec0283:
592
588
case 0x10ec0285:
···
6858
6856
}
6859
6857
}
6860
6858
6859
+
static void cs35lxx_autodet_fixup(struct hda_codec *cdc,
6860
+
const struct hda_fixup *fix,
6861
+
int action)
6862
+
{
6863
+
struct device *dev = hda_codec_dev(cdc);
6864
+
struct acpi_device *adev;
6865
+
struct fwnode_handle *fwnode __free(fwnode_handle) = NULL;
6866
+
const char *bus = NULL;
6867
+
static const struct {
6868
+
const char *hid;
6869
+
const char *name;
6870
+
} acpi_ids[] = {{ "CSC3554", "cs35l54-hda" },
6871
+
{ "CSC3556", "cs35l56-hda" },
6872
+
{ "CSC3557", "cs35l57-hda" }};
6873
+
char *match;
6874
+
int i, count = 0, count_devindex = 0;
6875
+
6876
+
switch (action) {
6877
+
case HDA_FIXUP_ACT_PRE_PROBE:
6878
+
for (i = 0; i < ARRAY_SIZE(acpi_ids); ++i) {
6879
+
adev = acpi_dev_get_first_match_dev(acpi_ids[i].hid, NULL, -1);
6880
+
if (adev)
6881
+
break;
6882
+
}
6883
+
if (!adev) {
6884
+
dev_err(dev, "Failed to find ACPI entry for a Cirrus Amp\n");
6885
+
return;
6886
+
}
6887
+
6888
+
count = i2c_acpi_client_count(adev);
6889
+
if (count > 0) {
6890
+
bus = "i2c";
6891
+
} else {
6892
+
count = acpi_spi_count_resources(adev);
6893
+
if (count > 0)
6894
+
bus = "spi";
6895
+
}
6896
+
6897
+
fwnode = fwnode_handle_get(acpi_fwnode_handle(adev));
6898
+
acpi_dev_put(adev);
6899
+
6900
+
if (!bus) {
6901
+
dev_err(dev, "Did not find any buses for %s\n", acpi_ids[i].hid);
6902
+
return;
6903
+
}
6904
+
6905
+
if (!fwnode) {
6906
+
dev_err(dev, "Could not get fwnode for %s\n", acpi_ids[i].hid);
6907
+
return;
6908
+
}
6909
+
6910
+
/*
6911
+
* When available the cirrus,dev-index property is an accurate
6912
+
* count of the amps in a system and is used in preference to
6913
+
* the count of bus devices that can contain additional address
6914
+
* alias entries.
6915
+
*/
6916
+
count_devindex = fwnode_property_count_u32(fwnode, "cirrus,dev-index");
6917
+
if (count_devindex > 0)
6918
+
count = count_devindex;
6919
+
6920
+
match = devm_kasprintf(dev, GFP_KERNEL, "-%%s:00-%s.%%d", acpi_ids[i].name);
6921
+
if (!match)
6922
+
return;
6923
+
dev_info(dev, "Found %d %s on %s (%s)\n", count, acpi_ids[i].hid, bus, match);
6924
+
comp_generic_fixup(cdc, action, bus, acpi_ids[i].hid, match, count);
6925
+
6926
+
break;
6927
+
case HDA_FIXUP_ACT_FREE:
6928
+
/*
6929
+
* Pass the action on to comp_generic_fixup() so that
6930
+
* hda_component_manager functions can be called in just once
6931
+
* place. In this context the bus, hid, match_str or count
6932
+
* values do not need to be calculated.
6933
+
*/
6934
+
comp_generic_fixup(cdc, action, NULL, NULL, NULL, 0);
6935
+
break;
6936
+
}
6937
+
}
6938
+
6861
6939
static void cs35l41_fixup_i2c_two(struct hda_codec *cdc, const struct hda_fixup *fix, int action)
6862
6940
{
6863
6941
comp_generic_fixup(cdc, action, "i2c", "CSC3551", "-%s:00-cs35l41-hda.%d", 2);
···
7610
7528
ALC256_FIXUP_CHROME_BOOK,
7611
7529
ALC287_FIXUP_LENOVO_14ARP8_LEGION_IAH7,
7612
7530
ALC287_FIXUP_LENOVO_SSID_17AA3820,
7531
+
ALCXXX_FIXUP_CS35LXX,
7613
7532
};
7614
7533
7615
7534
/* A special fixup for Lenovo C940 and Yoga Duet 7;
···
9940
9857
.type = HDA_FIXUP_FUNC,
9941
9858
.v.func = alc287_fixup_lenovo_ssid_17aa3820,
9942
9859
},
9860
+
[ALCXXX_FIXUP_CS35LXX] = {
9861
+
.type = HDA_FIXUP_FUNC,
9862
+
.v.func = cs35lxx_autodet_fixup,
9863
+
},
9943
9864
};
9944
9865
9945
9866
static const struct snd_pci_quirk alc269_fixup_tbl[] = {
···
10358
10271
SND_PCI_QUIRK(0x103c, 0x8cdf, "HP SnowWhite", ALC287_FIXUP_CS35L41_I2C_2_HP_GPIO_LED),
10359
10272
SND_PCI_QUIRK(0x103c, 0x8ce0, "HP SnowWhite", ALC287_FIXUP_CS35L41_I2C_2_HP_GPIO_LED),
10360
10273
SND_PCI_QUIRK(0x103c, 0x8cf5, "HP ZBook Studio 16", ALC245_FIXUP_CS35L41_SPI_4_HP_GPIO_LED),
10274
+
SND_PCI_QUIRK(0x103c, 0x8d01, "HP ZBook Power 14 G12", ALCXXX_FIXUP_CS35LXX),
10275
+
SND_PCI_QUIRK(0x103c, 0x8d08, "HP EliteBook 1045 14 G12", ALCXXX_FIXUP_CS35LXX),
10276
+
SND_PCI_QUIRK(0x103c, 0x8d85, "HP EliteBook 1040 14 G12", ALCXXX_FIXUP_CS35LXX),
10277
+
SND_PCI_QUIRK(0x103c, 0x8d86, "HP Elite x360 1040 14 G12", ALCXXX_FIXUP_CS35LXX),
10278
+
SND_PCI_QUIRK(0x103c, 0x8d8c, "HP EliteBook 830 13 G12", ALCXXX_FIXUP_CS35LXX),
10279
+
SND_PCI_QUIRK(0x103c, 0x8d8d, "HP Elite x360 830 13 G12", ALCXXX_FIXUP_CS35LXX),
10280
+
SND_PCI_QUIRK(0x103c, 0x8d8e, "HP EliteBook 840 14 G12", ALCXXX_FIXUP_CS35LXX),
10281
+
SND_PCI_QUIRK(0x103c, 0x8d8f, "HP EliteBook 840 14 G12", ALCXXX_FIXUP_CS35LXX),
10282
+
SND_PCI_QUIRK(0x103c, 0x8d90, "HP EliteBook 860 16 G12", ALCXXX_FIXUP_CS35LXX),
10283
+
SND_PCI_QUIRK(0x103c, 0x8d91, "HP ZBook Firefly 14 G12", ALCXXX_FIXUP_CS35LXX),
10284
+
SND_PCI_QUIRK(0x103c, 0x8d92, "HP ZBook Firefly 16 G12", ALCXXX_FIXUP_CS35LXX),
10361
10285
SND_PCI_QUIRK(0x1043, 0x103e, "ASUS X540SA", ALC256_FIXUP_ASUS_MIC),
10362
10286
SND_PCI_QUIRK(0x1043, 0x103f, "ASUS TX300", ALC282_FIXUP_ASUS_TX300),
10363
10287
SND_PCI_QUIRK(0x1043, 0x106d, "Asus K53BE", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
+9
-5
sound/pci/hda/tas2781_hda_i2c.c
+9
-5
sound/pci/hda/tas2781_hda_i2c.c
···
2
2
//
3
3
// TAS2781 HDA I2C driver
4
4
//
5
-
// Copyright 2023 Texas Instruments, Inc.
5
+
// Copyright 2023 - 2024 Texas Instruments, Inc.
6
6
//
7
7
// Author: Shenghao Ding <shenghao-ding@ti.com>
8
+
// Current maintainer: Baojun Xu <baojun.xu@ti.com>
8
9
10
+
#include <asm/unaligned.h>
9
11
#include <linux/acpi.h>
10
12
#include <linux/crc8.h>
11
13
#include <linux/crc32.h>
···
521
519
static const unsigned char rgno_array[CALIB_MAX] = {
522
520
0x74, 0x0c, 0x14, 0x70, 0x7c,
523
521
};
524
-
unsigned char *data;
522
+
int offset = 0;
525
523
int i, j, rc;
524
+
__be32 data;
526
525
527
526
for (i = 0; i < tas_priv->ndev; i++) {
528
-
data = tas_priv->cali_data.data +
529
-
i * TASDEVICE_SPEAKER_CALIBRATION_SIZE;
530
527
for (j = 0; j < CALIB_MAX; j++) {
528
+
data = cpu_to_be32(
529
+
*(uint32_t *)&tas_priv->cali_data.data[offset]);
531
530
rc = tasdevice_dev_bulk_write(tas_priv, i,
532
531
TASDEVICE_REG(0, page_array[j], rgno_array[j]),
533
-
&(data[4 * j]), 4);
532
+
(unsigned char *)&data, 4);
534
533
if (rc < 0)
535
534
dev_err(tas_priv->dev,
536
535
"chn %d calib %d bulk_wr err = %d\n",
537
536
i, j, rc);
537
+
offset += 4;
538
538
}
539
539
}
540
540
}
-1
sound/soc/codecs/cs42l42.c
-1
sound/soc/codecs/cs42l42.c
+1
sound/usb/quirks-table.h
+1
sound/usb/quirks-table.h
+2
sound/usb/quirks.c
+2
sound/usb/quirks.c
···
2221
2221
QUIRK_FLAG_GENERIC_IMPLICIT_FB),
2222
2222
DEVICE_FLG(0x2b53, 0x0031, /* Fiero SC-01 (firmware v1.1.0) */
2223
2223
QUIRK_FLAG_GENERIC_IMPLICIT_FB),
2224
+
DEVICE_FLG(0x2d95, 0x8021, /* VIVO USB-C-XE710 HEADSET */
2225
+
QUIRK_FLAG_CTL_MSG_DELAY_1M),
2224
2226
DEVICE_FLG(0x30be, 0x0101, /* Schiit Hel */
2225
2227
QUIRK_FLAG_IGNORE_CTL_ERROR),
2226
2228
DEVICE_FLG(0x413c, 0xa506, /* Dell AE515 sound bar */
+10
tools/arch/arm64/include/asm/cputype.h
+10
tools/arch/arm64/include/asm/cputype.h
···
86
86
#define ARM_CPU_PART_CORTEX_X2 0xD48
87
87
#define ARM_CPU_PART_NEOVERSE_N2 0xD49
88
88
#define ARM_CPU_PART_CORTEX_A78C 0xD4B
89
+
#define ARM_CPU_PART_CORTEX_X1C 0xD4C
90
+
#define ARM_CPU_PART_CORTEX_X3 0xD4E
89
91
#define ARM_CPU_PART_NEOVERSE_V2 0xD4F
92
+
#define ARM_CPU_PART_CORTEX_A720 0xD81
90
93
#define ARM_CPU_PART_CORTEX_X4 0xD82
91
94
#define ARM_CPU_PART_NEOVERSE_V3 0xD84
95
+
#define ARM_CPU_PART_CORTEX_X925 0xD85
96
+
#define ARM_CPU_PART_CORTEX_A725 0xD87
92
97
93
98
#define APM_CPU_PART_XGENE 0x000
94
99
#define APM_CPU_VAR_POTENZA 0x00
···
167
162
#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
168
163
#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
169
164
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
165
+
#define MIDR_CORTEX_X1C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1C)
166
+
#define MIDR_CORTEX_X3 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X3)
170
167
#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
168
+
#define MIDR_CORTEX_A720 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A720)
171
169
#define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4)
172
170
#define MIDR_NEOVERSE_V3 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V3)
171
+
#define MIDR_CORTEX_X925 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X925)
172
+
#define MIDR_CORTEX_A725 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A725)
173
173
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
174
174
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
175
175
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
+3
tools/arch/powerpc/include/uapi/asm/kvm.h
+3
tools/arch/powerpc/include/uapi/asm/kvm.h
···
645
645
#define KVM_REG_PPC_SIER3 (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc3)
646
646
#define KVM_REG_PPC_DAWR1 (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
647
647
#define KVM_REG_PPC_DAWRX1 (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
648
+
#define KVM_REG_PPC_DEXCR (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
649
+
#define KVM_REG_PPC_HASHKEYR (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc7)
650
+
#define KVM_REG_PPC_HASHPKEYR (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc8)
648
651
649
652
/* Transactional Memory checkpointed state:
650
653
* This is all GPRs, all VSX regs and a subset of SPRs
+403
-400
tools/arch/x86/include/asm/cpufeatures.h
+403
-400
tools/arch/x86/include/asm/cpufeatures.h
···
18
18
19
19
/*
20
20
* Note: If the comment begins with a quoted string, that string is used
21
-
* in /proc/cpuinfo instead of the macro name. If the string is "",
22
-
* this feature bit is not displayed in /proc/cpuinfo at all.
21
+
* in /proc/cpuinfo instead of the macro name. Otherwise, this feature
22
+
* bit is not displayed in /proc/cpuinfo at all.
23
23
*
24
24
* When adding new features here that depend on other features,
25
25
* please update the table in kernel/cpu/cpuid-deps.c as well.
26
26
*/
27
27
28
28
/* Intel-defined CPU features, CPUID level 0x00000001 (EDX), word 0 */
29
-
#define X86_FEATURE_FPU ( 0*32+ 0) /* Onboard FPU */
30
-
#define X86_FEATURE_VME ( 0*32+ 1) /* Virtual Mode Extensions */
31
-
#define X86_FEATURE_DE ( 0*32+ 2) /* Debugging Extensions */
32
-
#define X86_FEATURE_PSE ( 0*32+ 3) /* Page Size Extensions */
33
-
#define X86_FEATURE_TSC ( 0*32+ 4) /* Time Stamp Counter */
34
-
#define X86_FEATURE_MSR ( 0*32+ 5) /* Model-Specific Registers */
35
-
#define X86_FEATURE_PAE ( 0*32+ 6) /* Physical Address Extensions */
36
-
#define X86_FEATURE_MCE ( 0*32+ 7) /* Machine Check Exception */
37
-
#define X86_FEATURE_CX8 ( 0*32+ 8) /* CMPXCHG8 instruction */
38
-
#define X86_FEATURE_APIC ( 0*32+ 9) /* Onboard APIC */
39
-
#define X86_FEATURE_SEP ( 0*32+11) /* SYSENTER/SYSEXIT */
40
-
#define X86_FEATURE_MTRR ( 0*32+12) /* Memory Type Range Registers */
41
-
#define X86_FEATURE_PGE ( 0*32+13) /* Page Global Enable */
42
-
#define X86_FEATURE_MCA ( 0*32+14) /* Machine Check Architecture */
43
-
#define X86_FEATURE_CMOV ( 0*32+15) /* CMOV instructions (plus FCMOVcc, FCOMI with FPU) */
44
-
#define X86_FEATURE_PAT ( 0*32+16) /* Page Attribute Table */
45
-
#define X86_FEATURE_PSE36 ( 0*32+17) /* 36-bit PSEs */
46
-
#define X86_FEATURE_PN ( 0*32+18) /* Processor serial number */
47
-
#define X86_FEATURE_CLFLUSH ( 0*32+19) /* CLFLUSH instruction */
29
+
#define X86_FEATURE_FPU ( 0*32+ 0) /* "fpu" Onboard FPU */
30
+
#define X86_FEATURE_VME ( 0*32+ 1) /* "vme" Virtual Mode Extensions */
31
+
#define X86_FEATURE_DE ( 0*32+ 2) /* "de" Debugging Extensions */
32
+
#define X86_FEATURE_PSE ( 0*32+ 3) /* "pse" Page Size Extensions */
33
+
#define X86_FEATURE_TSC ( 0*32+ 4) /* "tsc" Time Stamp Counter */
34
+
#define X86_FEATURE_MSR ( 0*32+ 5) /* "msr" Model-Specific Registers */
35
+
#define X86_FEATURE_PAE ( 0*32+ 6) /* "pae" Physical Address Extensions */
36
+
#define X86_FEATURE_MCE ( 0*32+ 7) /* "mce" Machine Check Exception */
37
+
#define X86_FEATURE_CX8 ( 0*32+ 8) /* "cx8" CMPXCHG8 instruction */
38
+
#define X86_FEATURE_APIC ( 0*32+ 9) /* "apic" Onboard APIC */
39
+
#define X86_FEATURE_SEP ( 0*32+11) /* "sep" SYSENTER/SYSEXIT */
40
+
#define X86_FEATURE_MTRR ( 0*32+12) /* "mtrr" Memory Type Range Registers */
41
+
#define X86_FEATURE_PGE ( 0*32+13) /* "pge" Page Global Enable */
42
+
#define X86_FEATURE_MCA ( 0*32+14) /* "mca" Machine Check Architecture */
43
+
#define X86_FEATURE_CMOV ( 0*32+15) /* "cmov" CMOV instructions (plus FCMOVcc, FCOMI with FPU) */
44
+
#define X86_FEATURE_PAT ( 0*32+16) /* "pat" Page Attribute Table */
45
+
#define X86_FEATURE_PSE36 ( 0*32+17) /* "pse36" 36-bit PSEs */
46
+
#define X86_FEATURE_PN ( 0*32+18) /* "pn" Processor serial number */
47
+
#define X86_FEATURE_CLFLUSH ( 0*32+19) /* "clflush" CLFLUSH instruction */
48
48
#define X86_FEATURE_DS ( 0*32+21) /* "dts" Debug Store */
49
-
#define X86_FEATURE_ACPI ( 0*32+22) /* ACPI via MSR */
50
-
#define X86_FEATURE_MMX ( 0*32+23) /* Multimedia Extensions */
51
-
#define X86_FEATURE_FXSR ( 0*32+24) /* FXSAVE/FXRSTOR, CR4.OSFXSR */
49
+
#define X86_FEATURE_ACPI ( 0*32+22) /* "acpi" ACPI via MSR */
50
+
#define X86_FEATURE_MMX ( 0*32+23) /* "mmx" Multimedia Extensions */
51
+
#define X86_FEATURE_FXSR ( 0*32+24) /* "fxsr" FXSAVE/FXRSTOR, CR4.OSFXSR */
52
52
#define X86_FEATURE_XMM ( 0*32+25) /* "sse" */
53
53
#define X86_FEATURE_XMM2 ( 0*32+26) /* "sse2" */
54
54
#define X86_FEATURE_SELFSNOOP ( 0*32+27) /* "ss" CPU self snoop */
55
-
#define X86_FEATURE_HT ( 0*32+28) /* Hyper-Threading */
55
+
#define X86_FEATURE_HT ( 0*32+28) /* "ht" Hyper-Threading */
56
56
#define X86_FEATURE_ACC ( 0*32+29) /* "tm" Automatic clock control */
57
-
#define X86_FEATURE_IA64 ( 0*32+30) /* IA-64 processor */
58
-
#define X86_FEATURE_PBE ( 0*32+31) /* Pending Break Enable */
57
+
#define X86_FEATURE_IA64 ( 0*32+30) /* "ia64" IA-64 processor */
58
+
#define X86_FEATURE_PBE ( 0*32+31) /* "pbe" Pending Break Enable */
59
59
60
60
/* AMD-defined CPU features, CPUID level 0x80000001, word 1 */
61
61
/* Don't duplicate feature flags which are redundant with Intel! */
62
-
#define X86_FEATURE_SYSCALL ( 1*32+11) /* SYSCALL/SYSRET */
63
-
#define X86_FEATURE_MP ( 1*32+19) /* MP Capable */
64
-
#define X86_FEATURE_NX ( 1*32+20) /* Execute Disable */
65
-
#define X86_FEATURE_MMXEXT ( 1*32+22) /* AMD MMX extensions */
66
-
#define X86_FEATURE_FXSR_OPT ( 1*32+25) /* FXSAVE/FXRSTOR optimizations */
62
+
#define X86_FEATURE_SYSCALL ( 1*32+11) /* "syscall" SYSCALL/SYSRET */
63
+
#define X86_FEATURE_MP ( 1*32+19) /* "mp" MP Capable */
64
+
#define X86_FEATURE_NX ( 1*32+20) /* "nx" Execute Disable */
65
+
#define X86_FEATURE_MMXEXT ( 1*32+22) /* "mmxext" AMD MMX extensions */
66
+
#define X86_FEATURE_FXSR_OPT ( 1*32+25) /* "fxsr_opt" FXSAVE/FXRSTOR optimizations */
67
67
#define X86_FEATURE_GBPAGES ( 1*32+26) /* "pdpe1gb" GB pages */
68
-
#define X86_FEATURE_RDTSCP ( 1*32+27) /* RDTSCP */
69
-
#define X86_FEATURE_LM ( 1*32+29) /* Long Mode (x86-64, 64-bit support) */
70
-
#define X86_FEATURE_3DNOWEXT ( 1*32+30) /* AMD 3DNow extensions */
71
-
#define X86_FEATURE_3DNOW ( 1*32+31) /* 3DNow */
68
+
#define X86_FEATURE_RDTSCP ( 1*32+27) /* "rdtscp" RDTSCP */
69
+
#define X86_FEATURE_LM ( 1*32+29) /* "lm" Long Mode (x86-64, 64-bit support) */
70
+
#define X86_FEATURE_3DNOWEXT ( 1*32+30) /* "3dnowext" AMD 3DNow extensions */
71
+
#define X86_FEATURE_3DNOW ( 1*32+31) /* "3dnow" 3DNow */
72
72
73
73
/* Transmeta-defined CPU features, CPUID level 0x80860001, word 2 */
74
-
#define X86_FEATURE_RECOVERY ( 2*32+ 0) /* CPU in recovery mode */
75
-
#define X86_FEATURE_LONGRUN ( 2*32+ 1) /* Longrun power control */
76
-
#define X86_FEATURE_LRTI ( 2*32+ 3) /* LongRun table interface */
74
+
#define X86_FEATURE_RECOVERY ( 2*32+ 0) /* "recovery" CPU in recovery mode */
75
+
#define X86_FEATURE_LONGRUN ( 2*32+ 1) /* "longrun" Longrun power control */
76
+
#define X86_FEATURE_LRTI ( 2*32+ 3) /* "lrti" LongRun table interface */
77
77
78
78
/* Other features, Linux-defined mapping, word 3 */
79
79
/* This range is used for feature bits which conflict or are synthesized */
80
-
#define X86_FEATURE_CXMMX ( 3*32+ 0) /* Cyrix MMX extensions */
81
-
#define X86_FEATURE_K6_MTRR ( 3*32+ 1) /* AMD K6 nonstandard MTRRs */
82
-
#define X86_FEATURE_CYRIX_ARR ( 3*32+ 2) /* Cyrix ARRs (= MTRRs) */
83
-
#define X86_FEATURE_CENTAUR_MCR ( 3*32+ 3) /* Centaur MCRs (= MTRRs) */
84
-
#define X86_FEATURE_K8 ( 3*32+ 4) /* "" Opteron, Athlon64 */
85
-
#define X86_FEATURE_ZEN5 ( 3*32+ 5) /* "" CPU based on Zen5 microarchitecture */
86
-
#define X86_FEATURE_P3 ( 3*32+ 6) /* "" P3 */
87
-
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
88
-
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
89
-
#define X86_FEATURE_UP ( 3*32+ 9) /* SMP kernel running on UP */
90
-
#define X86_FEATURE_ART ( 3*32+10) /* Always running timer (ART) */
91
-
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
92
-
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
93
-
#define X86_FEATURE_BTS ( 3*32+13) /* Branch Trace Store */
94
-
#define X86_FEATURE_SYSCALL32 ( 3*32+14) /* "" syscall in IA32 userspace */
95
-
#define X86_FEATURE_SYSENTER32 ( 3*32+15) /* "" sysenter in IA32 userspace */
96
-
#define X86_FEATURE_REP_GOOD ( 3*32+16) /* REP microcode works well */
97
-
#define X86_FEATURE_AMD_LBR_V2 ( 3*32+17) /* AMD Last Branch Record Extension Version 2 */
98
-
#define X86_FEATURE_CLEAR_CPU_BUF ( 3*32+18) /* "" Clear CPU buffers using VERW */
99
-
#define X86_FEATURE_ACC_POWER ( 3*32+19) /* AMD Accumulated Power Mechanism */
100
-
#define X86_FEATURE_NOPL ( 3*32+20) /* The NOPL (0F 1F) instructions */
101
-
#define X86_FEATURE_ALWAYS ( 3*32+21) /* "" Always-present feature */
102
-
#define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* CPU topology enum extensions */
103
-
#define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* TSC is known to be reliable */
104
-
#define X86_FEATURE_NONSTOP_TSC ( 3*32+24) /* TSC does not stop in C states */
105
-
#define X86_FEATURE_CPUID ( 3*32+25) /* CPU has CPUID instruction itself */
106
-
#define X86_FEATURE_EXTD_APICID ( 3*32+26) /* Extended APICID (8 bits) */
107
-
#define X86_FEATURE_AMD_DCM ( 3*32+27) /* AMD multi-node processor */
108
-
#define X86_FEATURE_APERFMPERF ( 3*32+28) /* P-State hardware coordination feedback capability (APERF/MPERF MSRs) */
109
-
#define X86_FEATURE_RAPL ( 3*32+29) /* AMD/Hygon RAPL interface */
110
-
#define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30) /* TSC doesn't stop in S3 state */
111
-
#define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31) /* TSC has known frequency */
80
+
#define X86_FEATURE_CXMMX ( 3*32+ 0) /* "cxmmx" Cyrix MMX extensions */
81
+
#define X86_FEATURE_K6_MTRR ( 3*32+ 1) /* "k6_mtrr" AMD K6 nonstandard MTRRs */
82
+
#define X86_FEATURE_CYRIX_ARR ( 3*32+ 2) /* "cyrix_arr" Cyrix ARRs (= MTRRs) */
83
+
#define X86_FEATURE_CENTAUR_MCR ( 3*32+ 3) /* "centaur_mcr" Centaur MCRs (= MTRRs) */
84
+
#define X86_FEATURE_K8 ( 3*32+ 4) /* Opteron, Athlon64 */
85
+
#define X86_FEATURE_ZEN5 ( 3*32+ 5) /* CPU based on Zen5 microarchitecture */
86
+
#define X86_FEATURE_P3 ( 3*32+ 6) /* P3 */
87
+
#define X86_FEATURE_P4 ( 3*32+ 7) /* P4 */
88
+
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* "constant_tsc" TSC ticks at a constant rate */
89
+
#define X86_FEATURE_UP ( 3*32+ 9) /* "up" SMP kernel running on UP */
90
+
#define X86_FEATURE_ART ( 3*32+10) /* "art" Always running timer (ART) */
91
+
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* "arch_perfmon" Intel Architectural PerfMon */
92
+
#define X86_FEATURE_PEBS ( 3*32+12) /* "pebs" Precise-Event Based Sampling */
93
+
#define X86_FEATURE_BTS ( 3*32+13) /* "bts" Branch Trace Store */
94
+
#define X86_FEATURE_SYSCALL32 ( 3*32+14) /* syscall in IA32 userspace */
95
+
#define X86_FEATURE_SYSENTER32 ( 3*32+15) /* sysenter in IA32 userspace */
96
+
#define X86_FEATURE_REP_GOOD ( 3*32+16) /* "rep_good" REP microcode works well */
97
+
#define X86_FEATURE_AMD_LBR_V2 ( 3*32+17) /* "amd_lbr_v2" AMD Last Branch Record Extension Version 2 */
98
+
#define X86_FEATURE_CLEAR_CPU_BUF ( 3*32+18) /* Clear CPU buffers using VERW */
99
+
#define X86_FEATURE_ACC_POWER ( 3*32+19) /* "acc_power" AMD Accumulated Power Mechanism */
100
+
#define X86_FEATURE_NOPL ( 3*32+20) /* "nopl" The NOPL (0F 1F) instructions */
101
+
#define X86_FEATURE_ALWAYS ( 3*32+21) /* Always-present feature */
102
+
#define X86_FEATURE_XTOPOLOGY ( 3*32+22) /* "xtopology" CPU topology enum extensions */
103
+
#define X86_FEATURE_TSC_RELIABLE ( 3*32+23) /* "tsc_reliable" TSC is known to be reliable */
104
+
#define X86_FEATURE_NONSTOP_TSC ( 3*32+24) /* "nonstop_tsc" TSC does not stop in C states */
105
+
#define X86_FEATURE_CPUID ( 3*32+25) /* "cpuid" CPU has CPUID instruction itself */
106
+
#define X86_FEATURE_EXTD_APICID ( 3*32+26) /* "extd_apicid" Extended APICID (8 bits) */
107
+
#define X86_FEATURE_AMD_DCM ( 3*32+27) /* "amd_dcm" AMD multi-node processor */
108
+
#define X86_FEATURE_APERFMPERF ( 3*32+28) /* "aperfmperf" P-State hardware coordination feedback capability (APERF/MPERF MSRs) */
109
+
#define X86_FEATURE_RAPL ( 3*32+29) /* "rapl" AMD/Hygon RAPL interface */
110
+
#define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30) /* "nonstop_tsc_s3" TSC doesn't stop in S3 state */
111
+
#define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31) /* "tsc_known_freq" TSC has known frequency */
112
112
113
113
/* Intel-defined CPU features, CPUID level 0x00000001 (ECX), word 4 */
114
114
#define X86_FEATURE_XMM3 ( 4*32+ 0) /* "pni" SSE-3 */
115
-
#define X86_FEATURE_PCLMULQDQ ( 4*32+ 1) /* PCLMULQDQ instruction */
116
-
#define X86_FEATURE_DTES64 ( 4*32+ 2) /* 64-bit Debug Store */
115
+
#define X86_FEATURE_PCLMULQDQ ( 4*32+ 1) /* "pclmulqdq" PCLMULQDQ instruction */
116
+
#define X86_FEATURE_DTES64 ( 4*32+ 2) /* "dtes64" 64-bit Debug Store */
117
117
#define X86_FEATURE_MWAIT ( 4*32+ 3) /* "monitor" MONITOR/MWAIT support */
118
118
#define X86_FEATURE_DSCPL ( 4*32+ 4) /* "ds_cpl" CPL-qualified (filtered) Debug Store */
119
-
#define X86_FEATURE_VMX ( 4*32+ 5) /* Hardware virtualization */
120
-
#define X86_FEATURE_SMX ( 4*32+ 6) /* Safer Mode eXtensions */
121
-
#define X86_FEATURE_EST ( 4*32+ 7) /* Enhanced SpeedStep */
122
-
#define X86_FEATURE_TM2 ( 4*32+ 8) /* Thermal Monitor 2 */
123
-
#define X86_FEATURE_SSSE3 ( 4*32+ 9) /* Supplemental SSE-3 */
124
-
#define X86_FEATURE_CID ( 4*32+10) /* Context ID */
125
-
#define X86_FEATURE_SDBG ( 4*32+11) /* Silicon Debug */
126
-
#define X86_FEATURE_FMA ( 4*32+12) /* Fused multiply-add */
127
-
#define X86_FEATURE_CX16 ( 4*32+13) /* CMPXCHG16B instruction */
128
-
#define X86_FEATURE_XTPR ( 4*32+14) /* Send Task Priority Messages */
129
-
#define X86_FEATURE_PDCM ( 4*32+15) /* Perf/Debug Capabilities MSR */
130
-
#define X86_FEATURE_PCID ( 4*32+17) /* Process Context Identifiers */
131
-
#define X86_FEATURE_DCA ( 4*32+18) /* Direct Cache Access */
119
+
#define X86_FEATURE_VMX ( 4*32+ 5) /* "vmx" Hardware virtualization */
120
+
#define X86_FEATURE_SMX ( 4*32+ 6) /* "smx" Safer Mode eXtensions */
121
+
#define X86_FEATURE_EST ( 4*32+ 7) /* "est" Enhanced SpeedStep */
122
+
#define X86_FEATURE_TM2 ( 4*32+ 8) /* "tm2" Thermal Monitor 2 */
123
+
#define X86_FEATURE_SSSE3 ( 4*32+ 9) /* "ssse3" Supplemental SSE-3 */
124
+
#define X86_FEATURE_CID ( 4*32+10) /* "cid" Context ID */
125
+
#define X86_FEATURE_SDBG ( 4*32+11) /* "sdbg" Silicon Debug */
126
+
#define X86_FEATURE_FMA ( 4*32+12) /* "fma" Fused multiply-add */
127
+
#define X86_FEATURE_CX16 ( 4*32+13) /* "cx16" CMPXCHG16B instruction */
128
+
#define X86_FEATURE_XTPR ( 4*32+14) /* "xtpr" Send Task Priority Messages */
129
+
#define X86_FEATURE_PDCM ( 4*32+15) /* "pdcm" Perf/Debug Capabilities MSR */
130
+
#define X86_FEATURE_PCID ( 4*32+17) /* "pcid" Process Context Identifiers */
131
+
#define X86_FEATURE_DCA ( 4*32+18) /* "dca" Direct Cache Access */
132
132
#define X86_FEATURE_XMM4_1 ( 4*32+19) /* "sse4_1" SSE-4.1 */
133
133
#define X86_FEATURE_XMM4_2 ( 4*32+20) /* "sse4_2" SSE-4.2 */
134
-
#define X86_FEATURE_X2APIC ( 4*32+21) /* X2APIC */
135
-
#define X86_FEATURE_MOVBE ( 4*32+22) /* MOVBE instruction */
136
-
#define X86_FEATURE_POPCNT ( 4*32+23) /* POPCNT instruction */
137
-
#define X86_FEATURE_TSC_DEADLINE_TIMER ( 4*32+24) /* TSC deadline timer */
138
-
#define X86_FEATURE_AES ( 4*32+25) /* AES instructions */
139
-
#define X86_FEATURE_XSAVE ( 4*32+26) /* XSAVE/XRSTOR/XSETBV/XGETBV instructions */
140
-
#define X86_FEATURE_OSXSAVE ( 4*32+27) /* "" XSAVE instruction enabled in the OS */
141
-
#define X86_FEATURE_AVX ( 4*32+28) /* Advanced Vector Extensions */
142
-
#define X86_FEATURE_F16C ( 4*32+29) /* 16-bit FP conversions */
143
-
#define X86_FEATURE_RDRAND ( 4*32+30) /* RDRAND instruction */
144
-
#define X86_FEATURE_HYPERVISOR ( 4*32+31) /* Running on a hypervisor */
134
+
#define X86_FEATURE_X2APIC ( 4*32+21) /* "x2apic" X2APIC */
135
+
#define X86_FEATURE_MOVBE ( 4*32+22) /* "movbe" MOVBE instruction */
136
+
#define X86_FEATURE_POPCNT ( 4*32+23) /* "popcnt" POPCNT instruction */
137
+
#define X86_FEATURE_TSC_DEADLINE_TIMER ( 4*32+24) /* "tsc_deadline_timer" TSC deadline timer */
138
+
#define X86_FEATURE_AES ( 4*32+25) /* "aes" AES instructions */
139
+
#define X86_FEATURE_XSAVE ( 4*32+26) /* "xsave" XSAVE/XRSTOR/XSETBV/XGETBV instructions */
140
+
#define X86_FEATURE_OSXSAVE ( 4*32+27) /* XSAVE instruction enabled in the OS */
141
+
#define X86_FEATURE_AVX ( 4*32+28) /* "avx" Advanced Vector Extensions */
142
+
#define X86_FEATURE_F16C ( 4*32+29) /* "f16c" 16-bit FP conversions */
143
+
#define X86_FEATURE_RDRAND ( 4*32+30) /* "rdrand" RDRAND instruction */
144
+
#define X86_FEATURE_HYPERVISOR ( 4*32+31) /* "hypervisor" Running on a hypervisor */
145
145
146
146
/* VIA/Cyrix/Centaur-defined CPU features, CPUID level 0xC0000001, word 5 */
147
147
#define X86_FEATURE_XSTORE ( 5*32+ 2) /* "rng" RNG present (xstore) */
148
148
#define X86_FEATURE_XSTORE_EN ( 5*32+ 3) /* "rng_en" RNG enabled */
149
149
#define X86_FEATURE_XCRYPT ( 5*32+ 6) /* "ace" on-CPU crypto (xcrypt) */
150
150
#define X86_FEATURE_XCRYPT_EN ( 5*32+ 7) /* "ace_en" on-CPU crypto enabled */
151
-
#define X86_FEATURE_ACE2 ( 5*32+ 8) /* Advanced Cryptography Engine v2 */
152
-
#define X86_FEATURE_ACE2_EN ( 5*32+ 9) /* ACE v2 enabled */
153
-
#define X86_FEATURE_PHE ( 5*32+10) /* PadLock Hash Engine */
154
-
#define X86_FEATURE_PHE_EN ( 5*32+11) /* PHE enabled */
155
-
#define X86_FEATURE_PMM ( 5*32+12) /* PadLock Montgomery Multiplier */
156
-
#define X86_FEATURE_PMM_EN ( 5*32+13) /* PMM enabled */
151
+
#define X86_FEATURE_ACE2 ( 5*32+ 8) /* "ace2" Advanced Cryptography Engine v2 */
152
+
#define X86_FEATURE_ACE2_EN ( 5*32+ 9) /* "ace2_en" ACE v2 enabled */
153
+
#define X86_FEATURE_PHE ( 5*32+10) /* "phe" PadLock Hash Engine */
154
+
#define X86_FEATURE_PHE_EN ( 5*32+11) /* "phe_en" PHE enabled */
155
+
#define X86_FEATURE_PMM ( 5*32+12) /* "pmm" PadLock Montgomery Multiplier */
156
+
#define X86_FEATURE_PMM_EN ( 5*32+13) /* "pmm_en" PMM enabled */
157
157
158
158
/* More extended AMD flags: CPUID level 0x80000001, ECX, word 6 */
159
-
#define X86_FEATURE_LAHF_LM ( 6*32+ 0) /* LAHF/SAHF in long mode */
160
-
#define X86_FEATURE_CMP_LEGACY ( 6*32+ 1) /* If yes HyperThreading not valid */
161
-
#define X86_FEATURE_SVM ( 6*32+ 2) /* Secure Virtual Machine */
162
-
#define X86_FEATURE_EXTAPIC ( 6*32+ 3) /* Extended APIC space */
163
-
#define X86_FEATURE_CR8_LEGACY ( 6*32+ 4) /* CR8 in 32-bit mode */
164
-
#define X86_FEATURE_ABM ( 6*32+ 5) /* Advanced bit manipulation */
165
-
#define X86_FEATURE_SSE4A ( 6*32+ 6) /* SSE-4A */
166
-
#define X86_FEATURE_MISALIGNSSE ( 6*32+ 7) /* Misaligned SSE mode */
167
-
#define X86_FEATURE_3DNOWPREFETCH ( 6*32+ 8) /* 3DNow prefetch instructions */
168
-
#define X86_FEATURE_OSVW ( 6*32+ 9) /* OS Visible Workaround */
169
-
#define X86_FEATURE_IBS ( 6*32+10) /* Instruction Based Sampling */
170
-
#define X86_FEATURE_XOP ( 6*32+11) /* extended AVX instructions */
171
-
#define X86_FEATURE_SKINIT ( 6*32+12) /* SKINIT/STGI instructions */
172
-
#define X86_FEATURE_WDT ( 6*32+13) /* Watchdog timer */
173
-
#define X86_FEATURE_LWP ( 6*32+15) /* Light Weight Profiling */
174
-
#define X86_FEATURE_FMA4 ( 6*32+16) /* 4 operands MAC instructions */
175
-
#define X86_FEATURE_TCE ( 6*32+17) /* Translation Cache Extension */
176
-
#define X86_FEATURE_NODEID_MSR ( 6*32+19) /* NodeId MSR */
177
-
#define X86_FEATURE_TBM ( 6*32+21) /* Trailing Bit Manipulations */
178
-
#define X86_FEATURE_TOPOEXT ( 6*32+22) /* Topology extensions CPUID leafs */
179
-
#define X86_FEATURE_PERFCTR_CORE ( 6*32+23) /* Core performance counter extensions */
180
-
#define X86_FEATURE_PERFCTR_NB ( 6*32+24) /* NB performance counter extensions */
181
-
#define X86_FEATURE_BPEXT ( 6*32+26) /* Data breakpoint extension */
182
-
#define X86_FEATURE_PTSC ( 6*32+27) /* Performance time-stamp counter */
183
-
#define X86_FEATURE_PERFCTR_LLC ( 6*32+28) /* Last Level Cache performance counter extensions */
184
-
#define X86_FEATURE_MWAITX ( 6*32+29) /* MWAIT extension (MONITORX/MWAITX instructions) */
159
+
#define X86_FEATURE_LAHF_LM ( 6*32+ 0) /* "lahf_lm" LAHF/SAHF in long mode */
160
+
#define X86_FEATURE_CMP_LEGACY ( 6*32+ 1) /* "cmp_legacy" If yes HyperThreading not valid */
161
+
#define X86_FEATURE_SVM ( 6*32+ 2) /* "svm" Secure Virtual Machine */
162
+
#define X86_FEATURE_EXTAPIC ( 6*32+ 3) /* "extapic" Extended APIC space */
163
+
#define X86_FEATURE_CR8_LEGACY ( 6*32+ 4) /* "cr8_legacy" CR8 in 32-bit mode */
164
+
#define X86_FEATURE_ABM ( 6*32+ 5) /* "abm" Advanced bit manipulation */
165
+
#define X86_FEATURE_SSE4A ( 6*32+ 6) /* "sse4a" SSE-4A */
166
+
#define X86_FEATURE_MISALIGNSSE ( 6*32+ 7) /* "misalignsse" Misaligned SSE mode */
167
+
#define X86_FEATURE_3DNOWPREFETCH ( 6*32+ 8) /* "3dnowprefetch" 3DNow prefetch instructions */
168
+
#define X86_FEATURE_OSVW ( 6*32+ 9) /* "osvw" OS Visible Workaround */
169
+
#define X86_FEATURE_IBS ( 6*32+10) /* "ibs" Instruction Based Sampling */
170
+
#define X86_FEATURE_XOP ( 6*32+11) /* "xop" Extended AVX instructions */
171
+
#define X86_FEATURE_SKINIT ( 6*32+12) /* "skinit" SKINIT/STGI instructions */
172
+
#define X86_FEATURE_WDT ( 6*32+13) /* "wdt" Watchdog timer */
173
+
#define X86_FEATURE_LWP ( 6*32+15) /* "lwp" Light Weight Profiling */
174
+
#define X86_FEATURE_FMA4 ( 6*32+16) /* "fma4" 4 operands MAC instructions */
175
+
#define X86_FEATURE_TCE ( 6*32+17) /* "tce" Translation Cache Extension */
176
+
#define X86_FEATURE_NODEID_MSR ( 6*32+19) /* "nodeid_msr" NodeId MSR */
177
+
#define X86_FEATURE_TBM ( 6*32+21) /* "tbm" Trailing Bit Manipulations */
178
+
#define X86_FEATURE_TOPOEXT ( 6*32+22) /* "topoext" Topology extensions CPUID leafs */
179
+
#define X86_FEATURE_PERFCTR_CORE ( 6*32+23) /* "perfctr_core" Core performance counter extensions */
180
+
#define X86_FEATURE_PERFCTR_NB ( 6*32+24) /* "perfctr_nb" NB performance counter extensions */
181
+
#define X86_FEATURE_BPEXT ( 6*32+26) /* "bpext" Data breakpoint extension */
182
+
#define X86_FEATURE_PTSC ( 6*32+27) /* "ptsc" Performance time-stamp counter */
183
+
#define X86_FEATURE_PERFCTR_LLC ( 6*32+28) /* "perfctr_llc" Last Level Cache performance counter extensions */
184
+
#define X86_FEATURE_MWAITX ( 6*32+29) /* "mwaitx" MWAIT extension (MONITORX/MWAITX instructions) */
185
185
186
186
/*
187
187
* Auxiliary flags: Linux defined - For features scattered in various
···
189
189
*
190
190
* Reuse free bits when adding new feature flags!
191
191
*/
192
-
#define X86_FEATURE_RING3MWAIT ( 7*32+ 0) /* Ring 3 MONITOR/MWAIT instructions */
193
-
#define X86_FEATURE_CPUID_FAULT ( 7*32+ 1) /* Intel CPUID faulting */
194
-
#define X86_FEATURE_CPB ( 7*32+ 2) /* AMD Core Performance Boost */
195
-
#define X86_FEATURE_EPB ( 7*32+ 3) /* IA32_ENERGY_PERF_BIAS support */
196
-
#define X86_FEATURE_CAT_L3 ( 7*32+ 4) /* Cache Allocation Technology L3 */
197
-
#define X86_FEATURE_CAT_L2 ( 7*32+ 5) /* Cache Allocation Technology L2 */
198
-
#define X86_FEATURE_CDP_L3 ( 7*32+ 6) /* Code and Data Prioritization L3 */
199
-
#define X86_FEATURE_TDX_HOST_PLATFORM ( 7*32+ 7) /* Platform supports being a TDX host */
200
-
#define X86_FEATURE_HW_PSTATE ( 7*32+ 8) /* AMD HW-PState */
201
-
#define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
202
-
#define X86_FEATURE_XCOMPACTED ( 7*32+10) /* "" Use compacted XSTATE (XSAVES or XSAVEC) */
203
-
#define X86_FEATURE_PTI ( 7*32+11) /* Kernel Page Table Isolation enabled */
204
-
#define X86_FEATURE_KERNEL_IBRS ( 7*32+12) /* "" Set/clear IBRS on kernel entry/exit */
205
-
#define X86_FEATURE_RSB_VMEXIT ( 7*32+13) /* "" Fill RSB on VM-Exit */
206
-
#define X86_FEATURE_INTEL_PPIN ( 7*32+14) /* Intel Processor Inventory Number */
207
-
#define X86_FEATURE_CDP_L2 ( 7*32+15) /* Code and Data Prioritization L2 */
208
-
#define X86_FEATURE_MSR_SPEC_CTRL ( 7*32+16) /* "" MSR SPEC_CTRL is implemented */
209
-
#define X86_FEATURE_SSBD ( 7*32+17) /* Speculative Store Bypass Disable */
210
-
#define X86_FEATURE_MBA ( 7*32+18) /* Memory Bandwidth Allocation */
211
-
#define X86_FEATURE_RSB_CTXSW ( 7*32+19) /* "" Fill RSB on context switches */
212
-
#define X86_FEATURE_PERFMON_V2 ( 7*32+20) /* AMD Performance Monitoring Version 2 */
213
-
#define X86_FEATURE_USE_IBPB ( 7*32+21) /* "" Indirect Branch Prediction Barrier enabled */
214
-
#define X86_FEATURE_USE_IBRS_FW ( 7*32+22) /* "" Use IBRS during runtime firmware calls */
215
-
#define X86_FEATURE_SPEC_STORE_BYPASS_DISABLE ( 7*32+23) /* "" Disable Speculative Store Bypass. */
216
-
#define X86_FEATURE_LS_CFG_SSBD ( 7*32+24) /* "" AMD SSBD implementation via LS_CFG MSR */
217
-
#define X86_FEATURE_IBRS ( 7*32+25) /* Indirect Branch Restricted Speculation */
218
-
#define X86_FEATURE_IBPB ( 7*32+26) /* Indirect Branch Prediction Barrier */
219
-
#define X86_FEATURE_STIBP ( 7*32+27) /* Single Thread Indirect Branch Predictors */
220
-
#define X86_FEATURE_ZEN ( 7*32+28) /* "" Generic flag for all Zen and newer */
221
-
#define X86_FEATURE_L1TF_PTEINV ( 7*32+29) /* "" L1TF workaround PTE inversion */
222
-
#define X86_FEATURE_IBRS_ENHANCED ( 7*32+30) /* Enhanced IBRS */
223
-
#define X86_FEATURE_MSR_IA32_FEAT_CTL ( 7*32+31) /* "" MSR IA32_FEAT_CTL configured */
192
+
#define X86_FEATURE_RING3MWAIT ( 7*32+ 0) /* "ring3mwait" Ring 3 MONITOR/MWAIT instructions */
193
+
#define X86_FEATURE_CPUID_FAULT ( 7*32+ 1) /* "cpuid_fault" Intel CPUID faulting */
194
+
#define X86_FEATURE_CPB ( 7*32+ 2) /* "cpb" AMD Core Performance Boost */
195
+
#define X86_FEATURE_EPB ( 7*32+ 3) /* "epb" IA32_ENERGY_PERF_BIAS support */
196
+
#define X86_FEATURE_CAT_L3 ( 7*32+ 4) /* "cat_l3" Cache Allocation Technology L3 */
197
+
#define X86_FEATURE_CAT_L2 ( 7*32+ 5) /* "cat_l2" Cache Allocation Technology L2 */
198
+
#define X86_FEATURE_CDP_L3 ( 7*32+ 6) /* "cdp_l3" Code and Data Prioritization L3 */
199
+
#define X86_FEATURE_TDX_HOST_PLATFORM ( 7*32+ 7) /* "tdx_host_platform" Platform supports being a TDX host */
200
+
#define X86_FEATURE_HW_PSTATE ( 7*32+ 8) /* "hw_pstate" AMD HW-PState */
201
+
#define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* "proc_feedback" AMD ProcFeedbackInterface */
202
+
#define X86_FEATURE_XCOMPACTED ( 7*32+10) /* Use compacted XSTATE (XSAVES or XSAVEC) */
203
+
#define X86_FEATURE_PTI ( 7*32+11) /* "pti" Kernel Page Table Isolation enabled */
204
+
#define X86_FEATURE_KERNEL_IBRS ( 7*32+12) /* Set/clear IBRS on kernel entry/exit */
205
+
#define X86_FEATURE_RSB_VMEXIT ( 7*32+13) /* Fill RSB on VM-Exit */
206
+
#define X86_FEATURE_INTEL_PPIN ( 7*32+14) /* "intel_ppin" Intel Processor Inventory Number */
207
+
#define X86_FEATURE_CDP_L2 ( 7*32+15) /* "cdp_l2" Code and Data Prioritization L2 */
208
+
#define X86_FEATURE_MSR_SPEC_CTRL ( 7*32+16) /* MSR SPEC_CTRL is implemented */
209
+
#define X86_FEATURE_SSBD ( 7*32+17) /* "ssbd" Speculative Store Bypass Disable */
210
+
#define X86_FEATURE_MBA ( 7*32+18) /* "mba" Memory Bandwidth Allocation */
211
+
#define X86_FEATURE_RSB_CTXSW ( 7*32+19) /* Fill RSB on context switches */
212
+
#define X86_FEATURE_PERFMON_V2 ( 7*32+20) /* "perfmon_v2" AMD Performance Monitoring Version 2 */
213
+
#define X86_FEATURE_USE_IBPB ( 7*32+21) /* Indirect Branch Prediction Barrier enabled */
214
+
#define X86_FEATURE_USE_IBRS_FW ( 7*32+22) /* Use IBRS during runtime firmware calls */
215
+
#define X86_FEATURE_SPEC_STORE_BYPASS_DISABLE ( 7*32+23) /* Disable Speculative Store Bypass. */
216
+
#define X86_FEATURE_LS_CFG_SSBD ( 7*32+24) /* AMD SSBD implementation via LS_CFG MSR */
217
+
#define X86_FEATURE_IBRS ( 7*32+25) /* "ibrs" Indirect Branch Restricted Speculation */
218
+
#define X86_FEATURE_IBPB ( 7*32+26) /* "ibpb" Indirect Branch Prediction Barrier */
219
+
#define X86_FEATURE_STIBP ( 7*32+27) /* "stibp" Single Thread Indirect Branch Predictors */
220
+
#define X86_FEATURE_ZEN ( 7*32+28) /* Generic flag for all Zen and newer */
221
+
#define X86_FEATURE_L1TF_PTEINV ( 7*32+29) /* L1TF workaround PTE inversion */
222
+
#define X86_FEATURE_IBRS_ENHANCED ( 7*32+30) /* "ibrs_enhanced" Enhanced IBRS */
223
+
#define X86_FEATURE_MSR_IA32_FEAT_CTL ( 7*32+31) /* MSR IA32_FEAT_CTL configured */
224
224
225
225
/* Virtualization flags: Linux defined, word 8 */
226
-
#define X86_FEATURE_TPR_SHADOW ( 8*32+ 0) /* Intel TPR Shadow */
227
-
#define X86_FEATURE_FLEXPRIORITY ( 8*32+ 1) /* Intel FlexPriority */
228
-
#define X86_FEATURE_EPT ( 8*32+ 2) /* Intel Extended Page Table */
229
-
#define X86_FEATURE_VPID ( 8*32+ 3) /* Intel Virtual Processor ID */
226
+
#define X86_FEATURE_TPR_SHADOW ( 8*32+ 0) /* "tpr_shadow" Intel TPR Shadow */
227
+
#define X86_FEATURE_FLEXPRIORITY ( 8*32+ 1) /* "flexpriority" Intel FlexPriority */
228
+
#define X86_FEATURE_EPT ( 8*32+ 2) /* "ept" Intel Extended Page Table */
229
+
#define X86_FEATURE_VPID ( 8*32+ 3) /* "vpid" Intel Virtual Processor ID */
230
230
231
-
#define X86_FEATURE_VMMCALL ( 8*32+15) /* Prefer VMMCALL to VMCALL */
232
-
#define X86_FEATURE_XENPV ( 8*32+16) /* "" Xen paravirtual guest */
233
-
#define X86_FEATURE_EPT_AD ( 8*32+17) /* Intel Extended Page Table access-dirty bit */
234
-
#define X86_FEATURE_VMCALL ( 8*32+18) /* "" Hypervisor supports the VMCALL instruction */
235
-
#define X86_FEATURE_VMW_VMMCALL ( 8*32+19) /* "" VMware prefers VMMCALL hypercall instruction */
236
-
#define X86_FEATURE_PVUNLOCK ( 8*32+20) /* "" PV unlock function */
237
-
#define X86_FEATURE_VCPUPREEMPT ( 8*32+21) /* "" PV vcpu_is_preempted function */
238
-
#define X86_FEATURE_TDX_GUEST ( 8*32+22) /* Intel Trust Domain Extensions Guest */
231
+
#define X86_FEATURE_VMMCALL ( 8*32+15) /* "vmmcall" Prefer VMMCALL to VMCALL */
232
+
#define X86_FEATURE_XENPV ( 8*32+16) /* Xen paravirtual guest */
233
+
#define X86_FEATURE_EPT_AD ( 8*32+17) /* "ept_ad" Intel Extended Page Table access-dirty bit */
234
+
#define X86_FEATURE_VMCALL ( 8*32+18) /* Hypervisor supports the VMCALL instruction */
235
+
#define X86_FEATURE_VMW_VMMCALL ( 8*32+19) /* VMware prefers VMMCALL hypercall instruction */
236
+
#define X86_FEATURE_PVUNLOCK ( 8*32+20) /* PV unlock function */
237
+
#define X86_FEATURE_VCPUPREEMPT ( 8*32+21) /* PV vcpu_is_preempted function */
238
+
#define X86_FEATURE_TDX_GUEST ( 8*32+22) /* "tdx_guest" Intel Trust Domain Extensions Guest */
239
239
240
240
/* Intel-defined CPU features, CPUID level 0x00000007:0 (EBX), word 9 */
241
-
#define X86_FEATURE_FSGSBASE ( 9*32+ 0) /* RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/
242
-
#define X86_FEATURE_TSC_ADJUST ( 9*32+ 1) /* TSC adjustment MSR 0x3B */
243
-
#define X86_FEATURE_SGX ( 9*32+ 2) /* Software Guard Extensions */
244
-
#define X86_FEATURE_BMI1 ( 9*32+ 3) /* 1st group bit manipulation extensions */
245
-
#define X86_FEATURE_HLE ( 9*32+ 4) /* Hardware Lock Elision */
246
-
#define X86_FEATURE_AVX2 ( 9*32+ 5) /* AVX2 instructions */
247
-
#define X86_FEATURE_FDP_EXCPTN_ONLY ( 9*32+ 6) /* "" FPU data pointer updated only on x87 exceptions */
248
-
#define X86_FEATURE_SMEP ( 9*32+ 7) /* Supervisor Mode Execution Protection */
249
-
#define X86_FEATURE_BMI2 ( 9*32+ 8) /* 2nd group bit manipulation extensions */
250
-
#define X86_FEATURE_ERMS ( 9*32+ 9) /* Enhanced REP MOVSB/STOSB instructions */
251
-
#define X86_FEATURE_INVPCID ( 9*32+10) /* Invalidate Processor Context ID */
252
-
#define X86_FEATURE_RTM ( 9*32+11) /* Restricted Transactional Memory */
253
-
#define X86_FEATURE_CQM ( 9*32+12) /* Cache QoS Monitoring */
254
-
#define X86_FEATURE_ZERO_FCS_FDS ( 9*32+13) /* "" Zero out FPU CS and FPU DS */
255
-
#define X86_FEATURE_MPX ( 9*32+14) /* Memory Protection Extension */
256
-
#define X86_FEATURE_RDT_A ( 9*32+15) /* Resource Director Technology Allocation */
257
-
#define X86_FEATURE_AVX512F ( 9*32+16) /* AVX-512 Foundation */
258
-
#define X86_FEATURE_AVX512DQ ( 9*32+17) /* AVX-512 DQ (Double/Quad granular) Instructions */
259
-
#define X86_FEATURE_RDSEED ( 9*32+18) /* RDSEED instruction */
260
-
#define X86_FEATURE_ADX ( 9*32+19) /* ADCX and ADOX instructions */
261
-
#define X86_FEATURE_SMAP ( 9*32+20) /* Supervisor Mode Access Prevention */
262
-
#define X86_FEATURE_AVX512IFMA ( 9*32+21) /* AVX-512 Integer Fused Multiply-Add instructions */
263
-
#define X86_FEATURE_CLFLUSHOPT ( 9*32+23) /* CLFLUSHOPT instruction */
264
-
#define X86_FEATURE_CLWB ( 9*32+24) /* CLWB instruction */
265
-
#define X86_FEATURE_INTEL_PT ( 9*32+25) /* Intel Processor Trace */
266
-
#define X86_FEATURE_AVX512PF ( 9*32+26) /* AVX-512 Prefetch */
267
-
#define X86_FEATURE_AVX512ER ( 9*32+27) /* AVX-512 Exponential and Reciprocal */
268
-
#define X86_FEATURE_AVX512CD ( 9*32+28) /* AVX-512 Conflict Detection */
269
-
#define X86_FEATURE_SHA_NI ( 9*32+29) /* SHA1/SHA256 Instruction Extensions */
270
-
#define X86_FEATURE_AVX512BW ( 9*32+30) /* AVX-512 BW (Byte/Word granular) Instructions */
271
-
#define X86_FEATURE_AVX512VL ( 9*32+31) /* AVX-512 VL (128/256 Vector Length) Extensions */
241
+
#define X86_FEATURE_FSGSBASE ( 9*32+ 0) /* "fsgsbase" RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/
242
+
#define X86_FEATURE_TSC_ADJUST ( 9*32+ 1) /* "tsc_adjust" TSC adjustment MSR 0x3B */
243
+
#define X86_FEATURE_SGX ( 9*32+ 2) /* "sgx" Software Guard Extensions */
244
+
#define X86_FEATURE_BMI1 ( 9*32+ 3) /* "bmi1" 1st group bit manipulation extensions */
245
+
#define X86_FEATURE_HLE ( 9*32+ 4) /* "hle" Hardware Lock Elision */
246
+
#define X86_FEATURE_AVX2 ( 9*32+ 5) /* "avx2" AVX2 instructions */
247
+
#define X86_FEATURE_FDP_EXCPTN_ONLY ( 9*32+ 6) /* FPU data pointer updated only on x87 exceptions */
248
+
#define X86_FEATURE_SMEP ( 9*32+ 7) /* "smep" Supervisor Mode Execution Protection */
249
+
#define X86_FEATURE_BMI2 ( 9*32+ 8) /* "bmi2" 2nd group bit manipulation extensions */
250
+
#define X86_FEATURE_ERMS ( 9*32+ 9) /* "erms" Enhanced REP MOVSB/STOSB instructions */
251
+
#define X86_FEATURE_INVPCID ( 9*32+10) /* "invpcid" Invalidate Processor Context ID */
252
+
#define X86_FEATURE_RTM ( 9*32+11) /* "rtm" Restricted Transactional Memory */
253
+
#define X86_FEATURE_CQM ( 9*32+12) /* "cqm" Cache QoS Monitoring */
254
+
#define X86_FEATURE_ZERO_FCS_FDS ( 9*32+13) /* Zero out FPU CS and FPU DS */
255
+
#define X86_FEATURE_MPX ( 9*32+14) /* "mpx" Memory Protection Extension */
256
+
#define X86_FEATURE_RDT_A ( 9*32+15) /* "rdt_a" Resource Director Technology Allocation */
257
+
#define X86_FEATURE_AVX512F ( 9*32+16) /* "avx512f" AVX-512 Foundation */
258
+
#define X86_FEATURE_AVX512DQ ( 9*32+17) /* "avx512dq" AVX-512 DQ (Double/Quad granular) Instructions */
259
+
#define X86_FEATURE_RDSEED ( 9*32+18) /* "rdseed" RDSEED instruction */
260
+
#define X86_FEATURE_ADX ( 9*32+19) /* "adx" ADCX and ADOX instructions */
261
+
#define X86_FEATURE_SMAP ( 9*32+20) /* "smap" Supervisor Mode Access Prevention */
262
+
#define X86_FEATURE_AVX512IFMA ( 9*32+21) /* "avx512ifma" AVX-512 Integer Fused Multiply-Add instructions */
263
+
#define X86_FEATURE_CLFLUSHOPT ( 9*32+23) /* "clflushopt" CLFLUSHOPT instruction */
264
+
#define X86_FEATURE_CLWB ( 9*32+24) /* "clwb" CLWB instruction */
265
+
#define X86_FEATURE_INTEL_PT ( 9*32+25) /* "intel_pt" Intel Processor Trace */
266
+
#define X86_FEATURE_AVX512PF ( 9*32+26) /* "avx512pf" AVX-512 Prefetch */
267
+
#define X86_FEATURE_AVX512ER ( 9*32+27) /* "avx512er" AVX-512 Exponential and Reciprocal */
268
+
#define X86_FEATURE_AVX512CD ( 9*32+28) /* "avx512cd" AVX-512 Conflict Detection */
269
+
#define X86_FEATURE_SHA_NI ( 9*32+29) /* "sha_ni" SHA1/SHA256 Instruction Extensions */
270
+
#define X86_FEATURE_AVX512BW ( 9*32+30) /* "avx512bw" AVX-512 BW (Byte/Word granular) Instructions */
271
+
#define X86_FEATURE_AVX512VL ( 9*32+31) /* "avx512vl" AVX-512 VL (128/256 Vector Length) Extensions */
272
272
273
273
/* Extended state features, CPUID level 0x0000000d:1 (EAX), word 10 */
274
-
#define X86_FEATURE_XSAVEOPT (10*32+ 0) /* XSAVEOPT instruction */
275
-
#define X86_FEATURE_XSAVEC (10*32+ 1) /* XSAVEC instruction */
276
-
#define X86_FEATURE_XGETBV1 (10*32+ 2) /* XGETBV with ECX = 1 instruction */
277
-
#define X86_FEATURE_XSAVES (10*32+ 3) /* XSAVES/XRSTORS instructions */
278
-
#define X86_FEATURE_XFD (10*32+ 4) /* "" eXtended Feature Disabling */
274
+
#define X86_FEATURE_XSAVEOPT (10*32+ 0) /* "xsaveopt" XSAVEOPT instruction */
275
+
#define X86_FEATURE_XSAVEC (10*32+ 1) /* "xsavec" XSAVEC instruction */
276
+
#define X86_FEATURE_XGETBV1 (10*32+ 2) /* "xgetbv1" XGETBV with ECX = 1 instruction */
277
+
#define X86_FEATURE_XSAVES (10*32+ 3) /* "xsaves" XSAVES/XRSTORS instructions */
278
+
#define X86_FEATURE_XFD (10*32+ 4) /* eXtended Feature Disabling */
279
279
280
280
/*
281
281
* Extended auxiliary flags: Linux defined - for features scattered in various
···
283
283
*
284
284
* Reuse free bits when adding new feature flags!
285
285
*/
286
-
#define X86_FEATURE_CQM_LLC (11*32+ 0) /* LLC QoS if 1 */
287
-
#define X86_FEATURE_CQM_OCCUP_LLC (11*32+ 1) /* LLC occupancy monitoring */
288
-
#define X86_FEATURE_CQM_MBM_TOTAL (11*32+ 2) /* LLC Total MBM monitoring */
289
-
#define X86_FEATURE_CQM_MBM_LOCAL (11*32+ 3) /* LLC Local MBM monitoring */
290
-
#define X86_FEATURE_FENCE_SWAPGS_USER (11*32+ 4) /* "" LFENCE in user entry SWAPGS path */
291
-
#define X86_FEATURE_FENCE_SWAPGS_KERNEL (11*32+ 5) /* "" LFENCE in kernel entry SWAPGS path */
292
-
#define X86_FEATURE_SPLIT_LOCK_DETECT (11*32+ 6) /* #AC for split lock */
293
-
#define X86_FEATURE_PER_THREAD_MBA (11*32+ 7) /* "" Per-thread Memory Bandwidth Allocation */
294
-
#define X86_FEATURE_SGX1 (11*32+ 8) /* "" Basic SGX */
295
-
#define X86_FEATURE_SGX2 (11*32+ 9) /* "" SGX Enclave Dynamic Memory Management (EDMM) */
296
-
#define X86_FEATURE_ENTRY_IBPB (11*32+10) /* "" Issue an IBPB on kernel entry */
297
-
#define X86_FEATURE_RRSBA_CTRL (11*32+11) /* "" RET prediction control */
298
-
#define X86_FEATURE_RETPOLINE (11*32+12) /* "" Generic Retpoline mitigation for Spectre variant 2 */
299
-
#define X86_FEATURE_RETPOLINE_LFENCE (11*32+13) /* "" Use LFENCE for Spectre variant 2 */
300
-
#define X86_FEATURE_RETHUNK (11*32+14) /* "" Use REturn THUNK */
301
-
#define X86_FEATURE_UNRET (11*32+15) /* "" AMD BTB untrain return */
302
-
#define X86_FEATURE_USE_IBPB_FW (11*32+16) /* "" Use IBPB during runtime firmware calls */
303
-
#define X86_FEATURE_RSB_VMEXIT_LITE (11*32+17) /* "" Fill RSB on VM exit when EIBRS is enabled */
304
-
#define X86_FEATURE_SGX_EDECCSSA (11*32+18) /* "" SGX EDECCSSA user leaf function */
305
-
#define X86_FEATURE_CALL_DEPTH (11*32+19) /* "" Call depth tracking for RSB stuffing */
306
-
#define X86_FEATURE_MSR_TSX_CTRL (11*32+20) /* "" MSR IA32_TSX_CTRL (Intel) implemented */
307
-
#define X86_FEATURE_SMBA (11*32+21) /* "" Slow Memory Bandwidth Allocation */
308
-
#define X86_FEATURE_BMEC (11*32+22) /* "" Bandwidth Monitoring Event Configuration */
309
-
#define X86_FEATURE_USER_SHSTK (11*32+23) /* Shadow stack support for user mode applications */
310
-
#define X86_FEATURE_SRSO (11*32+24) /* "" AMD BTB untrain RETs */
311
-
#define X86_FEATURE_SRSO_ALIAS (11*32+25) /* "" AMD BTB untrain RETs through aliasing */
312
-
#define X86_FEATURE_IBPB_ON_VMEXIT (11*32+26) /* "" Issue an IBPB only on VMEXIT */
313
-
#define X86_FEATURE_APIC_MSRS_FENCE (11*32+27) /* "" IA32_TSC_DEADLINE and X2APIC MSRs need fencing */
314
-
#define X86_FEATURE_ZEN2 (11*32+28) /* "" CPU based on Zen2 microarchitecture */
315
-
#define X86_FEATURE_ZEN3 (11*32+29) /* "" CPU based on Zen3 microarchitecture */
316
-
#define X86_FEATURE_ZEN4 (11*32+30) /* "" CPU based on Zen4 microarchitecture */
317
-
#define X86_FEATURE_ZEN1 (11*32+31) /* "" CPU based on Zen1 microarchitecture */
286
+
#define X86_FEATURE_CQM_LLC (11*32+ 0) /* "cqm_llc" LLC QoS if 1 */
287
+
#define X86_FEATURE_CQM_OCCUP_LLC (11*32+ 1) /* "cqm_occup_llc" LLC occupancy monitoring */
288
+
#define X86_FEATURE_CQM_MBM_TOTAL (11*32+ 2) /* "cqm_mbm_total" LLC Total MBM monitoring */
289
+
#define X86_FEATURE_CQM_MBM_LOCAL (11*32+ 3) /* "cqm_mbm_local" LLC Local MBM monitoring */
290
+
#define X86_FEATURE_FENCE_SWAPGS_USER (11*32+ 4) /* LFENCE in user entry SWAPGS path */
291
+
#define X86_FEATURE_FENCE_SWAPGS_KERNEL (11*32+ 5) /* LFENCE in kernel entry SWAPGS path */
292
+
#define X86_FEATURE_SPLIT_LOCK_DETECT (11*32+ 6) /* "split_lock_detect" #AC for split lock */
293
+
#define X86_FEATURE_PER_THREAD_MBA (11*32+ 7) /* Per-thread Memory Bandwidth Allocation */
294
+
#define X86_FEATURE_SGX1 (11*32+ 8) /* Basic SGX */
295
+
#define X86_FEATURE_SGX2 (11*32+ 9) /* SGX Enclave Dynamic Memory Management (EDMM) */
296
+
#define X86_FEATURE_ENTRY_IBPB (11*32+10) /* Issue an IBPB on kernel entry */
297
+
#define X86_FEATURE_RRSBA_CTRL (11*32+11) /* RET prediction control */
298
+
#define X86_FEATURE_RETPOLINE (11*32+12) /* Generic Retpoline mitigation for Spectre variant 2 */
299
+
#define X86_FEATURE_RETPOLINE_LFENCE (11*32+13) /* Use LFENCE for Spectre variant 2 */
300
+
#define X86_FEATURE_RETHUNK (11*32+14) /* Use REturn THUNK */
301
+
#define X86_FEATURE_UNRET (11*32+15) /* AMD BTB untrain return */
302
+
#define X86_FEATURE_USE_IBPB_FW (11*32+16) /* Use IBPB during runtime firmware calls */
303
+
#define X86_FEATURE_RSB_VMEXIT_LITE (11*32+17) /* Fill RSB on VM exit when EIBRS is enabled */
304
+
#define X86_FEATURE_SGX_EDECCSSA (11*32+18) /* SGX EDECCSSA user leaf function */
305
+
#define X86_FEATURE_CALL_DEPTH (11*32+19) /* Call depth tracking for RSB stuffing */
306
+
#define X86_FEATURE_MSR_TSX_CTRL (11*32+20) /* MSR IA32_TSX_CTRL (Intel) implemented */
307
+
#define X86_FEATURE_SMBA (11*32+21) /* Slow Memory Bandwidth Allocation */
308
+
#define X86_FEATURE_BMEC (11*32+22) /* Bandwidth Monitoring Event Configuration */
309
+
#define X86_FEATURE_USER_SHSTK (11*32+23) /* "user_shstk" Shadow stack support for user mode applications */
310
+
#define X86_FEATURE_SRSO (11*32+24) /* AMD BTB untrain RETs */
311
+
#define X86_FEATURE_SRSO_ALIAS (11*32+25) /* AMD BTB untrain RETs through aliasing */
312
+
#define X86_FEATURE_IBPB_ON_VMEXIT (11*32+26) /* Issue an IBPB only on VMEXIT */
313
+
#define X86_FEATURE_APIC_MSRS_FENCE (11*32+27) /* IA32_TSC_DEADLINE and X2APIC MSRs need fencing */
314
+
#define X86_FEATURE_ZEN2 (11*32+28) /* CPU based on Zen2 microarchitecture */
315
+
#define X86_FEATURE_ZEN3 (11*32+29) /* CPU based on Zen3 microarchitecture */
316
+
#define X86_FEATURE_ZEN4 (11*32+30) /* CPU based on Zen4 microarchitecture */
317
+
#define X86_FEATURE_ZEN1 (11*32+31) /* CPU based on Zen1 microarchitecture */
318
318
319
319
/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */
320
-
#define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */
321
-
#define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* AVX512 BFLOAT16 instructions */
322
-
#define X86_FEATURE_CMPCCXADD (12*32+ 7) /* "" CMPccXADD instructions */
323
-
#define X86_FEATURE_ARCH_PERFMON_EXT (12*32+ 8) /* "" Intel Architectural PerfMon Extension */
324
-
#define X86_FEATURE_FZRM (12*32+10) /* "" Fast zero-length REP MOVSB */
325
-
#define X86_FEATURE_FSRS (12*32+11) /* "" Fast short REP STOSB */
326
-
#define X86_FEATURE_FSRC (12*32+12) /* "" Fast short REP {CMPSB,SCASB} */
327
-
#define X86_FEATURE_FRED (12*32+17) /* Flexible Return and Event Delivery */
328
-
#define X86_FEATURE_LKGS (12*32+18) /* "" Load "kernel" (userspace) GS */
329
-
#define X86_FEATURE_WRMSRNS (12*32+19) /* "" Non-serializing WRMSR */
330
-
#define X86_FEATURE_AMX_FP16 (12*32+21) /* "" AMX fp16 Support */
331
-
#define X86_FEATURE_AVX_IFMA (12*32+23) /* "" Support for VPMADD52[H,L]UQ */
332
-
#define X86_FEATURE_LAM (12*32+26) /* Linear Address Masking */
320
+
#define X86_FEATURE_AVX_VNNI (12*32+ 4) /* "avx_vnni" AVX VNNI instructions */
321
+
#define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* "avx512_bf16" AVX512 BFLOAT16 instructions */
322
+
#define X86_FEATURE_CMPCCXADD (12*32+ 7) /* CMPccXADD instructions */
323
+
#define X86_FEATURE_ARCH_PERFMON_EXT (12*32+ 8) /* Intel Architectural PerfMon Extension */
324
+
#define X86_FEATURE_FZRM (12*32+10) /* Fast zero-length REP MOVSB */
325
+
#define X86_FEATURE_FSRS (12*32+11) /* Fast short REP STOSB */
326
+
#define X86_FEATURE_FSRC (12*32+12) /* Fast short REP {CMPSB,SCASB} */
327
+
#define X86_FEATURE_FRED (12*32+17) /* "fred" Flexible Return and Event Delivery */
328
+
#define X86_FEATURE_LKGS (12*32+18) /* Load "kernel" (userspace) GS */
329
+
#define X86_FEATURE_WRMSRNS (12*32+19) /* Non-serializing WRMSR */
330
+
#define X86_FEATURE_AMX_FP16 (12*32+21) /* AMX fp16 Support */
331
+
#define X86_FEATURE_AVX_IFMA (12*32+23) /* Support for VPMADD52[H,L]UQ */
332
+
#define X86_FEATURE_LAM (12*32+26) /* "lam" Linear Address Masking */
333
333
334
334
/* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */
335
-
#define X86_FEATURE_CLZERO (13*32+ 0) /* CLZERO instruction */
336
-
#define X86_FEATURE_IRPERF (13*32+ 1) /* Instructions Retired Count */
337
-
#define X86_FEATURE_XSAVEERPTR (13*32+ 2) /* Always save/restore FP error pointers */
338
-
#define X86_FEATURE_RDPRU (13*32+ 4) /* Read processor register at user level */
339
-
#define X86_FEATURE_WBNOINVD (13*32+ 9) /* WBNOINVD instruction */
340
-
#define X86_FEATURE_AMD_IBPB (13*32+12) /* "" Indirect Branch Prediction Barrier */
341
-
#define X86_FEATURE_AMD_IBRS (13*32+14) /* "" Indirect Branch Restricted Speculation */
342
-
#define X86_FEATURE_AMD_STIBP (13*32+15) /* "" Single Thread Indirect Branch Predictors */
343
-
#define X86_FEATURE_AMD_STIBP_ALWAYS_ON (13*32+17) /* "" Single Thread Indirect Branch Predictors always-on preferred */
344
-
#define X86_FEATURE_AMD_PPIN (13*32+23) /* Protected Processor Inventory Number */
345
-
#define X86_FEATURE_AMD_SSBD (13*32+24) /* "" Speculative Store Bypass Disable */
346
-
#define X86_FEATURE_VIRT_SSBD (13*32+25) /* Virtualized Speculative Store Bypass Disable */
347
-
#define X86_FEATURE_AMD_SSB_NO (13*32+26) /* "" Speculative Store Bypass is fixed in hardware. */
348
-
#define X86_FEATURE_CPPC (13*32+27) /* Collaborative Processor Performance Control */
349
-
#define X86_FEATURE_AMD_PSFD (13*32+28) /* "" Predictive Store Forwarding Disable */
350
-
#define X86_FEATURE_BTC_NO (13*32+29) /* "" Not vulnerable to Branch Type Confusion */
351
-
#define X86_FEATURE_BRS (13*32+31) /* Branch Sampling available */
335
+
#define X86_FEATURE_CLZERO (13*32+ 0) /* "clzero" CLZERO instruction */
336
+
#define X86_FEATURE_IRPERF (13*32+ 1) /* "irperf" Instructions Retired Count */
337
+
#define X86_FEATURE_XSAVEERPTR (13*32+ 2) /* "xsaveerptr" Always save/restore FP error pointers */
338
+
#define X86_FEATURE_RDPRU (13*32+ 4) /* "rdpru" Read processor register at user level */
339
+
#define X86_FEATURE_WBNOINVD (13*32+ 9) /* "wbnoinvd" WBNOINVD instruction */
340
+
#define X86_FEATURE_AMD_IBPB (13*32+12) /* Indirect Branch Prediction Barrier */
341
+
#define X86_FEATURE_AMD_IBRS (13*32+14) /* Indirect Branch Restricted Speculation */
342
+
#define X86_FEATURE_AMD_STIBP (13*32+15) /* Single Thread Indirect Branch Predictors */
343
+
#define X86_FEATURE_AMD_STIBP_ALWAYS_ON (13*32+17) /* Single Thread Indirect Branch Predictors always-on preferred */
344
+
#define X86_FEATURE_AMD_PPIN (13*32+23) /* "amd_ppin" Protected Processor Inventory Number */
345
+
#define X86_FEATURE_AMD_SSBD (13*32+24) /* Speculative Store Bypass Disable */
346
+
#define X86_FEATURE_VIRT_SSBD (13*32+25) /* "virt_ssbd" Virtualized Speculative Store Bypass Disable */
347
+
#define X86_FEATURE_AMD_SSB_NO (13*32+26) /* Speculative Store Bypass is fixed in hardware. */
348
+
#define X86_FEATURE_CPPC (13*32+27) /* "cppc" Collaborative Processor Performance Control */
349
+
#define X86_FEATURE_AMD_PSFD (13*32+28) /* Predictive Store Forwarding Disable */
350
+
#define X86_FEATURE_BTC_NO (13*32+29) /* Not vulnerable to Branch Type Confusion */
351
+
#define X86_FEATURE_BRS (13*32+31) /* "brs" Branch Sampling available */
352
352
353
353
/* Thermal and Power Management Leaf, CPUID level 0x00000006 (EAX), word 14 */
354
-
#define X86_FEATURE_DTHERM (14*32+ 0) /* Digital Thermal Sensor */
355
-
#define X86_FEATURE_IDA (14*32+ 1) /* Intel Dynamic Acceleration */
356
-
#define X86_FEATURE_ARAT (14*32+ 2) /* Always Running APIC Timer */
357
-
#define X86_FEATURE_PLN (14*32+ 4) /* Intel Power Limit Notification */
358
-
#define X86_FEATURE_PTS (14*32+ 6) /* Intel Package Thermal Status */
359
-
#define X86_FEATURE_HWP (14*32+ 7) /* Intel Hardware P-states */
360
-
#define X86_FEATURE_HWP_NOTIFY (14*32+ 8) /* HWP Notification */
361
-
#define X86_FEATURE_HWP_ACT_WINDOW (14*32+ 9) /* HWP Activity Window */
362
-
#define X86_FEATURE_HWP_EPP (14*32+10) /* HWP Energy Perf. Preference */
363
-
#define X86_FEATURE_HWP_PKG_REQ (14*32+11) /* HWP Package Level Request */
364
-
#define X86_FEATURE_HFI (14*32+19) /* Hardware Feedback Interface */
354
+
#define X86_FEATURE_DTHERM (14*32+ 0) /* "dtherm" Digital Thermal Sensor */
355
+
#define X86_FEATURE_IDA (14*32+ 1) /* "ida" Intel Dynamic Acceleration */
356
+
#define X86_FEATURE_ARAT (14*32+ 2) /* "arat" Always Running APIC Timer */
357
+
#define X86_FEATURE_PLN (14*32+ 4) /* "pln" Intel Power Limit Notification */
358
+
#define X86_FEATURE_PTS (14*32+ 6) /* "pts" Intel Package Thermal Status */
359
+
#define X86_FEATURE_HWP (14*32+ 7) /* "hwp" Intel Hardware P-states */
360
+
#define X86_FEATURE_HWP_NOTIFY (14*32+ 8) /* "hwp_notify" HWP Notification */
361
+
#define X86_FEATURE_HWP_ACT_WINDOW (14*32+ 9) /* "hwp_act_window" HWP Activity Window */
362
+
#define X86_FEATURE_HWP_EPP (14*32+10) /* "hwp_epp" HWP Energy Perf. Preference */
363
+
#define X86_FEATURE_HWP_PKG_REQ (14*32+11) /* "hwp_pkg_req" HWP Package Level Request */
364
+
#define X86_FEATURE_HWP_HIGHEST_PERF_CHANGE (14*32+15) /* HWP Highest perf change */
365
+
#define X86_FEATURE_HFI (14*32+19) /* "hfi" Hardware Feedback Interface */
365
366
366
367
/* AMD SVM Feature Identification, CPUID level 0x8000000a (EDX), word 15 */
367
-
#define X86_FEATURE_NPT (15*32+ 0) /* Nested Page Table support */
368
-
#define X86_FEATURE_LBRV (15*32+ 1) /* LBR Virtualization support */
368
+
#define X86_FEATURE_NPT (15*32+ 0) /* "npt" Nested Page Table support */
369
+
#define X86_FEATURE_LBRV (15*32+ 1) /* "lbrv" LBR Virtualization support */
369
370
#define X86_FEATURE_SVML (15*32+ 2) /* "svm_lock" SVM locking MSR */
370
371
#define X86_FEATURE_NRIPS (15*32+ 3) /* "nrip_save" SVM next_rip save */
371
372
#define X86_FEATURE_TSCRATEMSR (15*32+ 4) /* "tsc_scale" TSC scaling support */
372
373
#define X86_FEATURE_VMCBCLEAN (15*32+ 5) /* "vmcb_clean" VMCB clean bits support */
373
-
#define X86_FEATURE_FLUSHBYASID (15*32+ 6) /* flush-by-ASID support */
374
-
#define X86_FEATURE_DECODEASSISTS (15*32+ 7) /* Decode Assists support */
375
-
#define X86_FEATURE_PAUSEFILTER (15*32+10) /* filtered pause intercept */
376
-
#define X86_FEATURE_PFTHRESHOLD (15*32+12) /* pause filter threshold */
377
-
#define X86_FEATURE_AVIC (15*32+13) /* Virtual Interrupt Controller */
378
-
#define X86_FEATURE_V_VMSAVE_VMLOAD (15*32+15) /* Virtual VMSAVE VMLOAD */
379
-
#define X86_FEATURE_VGIF (15*32+16) /* Virtual GIF */
380
-
#define X86_FEATURE_X2AVIC (15*32+18) /* Virtual x2apic */
381
-
#define X86_FEATURE_V_SPEC_CTRL (15*32+20) /* Virtual SPEC_CTRL */
382
-
#define X86_FEATURE_VNMI (15*32+25) /* Virtual NMI */
383
-
#define X86_FEATURE_SVME_ADDR_CHK (15*32+28) /* "" SVME addr check */
374
+
#define X86_FEATURE_FLUSHBYASID (15*32+ 6) /* "flushbyasid" Flush-by-ASID support */
375
+
#define X86_FEATURE_DECODEASSISTS (15*32+ 7) /* "decodeassists" Decode Assists support */
376
+
#define X86_FEATURE_PAUSEFILTER (15*32+10) /* "pausefilter" Filtered pause intercept */
377
+
#define X86_FEATURE_PFTHRESHOLD (15*32+12) /* "pfthreshold" Pause filter threshold */
378
+
#define X86_FEATURE_AVIC (15*32+13) /* "avic" Virtual Interrupt Controller */
379
+
#define X86_FEATURE_V_VMSAVE_VMLOAD (15*32+15) /* "v_vmsave_vmload" Virtual VMSAVE VMLOAD */
380
+
#define X86_FEATURE_VGIF (15*32+16) /* "vgif" Virtual GIF */
381
+
#define X86_FEATURE_X2AVIC (15*32+18) /* "x2avic" Virtual x2apic */
382
+
#define X86_FEATURE_V_SPEC_CTRL (15*32+20) /* "v_spec_ctrl" Virtual SPEC_CTRL */
383
+
#define X86_FEATURE_VNMI (15*32+25) /* "vnmi" Virtual NMI */
384
+
#define X86_FEATURE_SVME_ADDR_CHK (15*32+28) /* SVME addr check */
384
385
385
386
/* Intel-defined CPU features, CPUID level 0x00000007:0 (ECX), word 16 */
386
-
#define X86_FEATURE_AVX512VBMI (16*32+ 1) /* AVX512 Vector Bit Manipulation instructions*/
387
-
#define X86_FEATURE_UMIP (16*32+ 2) /* User Mode Instruction Protection */
388
-
#define X86_FEATURE_PKU (16*32+ 3) /* Protection Keys for Userspace */
389
-
#define X86_FEATURE_OSPKE (16*32+ 4) /* OS Protection Keys Enable */
390
-
#define X86_FEATURE_WAITPKG (16*32+ 5) /* UMONITOR/UMWAIT/TPAUSE Instructions */
391
-
#define X86_FEATURE_AVX512_VBMI2 (16*32+ 6) /* Additional AVX512 Vector Bit Manipulation Instructions */
392
-
#define X86_FEATURE_SHSTK (16*32+ 7) /* "" Shadow stack */
393
-
#define X86_FEATURE_GFNI (16*32+ 8) /* Galois Field New Instructions */
394
-
#define X86_FEATURE_VAES (16*32+ 9) /* Vector AES */
395
-
#define X86_FEATURE_VPCLMULQDQ (16*32+10) /* Carry-Less Multiplication Double Quadword */
396
-
#define X86_FEATURE_AVX512_VNNI (16*32+11) /* Vector Neural Network Instructions */
397
-
#define X86_FEATURE_AVX512_BITALG (16*32+12) /* Support for VPOPCNT[B,W] and VPSHUF-BITQMB instructions */
398
-
#define X86_FEATURE_TME (16*32+13) /* Intel Total Memory Encryption */
399
-
#define X86_FEATURE_AVX512_VPOPCNTDQ (16*32+14) /* POPCNT for vectors of DW/QW */
400
-
#define X86_FEATURE_LA57 (16*32+16) /* 5-level page tables */
401
-
#define X86_FEATURE_RDPID (16*32+22) /* RDPID instruction */
402
-
#define X86_FEATURE_BUS_LOCK_DETECT (16*32+24) /* Bus Lock detect */
403
-
#define X86_FEATURE_CLDEMOTE (16*32+25) /* CLDEMOTE instruction */
404
-
#define X86_FEATURE_MOVDIRI (16*32+27) /* MOVDIRI instruction */
405
-
#define X86_FEATURE_MOVDIR64B (16*32+28) /* MOVDIR64B instruction */
406
-
#define X86_FEATURE_ENQCMD (16*32+29) /* ENQCMD and ENQCMDS instructions */
407
-
#define X86_FEATURE_SGX_LC (16*32+30) /* Software Guard Extensions Launch Control */
387
+
#define X86_FEATURE_AVX512VBMI (16*32+ 1) /* "avx512vbmi" AVX512 Vector Bit Manipulation instructions*/
388
+
#define X86_FEATURE_UMIP (16*32+ 2) /* "umip" User Mode Instruction Protection */
389
+
#define X86_FEATURE_PKU (16*32+ 3) /* "pku" Protection Keys for Userspace */
390
+
#define X86_FEATURE_OSPKE (16*32+ 4) /* "ospke" OS Protection Keys Enable */
391
+
#define X86_FEATURE_WAITPKG (16*32+ 5) /* "waitpkg" UMONITOR/UMWAIT/TPAUSE Instructions */
392
+
#define X86_FEATURE_AVX512_VBMI2 (16*32+ 6) /* "avx512_vbmi2" Additional AVX512 Vector Bit Manipulation Instructions */
393
+
#define X86_FEATURE_SHSTK (16*32+ 7) /* Shadow stack */
394
+
#define X86_FEATURE_GFNI (16*32+ 8) /* "gfni" Galois Field New Instructions */
395
+
#define X86_FEATURE_VAES (16*32+ 9) /* "vaes" Vector AES */
396
+
#define X86_FEATURE_VPCLMULQDQ (16*32+10) /* "vpclmulqdq" Carry-Less Multiplication Double Quadword */
397
+
#define X86_FEATURE_AVX512_VNNI (16*32+11) /* "avx512_vnni" Vector Neural Network Instructions */
398
+
#define X86_FEATURE_AVX512_BITALG (16*32+12) /* "avx512_bitalg" Support for VPOPCNT[B,W] and VPSHUF-BITQMB instructions */
399
+
#define X86_FEATURE_TME (16*32+13) /* "tme" Intel Total Memory Encryption */
400
+
#define X86_FEATURE_AVX512_VPOPCNTDQ (16*32+14) /* "avx512_vpopcntdq" POPCNT for vectors of DW/QW */
401
+
#define X86_FEATURE_LA57 (16*32+16) /* "la57" 5-level page tables */
402
+
#define X86_FEATURE_RDPID (16*32+22) /* "rdpid" RDPID instruction */
403
+
#define X86_FEATURE_BUS_LOCK_DETECT (16*32+24) /* "bus_lock_detect" Bus Lock detect */
404
+
#define X86_FEATURE_CLDEMOTE (16*32+25) /* "cldemote" CLDEMOTE instruction */
405
+
#define X86_FEATURE_MOVDIRI (16*32+27) /* "movdiri" MOVDIRI instruction */
406
+
#define X86_FEATURE_MOVDIR64B (16*32+28) /* "movdir64b" MOVDIR64B instruction */
407
+
#define X86_FEATURE_ENQCMD (16*32+29) /* "enqcmd" ENQCMD and ENQCMDS instructions */
408
+
#define X86_FEATURE_SGX_LC (16*32+30) /* "sgx_lc" Software Guard Extensions Launch Control */
408
409
409
410
/* AMD-defined CPU features, CPUID level 0x80000007 (EBX), word 17 */
410
-
#define X86_FEATURE_OVERFLOW_RECOV (17*32+ 0) /* MCA overflow recovery support */
411
-
#define X86_FEATURE_SUCCOR (17*32+ 1) /* Uncorrectable error containment and recovery */
412
-
#define X86_FEATURE_SMCA (17*32+ 3) /* Scalable MCA */
411
+
#define X86_FEATURE_OVERFLOW_RECOV (17*32+ 0) /* "overflow_recov" MCA overflow recovery support */
412
+
#define X86_FEATURE_SUCCOR (17*32+ 1) /* "succor" Uncorrectable error containment and recovery */
413
+
#define X86_FEATURE_SMCA (17*32+ 3) /* "smca" Scalable MCA */
413
414
414
415
/* Intel-defined CPU features, CPUID level 0x00000007:0 (EDX), word 18 */
415
-
#define X86_FEATURE_AVX512_4VNNIW (18*32+ 2) /* AVX-512 Neural Network Instructions */
416
-
#define X86_FEATURE_AVX512_4FMAPS (18*32+ 3) /* AVX-512 Multiply Accumulation Single precision */
417
-
#define X86_FEATURE_FSRM (18*32+ 4) /* Fast Short Rep Mov */
418
-
#define X86_FEATURE_AVX512_VP2INTERSECT (18*32+ 8) /* AVX-512 Intersect for D/Q */
419
-
#define X86_FEATURE_SRBDS_CTRL (18*32+ 9) /* "" SRBDS mitigation MSR available */
420
-
#define X86_FEATURE_MD_CLEAR (18*32+10) /* VERW clears CPU buffers */
421
-
#define X86_FEATURE_RTM_ALWAYS_ABORT (18*32+11) /* "" RTM transaction always aborts */
422
-
#define X86_FEATURE_TSX_FORCE_ABORT (18*32+13) /* "" TSX_FORCE_ABORT */
423
-
#define X86_FEATURE_SERIALIZE (18*32+14) /* SERIALIZE instruction */
424
-
#define X86_FEATURE_HYBRID_CPU (18*32+15) /* "" This part has CPUs of more than one type */
425
-
#define X86_FEATURE_TSXLDTRK (18*32+16) /* TSX Suspend Load Address Tracking */
426
-
#define X86_FEATURE_PCONFIG (18*32+18) /* Intel PCONFIG */
427
-
#define X86_FEATURE_ARCH_LBR (18*32+19) /* Intel ARCH LBR */
428
-
#define X86_FEATURE_IBT (18*32+20) /* Indirect Branch Tracking */
429
-
#define X86_FEATURE_AMX_BF16 (18*32+22) /* AMX bf16 Support */
430
-
#define X86_FEATURE_AVX512_FP16 (18*32+23) /* AVX512 FP16 */
431
-
#define X86_FEATURE_AMX_TILE (18*32+24) /* AMX tile Support */
432
-
#define X86_FEATURE_AMX_INT8 (18*32+25) /* AMX int8 Support */
433
-
#define X86_FEATURE_SPEC_CTRL (18*32+26) /* "" Speculation Control (IBRS + IBPB) */
434
-
#define X86_FEATURE_INTEL_STIBP (18*32+27) /* "" Single Thread Indirect Branch Predictors */
435
-
#define X86_FEATURE_FLUSH_L1D (18*32+28) /* Flush L1D cache */
436
-
#define X86_FEATURE_ARCH_CAPABILITIES (18*32+29) /* IA32_ARCH_CAPABILITIES MSR (Intel) */
437
-
#define X86_FEATURE_CORE_CAPABILITIES (18*32+30) /* "" IA32_CORE_CAPABILITIES MSR */
438
-
#define X86_FEATURE_SPEC_CTRL_SSBD (18*32+31) /* "" Speculative Store Bypass Disable */
416
+
#define X86_FEATURE_AVX512_4VNNIW (18*32+ 2) /* "avx512_4vnniw" AVX-512 Neural Network Instructions */
417
+
#define X86_FEATURE_AVX512_4FMAPS (18*32+ 3) /* "avx512_4fmaps" AVX-512 Multiply Accumulation Single precision */
418
+
#define X86_FEATURE_FSRM (18*32+ 4) /* "fsrm" Fast Short Rep Mov */
419
+
#define X86_FEATURE_AVX512_VP2INTERSECT (18*32+ 8) /* "avx512_vp2intersect" AVX-512 Intersect for D/Q */
420
+
#define X86_FEATURE_SRBDS_CTRL (18*32+ 9) /* SRBDS mitigation MSR available */
421
+
#define X86_FEATURE_MD_CLEAR (18*32+10) /* "md_clear" VERW clears CPU buffers */
422
+
#define X86_FEATURE_RTM_ALWAYS_ABORT (18*32+11) /* RTM transaction always aborts */
423
+
#define X86_FEATURE_TSX_FORCE_ABORT (18*32+13) /* TSX_FORCE_ABORT */
424
+
#define X86_FEATURE_SERIALIZE (18*32+14) /* "serialize" SERIALIZE instruction */
425
+
#define X86_FEATURE_HYBRID_CPU (18*32+15) /* This part has CPUs of more than one type */
426
+
#define X86_FEATURE_TSXLDTRK (18*32+16) /* "tsxldtrk" TSX Suspend Load Address Tracking */
427
+
#define X86_FEATURE_PCONFIG (18*32+18) /* "pconfig" Intel PCONFIG */
428
+
#define X86_FEATURE_ARCH_LBR (18*32+19) /* "arch_lbr" Intel ARCH LBR */
429
+
#define X86_FEATURE_IBT (18*32+20) /* "ibt" Indirect Branch Tracking */
430
+
#define X86_FEATURE_AMX_BF16 (18*32+22) /* "amx_bf16" AMX bf16 Support */
431
+
#define X86_FEATURE_AVX512_FP16 (18*32+23) /* "avx512_fp16" AVX512 FP16 */
432
+
#define X86_FEATURE_AMX_TILE (18*32+24) /* "amx_tile" AMX tile Support */
433
+
#define X86_FEATURE_AMX_INT8 (18*32+25) /* "amx_int8" AMX int8 Support */
434
+
#define X86_FEATURE_SPEC_CTRL (18*32+26) /* Speculation Control (IBRS + IBPB) */
435
+
#define X86_FEATURE_INTEL_STIBP (18*32+27) /* Single Thread Indirect Branch Predictors */
436
+
#define X86_FEATURE_FLUSH_L1D (18*32+28) /* "flush_l1d" Flush L1D cache */
437
+
#define X86_FEATURE_ARCH_CAPABILITIES (18*32+29) /* "arch_capabilities" IA32_ARCH_CAPABILITIES MSR (Intel) */
438
+
#define X86_FEATURE_CORE_CAPABILITIES (18*32+30) /* IA32_CORE_CAPABILITIES MSR */
439
+
#define X86_FEATURE_SPEC_CTRL_SSBD (18*32+31) /* Speculative Store Bypass Disable */
439
440
440
441
/* AMD-defined memory encryption features, CPUID level 0x8000001f (EAX), word 19 */
441
-
#define X86_FEATURE_SME (19*32+ 0) /* AMD Secure Memory Encryption */
442
-
#define X86_FEATURE_SEV (19*32+ 1) /* AMD Secure Encrypted Virtualization */
443
-
#define X86_FEATURE_VM_PAGE_FLUSH (19*32+ 2) /* "" VM Page Flush MSR is supported */
444
-
#define X86_FEATURE_SEV_ES (19*32+ 3) /* AMD Secure Encrypted Virtualization - Encrypted State */
445
-
#define X86_FEATURE_SEV_SNP (19*32+ 4) /* AMD Secure Encrypted Virtualization - Secure Nested Paging */
446
-
#define X86_FEATURE_V_TSC_AUX (19*32+ 9) /* "" Virtual TSC_AUX */
447
-
#define X86_FEATURE_SME_COHERENT (19*32+10) /* "" AMD hardware-enforced cache coherency */
448
-
#define X86_FEATURE_DEBUG_SWAP (19*32+14) /* AMD SEV-ES full debug state swap support */
442
+
#define X86_FEATURE_SME (19*32+ 0) /* "sme" AMD Secure Memory Encryption */
443
+
#define X86_FEATURE_SEV (19*32+ 1) /* "sev" AMD Secure Encrypted Virtualization */
444
+
#define X86_FEATURE_VM_PAGE_FLUSH (19*32+ 2) /* VM Page Flush MSR is supported */
445
+
#define X86_FEATURE_SEV_ES (19*32+ 3) /* "sev_es" AMD Secure Encrypted Virtualization - Encrypted State */
446
+
#define X86_FEATURE_SEV_SNP (19*32+ 4) /* "sev_snp" AMD Secure Encrypted Virtualization - Secure Nested Paging */
447
+
#define X86_FEATURE_V_TSC_AUX (19*32+ 9) /* Virtual TSC_AUX */
448
+
#define X86_FEATURE_SME_COHERENT (19*32+10) /* AMD hardware-enforced cache coherency */
449
+
#define X86_FEATURE_DEBUG_SWAP (19*32+14) /* "debug_swap" AMD SEV-ES full debug state swap support */
450
+
#define X86_FEATURE_SVSM (19*32+28) /* "svsm" SVSM present */
449
451
450
452
/* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX), word 20 */
451
-
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" No Nested Data Breakpoints */
452
-
#define X86_FEATURE_WRMSR_XX_BASE_NS (20*32+ 1) /* "" WRMSR to {FS,GS,KERNEL_GS}_BASE is non-serializing */
453
-
#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
454
-
#define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* "" Null Selector Clears Base */
455
-
#define X86_FEATURE_AUTOIBRS (20*32+ 8) /* "" Automatic IBRS */
456
-
#define X86_FEATURE_NO_SMM_CTL_MSR (20*32+ 9) /* "" SMM_CTL MSR is not present */
453
+
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* No Nested Data Breakpoints */
454
+
#define X86_FEATURE_WRMSR_XX_BASE_NS (20*32+ 1) /* WRMSR to {FS,GS,KERNEL_GS}_BASE is non-serializing */
455
+
#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* LFENCE always serializing / synchronizes RDTSC */
456
+
#define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* Null Selector Clears Base */
457
+
#define X86_FEATURE_AUTOIBRS (20*32+ 8) /* Automatic IBRS */
458
+
#define X86_FEATURE_NO_SMM_CTL_MSR (20*32+ 9) /* SMM_CTL MSR is not present */
457
459
458
-
#define X86_FEATURE_SBPB (20*32+27) /* "" Selective Branch Prediction Barrier */
459
-
#define X86_FEATURE_IBPB_BRTYPE (20*32+28) /* "" MSR_PRED_CMD[IBPB] flushes all branch type predictions */
460
-
#define X86_FEATURE_SRSO_NO (20*32+29) /* "" CPU is not affected by SRSO */
460
+
#define X86_FEATURE_SBPB (20*32+27) /* Selective Branch Prediction Barrier */
461
+
#define X86_FEATURE_IBPB_BRTYPE (20*32+28) /* MSR_PRED_CMD[IBPB] flushes all branch type predictions */
462
+
#define X86_FEATURE_SRSO_NO (20*32+29) /* CPU is not affected by SRSO */
461
463
462
464
/*
463
465
* Extended auxiliary flags: Linux defined - for features scattered in various
···
467
465
*
468
466
* Reuse free bits when adding new feature flags!
469
467
*/
470
-
#define X86_FEATURE_AMD_LBR_PMC_FREEZE (21*32+ 0) /* AMD LBR and PMC Freeze */
471
-
#define X86_FEATURE_CLEAR_BHB_LOOP (21*32+ 1) /* "" Clear branch history at syscall entry using SW loop */
472
-
#define X86_FEATURE_BHI_CTRL (21*32+ 2) /* "" BHI_DIS_S HW control available */
473
-
#define X86_FEATURE_CLEAR_BHB_HW (21*32+ 3) /* "" BHI_DIS_S HW control enabled */
474
-
#define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* "" Clear branch history at vmexit using SW loop */
468
+
#define X86_FEATURE_AMD_LBR_PMC_FREEZE (21*32+ 0) /* "amd_lbr_pmc_freeze" AMD LBR and PMC Freeze */
469
+
#define X86_FEATURE_CLEAR_BHB_LOOP (21*32+ 1) /* Clear branch history at syscall entry using SW loop */
470
+
#define X86_FEATURE_BHI_CTRL (21*32+ 2) /* BHI_DIS_S HW control available */
471
+
#define X86_FEATURE_CLEAR_BHB_HW (21*32+ 3) /* BHI_DIS_S HW control enabled */
472
+
#define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* Clear branch history at vmexit using SW loop */
473
+
#define X86_FEATURE_FAST_CPPC (21*32 + 5) /* AMD Fast CPPC */
475
474
476
475
/*
477
476
* BUG word(s)
478
477
*/
479
478
#define X86_BUG(x) (NCAPINTS*32 + (x))
480
479
481
-
#define X86_BUG_F00F X86_BUG(0) /* Intel F00F */
482
-
#define X86_BUG_FDIV X86_BUG(1) /* FPU FDIV */
483
-
#define X86_BUG_COMA X86_BUG(2) /* Cyrix 6x86 coma */
480
+
#define X86_BUG_F00F X86_BUG(0) /* "f00f" Intel F00F */
481
+
#define X86_BUG_FDIV X86_BUG(1) /* "fdiv" FPU FDIV */
482
+
#define X86_BUG_COMA X86_BUG(2) /* "coma" Cyrix 6x86 coma */
484
483
#define X86_BUG_AMD_TLB_MMATCH X86_BUG(3) /* "tlb_mmatch" AMD Erratum 383 */
485
484
#define X86_BUG_AMD_APIC_C1E X86_BUG(4) /* "apic_c1e" AMD Erratum 400 */
486
-
#define X86_BUG_11AP X86_BUG(5) /* Bad local APIC aka 11AP */
487
-
#define X86_BUG_FXSAVE_LEAK X86_BUG(6) /* FXSAVE leaks FOP/FIP/FOP */
488
-
#define X86_BUG_CLFLUSH_MONITOR X86_BUG(7) /* AAI65, CLFLUSH required before MONITOR */
489
-
#define X86_BUG_SYSRET_SS_ATTRS X86_BUG(8) /* SYSRET doesn't fix up SS attrs */
485
+
#define X86_BUG_11AP X86_BUG(5) /* "11ap" Bad local APIC aka 11AP */
486
+
#define X86_BUG_FXSAVE_LEAK X86_BUG(6) /* "fxsave_leak" FXSAVE leaks FOP/FIP/FOP */
487
+
#define X86_BUG_CLFLUSH_MONITOR X86_BUG(7) /* "clflush_monitor" AAI65, CLFLUSH required before MONITOR */
488
+
#define X86_BUG_SYSRET_SS_ATTRS X86_BUG(8) /* "sysret_ss_attrs" SYSRET doesn't fix up SS attrs */
490
489
#ifdef CONFIG_X86_32
491
490
/*
492
491
* 64-bit kernels don't use X86_BUG_ESPFIX. Make the define conditional
493
492
* to avoid confusion.
494
493
*/
495
-
#define X86_BUG_ESPFIX X86_BUG(9) /* "" IRET to 16-bit SS corrupts ESP/RSP high bits */
494
+
#define X86_BUG_ESPFIX X86_BUG(9) /* IRET to 16-bit SS corrupts ESP/RSP high bits */
496
495
#endif
497
-
#define X86_BUG_NULL_SEG X86_BUG(10) /* Nulling a selector preserves the base */
498
-
#define X86_BUG_SWAPGS_FENCE X86_BUG(11) /* SWAPGS without input dep on GS */
499
-
#define X86_BUG_MONITOR X86_BUG(12) /* IPI required to wake up remote CPU */
500
-
#define X86_BUG_AMD_E400 X86_BUG(13) /* CPU is among the affected by Erratum 400 */
501
-
#define X86_BUG_CPU_MELTDOWN X86_BUG(14) /* CPU is affected by meltdown attack and needs kernel page table isolation */
502
-
#define X86_BUG_SPECTRE_V1 X86_BUG(15) /* CPU is affected by Spectre variant 1 attack with conditional branches */
503
-
#define X86_BUG_SPECTRE_V2 X86_BUG(16) /* CPU is affected by Spectre variant 2 attack with indirect branches */
504
-
#define X86_BUG_SPEC_STORE_BYPASS X86_BUG(17) /* CPU is affected by speculative store bypass attack */
505
-
#define X86_BUG_L1TF X86_BUG(18) /* CPU is affected by L1 Terminal Fault */
506
-
#define X86_BUG_MDS X86_BUG(19) /* CPU is affected by Microarchitectural data sampling */
507
-
#define X86_BUG_MSBDS_ONLY X86_BUG(20) /* CPU is only affected by the MSDBS variant of BUG_MDS */
508
-
#define X86_BUG_SWAPGS X86_BUG(21) /* CPU is affected by speculation through SWAPGS */
509
-
#define X86_BUG_TAA X86_BUG(22) /* CPU is affected by TSX Async Abort(TAA) */
510
-
#define X86_BUG_ITLB_MULTIHIT X86_BUG(23) /* CPU may incur MCE during certain page attribute changes */
511
-
#define X86_BUG_SRBDS X86_BUG(24) /* CPU may leak RNG bits if not mitigated */
512
-
#define X86_BUG_MMIO_STALE_DATA X86_BUG(25) /* CPU is affected by Processor MMIO Stale Data vulnerabilities */
513
-
#define X86_BUG_MMIO_UNKNOWN X86_BUG(26) /* CPU is too old and its MMIO Stale Data status is unknown */
514
-
#define X86_BUG_RETBLEED X86_BUG(27) /* CPU is affected by RETBleed */
515
-
#define X86_BUG_EIBRS_PBRSB X86_BUG(28) /* EIBRS is vulnerable to Post Barrier RSB Predictions */
516
-
#define X86_BUG_SMT_RSB X86_BUG(29) /* CPU is vulnerable to Cross-Thread Return Address Predictions */
517
-
#define X86_BUG_GDS X86_BUG(30) /* CPU is affected by Gather Data Sampling */
518
-
#define X86_BUG_TDX_PW_MCE X86_BUG(31) /* CPU may incur #MC if non-TD software does partial write to TDX private memory */
496
+
#define X86_BUG_NULL_SEG X86_BUG(10) /* "null_seg" Nulling a selector preserves the base */
497
+
#define X86_BUG_SWAPGS_FENCE X86_BUG(11) /* "swapgs_fence" SWAPGS without input dep on GS */
498
+
#define X86_BUG_MONITOR X86_BUG(12) /* "monitor" IPI required to wake up remote CPU */
499
+
#define X86_BUG_AMD_E400 X86_BUG(13) /* "amd_e400" CPU is among the affected by Erratum 400 */
500
+
#define X86_BUG_CPU_MELTDOWN X86_BUG(14) /* "cpu_meltdown" CPU is affected by meltdown attack and needs kernel page table isolation */
501
+
#define X86_BUG_SPECTRE_V1 X86_BUG(15) /* "spectre_v1" CPU is affected by Spectre variant 1 attack with conditional branches */
502
+
#define X86_BUG_SPECTRE_V2 X86_BUG(16) /* "spectre_v2" CPU is affected by Spectre variant 2 attack with indirect branches */
503
+
#define X86_BUG_SPEC_STORE_BYPASS X86_BUG(17) /* "spec_store_bypass" CPU is affected by speculative store bypass attack */
504
+
#define X86_BUG_L1TF X86_BUG(18) /* "l1tf" CPU is affected by L1 Terminal Fault */
505
+
#define X86_BUG_MDS X86_BUG(19) /* "mds" CPU is affected by Microarchitectural data sampling */
506
+
#define X86_BUG_MSBDS_ONLY X86_BUG(20) /* "msbds_only" CPU is only affected by the MSDBS variant of BUG_MDS */
507
+
#define X86_BUG_SWAPGS X86_BUG(21) /* "swapgs" CPU is affected by speculation through SWAPGS */
508
+
#define X86_BUG_TAA X86_BUG(22) /* "taa" CPU is affected by TSX Async Abort(TAA) */
509
+
#define X86_BUG_ITLB_MULTIHIT X86_BUG(23) /* "itlb_multihit" CPU may incur MCE during certain page attribute changes */
510
+
#define X86_BUG_SRBDS X86_BUG(24) /* "srbds" CPU may leak RNG bits if not mitigated */
511
+
#define X86_BUG_MMIO_STALE_DATA X86_BUG(25) /* "mmio_stale_data" CPU is affected by Processor MMIO Stale Data vulnerabilities */
512
+
#define X86_BUG_MMIO_UNKNOWN X86_BUG(26) /* "mmio_unknown" CPU is too old and its MMIO Stale Data status is unknown */
513
+
#define X86_BUG_RETBLEED X86_BUG(27) /* "retbleed" CPU is affected by RETBleed */
514
+
#define X86_BUG_EIBRS_PBRSB X86_BUG(28) /* "eibrs_pbrsb" EIBRS is vulnerable to Post Barrier RSB Predictions */
515
+
#define X86_BUG_SMT_RSB X86_BUG(29) /* "smt_rsb" CPU is vulnerable to Cross-Thread Return Address Predictions */
516
+
#define X86_BUG_GDS X86_BUG(30) /* "gds" CPU is affected by Gather Data Sampling */
517
+
#define X86_BUG_TDX_PW_MCE X86_BUG(31) /* "tdx_pw_mce" CPU may incur #MC if non-TD software does partial write to TDX private memory */
519
518
520
519
/* BUG word 2 */
521
-
#define X86_BUG_SRSO X86_BUG(1*32 + 0) /* AMD SRSO bug */
522
-
#define X86_BUG_DIV0 X86_BUG(1*32 + 1) /* AMD DIV0 speculation bug */
523
-
#define X86_BUG_RFDS X86_BUG(1*32 + 2) /* CPU is vulnerable to Register File Data Sampling */
524
-
#define X86_BUG_BHI X86_BUG(1*32 + 3) /* CPU is affected by Branch History Injection */
520
+
#define X86_BUG_SRSO X86_BUG(1*32 + 0) /* "srso" AMD SRSO bug */
521
+
#define X86_BUG_DIV0 X86_BUG(1*32 + 1) /* "div0" AMD DIV0 speculation bug */
522
+
#define X86_BUG_RFDS X86_BUG(1*32 + 2) /* "rfds" CPU is vulnerable to Register File Data Sampling */
523
+
#define X86_BUG_BHI X86_BUG(1*32 + 3) /* "bhi" CPU is affected by Branch History Injection */
525
524
#endif /* _ASM_X86_CPUFEATURES_H */
+11
tools/arch/x86/include/asm/msr-index.h
+11
tools/arch/x86/include/asm/msr-index.h
···
566
566
#define MSR_RELOAD_PMC0 0x000014c1
567
567
#define MSR_RELOAD_FIXED_CTR0 0x00001309
568
568
569
+
/* V6 PMON MSR range */
570
+
#define MSR_IA32_PMC_V6_GP0_CTR 0x1900
571
+
#define MSR_IA32_PMC_V6_GP0_CFG_A 0x1901
572
+
#define MSR_IA32_PMC_V6_FX0_CTR 0x1980
573
+
#define MSR_IA32_PMC_V6_STEP 4
574
+
569
575
/* KeyID partitioning between MKTME and TDX */
570
576
#define MSR_IA32_MKTME_KEYID_PARTITIONING 0x00000087
571
577
···
665
659
666
660
#define MSR_AMD64_RMP_BASE 0xc0010132
667
661
#define MSR_AMD64_RMP_END 0xc0010133
662
+
663
+
#define MSR_SVSM_CAA 0xc001f000
668
664
669
665
/* AMD Collaborative Processor Performance Control MSRs */
670
666
#define MSR_AMD_CPPC_CAP1 0xc00102b0
···
789
781
#define MSR_K7_HWCR_IRPERF_EN BIT_ULL(MSR_K7_HWCR_IRPERF_EN_BIT)
790
782
#define MSR_K7_FID_VID_CTL 0xc0010041
791
783
#define MSR_K7_FID_VID_STATUS 0xc0010042
784
+
#define MSR_K7_HWCR_CPB_DIS_BIT 25
785
+
#define MSR_K7_HWCR_CPB_DIS BIT_ULL(MSR_K7_HWCR_CPB_DIS_BIT)
792
786
793
787
/* K6 MSRs */
794
788
#define MSR_K6_WHCR 0xc0000082
···
1174
1164
#define MSR_IA32_QM_CTR 0xc8e
1175
1165
#define MSR_IA32_PQR_ASSOC 0xc8f
1176
1166
#define MSR_IA32_L3_CBM_BASE 0xc90
1167
+
#define MSR_RMID_SNC_CONFIG 0xca0
1177
1168
#define MSR_IA32_L2_CBM_BASE 0xd10
1178
1169
#define MSR_IA32_MBA_THRTL_BASE 0xd50
1179
1170
+49
tools/arch/x86/include/uapi/asm/kvm.h
+49
tools/arch/x86/include/uapi/asm/kvm.h
···
106
106
107
107
#define KVM_RUN_X86_SMM (1 << 0)
108
108
#define KVM_RUN_X86_BUS_LOCK (1 << 1)
109
+
#define KVM_RUN_X86_GUEST_MODE (1 << 2)
109
110
110
111
/* for KVM_GET_REGS and KVM_SET_REGS */
111
112
struct kvm_regs {
···
698
697
/* Second time is the charm; improved versions of the above ioctls. */
699
698
KVM_SEV_INIT2,
700
699
700
+
/* SNP-specific commands */
701
+
KVM_SEV_SNP_LAUNCH_START = 100,
702
+
KVM_SEV_SNP_LAUNCH_UPDATE,
703
+
KVM_SEV_SNP_LAUNCH_FINISH,
704
+
701
705
KVM_SEV_NR_MAX,
702
706
};
703
707
···
830
824
__u32 pad2;
831
825
};
832
826
827
+
struct kvm_sev_snp_launch_start {
828
+
__u64 policy;
829
+
__u8 gosvw[16];
830
+
__u16 flags;
831
+
__u8 pad0[6];
832
+
__u64 pad1[4];
833
+
};
834
+
835
+
/* Kept in sync with firmware values for simplicity. */
836
+
#define KVM_SEV_SNP_PAGE_TYPE_NORMAL 0x1
837
+
#define KVM_SEV_SNP_PAGE_TYPE_ZERO 0x3
838
+
#define KVM_SEV_SNP_PAGE_TYPE_UNMEASURED 0x4
839
+
#define KVM_SEV_SNP_PAGE_TYPE_SECRETS 0x5
840
+
#define KVM_SEV_SNP_PAGE_TYPE_CPUID 0x6
841
+
842
+
struct kvm_sev_snp_launch_update {
843
+
__u64 gfn_start;
844
+
__u64 uaddr;
845
+
__u64 len;
846
+
__u8 type;
847
+
__u8 pad0;
848
+
__u16 flags;
849
+
__u32 pad1;
850
+
__u64 pad2[4];
851
+
};
852
+
853
+
#define KVM_SEV_SNP_ID_BLOCK_SIZE 96
854
+
#define KVM_SEV_SNP_ID_AUTH_SIZE 4096
855
+
#define KVM_SEV_SNP_FINISH_DATA_SIZE 32
856
+
857
+
struct kvm_sev_snp_launch_finish {
858
+
__u64 id_block_uaddr;
859
+
__u64 id_auth_uaddr;
860
+
__u8 id_block_en;
861
+
__u8 auth_key_en;
862
+
__u8 vcek_disabled;
863
+
__u8 host_data[KVM_SEV_SNP_FINISH_DATA_SIZE];
864
+
__u8 pad0[3];
865
+
__u16 flags;
866
+
__u64 pad1[4];
867
+
};
868
+
833
869
#define KVM_X2APIC_API_USE_32BIT_IDS (1ULL << 0)
834
870
#define KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK (1ULL << 1)
835
871
···
922
874
#define KVM_X86_SW_PROTECTED_VM 1
923
875
#define KVM_X86_SEV_VM 2
924
876
#define KVM_X86_SEV_ES_VM 3
877
+
#define KVM_X86_SNP_VM 4
925
878
926
879
#endif /* _ASM_X86_KVM_H */
+1
tools/arch/x86/include/uapi/asm/svm.h
+1
tools/arch/x86/include/uapi/asm/svm.h
···
115
115
#define SVM_VMGEXIT_AP_CREATE_ON_INIT 0
116
116
#define SVM_VMGEXIT_AP_CREATE 1
117
117
#define SVM_VMGEXIT_AP_DESTROY 2
118
+
#define SVM_VMGEXIT_SNP_RUN_VMPL 0x80000018
118
119
#define SVM_VMGEXIT_HV_FEATURES 0x8000fffd
119
120
#define SVM_VMGEXIT_TERM_REQUEST 0x8000fffe
120
121
#define SVM_VMGEXIT_TERM_REASON(reason_set, reason_code) \
+73
tools/include/uapi/README
+73
tools/include/uapi/README
···
1
+
Why we want a copy of kernel headers in tools?
2
+
==============================================
3
+
4
+
There used to be no copies, with tools/ code using kernel headers
5
+
directly. From time to time tools/perf/ broke due to legitimate kernel
6
+
hacking. At some point Linus complained about such direct usage. Then we
7
+
adopted the current model.
8
+
9
+
The way these headers are used in perf are not restricted to just
10
+
including them to compile something.
11
+
12
+
There are sometimes used in scripts that convert defines into string
13
+
tables, etc, so some change may break one of these scripts, or new MSRs
14
+
may use some different #define pattern, etc.
15
+
16
+
E.g.:
17
+
18
+
$ ls -1 tools/perf/trace/beauty/*.sh | head -5
19
+
tools/perf/trace/beauty/arch_errno_names.sh
20
+
tools/perf/trace/beauty/drm_ioctl.sh
21
+
tools/perf/trace/beauty/fadvise.sh
22
+
tools/perf/trace/beauty/fsconfig.sh
23
+
tools/perf/trace/beauty/fsmount.sh
24
+
$
25
+
$ tools/perf/trace/beauty/fadvise.sh
26
+
static const char *fadvise_advices[] = {
27
+
[0] = "NORMAL",
28
+
[1] = "RANDOM",
29
+
[2] = "SEQUENTIAL",
30
+
[3] = "WILLNEED",
31
+
[4] = "DONTNEED",
32
+
[5] = "NOREUSE",
33
+
};
34
+
$
35
+
36
+
The tools/perf/check-headers.sh script, part of the tools/ build
37
+
process, points out changes in the original files.
38
+
39
+
So its important not to touch the copies in tools/ when doing changes in
40
+
the original kernel headers, that will be done later, when
41
+
check-headers.sh inform about the change to the perf tools hackers.
42
+
43
+
Another explanation from Ingo Molnar:
44
+
It's better than all the alternatives we tried so far:
45
+
46
+
- Symbolic links and direct #includes: this was the original approach but
47
+
was pushed back on from the kernel side, when tooling modified the
48
+
headers and broke them accidentally for kernel builds.
49
+
50
+
- Duplicate self-defined ABI headers like glibc: double the maintenance
51
+
burden, double the chance for mistakes, plus there's no tech-driven
52
+
notification mechanism to look at new kernel side changes.
53
+
54
+
What we are doing now is a third option:
55
+
56
+
- A software-enforced copy-on-write mechanism of kernel headers to
57
+
tooling, driven by non-fatal warnings on the tooling side build when
58
+
kernel headers get modified:
59
+
60
+
Warning: Kernel ABI header differences:
61
+
diff -u tools/include/uapi/drm/i915_drm.h include/uapi/drm/i915_drm.h
62
+
diff -u tools/include/uapi/linux/fs.h include/uapi/linux/fs.h
63
+
diff -u tools/include/uapi/linux/kvm.h include/uapi/linux/kvm.h
64
+
...
65
+
66
+
The tooling policy is to always pick up the kernel side headers as-is,
67
+
and integate them into the tooling build. The warnings above serve as a
68
+
notification to tooling maintainers that there's changes on the kernel
69
+
side.
70
+
71
+
We've been using this for many years now, and it might seem hacky, but
72
+
works surprisingly well.
73
+
+1
-1
tools/include/uapi/asm-generic/unistd.h
+1
-1
tools/include/uapi/asm-generic/unistd.h
···
737
737
#define __NR_ppoll_time64 414
738
738
__SC_COMP(__NR_ppoll_time64, sys_ppoll, compat_sys_ppoll_time64)
739
739
#define __NR_io_pgetevents_time64 416
740
-
__SYSCALL(__NR_io_pgetevents_time64, sys_io_pgetevents)
740
+
__SC_COMP(__NR_io_pgetevents_time64, sys_io_pgetevents, compat_sys_io_pgetevents_time64)
741
741
#define __NR_recvmmsg_time64 417
742
742
__SC_COMP(__NR_recvmmsg_time64, sys_recvmmsg, compat_sys_recvmmsg_time64)
743
743
#define __NR_mq_timedsend_time64 418
+27
tools/include/uapi/drm/i915_drm.h
+27
tools/include/uapi/drm/i915_drm.h
···
2163
2163
* supports this per context flag.
2164
2164
*/
2165
2165
#define I915_CONTEXT_PARAM_LOW_LATENCY 0xe
2166
+
2167
+
/*
2168
+
* I915_CONTEXT_PARAM_CONTEXT_IMAGE:
2169
+
*
2170
+
* Allows userspace to provide own context images.
2171
+
*
2172
+
* Note that this is a debug API not available on production kernel builds.
2173
+
*/
2174
+
#define I915_CONTEXT_PARAM_CONTEXT_IMAGE 0xf
2166
2175
/* Must be kept compact -- no holes and well documented */
2167
2176
2168
2177
/** @value: Context parameter value to be set or queried */
···
2572
2563
__u64 extensions; \
2573
2564
struct i915_engine_class_instance engines[N__]; \
2574
2565
} __attribute__((packed)) name__
2566
+
2567
+
struct i915_gem_context_param_context_image {
2568
+
/** @engine: Engine class & instance to be configured. */
2569
+
struct i915_engine_class_instance engine;
2570
+
2571
+
/** @flags: One of the supported flags or zero. */
2572
+
__u32 flags;
2573
+
#define I915_CONTEXT_IMAGE_FLAG_ENGINE_INDEX (1u << 0)
2574
+
2575
+
/** @size: Size of the image blob pointed to by @image. */
2576
+
__u32 size;
2577
+
2578
+
/** @mbz: Must be zero. */
2579
+
__u32 mbz;
2580
+
2581
+
/** @image: Userspace memory containing the context image. */
2582
+
__u64 image;
2583
+
} __attribute__((packed));
2575
2584
2576
2585
/**
2577
2586
* struct drm_i915_gem_context_create_ext_setparam - Context parameter
+2
tools/include/uapi/linux/in.h
+2
tools/include/uapi/linux/in.h
···
81
81
#define IPPROTO_ETHERNET IPPROTO_ETHERNET
82
82
IPPROTO_RAW = 255, /* Raw IP packets */
83
83
#define IPPROTO_RAW IPPROTO_RAW
84
+
IPPROTO_SMC = 256, /* Shared Memory Communications */
85
+
#define IPPROTO_SMC IPPROTO_SMC
84
86
IPPROTO_MPTCP = 262, /* Multipath TCP connection */
85
87
#define IPPROTO_MPTCP IPPROTO_MPTCP
86
88
IPPROTO_MAX
+16
-1
tools/include/uapi/linux/kvm.h
+16
-1
tools/include/uapi/linux/kvm.h
···
192
192
/* Flags that describe what fields in emulation_failure hold valid data. */
193
193
#define KVM_INTERNAL_ERROR_EMULATION_FLAG_INSTRUCTION_BYTES (1ULL << 0)
194
194
195
+
/*
196
+
* struct kvm_run can be modified by userspace at any time, so KVM must be
197
+
* careful to avoid TOCTOU bugs. In order to protect KVM, HINT_UNSAFE_IN_KVM()
198
+
* renames fields in struct kvm_run from <symbol> to <symbol>__unsafe when
199
+
* compiled into the kernel, ensuring that any use within KVM is obvious and
200
+
* gets extra scrutiny.
201
+
*/
202
+
#ifdef __KERNEL__
203
+
#define HINT_UNSAFE_IN_KVM(_symbol) _symbol##__unsafe
204
+
#else
205
+
#define HINT_UNSAFE_IN_KVM(_symbol) _symbol
206
+
#endif
207
+
195
208
/* for KVM_RUN, returned by mmap(vcpu_fd, offset=0) */
196
209
struct kvm_run {
197
210
/* in */
198
211
__u8 request_interrupt_window;
199
-
__u8 immediate_exit;
212
+
__u8 HINT_UNSAFE_IN_KVM(immediate_exit);
200
213
__u8 padding1[6];
201
214
202
215
/* out */
···
931
918
#define KVM_CAP_GUEST_MEMFD 234
932
919
#define KVM_CAP_VM_TYPES 235
933
920
#define KVM_CAP_PRE_FAULT_MEMORY 236
921
+
#define KVM_CAP_X86_APIC_BUS_CYCLES_NS 237
922
+
#define KVM_CAP_X86_GUEST_MODE 238
934
923
935
924
struct kvm_irq_routing_irqchip {
936
925
__u32 irqchip;
+4
-2
tools/include/uapi/linux/perf_event.h
+4
-2
tools/include/uapi/linux/perf_event.h
···
1349
1349
#define PERF_MEM_LVLNUM_L2 0x02 /* L2 */
1350
1350
#define PERF_MEM_LVLNUM_L3 0x03 /* L3 */
1351
1351
#define PERF_MEM_LVLNUM_L4 0x04 /* L4 */
1352
-
/* 5-0x7 available */
1352
+
#define PERF_MEM_LVLNUM_L2_MHB 0x05 /* L2 Miss Handling Buffer */
1353
+
#define PERF_MEM_LVLNUM_MSC 0x06 /* Memory-side Cache */
1354
+
/* 0x7 available */
1353
1355
#define PERF_MEM_LVLNUM_UNC 0x08 /* Uncached */
1354
1356
#define PERF_MEM_LVLNUM_CXL 0x09 /* CXL */
1355
1357
#define PERF_MEM_LVLNUM_IO 0x0a /* I/O */
1356
1358
#define PERF_MEM_LVLNUM_ANY_CACHE 0x0b /* Any cache */
1357
-
#define PERF_MEM_LVLNUM_LFB 0x0c /* LFB */
1359
+
#define PERF_MEM_LVLNUM_LFB 0x0c /* LFB / L1 Miss Handling Buffer */
1358
1360
#define PERF_MEM_LVLNUM_RAM 0x0d /* RAM */
1359
1361
#define PERF_MEM_LVLNUM_PMEM 0x0e /* PMEM */
1360
1362
#define PERF_MEM_LVLNUM_NA 0x0f /* N/A */
+10
-2
tools/include/uapi/linux/stat.h
+10
-2
tools/include/uapi/linux/stat.h
···
126
126
__u64 stx_mnt_id;
127
127
__u32 stx_dio_mem_align; /* Memory buffer alignment for direct I/O */
128
128
__u32 stx_dio_offset_align; /* File offset alignment for direct I/O */
129
-
__u64 stx_subvol; /* Subvolume identifier */
130
129
/* 0xa0 */
131
-
__u64 __spare3[11]; /* Spare space for future expansion */
130
+
__u64 stx_subvol; /* Subvolume identifier */
131
+
__u32 stx_atomic_write_unit_min; /* Min atomic write unit in bytes */
132
+
__u32 stx_atomic_write_unit_max; /* Max atomic write unit in bytes */
133
+
/* 0xb0 */
134
+
__u32 stx_atomic_write_segments_max; /* Max atomic write segment count */
135
+
__u32 __spare1[1];
136
+
/* 0xb8 */
137
+
__u64 __spare3[9]; /* Spare space for future expansion */
132
138
/* 0x100 */
133
139
};
134
140
···
163
157
#define STATX_DIOALIGN 0x00002000U /* Want/got direct I/O alignment info */
164
158
#define STATX_MNT_ID_UNIQUE 0x00004000U /* Want/got extended stx_mount_id */
165
159
#define STATX_SUBVOL 0x00008000U /* Want/got stx_subvol */
160
+
#define STATX_WRITE_ATOMIC 0x00010000U /* Want/got atomic_write_* fields */
166
161
167
162
#define STATX__RESERVED 0x80000000U /* Reserved for future struct statx expansion */
168
163
···
199
192
#define STATX_ATTR_MOUNT_ROOT 0x00002000 /* Root of a mount */
200
193
#define STATX_ATTR_VERITY 0x00100000 /* [I] Verity protected file */
201
194
#define STATX_ATTR_DAX 0x00200000 /* File is currently in DAX state */
195
+
#define STATX_ATTR_WRITE_ATOMIC 0x00400000 /* File supports atomic write operations */
202
196
203
197
204
198
#endif /* _UAPI_LINUX_STAT_H */
+5
-1
tools/perf/arch/powerpc/entry/syscalls/syscall.tbl
+5
-1
tools/perf/arch/powerpc/entry/syscalls/syscall.tbl
···
230
230
178 nospu rt_sigsuspend sys_rt_sigsuspend compat_sys_rt_sigsuspend
231
231
179 32 pread64 sys_ppc_pread64 compat_sys_ppc_pread64
232
232
179 64 pread64 sys_pread64
233
+
179 spu pread64 sys_pread64
233
234
180 32 pwrite64 sys_ppc_pwrite64 compat_sys_ppc_pwrite64
234
235
180 64 pwrite64 sys_pwrite64
236
+
180 spu pwrite64 sys_pwrite64
235
237
181 common chown sys_chown
236
238
182 common getcwd sys_getcwd
237
239
183 common capget sys_capget
···
248
246
190 common ugetrlimit sys_getrlimit compat_sys_getrlimit
249
247
191 32 readahead sys_ppc_readahead compat_sys_ppc_readahead
250
248
191 64 readahead sys_readahead
249
+
191 spu readahead sys_readahead
251
250
192 32 mmap2 sys_mmap2 compat_sys_mmap2
252
251
193 32 truncate64 sys_ppc_truncate64 compat_sys_ppc_truncate64
253
252
194 32 ftruncate64 sys_ppc_ftruncate64 compat_sys_ppc_ftruncate64
···
296
293
232 nospu set_tid_address sys_set_tid_address
297
294
233 32 fadvise64 sys_ppc32_fadvise64 compat_sys_ppc32_fadvise64
298
295
233 64 fadvise64 sys_fadvise64
296
+
233 spu fadvise64 sys_fadvise64
299
297
234 nospu exit_group sys_exit_group
300
298
235 nospu lookup_dcookie sys_ni_syscall
301
299
236 common epoll_create sys_epoll_create
···
506
502
412 32 utimensat_time64 sys_utimensat sys_utimensat
507
503
413 32 pselect6_time64 sys_pselect6 compat_sys_pselect6_time64
508
504
414 32 ppoll_time64 sys_ppoll compat_sys_ppoll_time64
509
-
416 32 io_pgetevents_time64 sys_io_pgetevents sys_io_pgetevents
505
+
416 32 io_pgetevents_time64 sys_io_pgetevents compat_sys_io_pgetevents_time64
510
506
417 32 recvmmsg_time64 sys_recvmmsg compat_sys_recvmmsg_time64
511
507
418 32 mq_timedsend_time64 sys_mq_timedsend sys_mq_timedsend
512
508
419 32 mq_timedreceive_time64 sys_mq_timedreceive sys_mq_timedreceive
+1
-1
tools/perf/arch/s390/entry/syscalls/syscall.tbl
+1
-1
tools/perf/arch/s390/entry/syscalls/syscall.tbl
···
418
418
412 32 utimensat_time64 - sys_utimensat
419
419
413 32 pselect6_time64 - compat_sys_pselect6_time64
420
420
414 32 ppoll_time64 - compat_sys_ppoll_time64
421
-
416 32 io_pgetevents_time64 - sys_io_pgetevents
421
+
416 32 io_pgetevents_time64 - compat_sys_io_pgetevents_time64
422
422
417 32 recvmmsg_time64 - compat_sys_recvmmsg_time64
423
423
418 32 mq_timedsend_time64 - sys_mq_timedsend
424
424
419 32 mq_timedreceive_time64 - sys_mq_timedreceive
+5
-3
tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
+5
-3
tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
···
1
+
# SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note
1
2
#
2
3
# 64-bit system call numbers and entry vectors
3
4
#
4
5
# The format is:
5
-
# <number> <abi> <name> <entry point>
6
+
# <number> <abi> <name> <entry point> [<compat entry point> [noreturn]]
6
7
#
7
8
# The __x64_sys_*() stubs are created on-the-fly for sys_*() system calls
8
9
#
···
69
68
57 common fork sys_fork
70
69
58 common vfork sys_vfork
71
70
59 64 execve sys_execve
72
-
60 common exit sys_exit
71
+
60 common exit sys_exit - noreturn
73
72
61 common wait4 sys_wait4
74
73
62 common kill sys_kill
75
74
63 common uname sys_newuname
···
240
239
228 common clock_gettime sys_clock_gettime
241
240
229 common clock_getres sys_clock_getres
242
241
230 common clock_nanosleep sys_clock_nanosleep
243
-
231 common exit_group sys_exit_group
242
+
231 common exit_group sys_exit_group - noreturn
244
243
232 common epoll_wait sys_epoll_wait
245
244
233 common epoll_ctl sys_epoll_ctl
246
245
234 common tgkill sys_tgkill
···
344
343
332 common statx sys_statx
345
344
333 common io_pgetevents sys_io_pgetevents
346
345
334 common rseq sys_rseq
346
+
335 common uretprobe sys_uretprobe
347
347
# don't use numbers 387 through 423, add new calls after the last
348
348
# 'common' entry
349
349
424 common pidfd_send_signal sys_pidfd_send_signal
+5
-4
tools/perf/builtin-daemon.c
+5
-4
tools/perf/builtin-daemon.c
···
1
1
// SPDX-License-Identifier: GPL-2.0
2
2
#include <internal/lib.h>
3
+
#include <inttypes.h>
3
4
#include <subcmd/parse-options.h>
4
5
#include <api/fd/array.h>
5
6
#include <api/fs/fs.h>
···
689
688
/* lock */
690
689
csv_sep, daemon->base, "lock");
691
690
692
-
fprintf(out, "%c%lu",
691
+
fprintf(out, "%c%" PRIu64,
693
692
/* session up time */
694
693
csv_sep, (curr - daemon->start) / 60);
695
694
···
701
700
daemon->base, SESSION_OUTPUT);
702
701
fprintf(out, " lock: %s/lock\n",
703
702
daemon->base);
704
-
fprintf(out, " up: %lu minutes\n",
703
+
fprintf(out, " up: %" PRIu64 " minutes\n",
705
704
(curr - daemon->start) / 60);
706
705
}
707
706
}
···
728
727
/* session ack */
729
728
csv_sep, session->base, SESSION_ACK);
730
729
731
-
fprintf(out, "%c%lu",
730
+
fprintf(out, "%c%" PRIu64,
732
731
/* session up time */
733
732
csv_sep, (curr - session->start) / 60);
734
733
···
746
745
session->base, SESSION_CONTROL);
747
746
fprintf(out, " ack: %s/%s\n",
748
747
session->base, SESSION_ACK);
749
-
fprintf(out, " up: %lu minutes\n",
748
+
fprintf(out, " up: %" PRIu64 " minutes\n",
750
749
(curr - session->start) / 60);
751
750
}
752
751
}
+4
-1
tools/perf/trace/beauty/include/linux/socket.h
+4
-1
tools/perf/trace/beauty/include/linux/socket.h
···
76
76
__kernel_size_t msg_controllen; /* ancillary data buffer length */
77
77
struct kiocb *msg_iocb; /* ptr to iocb for async requests */
78
78
struct ubuf_info *msg_ubuf;
79
-
int (*sg_from_iter)(struct sock *sk, struct sk_buff *skb,
79
+
int (*sg_from_iter)(struct sk_buff *skb,
80
80
struct iov_iter *from, size_t length);
81
81
};
82
82
···
442
442
extern int __sys_socket(int family, int type, int protocol);
443
443
extern struct file *__sys_socket_file(int family, int type, int protocol);
444
444
extern int __sys_bind(int fd, struct sockaddr __user *umyaddr, int addrlen);
445
+
extern int __sys_bind_socket(struct socket *sock, struct sockaddr_storage *address,
446
+
int addrlen);
445
447
extern int __sys_connect_file(struct file *file, struct sockaddr_storage *addr,
446
448
int addrlen, int file_flags);
447
449
extern int __sys_connect(int fd, struct sockaddr __user *uservaddr,
448
450
int addrlen);
449
451
extern int __sys_listen(int fd, int backlog);
452
+
extern int __sys_listen_socket(struct socket *sock, int backlog);
450
453
extern int __sys_getsockname(int fd, struct sockaddr __user *usockaddr,
451
454
int __user *usockaddr_len);
452
455
extern int __sys_getpeername(int fd, struct sockaddr __user *usockaddr,
+161
-2
tools/perf/trace/beauty/include/uapi/linux/fs.h
+161
-2
tools/perf/trace/beauty/include/uapi/linux/fs.h
···
329
329
/* per-IO negation of O_APPEND */
330
330
#define RWF_NOAPPEND ((__force __kernel_rwf_t)0x00000020)
331
331
332
+
/* Atomic Write */
333
+
#define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040)
334
+
332
335
/* mask of flags supported by the kernel */
333
336
#define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\
334
-
RWF_APPEND | RWF_NOAPPEND)
337
+
RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC)
338
+
339
+
#define PROCFS_IOCTL_MAGIC 'f'
335
340
336
341
/* Pagemap ioctl */
337
-
#define PAGEMAP_SCAN _IOWR('f', 16, struct pm_scan_arg)
342
+
#define PAGEMAP_SCAN _IOWR(PROCFS_IOCTL_MAGIC, 16, struct pm_scan_arg)
338
343
339
344
/* Bitmasks provided in pm_scan_args masks and reported in page_region.categories. */
340
345
#define PAGE_IS_WPALLOWED (1 << 0)
···
396
391
__u64 category_mask;
397
392
__u64 category_anyof_mask;
398
393
__u64 return_mask;
394
+
};
395
+
396
+
/* /proc/<pid>/maps ioctl */
397
+
#define PROCMAP_QUERY _IOWR(PROCFS_IOCTL_MAGIC, 17, struct procmap_query)
398
+
399
+
enum procmap_query_flags {
400
+
/*
401
+
* VMA permission flags.
402
+
*
403
+
* Can be used as part of procmap_query.query_flags field to look up
404
+
* only VMAs satisfying specified subset of permissions. E.g., specifying
405
+
* PROCMAP_QUERY_VMA_READABLE only will return both readable and read/write VMAs,
406
+
* while having PROCMAP_QUERY_VMA_READABLE | PROCMAP_QUERY_VMA_WRITABLE will only
407
+
* return read/write VMAs, though both executable/non-executable and
408
+
* private/shared will be ignored.
409
+
*
410
+
* PROCMAP_QUERY_VMA_* flags are also returned in procmap_query.vma_flags
411
+
* field to specify actual VMA permissions.
412
+
*/
413
+
PROCMAP_QUERY_VMA_READABLE = 0x01,
414
+
PROCMAP_QUERY_VMA_WRITABLE = 0x02,
415
+
PROCMAP_QUERY_VMA_EXECUTABLE = 0x04,
416
+
PROCMAP_QUERY_VMA_SHARED = 0x08,
417
+
/*
418
+
* Query modifier flags.
419
+
*
420
+
* By default VMA that covers provided address is returned, or -ENOENT
421
+
* is returned. With PROCMAP_QUERY_COVERING_OR_NEXT_VMA flag set, closest
422
+
* VMA with vma_start > addr will be returned if no covering VMA is
423
+
* found.
424
+
*
425
+
* PROCMAP_QUERY_FILE_BACKED_VMA instructs query to consider only VMAs that
426
+
* have file backing. Can be combined with PROCMAP_QUERY_COVERING_OR_NEXT_VMA
427
+
* to iterate all VMAs with file backing.
428
+
*/
429
+
PROCMAP_QUERY_COVERING_OR_NEXT_VMA = 0x10,
430
+
PROCMAP_QUERY_FILE_BACKED_VMA = 0x20,
431
+
};
432
+
433
+
/*
434
+
* Input/output argument structured passed into ioctl() call. It can be used
435
+
* to query a set of VMAs (Virtual Memory Areas) of a process.
436
+
*
437
+
* Each field can be one of three kinds, marked in a short comment to the
438
+
* right of the field:
439
+
* - "in", input argument, user has to provide this value, kernel doesn't modify it;
440
+
* - "out", output argument, kernel sets this field with VMA data;
441
+
* - "in/out", input and output argument; user provides initial value (used
442
+
* to specify maximum allowable buffer size), and kernel sets it to actual
443
+
* amount of data written (or zero, if there is no data).
444
+
*
445
+
* If matching VMA is found (according to criterias specified by
446
+
* query_addr/query_flags, all the out fields are filled out, and ioctl()
447
+
* returns 0. If there is no matching VMA, -ENOENT will be returned.
448
+
* In case of any other error, negative error code other than -ENOENT is
449
+
* returned.
450
+
*
451
+
* Most of the data is similar to the one returned as text in /proc/<pid>/maps
452
+
* file, but procmap_query provides more querying flexibility. There are no
453
+
* consistency guarantees between subsequent ioctl() calls, but data returned
454
+
* for matched VMA is self-consistent.
455
+
*/
456
+
struct procmap_query {
457
+
/* Query struct size, for backwards/forward compatibility */
458
+
__u64 size;
459
+
/*
460
+
* Query flags, a combination of enum procmap_query_flags values.
461
+
* Defines query filtering and behavior, see enum procmap_query_flags.
462
+
*
463
+
* Input argument, provided by user. Kernel doesn't modify it.
464
+
*/
465
+
__u64 query_flags; /* in */
466
+
/*
467
+
* Query address. By default, VMA that covers this address will
468
+
* be looked up. PROCMAP_QUERY_* flags above modify this default
469
+
* behavior further.
470
+
*
471
+
* Input argument, provided by user. Kernel doesn't modify it.
472
+
*/
473
+
__u64 query_addr; /* in */
474
+
/* VMA starting (inclusive) and ending (exclusive) address, if VMA is found. */
475
+
__u64 vma_start; /* out */
476
+
__u64 vma_end; /* out */
477
+
/* VMA permissions flags. A combination of PROCMAP_QUERY_VMA_* flags. */
478
+
__u64 vma_flags; /* out */
479
+
/* VMA backing page size granularity. */
480
+
__u64 vma_page_size; /* out */
481
+
/*
482
+
* VMA file offset. If VMA has file backing, this specifies offset
483
+
* within the file that VMA's start address corresponds to.
484
+
* Is set to zero if VMA has no backing file.
485
+
*/
486
+
__u64 vma_offset; /* out */
487
+
/* Backing file's inode number, or zero, if VMA has no backing file. */
488
+
__u64 inode; /* out */
489
+
/* Backing file's device major/minor number, or zero, if VMA has no backing file. */
490
+
__u32 dev_major; /* out */
491
+
__u32 dev_minor; /* out */
492
+
/*
493
+
* If set to non-zero value, signals the request to return VMA name
494
+
* (i.e., VMA's backing file's absolute path, with " (deleted)" suffix
495
+
* appended, if file was unlinked from FS) for matched VMA. VMA name
496
+
* can also be some special name (e.g., "[heap]", "[stack]") or could
497
+
* be even user-supplied with prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME).
498
+
*
499
+
* Kernel will set this field to zero, if VMA has no associated name.
500
+
* Otherwise kernel will return actual amount of bytes filled in
501
+
* user-supplied buffer (see vma_name_addr field below), including the
502
+
* terminating zero.
503
+
*
504
+
* If VMA name is longer that user-supplied maximum buffer size,
505
+
* -E2BIG error is returned.
506
+
*
507
+
* If this field is set to non-zero value, vma_name_addr should point
508
+
* to valid user space memory buffer of at least vma_name_size bytes.
509
+
* If set to zero, vma_name_addr should be set to zero as well
510
+
*/
511
+
__u32 vma_name_size; /* in/out */
512
+
/*
513
+
* If set to non-zero value, signals the request to extract and return
514
+
* VMA's backing file's build ID, if the backing file is an ELF file
515
+
* and it contains embedded build ID.
516
+
*
517
+
* Kernel will set this field to zero, if VMA has no backing file,
518
+
* backing file is not an ELF file, or ELF file has no build ID
519
+
* embedded.
520
+
*
521
+
* Build ID is a binary value (not a string). Kernel will set
522
+
* build_id_size field to exact number of bytes used for build ID.
523
+
* If build ID is requested and present, but needs more bytes than
524
+
* user-supplied maximum buffer size (see build_id_addr field below),
525
+
* -E2BIG error will be returned.
526
+
*
527
+
* If this field is set to non-zero value, build_id_addr should point
528
+
* to valid user space memory buffer of at least build_id_size bytes.
529
+
* If set to zero, build_id_addr should be set to zero as well
530
+
*/
531
+
__u32 build_id_size; /* in/out */
532
+
/*
533
+
* User-supplied address of a buffer of at least vma_name_size bytes
534
+
* for kernel to fill with matched VMA's name (see vma_name_size field
535
+
* description above for details).
536
+
*
537
+
* Should be set to zero if VMA name should not be returned.
538
+
*/
539
+
__u64 vma_name_addr; /* in */
540
+
/*
541
+
* User-supplied address of a buffer of at least build_id_size bytes
542
+
* for kernel to fill with matched VMA's ELF build ID, if available
543
+
* (see build_id_size field description above for details).
544
+
*
545
+
* Should be set to zero if build ID should not be returned.
546
+
*/
547
+
__u64 build_id_addr; /* in */
399
548
};
400
549
401
550
#endif /* _UAPI_LINUX_FS_H */
+8
-2
tools/perf/trace/beauty/include/uapi/linux/mount.h
+8
-2
tools/perf/trace/beauty/include/uapi/linux/mount.h
···
154
154
*/
155
155
struct statmount {
156
156
__u32 size; /* Total size, including strings */
157
-
__u32 __spare1;
157
+
__u32 mnt_opts; /* [str] Mount options of the mount */
158
158
__u64 mask; /* What results were written */
159
159
__u32 sb_dev_major; /* Device ID */
160
160
__u32 sb_dev_minor;
···
172
172
__u64 propagate_from; /* Propagation from in current namespace */
173
173
__u32 mnt_root; /* [str] Root of mount relative to root of fs */
174
174
__u32 mnt_point; /* [str] Mountpoint relative to current root */
175
-
__u64 __spare2[50];
175
+
__u64 mnt_ns_id; /* ID of the mount namespace */
176
+
__u64 __spare2[49];
176
177
char str[]; /* Variable size part containing strings */
177
178
};
178
179
···
189
188
__u32 spare;
190
189
__u64 mnt_id;
191
190
__u64 param;
191
+
__u64 mnt_ns_id;
192
192
};
193
193
194
194
/* List of all mnt_id_req versions. */
195
195
#define MNT_ID_REQ_SIZE_VER0 24 /* sizeof first published struct */
196
+
#define MNT_ID_REQ_SIZE_VER1 32 /* sizeof second published struct */
196
197
197
198
/*
198
199
* @mask bits for statmount(2)
···
205
202
#define STATMOUNT_MNT_ROOT 0x00000008U /* Want/got mnt_root */
206
203
#define STATMOUNT_MNT_POINT 0x00000010U /* Want/got mnt_point */
207
204
#define STATMOUNT_FS_TYPE 0x00000020U /* Want/got fs_type */
205
+
#define STATMOUNT_MNT_NS_ID 0x00000040U /* Want/got mnt_ns_id */
206
+
#define STATMOUNT_MNT_OPTS 0x00000080U /* Want/got mnt_opts */
208
207
209
208
/*
210
209
* Special @mnt_id values that can be passed to listmount
211
210
*/
212
211
#define LSMT_ROOT 0xffffffffffffffff /* root mount */
212
+
#define LISTMOUNT_REVERSE (1 << 0) /* List later mounts first */
213
213
214
214
#endif /* _UAPI_LINUX_MOUNT_H */
+10
-2
tools/perf/trace/beauty/include/uapi/linux/stat.h
+10
-2
tools/perf/trace/beauty/include/uapi/linux/stat.h
···
126
126
__u64 stx_mnt_id;
127
127
__u32 stx_dio_mem_align; /* Memory buffer alignment for direct I/O */
128
128
__u32 stx_dio_offset_align; /* File offset alignment for direct I/O */
129
-
__u64 stx_subvol; /* Subvolume identifier */
130
129
/* 0xa0 */
131
-
__u64 __spare3[11]; /* Spare space for future expansion */
130
+
__u64 stx_subvol; /* Subvolume identifier */
131
+
__u32 stx_atomic_write_unit_min; /* Min atomic write unit in bytes */
132
+
__u32 stx_atomic_write_unit_max; /* Max atomic write unit in bytes */
133
+
/* 0xb0 */
134
+
__u32 stx_atomic_write_segments_max; /* Max atomic write segment count */
135
+
__u32 __spare1[1];
136
+
/* 0xb8 */
137
+
__u64 __spare3[9]; /* Spare space for future expansion */
132
138
/* 0x100 */
133
139
};
134
140
···
163
157
#define STATX_DIOALIGN 0x00002000U /* Want/got direct I/O alignment info */
164
158
#define STATX_MNT_ID_UNIQUE 0x00004000U /* Want/got extended stx_mount_id */
165
159
#define STATX_SUBVOL 0x00008000U /* Want/got stx_subvol */
160
+
#define STATX_WRITE_ATOMIC 0x00010000U /* Want/got atomic_write_* fields */
166
161
167
162
#define STATX__RESERVED 0x80000000U /* Reserved for future struct statx expansion */
168
163
···
199
192
#define STATX_ATTR_MOUNT_ROOT 0x00002000 /* Root of a mount */
200
193
#define STATX_ATTR_VERITY 0x00100000 /* [I] Verity protected file */
201
194
#define STATX_ATTR_DAX 0x00200000 /* File is currently in DAX state */
195
+
#define STATX_ATTR_WRITE_ATOMIC 0x00400000 /* File supports atomic write operations */
202
196
203
197
204
198
#endif /* _UAPI_LINUX_STAT_H */
+5
-4
tools/perf/trace/beauty/include/uapi/sound/asound.h
+5
-4
tools/perf/trace/beauty/include/uapi/sound/asound.h
···
142
142
* *
143
143
*****************************************************************************/
144
144
145
-
#define SNDRV_PCM_VERSION SNDRV_PROTOCOL_VERSION(2, 0, 17)
145
+
#define SNDRV_PCM_VERSION SNDRV_PROTOCOL_VERSION(2, 0, 18)
146
146
147
147
typedef unsigned long snd_pcm_uframes_t;
148
148
typedef signed long snd_pcm_sframes_t;
···
334
334
unsigned char id[16];
335
335
unsigned short id16[8];
336
336
unsigned int id32[4];
337
-
};
337
+
} __attribute__((deprecated));
338
338
339
339
struct snd_pcm_info {
340
340
unsigned int device; /* RO/WR (control): device number */
···
348
348
int dev_subclass; /* SNDRV_PCM_SUBCLASS_* */
349
349
unsigned int subdevices_count;
350
350
unsigned int subdevices_avail;
351
-
union snd_pcm_sync_id sync; /* hardware synchronization ID */
351
+
unsigned char pad1[16]; /* was: hardware synchronization ID */
352
352
unsigned char reserved[64]; /* reserved for future... */
353
353
};
354
354
···
420
420
unsigned int rate_num; /* R: rate numerator */
421
421
unsigned int rate_den; /* R: rate denominator */
422
422
snd_pcm_uframes_t fifo_size; /* R: chip FIFO size in frames */
423
-
unsigned char reserved[64]; /* reserved for future */
423
+
unsigned char sync[16]; /* R: synchronization ID (perfect sync - one clock source) */
424
+
unsigned char reserved[48]; /* reserved for future */
424
425
};
425
426
426
427
enum {
+1
tools/testing/cxl/Kbuild
+1
tools/testing/cxl/Kbuild
+12
tools/testing/cxl/test/mock.c
+12
tools/testing/cxl/test/mock.c
···
299
299
}
300
300
EXPORT_SYMBOL_NS_GPL(__wrap_cxl_endpoint_parse_cdat, CXL);
301
301
302
+
void __wrap_cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport)
303
+
{
304
+
int index;
305
+
struct cxl_mock_ops *ops = get_cxl_mock_ops(&index);
306
+
307
+
if (!ops || !ops->is_mock_port(dport->dport_dev))
308
+
cxl_setup_parent_dport(host, dport);
309
+
310
+
put_cxl_mock_ops(index);
311
+
}
312
+
EXPORT_SYMBOL_NS_GPL(__wrap_cxl_setup_parent_dport, CXL);
313
+
302
314
MODULE_LICENSE("GPL v2");
303
315
MODULE_IMPORT_NS(ACPI);
304
316
MODULE_IMPORT_NS(CXL);
+2
-1
tools/testing/selftests/drivers/net/mlxsw/ethtool_lanes.sh
+2
-1
tools/testing/selftests/drivers/net/mlxsw/ethtool_lanes.sh
···
2
2
# SPDX-License-Identifier: GPL-2.0
3
3
4
4
lib_dir=$(dirname $0)/../../../net/forwarding
5
+
ethtool_lib_dir=$(dirname $0)/../hw
5
6
6
7
ALL_TESTS="
7
8
autoneg
···
12
11
NUM_NETIFS=2
13
12
: ${TIMEOUT:=30000} # ms
14
13
source $lib_dir/lib.sh
15
-
source $lib_dir/ethtool_lib.sh
14
+
source $ethtool_lib_dir/ethtool_lib.sh
16
15
17
16
setup_prepare()
18
17
{
+2
tools/testing/selftests/mm/Makefile
+2
tools/testing/selftests/mm/Makefile
···
53
53
TEST_GEN_FILES += map_fixed_noreplace
54
54
TEST_GEN_FILES += map_hugetlb
55
55
TEST_GEN_FILES += map_populate
56
+
ifneq (,$(filter $(ARCH),arm64 riscv riscv64 x86 x86_64))
56
57
TEST_GEN_FILES += memfd_secret
58
+
endif
57
59
TEST_GEN_FILES += migration
58
60
TEST_GEN_FILES += mkdirty
59
61
TEST_GEN_FILES += mlock-random-test
+3
-2
tools/testing/selftests/mm/compaction_test.c
+3
-2
tools/testing/selftests/mm/compaction_test.c
···
89
89
int fd, ret = -1;
90
90
int compaction_index = 0;
91
91
char nr_hugepages[20] = {0};
92
-
char init_nr_hugepages[20] = {0};
92
+
char init_nr_hugepages[24] = {0};
93
93
94
-
sprintf(init_nr_hugepages, "%lu", initial_nr_hugepages);
94
+
snprintf(init_nr_hugepages, sizeof(init_nr_hugepages),
95
+
"%lu", initial_nr_hugepages);
95
96
96
97
/* We want to test with 80% of available memory. Else, OOM killer comes
97
98
in to play */
+3
tools/testing/selftests/mm/run_vmtests.sh
+3
tools/testing/selftests/mm/run_vmtests.sh
···
374
374
# MADV_POPULATE_READ and MADV_POPULATE_WRITE tests
375
375
CATEGORY="madv_populate" run_test ./madv_populate
376
376
377
+
if [ -x ./memfd_secret ]
378
+
then
377
379
(echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope 2>&1) | tap_prefix
378
380
CATEGORY="memfd_secret" run_test ./memfd_secret
381
+
fi
379
382
380
383
# KSM KSM_MERGE_TIME_HUGE_PAGES test with size of 100
381
384
CATEGORY="ksm" run_test ./ksm_tests -H -s 100
+53
-1
tools/testing/selftests/net/forwarding/bridge_vlan_aware.sh
+53
-1
tools/testing/selftests/net/forwarding/bridge_vlan_aware.sh
···
1
1
#!/bin/bash
2
2
# SPDX-License-Identifier: GPL-2.0
3
3
4
-
ALL_TESTS="ping_ipv4 ping_ipv6 learning flooding vlan_deletion extern_learn"
4
+
ALL_TESTS="ping_ipv4 ping_ipv6 learning flooding vlan_deletion extern_learn other_tpid"
5
5
NUM_NETIFS=4
6
6
CHECK_TC="yes"
7
7
source lib.sh
···
140
140
141
141
bridge fdb del de:ad:be:ef:13:37 dev $swp2 master vlan 1 &> /dev/null
142
142
bridge fdb del de:ad:be:ef:13:37 dev $swp1 master vlan 1 &> /dev/null
143
+
}
144
+
145
+
other_tpid()
146
+
{
147
+
local mac=de:ad:be:ef:13:37
148
+
149
+
# Test that packets with TPID 802.1ad VID 3 + TPID 802.1Q VID 5 are
150
+
# classified as untagged by a bridge with vlan_protocol 802.1Q, and
151
+
# are processed in the PVID of the ingress port (here 1). Not VID 3,
152
+
# and not VID 5.
153
+
RET=0
154
+
155
+
tc qdisc add dev $h2 clsact
156
+
tc filter add dev $h2 ingress protocol all pref 1 handle 101 \
157
+
flower dst_mac $mac action drop
158
+
ip link set $h2 promisc on
159
+
ethtool -K $h2 rx-vlan-filter off rx-vlan-stag-filter off
160
+
161
+
$MZ -q $h1 -c 1 -b $mac -a own "88:a8 00:03 81:00 00:05 08:00 aa-aa-aa-aa-aa-aa-aa-aa-aa"
162
+
sleep 1
163
+
164
+
# Match on 'self' addresses as well, for those drivers which
165
+
# do not push their learned addresses to the bridge software
166
+
# database
167
+
bridge -j fdb show $swp1 | \
168
+
jq -e ".[] | select(.mac == \"$(mac_get $h1)\") | select(.vlan == 1)" &> /dev/null
169
+
check_err $? "FDB entry was not learned when it should"
170
+
171
+
log_test "FDB entry in PVID for VLAN-tagged with other TPID"
172
+
173
+
RET=0
174
+
tc -j -s filter show dev $h2 ingress \
175
+
| jq -e ".[] | select(.options.handle == 101) \
176
+
| select(.options.actions[0].stats.packets == 1)" &> /dev/null
177
+
check_err $? "Packet was not forwarded when it should"
178
+
log_test "Reception of VLAN with other TPID as untagged"
179
+
180
+
bridge vlan del dev $swp1 vid 1
181
+
182
+
$MZ -q $h1 -c 1 -b $mac -a own "88:a8 00:03 81:00 00:05 08:00 aa-aa-aa-aa-aa-aa-aa-aa-aa"
183
+
sleep 1
184
+
185
+
RET=0
186
+
tc -j -s filter show dev $h2 ingress \
187
+
| jq -e ".[] | select(.options.handle == 101) \
188
+
| select(.options.actions[0].stats.packets == 1)" &> /dev/null
189
+
check_err $? "Packet was forwarded when should not"
190
+
log_test "Reception of VLAN with other TPID as untagged (no PVID)"
191
+
192
+
bridge vlan add dev $swp1 vid 1 pvid untagged
193
+
ip link set $h2 promisc off
194
+
tc qdisc del dev $h2 clsact
143
195
}
144
196
145
197
trap cleanup EXIT
+57
tools/testing/selftests/net/forwarding/lib.sh
+57
tools/testing/selftests/net/forwarding/lib.sh
···
500
500
fi
501
501
}
502
502
503
+
xfail()
504
+
{
505
+
FAIL_TO_XFAIL=yes "$@"
506
+
}
507
+
503
508
xfail_on_slow()
504
509
{
505
510
if [[ $KSFT_MACHINE_SLOW = yes ]]; then
···
1123
1118
local if_name=$1
1124
1119
1125
1120
ip -j link show dev $if_name | jq -r '.[]["address"]'
1121
+
}
1122
+
1123
+
ether_addr_to_u64()
1124
+
{
1125
+
local addr="$1"
1126
+
local order="$((1 << 40))"
1127
+
local val=0
1128
+
local byte
1129
+
1130
+
addr="${addr//:/ }"
1131
+
1132
+
for byte in $addr; do
1133
+
byte="0x$byte"
1134
+
val=$((val + order * byte))
1135
+
order=$((order >> 8))
1136
+
done
1137
+
1138
+
printf "0x%x" $val
1139
+
}
1140
+
1141
+
u64_to_ether_addr()
1142
+
{
1143
+
local val=$1
1144
+
local byte
1145
+
local i
1146
+
1147
+
for ((i = 40; i >= 0; i -= 8)); do
1148
+
byte=$(((val & (0xff << i)) >> i))
1149
+
printf "%02x" $byte
1150
+
if [ $i -ne 0 ]; then
1151
+
printf ":"
1152
+
fi
1153
+
done
1126
1154
}
1127
1155
1128
1156
ipv6_lladdr_get()
···
2273
2235
local v=$1; shift
2274
2236
2275
2237
echo $((v > 0 ? v : -v))
2238
+
}
2239
+
2240
+
has_unicast_flt()
2241
+
{
2242
+
local dev=$1; shift
2243
+
local mac_addr=$(mac_get $dev)
2244
+
local tmp=$(ether_addr_to_u64 $mac_addr)
2245
+
local promisc
2246
+
2247
+
ip link set $dev up
2248
+
ip link add link $dev name macvlan-tmp type macvlan mode private
2249
+
ip link set macvlan-tmp address $(u64_to_ether_addr $((tmp + 1)))
2250
+
ip link set macvlan-tmp up
2251
+
2252
+
promisc=$(ip -j -d link show dev $dev | jq -r '.[].promiscuity')
2253
+
2254
+
ip link del macvlan-tmp
2255
+
2256
+
[[ $promisc == 1 ]] && echo "no" || echo "yes"
2276
2257
}
+361
-74
tools/testing/selftests/net/forwarding/local_termination.sh
+361
-74
tools/testing/selftests/net/forwarding/local_termination.sh
···
1
1
#!/bin/bash
2
2
# SPDX-License-Identifier: GPL-2.0
3
3
4
-
ALL_TESTS="standalone bridge"
4
+
ALL_TESTS="standalone vlan_unaware_bridge vlan_aware_bridge test_vlan \
5
+
vlan_over_vlan_unaware_bridged_port vlan_over_vlan_aware_bridged_port \
6
+
vlan_over_vlan_unaware_bridge vlan_over_vlan_aware_bridge"
5
7
NUM_NETIFS=2
6
8
PING_COUNT=1
7
9
REQUIRE_MTOOLS=yes
···
39
37
UNKNOWN_MACV6_MC_ADDR2="33:33:01:02:03:06"
40
38
UNKNOWN_MACV6_MC_ADDR3="33:33:01:02:03:07"
41
39
42
-
NON_IP_MC="01:02:03:04:05:06"
43
-
NON_IP_PKT="00:04 48:45:4c:4f"
44
-
BC="ff:ff:ff:ff:ff:ff"
40
+
PTP_1588_L2_SYNC=" \
41
+
01:1b:19:00:00:00 00:00:de:ad:be:ef 88:f7 00 02 \
42
+
00 2c 00 00 02 00 00 00 00 00 00 00 00 00 00 00 \
43
+
00 00 3e 37 63 ff fe cf 17 0e 00 01 00 00 00 00 \
44
+
00 00 00 00 00 00 00 00 00 00"
45
+
PTP_1588_L2_FOLLOW_UP=" \
46
+
01:1b:19:00:00:00 00:00:de:ad:be:ef 88:f7 08 02 \
47
+
00 2c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \
48
+
00 00 3e 37 63 ff fe cf 17 0e 00 01 00 00 02 00 \
49
+
00 00 66 83 c5 f1 17 97 ed f0"
50
+
PTP_1588_L2_PDELAY_REQ=" \
51
+
01:80:c2:00:00:0e 00:00:de:ad:be:ef 88:f7 02 02 \
52
+
00 36 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \
53
+
00 00 3e 37 63 ff fe cf 17 0e 00 01 00 06 05 7f \
54
+
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \
55
+
00 00 00 00"
56
+
PTP_1588_IPV4_SYNC=" \
57
+
01:00:5e:00:01:81 00:00:de:ad:be:ef 08:00 45 00 \
58
+
00 48 0a 9a 40 00 01 11 cb 88 c0 00 02 01 e0 00 \
59
+
01 81 01 3f 01 3f 00 34 a3 c8 00 02 00 2c 00 00 \
60
+
02 00 00 00 00 00 00 00 00 00 00 00 00 00 3e 37 \
61
+
63 ff fe cf 17 0e 00 01 00 00 00 00 00 00 00 00 \
62
+
00 00 00 00 00 00"
63
+
PTP_1588_IPV4_FOLLOW_UP="
64
+
01:00:5e:00:01:81 00:00:de:ad:be:ef 08:00 45 00 \
65
+
00 48 0a 9b 40 00 01 11 cb 87 c0 00 02 01 e0 00 \
66
+
01 81 01 40 01 40 00 34 a3 c8 08 02 00 2c 00 00 \
67
+
00 00 00 00 00 00 00 00 00 00 00 00 00 00 3e 37 \
68
+
63 ff fe cf 17 0e 00 01 00 00 02 00 00 00 66 83 \
69
+
c6 0f 1d 9a 61 87"
70
+
PTP_1588_IPV4_PDELAY_REQ=" \
71
+
01:00:5e:00:00:6b 00:00:de:ad:be:ef 08:00 45 00 \
72
+
00 52 35 a9 40 00 01 11 a1 85 c0 00 02 01 e0 00 \
73
+
00 6b 01 3f 01 3f 00 3e a2 bc 02 02 00 36 00 00 \
74
+
00 00 00 00 00 00 00 00 00 00 00 00 00 00 3e 37 \
75
+
63 ff fe cf 17 0e 00 01 00 01 05 7f 00 00 00 00 \
76
+
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00"
77
+
PTP_1588_IPV6_SYNC=" \
78
+
33:33:00:00:01:81 00:00:de:ad:be:ef 86:dd 60 06 \
79
+
7c 2f 00 36 11 01 20 01 0d b8 00 01 00 00 00 00 \
80
+
00 00 00 00 00 01 ff 0e 00 00 00 00 00 00 00 00 \
81
+
00 00 00 00 01 81 01 3f 01 3f 00 36 2e 92 00 02 \
82
+
00 2c 00 00 02 00 00 00 00 00 00 00 00 00 00 00 \
83
+
00 00 3e 37 63 ff fe cf 17 0e 00 01 00 00 00 00 \
84
+
00 00 00 00 00 00 00 00 00 00 00 00"
85
+
PTP_1588_IPV6_FOLLOW_UP=" \
86
+
33:33:00:00:01:81 00:00:de:ad:be:ef 86:dd 60 0a \
87
+
00 bc 00 36 11 01 20 01 0d b8 00 01 00 00 00 00 \
88
+
00 00 00 00 00 01 ff 0e 00 00 00 00 00 00 00 00 \
89
+
00 00 00 00 01 81 01 40 01 40 00 36 2e 92 08 02 \
90
+
00 2c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \
91
+
00 00 3e 37 63 ff fe cf 17 0e 00 01 00 00 02 00 \
92
+
00 00 66 83 c6 2a 32 09 bd 74 00 00"
93
+
PTP_1588_IPV6_PDELAY_REQ=" \
94
+
33:33:00:00:00:6b 00:00:de:ad:be:ef 86:dd 60 0c \
95
+
5c fd 00 40 11 01 fe 80 00 00 00 00 00 00 3c 37 \
96
+
63 ff fe cf 17 0e ff 02 00 00 00 00 00 00 00 00 \
97
+
00 00 00 00 00 6b 01 3f 01 3f 00 40 b4 54 02 02 \
98
+
00 36 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \
99
+
00 00 3e 37 63 ff fe cf 17 0e 00 01 00 01 05 7f \
100
+
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 \
101
+
00 00 00 00 00 00"
45
102
46
103
# Disable promisc to ensure we don't receive unknown MAC DA packets
47
104
export TCPDUMP_EXTRA_FLAGS="-pl"
···
108
47
h1=${NETIFS[p1]}
109
48
h2=${NETIFS[p2]}
110
49
111
-
send_non_ip()
50
+
send_raw()
112
51
{
113
-
local if_name=$1
114
-
local smac=$2
115
-
local dmac=$3
52
+
local if_name=$1; shift
53
+
local pkt="$1"; shift
54
+
local smac=$(mac_get $if_name)
116
55
117
-
$MZ -q $if_name "$dmac $smac $NON_IP_PKT"
56
+
pkt="${pkt/00:00:de:ad:be:ef/$smac}"
57
+
58
+
$MZ -q $if_name "$pkt"
118
59
}
119
60
120
61
send_uc_ipv4()
···
131
68
132
69
check_rcv()
133
70
{
134
-
local if_name=$1
135
-
local type=$2
136
-
local pattern=$3
137
-
local should_receive=$4
71
+
local if_name=$1; shift
72
+
local type=$1; shift
73
+
local pattern=$1; shift
74
+
local should_receive=$1; shift
75
+
local test_name="$1"; shift
138
76
local should_fail=
139
77
140
78
[ $should_receive = true ] && should_fail=0 || should_fail=1
···
145
81
146
82
check_err_fail "$should_fail" "$?" "reception"
147
83
148
-
log_test "$if_name: $type"
84
+
log_test "$test_name: $type"
149
85
}
150
86
151
87
mc_route_prepare()
···
168
104
169
105
run_test()
170
106
{
171
-
local rcv_if_name=$1
172
-
local smac=$(mac_get $h1)
107
+
local send_if_name=$1; shift
108
+
local rcv_if_name=$1; shift
109
+
local skip_ptp=$1; shift
110
+
local no_unicast_flt=$1; shift
111
+
local test_name="$1"; shift
112
+
local smac=$(mac_get $send_if_name)
173
113
local rcv_dmac=$(mac_get $rcv_if_name)
114
+
local should_receive
174
115
175
116
tcpdump_start $rcv_if_name
176
117
177
-
mc_route_prepare $h1
118
+
mc_route_prepare $send_if_name
178
119
mc_route_prepare $rcv_if_name
179
120
180
-
send_uc_ipv4 $h1 $rcv_dmac
181
-
send_uc_ipv4 $h1 $MACVLAN_ADDR
182
-
send_uc_ipv4 $h1 $UNKNOWN_UC_ADDR1
121
+
send_uc_ipv4 $send_if_name $rcv_dmac
122
+
send_uc_ipv4 $send_if_name $MACVLAN_ADDR
123
+
send_uc_ipv4 $send_if_name $UNKNOWN_UC_ADDR1
183
124
184
125
ip link set dev $rcv_if_name promisc on
185
-
send_uc_ipv4 $h1 $UNKNOWN_UC_ADDR2
186
-
mc_send $h1 $UNKNOWN_IPV4_MC_ADDR2
187
-
mc_send $h1 $UNKNOWN_IPV6_MC_ADDR2
126
+
send_uc_ipv4 $send_if_name $UNKNOWN_UC_ADDR2
127
+
mc_send $send_if_name $UNKNOWN_IPV4_MC_ADDR2
128
+
mc_send $send_if_name $UNKNOWN_IPV6_MC_ADDR2
188
129
ip link set dev $rcv_if_name promisc off
189
130
190
131
mc_join $rcv_if_name $JOINED_IPV4_MC_ADDR
191
-
mc_send $h1 $JOINED_IPV4_MC_ADDR
132
+
mc_send $send_if_name $JOINED_IPV4_MC_ADDR
192
133
mc_leave
193
134
194
135
mc_join $rcv_if_name $JOINED_IPV6_MC_ADDR
195
-
mc_send $h1 $JOINED_IPV6_MC_ADDR
136
+
mc_send $send_if_name $JOINED_IPV6_MC_ADDR
196
137
mc_leave
197
138
198
-
mc_send $h1 $UNKNOWN_IPV4_MC_ADDR1
199
-
mc_send $h1 $UNKNOWN_IPV6_MC_ADDR1
139
+
mc_send $send_if_name $UNKNOWN_IPV4_MC_ADDR1
140
+
mc_send $send_if_name $UNKNOWN_IPV6_MC_ADDR1
200
141
201
142
ip link set dev $rcv_if_name allmulticast on
202
-
send_uc_ipv4 $h1 $UNKNOWN_UC_ADDR3
203
-
mc_send $h1 $UNKNOWN_IPV4_MC_ADDR3
204
-
mc_send $h1 $UNKNOWN_IPV6_MC_ADDR3
143
+
send_uc_ipv4 $send_if_name $UNKNOWN_UC_ADDR3
144
+
mc_send $send_if_name $UNKNOWN_IPV4_MC_ADDR3
145
+
mc_send $send_if_name $UNKNOWN_IPV6_MC_ADDR3
205
146
ip link set dev $rcv_if_name allmulticast off
206
147
207
148
mc_route_destroy $rcv_if_name
208
-
mc_route_destroy $h1
149
+
mc_route_destroy $send_if_name
150
+
151
+
if [ $skip_ptp = false ]; then
152
+
ip maddress add 01:1b:19:00:00:00 dev $rcv_if_name
153
+
send_raw $send_if_name "$PTP_1588_L2_SYNC"
154
+
send_raw $send_if_name "$PTP_1588_L2_FOLLOW_UP"
155
+
ip maddress del 01:1b:19:00:00:00 dev $rcv_if_name
156
+
157
+
ip maddress add 01:80:c2:00:00:0e dev $rcv_if_name
158
+
send_raw $send_if_name "$PTP_1588_L2_PDELAY_REQ"
159
+
ip maddress del 01:80:c2:00:00:0e dev $rcv_if_name
160
+
161
+
mc_join $rcv_if_name 224.0.1.129
162
+
send_raw $send_if_name "$PTP_1588_IPV4_SYNC"
163
+
send_raw $send_if_name "$PTP_1588_IPV4_FOLLOW_UP"
164
+
mc_leave
165
+
166
+
mc_join $rcv_if_name 224.0.0.107
167
+
send_raw $send_if_name "$PTP_1588_IPV4_PDELAY_REQ"
168
+
mc_leave
169
+
170
+
mc_join $rcv_if_name ff0e::181
171
+
send_raw $send_if_name "$PTP_1588_IPV6_SYNC"
172
+
send_raw $send_if_name "$PTP_1588_IPV6_FOLLOW_UP"
173
+
mc_leave
174
+
175
+
mc_join $rcv_if_name ff02::6b
176
+
send_raw $send_if_name "$PTP_1588_IPV6_PDELAY_REQ"
177
+
mc_leave
178
+
fi
209
179
210
180
sleep 1
211
181
···
247
149
248
150
check_rcv $rcv_if_name "Unicast IPv4 to primary MAC address" \
249
151
"$smac > $rcv_dmac, ethertype IPv4 (0x0800)" \
250
-
true
152
+
true "$test_name"
251
153
252
154
check_rcv $rcv_if_name "Unicast IPv4 to macvlan MAC address" \
253
155
"$smac > $MACVLAN_ADDR, ethertype IPv4 (0x0800)" \
254
-
true
156
+
true "$test_name"
255
157
256
-
xfail_on_veth $h1 \
257
-
check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address" \
258
-
"$smac > $UNKNOWN_UC_ADDR1, ethertype IPv4 (0x0800)" \
259
-
false
158
+
[ $no_unicast_flt = true ] && should_receive=true || should_receive=false
159
+
check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address" \
160
+
"$smac > $UNKNOWN_UC_ADDR1, ethertype IPv4 (0x0800)" \
161
+
$should_receive "$test_name"
260
162
261
163
check_rcv $rcv_if_name "Unicast IPv4 to unknown MAC address, promisc" \
262
164
"$smac > $UNKNOWN_UC_ADDR2, ethertype IPv4 (0x0800)" \
263
-
true
165
+
true "$test_name"
264
166
265
-
xfail_on_veth $h1 \
266
-
check_rcv $rcv_if_name \
267
-
"Unicast IPv4 to unknown MAC address, allmulti" \
268
-
"$smac > $UNKNOWN_UC_ADDR3, ethertype IPv4 (0x0800)" \
269
-
false
167
+
[ $no_unicast_flt = true ] && should_receive=true || should_receive=false
168
+
check_rcv $rcv_if_name \
169
+
"Unicast IPv4 to unknown MAC address, allmulti" \
170
+
"$smac > $UNKNOWN_UC_ADDR3, ethertype IPv4 (0x0800)" \
171
+
$should_receive "$test_name"
270
172
271
173
check_rcv $rcv_if_name "Multicast IPv4 to joined group" \
272
174
"$smac > $JOINED_MACV4_MC_ADDR, ethertype IPv4 (0x0800)" \
273
-
true
175
+
true "$test_name"
274
176
275
-
xfail_on_veth $h1 \
177
+
xfail \
276
178
check_rcv $rcv_if_name \
277
179
"Multicast IPv4 to unknown group" \
278
180
"$smac > $UNKNOWN_MACV4_MC_ADDR1, ethertype IPv4 (0x0800)" \
279
-
false
181
+
false "$test_name"
280
182
281
183
check_rcv $rcv_if_name "Multicast IPv4 to unknown group, promisc" \
282
184
"$smac > $UNKNOWN_MACV4_MC_ADDR2, ethertype IPv4 (0x0800)" \
283
-
true
185
+
true "$test_name"
284
186
285
187
check_rcv $rcv_if_name "Multicast IPv4 to unknown group, allmulti" \
286
188
"$smac > $UNKNOWN_MACV4_MC_ADDR3, ethertype IPv4 (0x0800)" \
287
-
true
189
+
true "$test_name"
288
190
289
191
check_rcv $rcv_if_name "Multicast IPv6 to joined group" \
290
192
"$smac > $JOINED_MACV6_MC_ADDR, ethertype IPv6 (0x86dd)" \
291
-
true
193
+
true "$test_name"
292
194
293
-
xfail_on_veth $h1 \
195
+
xfail \
294
196
check_rcv $rcv_if_name "Multicast IPv6 to unknown group" \
295
197
"$smac > $UNKNOWN_MACV6_MC_ADDR1, ethertype IPv6 (0x86dd)" \
296
-
false
198
+
false "$test_name"
297
199
298
200
check_rcv $rcv_if_name "Multicast IPv6 to unknown group, promisc" \
299
201
"$smac > $UNKNOWN_MACV6_MC_ADDR2, ethertype IPv6 (0x86dd)" \
300
-
true
202
+
true "$test_name"
301
203
302
204
check_rcv $rcv_if_name "Multicast IPv6 to unknown group, allmulti" \
303
205
"$smac > $UNKNOWN_MACV6_MC_ADDR3, ethertype IPv6 (0x86dd)" \
304
-
true
206
+
true "$test_name"
207
+
208
+
if [ $skip_ptp = false ]; then
209
+
check_rcv $rcv_if_name "1588v2 over L2 transport, Sync" \
210
+
"ethertype PTP (0x88f7).* PTPv2.* msg type : sync msg" \
211
+
true "$test_name"
212
+
213
+
check_rcv $rcv_if_name "1588v2 over L2 transport, Follow-Up" \
214
+
"ethertype PTP (0x88f7).* PTPv2.* msg type : follow up msg" \
215
+
true "$test_name"
216
+
217
+
check_rcv $rcv_if_name "1588v2 over L2 transport, Peer Delay Request" \
218
+
"ethertype PTP (0x88f7).* PTPv2.* msg type : peer delay req msg" \
219
+
true "$test_name"
220
+
221
+
check_rcv $rcv_if_name "1588v2 over IPv4, Sync" \
222
+
"ethertype IPv4 (0x0800).* PTPv2.* msg type : sync msg" \
223
+
true "$test_name"
224
+
225
+
check_rcv $rcv_if_name "1588v2 over IPv4, Follow-Up" \
226
+
"ethertype IPv4 (0x0800).* PTPv2.* msg type : follow up msg" \
227
+
true "$test_name"
228
+
229
+
check_rcv $rcv_if_name "1588v2 over IPv4, Peer Delay Request" \
230
+
"ethertype IPv4 (0x0800).* PTPv2.* msg type : peer delay req msg" \
231
+
true "$test_name"
232
+
233
+
check_rcv $rcv_if_name "1588v2 over IPv6, Sync" \
234
+
"ethertype IPv6 (0x86dd).* PTPv2.* msg type : sync msg" \
235
+
true "$test_name"
236
+
237
+
check_rcv $rcv_if_name "1588v2 over IPv6, Follow-Up" \
238
+
"ethertype IPv6 (0x86dd).* PTPv2.* msg type : follow up msg" \
239
+
true "$test_name"
240
+
241
+
check_rcv $rcv_if_name "1588v2 over IPv6, Peer Delay Request" \
242
+
"ethertype IPv6 (0x86dd).* PTPv2.* msg type : peer delay req msg" \
243
+
true "$test_name"
244
+
fi
305
245
306
246
tcpdump_cleanup $rcv_if_name
307
247
}
···
364
228
simple_if_fini $h2 $H2_IPV4/24 $H2_IPV6/64
365
229
}
366
230
231
+
h1_vlan_create()
232
+
{
233
+
simple_if_init $h1
234
+
vlan_create $h1 100 v$h1 $H1_IPV4/24 $H1_IPV6/64
235
+
}
236
+
237
+
h1_vlan_destroy()
238
+
{
239
+
vlan_destroy $h1 100
240
+
simple_if_fini $h1
241
+
}
242
+
243
+
h2_vlan_create()
244
+
{
245
+
simple_if_init $h2
246
+
vlan_create $h2 100 v$h2 $H2_IPV4/24 $H2_IPV6/64
247
+
}
248
+
249
+
h2_vlan_destroy()
250
+
{
251
+
vlan_destroy $h2 100
252
+
simple_if_fini $h2
253
+
}
254
+
367
255
bridge_create()
368
256
{
369
-
ip link add br0 type bridge
257
+
local vlan_filtering=$1
258
+
259
+
ip link add br0 type bridge vlan_filtering $vlan_filtering
370
260
ip link set br0 address $BRIDGE_ADDR
371
261
ip link set br0 up
372
262
373
263
ip link set $h2 master br0
374
264
ip link set $h2 up
375
-
376
-
simple_if_init br0 $H2_IPV4/24 $H2_IPV6/64
377
265
}
378
266
379
267
bridge_destroy()
380
268
{
381
-
simple_if_fini br0 $H2_IPV4/24 $H2_IPV6/64
382
-
383
269
ip link del br0
270
+
}
271
+
272
+
macvlan_create()
273
+
{
274
+
local lower=$1
275
+
276
+
ip link add link $lower name macvlan0 type macvlan mode private
277
+
ip link set macvlan0 address $MACVLAN_ADDR
278
+
ip link set macvlan0 up
279
+
}
280
+
281
+
macvlan_destroy()
282
+
{
283
+
ip link del macvlan0
384
284
}
385
285
386
286
standalone()
387
287
{
288
+
local no_unicast_flt=true
289
+
local skip_ptp=false
290
+
291
+
if [ $(has_unicast_flt $h2) = yes ]; then
292
+
no_unicast_flt=false
293
+
fi
294
+
388
295
h1_create
389
296
h2_create
297
+
macvlan_create $h2
390
298
391
-
ip link add link $h2 name macvlan0 type macvlan mode private
392
-
ip link set macvlan0 address $MACVLAN_ADDR
393
-
ip link set macvlan0 up
299
+
run_test $h1 $h2 $skip_ptp $no_unicast_flt "$h2"
394
300
395
-
run_test $h2
396
-
397
-
ip link del macvlan0
398
-
301
+
macvlan_destroy
399
302
h2_destroy
400
303
h1_destroy
401
304
}
402
305
403
-
bridge()
306
+
test_bridge()
404
307
{
308
+
local no_unicast_flt=true
309
+
local vlan_filtering=$1
310
+
local skip_ptp=true
311
+
405
312
h1_create
406
-
bridge_create
313
+
bridge_create $vlan_filtering
314
+
simple_if_init br0 $H2_IPV4/24 $H2_IPV6/64
315
+
macvlan_create br0
407
316
408
-
ip link add link br0 name macvlan0 type macvlan mode private
409
-
ip link set macvlan0 address $MACVLAN_ADDR
410
-
ip link set macvlan0 up
317
+
run_test $h1 br0 $skip_ptp $no_unicast_flt \
318
+
"vlan_filtering=$vlan_filtering bridge"
411
319
412
-
run_test br0
413
-
414
-
ip link del macvlan0
415
-
320
+
macvlan_destroy
321
+
simple_if_fini br0 $H2_IPV4/24 $H2_IPV6/64
416
322
bridge_destroy
417
323
h1_destroy
324
+
}
325
+
326
+
vlan_unaware_bridge()
327
+
{
328
+
test_bridge 0
329
+
}
330
+
331
+
vlan_aware_bridge()
332
+
{
333
+
test_bridge 1
334
+
}
335
+
336
+
test_vlan()
337
+
{
338
+
local no_unicast_flt=true
339
+
local skip_ptp=false
340
+
341
+
if [ $(has_unicast_flt $h2) = yes ]; then
342
+
no_unicast_flt=false
343
+
fi
344
+
345
+
h1_vlan_create
346
+
h2_vlan_create
347
+
macvlan_create $h2.100
348
+
349
+
run_test $h1.100 $h2.100 $skip_ptp $no_unicast_flt "VLAN upper"
350
+
351
+
macvlan_destroy
352
+
h2_vlan_destroy
353
+
h1_vlan_destroy
354
+
}
355
+
356
+
vlan_over_bridged_port()
357
+
{
358
+
local no_unicast_flt=true
359
+
local vlan_filtering=$1
360
+
local skip_ptp=false
361
+
362
+
# br_manage_promisc() will not force a single vlan_filtering port to
363
+
# promiscuous mode, so we should still expect unicast filtering to take
364
+
# place if the device can do it.
365
+
if [ $(has_unicast_flt $h2) = yes ] && [ $vlan_filtering = 1 ]; then
366
+
no_unicast_flt=false
367
+
fi
368
+
369
+
h1_vlan_create
370
+
h2_vlan_create
371
+
bridge_create $vlan_filtering
372
+
macvlan_create $h2.100
373
+
374
+
run_test $h1.100 $h2.100 $skip_ptp $no_unicast_flt \
375
+
"VLAN over vlan_filtering=$vlan_filtering bridged port"
376
+
377
+
macvlan_destroy
378
+
bridge_destroy
379
+
h2_vlan_destroy
380
+
h1_vlan_destroy
381
+
}
382
+
383
+
vlan_over_vlan_unaware_bridged_port()
384
+
{
385
+
vlan_over_bridged_port 0
386
+
}
387
+
388
+
vlan_over_vlan_aware_bridged_port()
389
+
{
390
+
vlan_over_bridged_port 1
391
+
}
392
+
393
+
vlan_over_bridge()
394
+
{
395
+
local no_unicast_flt=true
396
+
local vlan_filtering=$1
397
+
local skip_ptp=true
398
+
399
+
h1_vlan_create
400
+
bridge_create $vlan_filtering
401
+
simple_if_init br0
402
+
vlan_create br0 100 vbr0 $H2_IPV4/24 $H2_IPV6/64
403
+
macvlan_create br0.100
404
+
405
+
if [ $vlan_filtering = 1 ]; then
406
+
bridge vlan add dev $h2 vid 100 master
407
+
bridge vlan add dev br0 vid 100 self
408
+
fi
409
+
410
+
run_test $h1.100 br0.100 $skip_ptp $no_unicast_flt \
411
+
"VLAN over vlan_filtering=$vlan_filtering bridge"
412
+
413
+
if [ $vlan_filtering = 1 ]; then
414
+
bridge vlan del dev br0 vid 100 self
415
+
bridge vlan del dev $h2 vid 100 master
416
+
fi
417
+
418
+
macvlan_destroy
419
+
vlan_destroy br0 100
420
+
simple_if_fini br0
421
+
bridge_destroy
422
+
h1_vlan_destroy
423
+
}
424
+
425
+
vlan_over_vlan_unaware_bridge()
426
+
{
427
+
vlan_over_bridge 0
428
+
}
429
+
430
+
vlan_over_vlan_aware_bridge()
431
+
{
432
+
vlan_over_bridge 1
418
433
}
419
434
420
435
cleanup()
+66
-10
tools/testing/selftests/net/mptcp/mptcp_join.sh
+66
-10
tools/testing/selftests/net/mptcp/mptcp_join.sh
···
436
436
local ns="${!1}"
437
437
local src="${2}"
438
438
local target="${3}"
439
+
local chain="${4:-INPUT}"
439
440
440
441
if ! ip netns exec "${ns}" ${iptables} \
441
-
-A INPUT \
442
+
-A "${chain}" \
442
443
-s "${src}" \
443
444
-p tcp \
444
445
-j "${target}"; then
···
3059
3058
pm_nl_set_limits $ns1 1 3
3060
3059
pm_nl_set_limits $ns2 1 3
3061
3060
pm_nl_add_endpoint $ns1 10.0.2.1 flags signal
3061
+
pm_nl_add_endpoint $ns2 10.0.1.2 flags subflow,fullmesh
3062
3062
fullmesh=1 speed=slow \
3063
3063
run_tests $ns1 $ns2 10.0.1.1
3064
3064
chk_join_nr 3 3 3
···
3573
3571
mptcp_lib_kill_wait $tests_pid
3574
3572
fi
3575
3573
3576
-
if reset "delete and re-add" &&
3574
+
if reset_with_tcp_filter "delete and re-add" ns2 10.0.3.2 REJECT OUTPUT &&
3577
3575
mptcp_lib_kallsyms_has "subflow_rebuild_header$"; then
3578
-
pm_nl_set_limits $ns1 1 1
3579
-
pm_nl_set_limits $ns2 1 1
3576
+
pm_nl_set_limits $ns1 0 2
3577
+
pm_nl_set_limits $ns2 0 2
3580
3578
pm_nl_add_endpoint $ns2 10.0.2.2 id 2 dev ns2eth2 flags subflow
3581
3579
test_linkfail=4 speed=20 \
3582
3580
run_tests $ns1 $ns2 10.0.1.1 &
···
3593
3591
chk_subflow_nr "after delete" 1
3594
3592
chk_mptcp_info subflows 0 subflows 0
3595
3593
3596
-
pm_nl_add_endpoint $ns2 10.0.2.2 dev ns2eth2 flags subflow
3594
+
pm_nl_add_endpoint $ns2 10.0.2.2 id 2 dev ns2eth2 flags subflow
3597
3595
wait_mpj $ns2
3598
3596
chk_subflow_nr "after re-add" 2
3599
3597
chk_mptcp_info subflows 1 subflows 1
3598
+
3599
+
pm_nl_add_endpoint $ns2 10.0.3.2 id 3 flags subflow
3600
+
wait_attempt_fail $ns2
3601
+
chk_subflow_nr "after new reject" 2
3602
+
chk_mptcp_info subflows 1 subflows 1
3603
+
3604
+
ip netns exec "${ns2}" ${iptables} -D OUTPUT -s "10.0.3.2" -p tcp -j REJECT
3605
+
pm_nl_del_endpoint $ns2 3 10.0.3.2
3606
+
pm_nl_add_endpoint $ns2 10.0.3.2 id 3 flags subflow
3607
+
wait_mpj $ns2
3608
+
chk_subflow_nr "after no reject" 3
3609
+
chk_mptcp_info subflows 2 subflows 2
3610
+
3600
3611
mptcp_lib_kill_wait $tests_pid
3612
+
3613
+
chk_join_nr 3 3 3
3614
+
chk_rm_nr 1 1
3601
3615
fi
3602
3616
3603
3617
# remove and re-add
3604
3618
if reset "delete re-add signal" &&
3605
3619
mptcp_lib_kallsyms_has "subflow_rebuild_header$"; then
3606
-
pm_nl_set_limits $ns1 1 1
3607
-
pm_nl_set_limits $ns2 1 1
3620
+
pm_nl_set_limits $ns1 0 2
3621
+
pm_nl_set_limits $ns2 2 2
3608
3622
pm_nl_add_endpoint $ns1 10.0.2.1 id 1 flags signal
3623
+
# broadcast IP: no packet for this address will be received on ns1
3624
+
pm_nl_add_endpoint $ns1 224.0.0.1 id 2 flags signal
3609
3625
test_linkfail=4 speed=20 \
3610
3626
run_tests $ns1 $ns2 10.0.1.1 &
3611
3627
local tests_pid=$!
···
3635
3615
chk_mptcp_info subflows 1 subflows 1
3636
3616
3637
3617
pm_nl_del_endpoint $ns1 1 10.0.2.1
3618
+
pm_nl_del_endpoint $ns1 2 224.0.0.1
3638
3619
sleep 0.5
3639
3620
chk_subflow_nr "after delete" 1
3640
3621
chk_mptcp_info subflows 0 subflows 0
3641
3622
3642
-
pm_nl_add_endpoint $ns1 10.0.2.1 flags signal
3623
+
pm_nl_add_endpoint $ns1 10.0.2.1 id 1 flags signal
3624
+
pm_nl_add_endpoint $ns1 10.0.3.1 id 2 flags signal
3643
3625
wait_mpj $ns2
3644
-
chk_subflow_nr "after re-add" 2
3645
-
chk_mptcp_info subflows 1 subflows 1
3626
+
chk_subflow_nr "after re-add" 3
3627
+
chk_mptcp_info subflows 2 subflows 2
3646
3628
mptcp_lib_kill_wait $tests_pid
3629
+
3630
+
chk_join_nr 3 3 3
3631
+
chk_add_nr 4 4
3632
+
chk_rm_nr 2 1 invert
3647
3633
fi
3648
3634
3635
+
# flush and re-add
3636
+
if reset_with_tcp_filter "flush re-add" ns2 10.0.3.2 REJECT OUTPUT &&
3637
+
mptcp_lib_kallsyms_has "subflow_rebuild_header$"; then
3638
+
pm_nl_set_limits $ns1 0 2
3639
+
pm_nl_set_limits $ns2 1 2
3640
+
# broadcast IP: no packet for this address will be received on ns1
3641
+
pm_nl_add_endpoint $ns1 224.0.0.1 id 2 flags signal
3642
+
pm_nl_add_endpoint $ns2 10.0.3.2 id 3 flags subflow
3643
+
test_linkfail=4 speed=20 \
3644
+
run_tests $ns1 $ns2 10.0.1.1 &
3645
+
local tests_pid=$!
3646
+
3647
+
wait_attempt_fail $ns2
3648
+
chk_subflow_nr "before flush" 1
3649
+
chk_mptcp_info subflows 0 subflows 0
3650
+
3651
+
pm_nl_flush_endpoint $ns2
3652
+
pm_nl_flush_endpoint $ns1
3653
+
wait_rm_addr $ns2 0
3654
+
ip netns exec "${ns2}" ${iptables} -D OUTPUT -s "10.0.3.2" -p tcp -j REJECT
3655
+
pm_nl_add_endpoint $ns2 10.0.3.2 id 3 flags subflow
3656
+
wait_mpj $ns2
3657
+
pm_nl_add_endpoint $ns1 10.0.3.1 id 2 flags signal
3658
+
wait_mpj $ns2
3659
+
mptcp_lib_kill_wait $tests_pid
3660
+
3661
+
chk_join_nr 2 2 2
3662
+
chk_add_nr 2 2
3663
+
chk_rm_nr 1 0 invert
3664
+
fi
3649
3665
}
3650
3666
3651
3667
# [$1: error message]
+28
-25
tools/testing/selftests/net/udpgro.sh
+28
-25
tools/testing/selftests/net/udpgro.sh
···
7
7
8
8
readonly PEER_NS="ns-peer-$(mktemp -u XXXXXX)"
9
9
10
-
BPF_FILE="xdp_dummy.bpf.o"
11
-
12
10
# set global exit status, but never reset nonzero one.
13
11
check_err()
14
12
{
···
36
38
ip -netns "${PEER_NS}" addr add dev veth1 192.168.1.1/24
37
39
ip -netns "${PEER_NS}" addr add dev veth1 2001:db8::1/64 nodad
38
40
ip -netns "${PEER_NS}" link set dev veth1 up
39
-
ip -n "${PEER_NS}" link set veth1 xdp object ${BPF_FILE} section xdp
41
+
ip netns exec "${PEER_NS}" ethtool -K veth1 gro on
40
42
}
41
43
42
44
run_one() {
···
44
46
local -r all="$@"
45
47
local -r tx_args=${all%rx*}
46
48
local -r rx_args=${all#*rx}
49
+
local ret=0
47
50
48
51
cfg_veth
49
52
50
-
ip netns exec "${PEER_NS}" ./udpgso_bench_rx -C 1000 -R 10 ${rx_args} && \
51
-
echo "ok" || \
52
-
echo "failed" &
53
+
ip netns exec "${PEER_NS}" ./udpgso_bench_rx -C 1000 -R 10 ${rx_args} &
54
+
local PID1=$!
53
55
54
56
wait_local_port_listen ${PEER_NS} 8000 udp
55
57
./udpgso_bench_tx ${tx_args}
56
-
ret=$?
57
-
wait $(jobs -p)
58
+
check_err $?
59
+
wait ${PID1}
60
+
check_err $?
61
+
[ "$ret" -eq 0 ] && echo "ok" || echo "failed"
58
62
return $ret
59
63
}
60
64
···
73
73
local -r all="$@"
74
74
local -r tx_args=${all%rx*}
75
75
local -r rx_args=${all#*rx}
76
+
local ret=0
76
77
77
78
if [[ ${tx_args} = *-4* ]]; then
78
79
ipt_cmd=iptables
···
94
93
# ... so that GRO will match the UDP_GRO enabled socket, but packets
95
94
# will land on the 'plain' one
96
95
ip netns exec "${PEER_NS}" ./udpgso_bench_rx -G ${family} -b ${addr1} -n 0 &
97
-
pid=$!
98
-
ip netns exec "${PEER_NS}" ./udpgso_bench_rx -C 1000 -R 10 ${family} -b ${addr2%/*} ${rx_args} && \
99
-
echo "ok" || \
100
-
echo "failed"&
96
+
local PID1=$!
97
+
ip netns exec "${PEER_NS}" ./udpgso_bench_rx -C 1000 -R 10 ${family} -b ${addr2%/*} ${rx_args} &
98
+
local PID2=$!
101
99
102
100
wait_local_port_listen "${PEER_NS}" 8000 udp
103
101
./udpgso_bench_tx ${tx_args}
104
-
ret=$?
105
-
kill -INT $pid
106
-
wait $(jobs -p)
102
+
check_err $?
103
+
kill -INT ${PID1}
104
+
wait ${PID2}
105
+
check_err $?
106
+
[ "$ret" -eq 0 ] && echo "ok" || echo "failed"
107
107
return $ret
108
108
}
109
109
···
113
111
local -r all="$@"
114
112
local -r tx_args=${all%rx*}
115
113
local -r rx_args=${all#*rx}
114
+
local ret=0
116
115
117
116
cfg_veth
118
117
119
118
ip netns exec "${PEER_NS}" ./udpgso_bench_rx -C 1000 -R 10 ${rx_args} -p 12345 &
120
-
ip netns exec "${PEER_NS}" ./udpgso_bench_rx -C 2000 -R 10 ${rx_args} && \
121
-
echo "ok" || \
122
-
echo "failed" &
119
+
local PID1=$!
120
+
ip netns exec "${PEER_NS}" ./udpgso_bench_rx -C 2000 -R 10 ${rx_args} &
121
+
local PID2=$!
123
122
124
123
wait_local_port_listen "${PEER_NS}" 12345 udp
125
124
./udpgso_bench_tx ${tx_args} -p 12345
125
+
check_err $?
126
126
wait_local_port_listen "${PEER_NS}" 8000 udp
127
127
./udpgso_bench_tx ${tx_args}
128
-
ret=$?
129
-
wait $(jobs -p)
128
+
check_err $?
129
+
wait ${PID1}
130
+
check_err $?
131
+
wait ${PID2}
132
+
check_err $?
133
+
[ "$ret" -eq 0 ] && echo "ok" || echo "failed"
130
134
return $ret
131
135
}
132
136
···
203
195
check_err $?
204
196
return $ret
205
197
}
206
-
207
-
if [ ! -f ${BPF_FILE} ]; then
208
-
echo "Missing ${BPF_FILE}. Run 'make' first"
209
-
exit -1
210
-
fi
211
198
212
199
if [[ $# -eq 0 ]]; then
213
200
run_all
-1
tools/testing/selftests/tc-testing/tdc.py
-1
tools/testing/selftests/tc-testing/tdc.py
+4
-7
tools/tracing/rtla/src/osnoise_top.c
+4
-7
tools/tracing/rtla/src/osnoise_top.c
···
651
651
return NULL;
652
652
653
653
tool->data = osnoise_alloc_top(nr_cpus);
654
-
if (!tool->data)
655
-
goto out_err;
654
+
if (!tool->data) {
655
+
osnoise_destroy_tool(tool);
656
+
return NULL;
657
+
}
656
658
657
659
tool->params = params;
658
660
···
662
660
osnoise_top_handler, NULL);
663
661
664
662
return tool;
665
-
666
-
out_err:
667
-
osnoise_free_top(tool->data);
668
-
osnoise_destroy_tool(tool);
669
-
return NULL;
670
663
}
671
664
672
665
static int stop_tracing;