Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Merge tag 'powerpc-6.7-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
Pull powerpc updates from Michael Ellerman:
- Add support for KVM running as a nested hypervisor under development
versions of PowerVM, using the new PAPR nested virtualisation API
- Add support for the BPF prog pack allocator
- A rework of the non-server MMU handling to support execute-only on
all platforms
- Some optimisations & cleanups for the powerpc qspinlock code
- Various other small features and fixes
Thanks to Aboorva Devarajan, Aditya Gupta, Amit Machhiwal, Benjamin
Gray, Christophe Leroy, Dr. David Alan Gilbert, Gaurav Batra, Gautam
Menghani, Geert Uytterhoeven, Haren Myneni, Hari Bathini, Joel Stanley,
Jordan Niethe, Julia Lawall, Kautuk Consul, Kuan-Wei Chiu, Michael
Neuling, Minjie Du, Muhammad Muzammil, Naveen N Rao, Nicholas Piggin,
Nick Child, Nysal Jan K.A, Peter Lafreniere, Rob Herring, Sachin Sant,
Sebastian Andrzej Siewior, Shrikanth Hegde, Srikar Dronamraju, Stanislav
Kinsburskii, Vaibhav Jain, Wang Yufen, Yang Yingliang, and Yuan Tan.
* tag 'powerpc-6.7-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux: (100 commits)
powerpc/vmcore: Add MMU information to vmcoreinfo
Revert "powerpc: add `cur_cpu_spec` symbol to vmcoreinfo"
powerpc/bpf: use bpf_jit_binary_pack_[alloc|finalize|free]
powerpc/bpf: rename powerpc64_jit_data to powerpc_jit_data
powerpc/bpf: implement bpf_arch_text_invalidate for bpf_prog_pack
powerpc/bpf: implement bpf_arch_text_copy
powerpc/code-patching: introduce patch_instructions()
powerpc/32s: Implement local_flush_tlb_page_psize()
powerpc/pseries: use kfree_sensitive() in plpks_gen_password()
powerpc/code-patching: Perform hwsync in __patch_instruction() in case of failure
powerpc/fsl_msi: Use device_get_match_data()
powerpc: Remove cpm_dp...() macros
powerpc/qspinlock: Rename yield_propagate_owner tunable
powerpc/qspinlock: Propagate sleepy if previous waiter is preempted
powerpc/qspinlock: don't propagate the not-sleepy state
powerpc/qspinlock: propagate owner preemptedness rather than CPU number
powerpc/qspinlock: stop queued waiters trying to set lock sleepy
powerpc/perf: Fix disabling BHRB and instruction sampling
powerpc/trace: Add support for HAVE_FUNCTION_ARG_ACCESS_API
powerpc/tools: Pass -mabi=elfv2 to gcc-check-mprofile-kernel.sh
...
+5794
-1681
+1
Documentation/arch/powerpc/index.rst
+1
Documentation/arch/powerpc/index.rst
+634
Documentation/arch/powerpc/kvm-nested.rst
+634
Documentation/arch/powerpc/kvm-nested.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
====================================
4
+
Nested KVM on POWER
5
+
====================================
6
+
7
+
Introduction
8
+
============
9
+
10
+
This document explains how a guest operating system can act as a
11
+
hypervisor and run nested guests through the use of hypercalls, if the
12
+
hypervisor has implemented them. The terms L0, L1, and L2 are used to
13
+
refer to different software entities. L0 is the hypervisor mode entity
14
+
that would normally be called the "host" or "hypervisor". L1 is a
15
+
guest virtual machine that is directly run under L0 and is initiated
16
+
and controlled by L0. L2 is a guest virtual machine that is initiated
17
+
and controlled by L1 acting as a hypervisor.
18
+
19
+
Existing API
20
+
============
21
+
22
+
Linux/KVM has had support for Nesting as an L0 or L1 since 2018
23
+
24
+
The L0 code was added::
25
+
26
+
commit 8e3f5fc1045dc49fd175b978c5457f5f51e7a2ce
27
+
Author: Paul Mackerras <paulus@ozlabs.org>
28
+
Date: Mon Oct 8 16:31:03 2018 +1100
29
+
KVM: PPC: Book3S HV: Framework and hcall stubs for nested virtualization
30
+
31
+
The L1 code was added::
32
+
33
+
commit 360cae313702cdd0b90f82c261a8302fecef030a
34
+
Author: Paul Mackerras <paulus@ozlabs.org>
35
+
Date: Mon Oct 8 16:31:04 2018 +1100
36
+
KVM: PPC: Book3S HV: Nested guest entry via hypercall
37
+
38
+
This API works primarily using a single hcall h_enter_nested(). This
39
+
call made by the L1 to tell the L0 to start an L2 vCPU with the given
40
+
state. The L0 then starts this L2 and runs until an L2 exit condition
41
+
is reached. Once the L2 exits, the state of the L2 is given back to
42
+
the L1 by the L0. The full L2 vCPU state is always transferred from
43
+
and to L1 when the L2 is run. The L0 doesn't keep any state on the L2
44
+
vCPU (except in the short sequence in the L0 on L1 -> L2 entry and L2
45
+
-> L1 exit).
46
+
47
+
The only state kept by the L0 is the partition table. The L1 registers
48
+
it's partition table using the h_set_partition_table() hcall. All
49
+
other state held by the L0 about the L2s is cached state (such as
50
+
shadow page tables).
51
+
52
+
The L1 may run any L2 or vCPU without first informing the L0. It
53
+
simply starts the vCPU using h_enter_nested(). The creation of L2s and
54
+
vCPUs is done implicitly whenever h_enter_nested() is called.
55
+
56
+
In this document, we call this existing API the v1 API.
57
+
58
+
New PAPR API
59
+
===============
60
+
61
+
The new PAPR API changes from the v1 API such that the creating L2 and
62
+
associated vCPUs is explicit. In this document, we call this the v2
63
+
API.
64
+
65
+
h_enter_nested() is replaced with H_GUEST_VCPU_RUN(). Before this can
66
+
be called the L1 must explicitly create the L2 using h_guest_create()
67
+
and any associated vCPUs() created with h_guest_create_vCPU(). Getting
68
+
and setting vCPU state can also be performed using h_guest_{g|s}et
69
+
hcall.
70
+
71
+
The basic execution flow is for an L1 to create an L2, run it, and
72
+
delete it is:
73
+
74
+
- L1 and L0 negotiate capabilities with H_GUEST_{G,S}ET_CAPABILITIES()
75
+
(normally at L1 boot time).
76
+
77
+
- L1 requests the L0 create an L2 with H_GUEST_CREATE() and receives a token
78
+
79
+
- L1 requests the L0 create an L2 vCPU with H_GUEST_CREATE_VCPU()
80
+
81
+
- L1 and L0 communicate the vCPU state using the H_GUEST_{G,S}ET() hcall
82
+
83
+
- L1 requests the L0 runs the vCPU running H_GUEST_VCPU_RUN() hcall
84
+
85
+
- L1 deletes L2 with H_GUEST_DELETE()
86
+
87
+
More details of the individual hcalls follows:
88
+
89
+
HCALL Details
90
+
=============
91
+
92
+
This documentation is provided to give an overall understating of the
93
+
API. It doesn't aim to provide all the details required to implement
94
+
an L1 or L0. Latest version of PAPR can be referred to for more details.
95
+
96
+
All these HCALLs are made by the L1 to the L0.
97
+
98
+
H_GUEST_GET_CAPABILITIES()
99
+
--------------------------
100
+
101
+
This is called to get the capabilities of the L0 nested
102
+
hypervisor. This includes capabilities such the CPU versions (eg
103
+
POWER9, POWER10) that are supported as L2s::
104
+
105
+
H_GUEST_GET_CAPABILITIES(uint64 flags)
106
+
107
+
Parameters:
108
+
Input:
109
+
flags: Reserved
110
+
Output:
111
+
R3: Return code
112
+
R4: Hypervisor Supported Capabilities bitmap 1
113
+
114
+
H_GUEST_SET_CAPABILITIES()
115
+
--------------------------
116
+
117
+
This is called to inform the L0 of the capabilities of the L1
118
+
hypervisor. The set of flags passed here are the same as
119
+
H_GUEST_GET_CAPABILITIES()
120
+
121
+
Typically, GET will be called first and then SET will be called with a
122
+
subset of the flags returned from GET. This process allows the L0 and
123
+
L1 to negotiate an agreed set of capabilities::
124
+
125
+
H_GUEST_SET_CAPABILITIES(uint64 flags,
126
+
uint64 capabilitiesBitmap1)
127
+
Parameters:
128
+
Input:
129
+
flags: Reserved
130
+
capabilitiesBitmap1: Only capabilities advertised through
131
+
H_GUEST_GET_CAPABILITIES
132
+
Output:
133
+
R3: Return code
134
+
R4: If R3 = H_P2: The number of invalid bitmaps
135
+
R5: If R3 = H_P2: The index of first invalid bitmap
136
+
137
+
H_GUEST_CREATE()
138
+
----------------
139
+
140
+
This is called to create an L2. A unique ID of the L2 created
141
+
(similar to an LPID) is returned, which can be used on subsequent HCALLs to
142
+
identify the L2::
143
+
144
+
H_GUEST_CREATE(uint64 flags,
145
+
uint64 continueToken);
146
+
Parameters:
147
+
Input:
148
+
flags: Reserved
149
+
continueToken: Initial call set to -1. Subsequent calls,
150
+
after H_Busy or H_LongBusyOrder has been
151
+
returned, value that was returned in R4.
152
+
Output:
153
+
R3: Return code. Notable:
154
+
H_Not_Enough_Resources: Unable to create Guest VCPU due to not
155
+
enough Hypervisor memory. See H_GUEST_CREATE_GET_STATE(flags =
156
+
takeOwnershipOfVcpuState)
157
+
R4: If R3 = H_Busy or_H_LongBusyOrder -> continueToken
158
+
159
+
H_GUEST_CREATE_VCPU()
160
+
---------------------
161
+
162
+
This is called to create a vCPU associated with an L2. The L2 id
163
+
(returned from H_GUEST_CREATE()) should be passed it. Also passed in
164
+
is a unique (for this L2) vCPUid. This vCPUid is allocated by the
165
+
L1::
166
+
167
+
H_GUEST_CREATE_VCPU(uint64 flags,
168
+
uint64 guestId,
169
+
uint64 vcpuId);
170
+
Parameters:
171
+
Input:
172
+
flags: Reserved
173
+
guestId: ID obtained from H_GUEST_CREATE
174
+
vcpuId: ID of the vCPU to be created. This must be within the
175
+
range of 0 to 2047
176
+
Output:
177
+
R3: Return code. Notable:
178
+
H_Not_Enough_Resources: Unable to create Guest VCPU due to not
179
+
enough Hypervisor memory. See H_GUEST_CREATE_GET_STATE(flags =
180
+
takeOwnershipOfVcpuState)
181
+
182
+
H_GUEST_GET_STATE()
183
+
-------------------
184
+
185
+
This is called to get state associated with an L2 (Guest-wide or vCPU specific).
186
+
This info is passed via the Guest State Buffer (GSB), a standard format as
187
+
explained later in this doc, necessary details below:
188
+
189
+
This can get either L2 wide or vcpu specific information. Examples of
190
+
L2 wide is the timebase offset or process scoped page table
191
+
info. Examples of vCPU specific are GPRs or VSRs. A bit in the flags
192
+
parameter specifies if this call is L2 wide or vCPU specific and the
193
+
IDs in the GSB must match this.
194
+
195
+
The L1 provides a pointer to the GSB as a parameter to this call. Also
196
+
provided is the L2 and vCPU IDs associated with the state to set.
197
+
198
+
The L1 writes only the IDs and sizes in the GSB. L0 writes the
199
+
associated values for each ID in the GSB::
200
+
201
+
H_GUEST_GET_STATE(uint64 flags,
202
+
uint64 guestId,
203
+
uint64 vcpuId,
204
+
uint64 dataBuffer,
205
+
uint64 dataBufferSizeInBytes);
206
+
Parameters:
207
+
Input:
208
+
flags:
209
+
Bit 0: getGuestWideState: Request state of the Guest instead
210
+
of an individual VCPU.
211
+
Bit 1: takeOwnershipOfVcpuState Indicate the L1 is taking
212
+
over ownership of the VCPU state and that the L0 can free
213
+
the storage holding the state. The VCPU state will need to
214
+
be returned to the Hypervisor via H_GUEST_SET_STATE prior
215
+
to H_GUEST_RUN_VCPU being called for this VCPU. The data
216
+
returned in the dataBuffer is in a Hypervisor internal
217
+
format.
218
+
Bits 2-63: Reserved
219
+
guestId: ID obtained from H_GUEST_CREATE
220
+
vcpuId: ID of the vCPU pass to H_GUEST_CREATE_VCPU
221
+
dataBuffer: A L1 real address of the GSB.
222
+
If takeOwnershipOfVcpuState, size must be at least the size
223
+
returned by ID=0x0001
224
+
dataBufferSizeInBytes: Size of dataBuffer
225
+
Output:
226
+
R3: Return code
227
+
R4: If R3 = H_Invalid_Element_Id: The array index of the bad
228
+
element ID.
229
+
If R3 = H_Invalid_Element_Size: The array index of the bad
230
+
element size.
231
+
If R3 = H_Invalid_Element_Value: The array index of the bad
232
+
element value.
233
+
234
+
H_GUEST_SET_STATE()
235
+
-------------------
236
+
237
+
This is called to set L2 wide or vCPU specific L2 state. This info is
238
+
passed via the Guest State Buffer (GSB), necessary details below:
239
+
240
+
This can set either L2 wide or vcpu specific information. Examples of
241
+
L2 wide is the timebase offset or process scoped page table
242
+
info. Examples of vCPU specific are GPRs or VSRs. A bit in the flags
243
+
parameter specifies if this call is L2 wide or vCPU specific and the
244
+
IDs in the GSB must match this.
245
+
246
+
The L1 provides a pointer to the GSB as a parameter to this call. Also
247
+
provided is the L2 and vCPU IDs associated with the state to set.
248
+
249
+
The L1 writes all values in the GSB and the L0 only reads the GSB for
250
+
this call::
251
+
252
+
H_GUEST_SET_STATE(uint64 flags,
253
+
uint64 guestId,
254
+
uint64 vcpuId,
255
+
uint64 dataBuffer,
256
+
uint64 dataBufferSizeInBytes);
257
+
Parameters:
258
+
Input:
259
+
flags:
260
+
Bit 0: getGuestWideState: Request state of the Guest instead
261
+
of an individual VCPU.
262
+
Bit 1: returnOwnershipOfVcpuState Return Guest VCPU state. See
263
+
GET_STATE takeOwnershipOfVcpuState
264
+
Bits 2-63: Reserved
265
+
guestId: ID obtained from H_GUEST_CREATE
266
+
vcpuId: ID of the vCPU pass to H_GUEST_CREATE_VCPU
267
+
dataBuffer: A L1 real address of the GSB.
268
+
If takeOwnershipOfVcpuState, size must be at least the size
269
+
returned by ID=0x0001
270
+
dataBufferSizeInBytes: Size of dataBuffer
271
+
Output:
272
+
R3: Return code
273
+
R4: If R3 = H_Invalid_Element_Id: The array index of the bad
274
+
element ID.
275
+
If R3 = H_Invalid_Element_Size: The array index of the bad
276
+
element size.
277
+
If R3 = H_Invalid_Element_Value: The array index of the bad
278
+
element value.
279
+
280
+
H_GUEST_RUN_VCPU()
281
+
------------------
282
+
283
+
This is called to run an L2 vCPU. The L2 and vCPU IDs are passed in as
284
+
parameters. The vCPU runs with the state set previously using
285
+
H_GUEST_SET_STATE(). When the L2 exits, the L1 will resume from this
286
+
hcall.
287
+
288
+
This hcall also has associated input and output GSBs. Unlike
289
+
H_GUEST_{S,G}ET_STATE(), these GSB pointers are not passed in as
290
+
parameters to the hcall (This was done in the interest of
291
+
performance). The locations of these GSBs must be preregistered using
292
+
the H_GUEST_SET_STATE() call with ID 0x0c00 and 0x0c01 (see table
293
+
below).
294
+
295
+
The input GSB may contain only VCPU specific elements to be set. This
296
+
GSB may also contain zero elements (ie 0 in the first 4 bytes of the
297
+
GSB) if nothing needs to be set.
298
+
299
+
On exit from the hcall, the output buffer is filled with elements
300
+
determined by the L0. The reason for the exit is contained in GPR4 (ie
301
+
NIP is put in GPR4). The elements returned depend on the exit
302
+
type. For example, if the exit reason is the L2 doing a hcall (GPR4 =
303
+
0xc00), then GPR3-12 are provided in the output GSB as this is the
304
+
state likely needed to service the hcall. If additional state is
305
+
needed, H_GUEST_GET_STATE() may be called by the L1.
306
+
307
+
To synthesize interrupts in the L2, when calling H_GUEST_RUN_VCPU()
308
+
the L1 may set a flag (as a hcall parameter) and the L0 will
309
+
synthesize the interrupt in the L2. Alternatively, the L1 may
310
+
synthesize the interrupt itself using H_GUEST_SET_STATE() or the
311
+
H_GUEST_RUN_VCPU() input GSB to set the state appropriately::
312
+
313
+
H_GUEST_RUN_VCPU(uint64 flags,
314
+
uint64 guestId,
315
+
uint64 vcpuId,
316
+
uint64 dataBuffer,
317
+
uint64 dataBufferSizeInBytes);
318
+
Parameters:
319
+
Input:
320
+
flags:
321
+
Bit 0: generateExternalInterrupt: Generate an external interrupt
322
+
Bit 1: generatePrivilegedDoorbell: Generate a Privileged Doorbell
323
+
Bit 2: sendToSystemReset”: Generate a System Reset Interrupt
324
+
Bits 3-63: Reserved
325
+
guestId: ID obtained from H_GUEST_CREATE
326
+
vcpuId: ID of the vCPU pass to H_GUEST_CREATE_VCPU
327
+
Output:
328
+
R3: Return code
329
+
R4: If R3 = H_Success: The reason L1 VCPU exited (ie. NIA)
330
+
0x000: The VCPU stopped running for an unspecified reason. An
331
+
example of this is the Hypervisor stopping a VCPU running
332
+
due to an outstanding interrupt for the Host Partition.
333
+
0x980: HDEC
334
+
0xC00: HCALL
335
+
0xE00: HDSI
336
+
0xE20: HISI
337
+
0xE40: HEA
338
+
0xF80: HV Fac Unavail
339
+
If R3 = H_Invalid_Element_Id, H_Invalid_Element_Size, or
340
+
H_Invalid_Element_Value: R4 is offset of the invalid element
341
+
in the input buffer.
342
+
343
+
H_GUEST_DELETE()
344
+
----------------
345
+
346
+
This is called to delete an L2. All associated vCPUs are also
347
+
deleted. No specific vCPU delete call is provided.
348
+
349
+
A flag may be provided to delete all guests. This is used to reset the
350
+
L0 in the case of kdump/kexec::
351
+
352
+
H_GUEST_DELETE(uint64 flags,
353
+
uint64 guestId)
354
+
Parameters:
355
+
Input:
356
+
flags:
357
+
Bit 0: deleteAllGuests: deletes all guests
358
+
Bits 1-63: Reserved
359
+
guestId: ID obtained from H_GUEST_CREATE
360
+
Output:
361
+
R3: Return code
362
+
363
+
Guest State Buffer
364
+
==================
365
+
366
+
The Guest State Buffer (GSB) is the main method of communicating state
367
+
about the L2 between the L1 and L0 via H_GUEST_{G,S}ET() and
368
+
H_GUEST_VCPU_RUN() calls.
369
+
370
+
State may be associated with a whole L2 (eg timebase offset) or a
371
+
specific L2 vCPU (eg. GPR state). Only L2 VCPU state maybe be set by
372
+
H_GUEST_VCPU_RUN().
373
+
374
+
All data in the GSB is big endian (as is standard in PAPR)
375
+
376
+
The Guest state buffer has a header which gives the number of
377
+
elements, followed by the GSB elements themselves.
378
+
379
+
GSB header:
380
+
381
+
+----------+----------+-------------------------------------------+
382
+
| Offset | Size | Purpose |
383
+
| Bytes | Bytes | |
384
+
+==========+==========+===========================================+
385
+
| 0 | 4 | Number of elements |
386
+
+----------+----------+-------------------------------------------+
387
+
| 4 | | Guest state buffer elements |
388
+
+----------+----------+-------------------------------------------+
389
+
390
+
GSB element:
391
+
392
+
+----------+----------+-------------------------------------------+
393
+
| Offset | Size | Purpose |
394
+
| Bytes | Bytes | |
395
+
+==========+==========+===========================================+
396
+
| 0 | 2 | ID |
397
+
+----------+----------+-------------------------------------------+
398
+
| 2 | 2 | Size of Value |
399
+
+----------+----------+-------------------------------------------+
400
+
| 4 | As above | Value |
401
+
+----------+----------+-------------------------------------------+
402
+
403
+
The ID in the GSB element specifies what is to be set. This includes
404
+
archtected state like GPRs, VSRs, SPRs, plus also some meta data about
405
+
the partition like the timebase offset and partition scoped page
406
+
table information.
407
+
408
+
+--------+-------+----+--------+----------------------------------+
409
+
| ID | Size | RW | Thread | Details |
410
+
| | Bytes | | Guest | |
411
+
| | | | Scope | |
412
+
+========+=======+====+========+==================================+
413
+
| 0x0000 | | RW | TG | NOP element |
414
+
+--------+-------+----+--------+----------------------------------+
415
+
| 0x0001 | 0x08 | R | G | Size of L0 vCPU state. See: |
416
+
| | | | | H_GUEST_GET_STATE: |
417
+
| | | | | flags = takeOwnershipOfVcpuState |
418
+
+--------+-------+----+--------+----------------------------------+
419
+
| 0x0002 | 0x08 | R | G | Size Run vCPU out buffer |
420
+
+--------+-------+----+--------+----------------------------------+
421
+
| 0x0003 | 0x04 | RW | G | Logical PVR |
422
+
+--------+-------+----+--------+----------------------------------+
423
+
| 0x0004 | 0x08 | RW | G | TB Offset (L1 relative) |
424
+
+--------+-------+----+--------+----------------------------------+
425
+
| 0x0005 | 0x18 | RW | G |Partition scoped page tbl info: |
426
+
| | | | | |
427
+
| | | | |- 0x00 Addr part scope table |
428
+
| | | | |- 0x08 Num addr bits |
429
+
| | | | |- 0x10 Size root dir |
430
+
+--------+-------+----+--------+----------------------------------+
431
+
| 0x0006 | 0x10 | RW | G |Process Table Information: |
432
+
| | | | | |
433
+
| | | | |- 0x0 Addr proc scope table |
434
+
| | | | |- 0x8 Table size. |
435
+
+--------+-------+----+--------+----------------------------------+
436
+
| 0x0007-| | | | Reserved |
437
+
| 0x0BFF | | | | |
438
+
+--------+-------+----+--------+----------------------------------+
439
+
| 0x0C00 | 0x10 | RW | T |Run vCPU Input Buffer: |
440
+
| | | | | |
441
+
| | | | |- 0x0 Addr of buffer |
442
+
| | | | |- 0x8 Buffer Size. |
443
+
+--------+-------+----+--------+----------------------------------+
444
+
| 0x0C01 | 0x10 | RW | T |Run vCPU Output Buffer: |
445
+
| | | | | |
446
+
| | | | |- 0x0 Addr of buffer |
447
+
| | | | |- 0x8 Buffer Size. |
448
+
+--------+-------+----+--------+----------------------------------+
449
+
| 0x0C02 | 0x08 | RW | T | vCPU VPA Address |
450
+
+--------+-------+----+--------+----------------------------------+
451
+
| 0x0C03-| | | | Reserved |
452
+
| 0x0FFF | | | | |
453
+
+--------+-------+----+--------+----------------------------------+
454
+
| 0x1000-| 0x08 | RW | T | GPR 0-31 |
455
+
| 0x101F | | | | |
456
+
+--------+-------+----+--------+----------------------------------+
457
+
| 0x1020 | 0x08 | T | T | HDEC expiry TB |
458
+
+--------+-------+----+--------+----------------------------------+
459
+
| 0x1021 | 0x08 | RW | T | NIA |
460
+
+--------+-------+----+--------+----------------------------------+
461
+
| 0x1022 | 0x08 | RW | T | MSR |
462
+
+--------+-------+----+--------+----------------------------------+
463
+
| 0x1023 | 0x08 | RW | T | LR |
464
+
+--------+-------+----+--------+----------------------------------+
465
+
| 0x1024 | 0x08 | RW | T | XER |
466
+
+--------+-------+----+--------+----------------------------------+
467
+
| 0x1025 | 0x08 | RW | T | CTR |
468
+
+--------+-------+----+--------+----------------------------------+
469
+
| 0x1026 | 0x08 | RW | T | CFAR |
470
+
+--------+-------+----+--------+----------------------------------+
471
+
| 0x1027 | 0x08 | RW | T | SRR0 |
472
+
+--------+-------+----+--------+----------------------------------+
473
+
| 0x1028 | 0x08 | RW | T | SRR1 |
474
+
+--------+-------+----+--------+----------------------------------+
475
+
| 0x1029 | 0x08 | RW | T | DAR |
476
+
+--------+-------+----+--------+----------------------------------+
477
+
| 0x102A | 0x08 | RW | T | DEC expiry TB |
478
+
+--------+-------+----+--------+----------------------------------+
479
+
| 0x102B | 0x08 | RW | T | VTB |
480
+
+--------+-------+----+--------+----------------------------------+
481
+
| 0x102C | 0x08 | RW | T | LPCR |
482
+
+--------+-------+----+--------+----------------------------------+
483
+
| 0x102D | 0x08 | RW | T | HFSCR |
484
+
+--------+-------+----+--------+----------------------------------+
485
+
| 0x102E | 0x08 | RW | T | FSCR |
486
+
+--------+-------+----+--------+----------------------------------+
487
+
| 0x102F | 0x08 | RW | T | FPSCR |
488
+
+--------+-------+----+--------+----------------------------------+
489
+
| 0x1030 | 0x08 | RW | T | DAWR0 |
490
+
+--------+-------+----+--------+----------------------------------+
491
+
| 0x1031 | 0x08 | RW | T | DAWR1 |
492
+
+--------+-------+----+--------+----------------------------------+
493
+
| 0x1032 | 0x08 | RW | T | CIABR |
494
+
+--------+-------+----+--------+----------------------------------+
495
+
| 0x1033 | 0x08 | RW | T | PURR |
496
+
+--------+-------+----+--------+----------------------------------+
497
+
| 0x1034 | 0x08 | RW | T | SPURR |
498
+
+--------+-------+----+--------+----------------------------------+
499
+
| 0x1035 | 0x08 | RW | T | IC |
500
+
+--------+-------+----+--------+----------------------------------+
501
+
| 0x1036-| 0x08 | RW | T | SPRG 0-3 |
502
+
| 0x1039 | | | | |
503
+
+--------+-------+----+--------+----------------------------------+
504
+
| 0x103A | 0x08 | W | T | PPR |
505
+
+--------+-------+----+--------+----------------------------------+
506
+
| 0x103B | 0x08 | RW | T | MMCR 0-3 |
507
+
| 0x103E | | | | |
508
+
+--------+-------+----+--------+----------------------------------+
509
+
| 0x103F | 0x08 | RW | T | MMCRA |
510
+
+--------+-------+----+--------+----------------------------------+
511
+
| 0x1040 | 0x08 | RW | T | SIER |
512
+
+--------+-------+----+--------+----------------------------------+
513
+
| 0x1041 | 0x08 | RW | T | SIER 2 |
514
+
+--------+-------+----+--------+----------------------------------+
515
+
| 0x1042 | 0x08 | RW | T | SIER 3 |
516
+
+--------+-------+----+--------+----------------------------------+
517
+
| 0x1043 | 0x08 | RW | T | BESCR |
518
+
+--------+-------+----+--------+----------------------------------+
519
+
| 0x1044 | 0x08 | RW | T | EBBHR |
520
+
+--------+-------+----+--------+----------------------------------+
521
+
| 0x1045 | 0x08 | RW | T | EBBRR |
522
+
+--------+-------+----+--------+----------------------------------+
523
+
| 0x1046 | 0x08 | RW | T | AMR |
524
+
+--------+-------+----+--------+----------------------------------+
525
+
| 0x1047 | 0x08 | RW | T | IAMR |
526
+
+--------+-------+----+--------+----------------------------------+
527
+
| 0x1048 | 0x08 | RW | T | AMOR |
528
+
+--------+-------+----+--------+----------------------------------+
529
+
| 0x1049 | 0x08 | RW | T | UAMOR |
530
+
+--------+-------+----+--------+----------------------------------+
531
+
| 0x104A | 0x08 | RW | T | SDAR |
532
+
+--------+-------+----+--------+----------------------------------+
533
+
| 0x104B | 0x08 | RW | T | SIAR |
534
+
+--------+-------+----+--------+----------------------------------+
535
+
| 0x104C | 0x08 | RW | T | DSCR |
536
+
+--------+-------+----+--------+----------------------------------+
537
+
| 0x104D | 0x08 | RW | T | TAR |
538
+
+--------+-------+----+--------+----------------------------------+
539
+
| 0x104E | 0x08 | RW | T | DEXCR |
540
+
+--------+-------+----+--------+----------------------------------+
541
+
| 0x104F | 0x08 | RW | T | HDEXCR |
542
+
+--------+-------+----+--------+----------------------------------+
543
+
| 0x1050 | 0x08 | RW | T | HASHKEYR |
544
+
+--------+-------+----+--------+----------------------------------+
545
+
| 0x1051 | 0x08 | RW | T | HASHPKEYR |
546
+
+--------+-------+----+--------+----------------------------------+
547
+
| 0x1052 | 0x08 | RW | T | CTRL |
548
+
+--------+-------+----+--------+----------------------------------+
549
+
| 0x1053-| | | | Reserved |
550
+
| 0x1FFF | | | | |
551
+
+--------+-------+----+--------+----------------------------------+
552
+
| 0x2000 | 0x04 | RW | T | CR |
553
+
+--------+-------+----+--------+----------------------------------+
554
+
| 0x2001 | 0x04 | RW | T | PIDR |
555
+
+--------+-------+----+--------+----------------------------------+
556
+
| 0x2002 | 0x04 | RW | T | DSISR |
557
+
+--------+-------+----+--------+----------------------------------+
558
+
| 0x2003 | 0x04 | RW | T | VSCR |
559
+
+--------+-------+----+--------+----------------------------------+
560
+
| 0x2004 | 0x04 | RW | T | VRSAVE |
561
+
+--------+-------+----+--------+----------------------------------+
562
+
| 0x2005 | 0x04 | RW | T | DAWRX0 |
563
+
+--------+-------+----+--------+----------------------------------+
564
+
| 0x2006 | 0x04 | RW | T | DAWRX1 |
565
+
+--------+-------+----+--------+----------------------------------+
566
+
| 0x2007-| 0x04 | RW | T | PMC 1-6 |
567
+
| 0x200c | | | | |
568
+
+--------+-------+----+--------+----------------------------------+
569
+
| 0x200D | 0x04 | RW | T | WORT |
570
+
+--------+-------+----+--------+----------------------------------+
571
+
| 0x200E | 0x04 | RW | T | PSPB |
572
+
+--------+-------+----+--------+----------------------------------+
573
+
| 0x200F-| | | | Reserved |
574
+
| 0x2FFF | | | | |
575
+
+--------+-------+----+--------+----------------------------------+
576
+
| 0x3000-| 0x10 | RW | T | VSR 0-63 |
577
+
| 0x303F | | | | |
578
+
+--------+-------+----+--------+----------------------------------+
579
+
| 0x3040-| | | | Reserved |
580
+
| 0xEFFF | | | | |
581
+
+--------+-------+----+--------+----------------------------------+
582
+
| 0xF000 | 0x08 | R | T | HDAR |
583
+
+--------+-------+----+--------+----------------------------------+
584
+
| 0xF001 | 0x04 | R | T | HDSISR |
585
+
+--------+-------+----+--------+----------------------------------+
586
+
| 0xF002 | 0x04 | R | T | HEIR |
587
+
+--------+-------+----+--------+----------------------------------+
588
+
| 0xF003 | 0x08 | R | T | ASDR |
589
+
+--------+-------+----+--------+----------------------------------+
590
+
591
+
592
+
Miscellaneous info
593
+
==================
594
+
595
+
State not in ptregs/hvregs
596
+
--------------------------
597
+
598
+
In the v1 API, some state is not in the ptregs/hvstate. This includes
599
+
the vector register and some SPRs. For the L1 to set this state for
600
+
the L2, the L1 loads up these hardware registers before the
601
+
h_enter_nested() call and the L0 ensures they end up as the L2 state
602
+
(by not touching them).
603
+
604
+
The v2 API removes this and explicitly sets this state via the GSB.
605
+
606
+
L1 Implementation details: Caching state
607
+
----------------------------------------
608
+
609
+
In the v1 API, all state is sent from the L1 to the L0 and vice versa
610
+
on every h_enter_nested() hcall. If the L0 is not currently running
611
+
any L2s, the L0 has no state information about them. The only
612
+
exception to this is the location of the partition table, registered
613
+
via h_set_partition_table().
614
+
615
+
The v2 API changes this so that the L0 retains the L2 state even when
616
+
it's vCPUs are no longer running. This means that the L1 only needs to
617
+
communicate with the L0 about L2 state when it needs to modify the L2
618
+
state, or when it's value is out of date. This provides an opportunity
619
+
for performance optimisation.
620
+
621
+
When a vCPU exits from a H_GUEST_RUN_VCPU() call, the L1 internally
622
+
marks all L2 state as invalid. This means that if the L1 wants to know
623
+
the L2 state (say via a kvm_get_one_reg() call), it needs call
624
+
H_GUEST_GET_STATE() to get that state. Once it's read, it's marked as
625
+
valid in L1 until the L2 is run again.
626
+
627
+
Also, when an L1 modifies L2 vcpu state, it doesn't need to write it
628
+
to the L0 until that L2 vcpu runs again. Hence when the L1 updates
629
+
state (say via a kvm_set_one_reg() call), it writes to an internal L1
630
+
copy and only flushes this copy to the L0 when the L2 runs again via
631
+
the H_GUEST_VCPU_RUN() input buffer.
632
+
633
+
This lazy updating of state by the L1 avoids unnecessary
634
+
H_GUEST_{G|S}ET_STATE() calls.
+1
arch/powerpc/Kconfig
+1
arch/powerpc/Kconfig
···
237
237
select HAVE_EFFICIENT_UNALIGNED_ACCESS
238
238
select HAVE_FAST_GUP
239
239
select HAVE_FTRACE_MCOUNT_RECORD
240
+
select HAVE_FUNCTION_ARG_ACCESS_API
240
241
select HAVE_FUNCTION_DESCRIPTORS if PPC64_ELF_ABI_V1
241
242
select HAVE_FUNCTION_ERROR_INJECTION
242
243
select HAVE_FUNCTION_GRAPH_TRACER
+14
arch/powerpc/Kconfig.debug
+14
arch/powerpc/Kconfig.debug
···
82
82
bool "Run self-tests of the MSI bitmap code"
83
83
depends on DEBUG_KERNEL
84
84
85
+
config GUEST_STATE_BUFFER_TEST
86
+
def_tristate n
87
+
prompt "Enable Guest State Buffer unit tests"
88
+
depends on KUNIT
89
+
depends on KVM_BOOK3S_HV_POSSIBLE
90
+
default KUNIT_ALL_TESTS
91
+
help
92
+
The Guest State Buffer is a data format specified in the PAPR.
93
+
It is by hcalls to communicate the state of L2 guests between
94
+
the L1 and L0 hypervisors. Enable unit tests for the library
95
+
used to create and use guest state buffers.
96
+
85
97
config PPC_IRQ_SOFT_MASK_DEBUG
86
98
bool "Include extra checks for powerpc irq soft masking"
87
99
depends on PPC64
···
159
147
config BOOTX_TEXT
160
148
bool "Support for early boot text console (BootX or OpenFirmware only)"
161
149
depends on PPC_BOOK3S
150
+
select FONT_SUN8x16
151
+
select FONT_SUPPORT
162
152
help
163
153
Say Y here to see progress messages from the boot firmware in text
164
154
mode. Requires either BootX or Open Firmware.
+10
-6
arch/powerpc/boot/install.sh
+10
-6
arch/powerpc/boot/install.sh
···
21
21
# this should work for both the pSeries zImage and the iSeries vmlinux.sm
22
22
image_name=`basename $2`
23
23
24
-
if [ -f $4/$image_name ]; then
25
-
mv $4/$image_name $4/$image_name.old
24
+
25
+
echo "Warning: '${INSTALLKERNEL}' command not available... Copying" \
26
+
"directly to $4/$image_name-$1" >&2
27
+
28
+
if [ -f $4/$image_name-$1 ]; then
29
+
mv $4/$image_name-$1 $4/$image_name-$1.old
26
30
fi
27
31
28
-
if [ -f $4/System.map ]; then
29
-
mv $4/System.map $4/System.old
32
+
if [ -f $4/System.map-$1 ]; then
33
+
mv $4/System.map-$1 $4/System-$1.old
30
34
fi
31
35
32
-
cat $2 > $4/$image_name
33
-
cp $3 $4/System.map
36
+
cat $2 > $4/$image_name-$1
37
+
cp $3 $4/System.map-$1
-1
arch/powerpc/configs/44x/sam440ep_defconfig
-1
arch/powerpc/configs/44x/sam440ep_defconfig
+4
arch/powerpc/configs/debug.config
+4
arch/powerpc/configs/debug.config
-4
arch/powerpc/configs/g5_defconfig
-4
arch/powerpc/configs/g5_defconfig
···
202
202
CONFIG_EXT4_FS=y
203
203
CONFIG_EXT4_FS_POSIX_ACL=y
204
204
CONFIG_EXT4_FS_SECURITY=y
205
-
CONFIG_REISERFS_FS=y
206
-
CONFIG_REISERFS_FS_XATTR=y
207
-
CONFIG_REISERFS_FS_POSIX_ACL=y
208
-
CONFIG_REISERFS_FS_SECURITY=y
209
205
CONFIG_XFS_FS=m
210
206
CONFIG_XFS_POSIX_ACL=y
211
207
CONFIG_FS_DAX=y
+1
-1
arch/powerpc/configs/pmac32_defconfig
+1
-1
arch/powerpc/configs/pmac32_defconfig
···
138
138
CONFIG_DM_MIRROR=m
139
139
CONFIG_DM_ZERO=m
140
140
CONFIG_ADB=y
141
-
CONFIG_ADB_CUDA=y
142
141
CONFIG_ADB_PMU=y
143
142
CONFIG_ADB_PMU_LED=y
144
143
CONFIG_ADB_PMU_LED_DISK=y
···
180
181
CONFIG_SERIAL_PMACZILOG_CONSOLE=y
181
182
CONFIG_NVRAM=y
182
183
CONFIG_I2C_CHARDEV=m
184
+
CONFIG_POWER_RESET=y
183
185
CONFIG_APM_POWER=y
184
186
CONFIG_BATTERY_PMU=y
185
187
CONFIG_HWMON=m
-4
arch/powerpc/configs/ppc64e_defconfig
-4
arch/powerpc/configs/ppc64e_defconfig
···
175
175
CONFIG_EXT4_FS=y
176
176
CONFIG_EXT4_FS_POSIX_ACL=y
177
177
CONFIG_EXT4_FS_SECURITY=y
178
-
CONFIG_REISERFS_FS=y
179
-
CONFIG_REISERFS_FS_XATTR=y
180
-
CONFIG_REISERFS_FS_POSIX_ACL=y
181
-
CONFIG_REISERFS_FS_SECURITY=y
182
178
CONFIG_JFS_FS=y
183
179
CONFIG_JFS_POSIX_ACL=y
184
180
CONFIG_JFS_SECURITY=y
-5
arch/powerpc/configs/ppc6xx_defconfig
-5
arch/powerpc/configs/ppc6xx_defconfig
···
954
954
CONFIG_EXT4_FS_POSIX_ACL=y
955
955
CONFIG_EXT4_FS_SECURITY=y
956
956
CONFIG_JBD2_DEBUG=y
957
-
CONFIG_REISERFS_FS=m
958
-
CONFIG_REISERFS_PROC_INFO=y
959
-
CONFIG_REISERFS_FS_XATTR=y
960
-
CONFIG_REISERFS_FS_POSIX_ACL=y
961
-
CONFIG_REISERFS_FS_SECURITY=y
962
957
CONFIG_JFS_FS=m
963
958
CONFIG_JFS_POSIX_ACL=y
964
959
CONFIG_JFS_SECURITY=y
+31
-52
arch/powerpc/include/asm/book3s/32/pgtable.h
+31
-52
arch/powerpc/include/asm/book3s/32/pgtable.h
···
20
20
21
21
#define _PAGE_PRESENT 0x001 /* software: pte contains a translation */
22
22
#define _PAGE_HASHPTE 0x002 /* hash_page has made an HPTE for this pte */
23
-
#define _PAGE_USER 0x004 /* usermode access allowed */
23
+
#define _PAGE_READ 0x004 /* software: read access allowed */
24
24
#define _PAGE_GUARDED 0x008 /* G: prohibit speculative access */
25
25
#define _PAGE_COHERENT 0x010 /* M: enforce memory coherence (SMP systems) */
26
26
#define _PAGE_NO_CACHE 0x020 /* I: cache inhibit */
···
28
28
#define _PAGE_DIRTY 0x080 /* C: page changed */
29
29
#define _PAGE_ACCESSED 0x100 /* R: page referenced */
30
30
#define _PAGE_EXEC 0x200 /* software: exec allowed */
31
-
#define _PAGE_RW 0x400 /* software: user write access allowed */
31
+
#define _PAGE_WRITE 0x400 /* software: user write access allowed */
32
32
#define _PAGE_SPECIAL 0x800 /* software: Special page */
33
33
34
34
#ifdef CONFIG_PTE_64BIT
···
42
42
#define _PMD_PRESENT_MASK (PAGE_MASK)
43
43
#define _PMD_BAD (~PAGE_MASK)
44
44
45
-
/* We borrow the _PAGE_USER bit to store the exclusive marker in swap PTEs. */
46
-
#define _PAGE_SWP_EXCLUSIVE _PAGE_USER
45
+
/* We borrow the _PAGE_READ bit to store the exclusive marker in swap PTEs. */
46
+
#define _PAGE_SWP_EXCLUSIVE _PAGE_READ
47
47
48
48
/* And here we include common definitions */
49
49
50
-
#define _PAGE_KERNEL_RO 0
51
-
#define _PAGE_KERNEL_ROX (_PAGE_EXEC)
52
-
#define _PAGE_KERNEL_RW (_PAGE_DIRTY | _PAGE_RW)
53
-
#define _PAGE_KERNEL_RWX (_PAGE_DIRTY | _PAGE_RW | _PAGE_EXEC)
54
-
55
50
#define _PAGE_HPTEFLAGS _PAGE_HASHPTE
56
-
57
-
#ifndef __ASSEMBLY__
58
-
59
-
static inline bool pte_user(pte_t pte)
60
-
{
61
-
return pte_val(pte) & _PAGE_USER;
62
-
}
63
-
#endif /* __ASSEMBLY__ */
64
51
65
52
/*
66
53
* Location of the PFN in the PTE. Most 32-bit platforms use the same
···
84
97
#define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED)
85
98
#define _PAGE_BASE (_PAGE_BASE_NC | _PAGE_COHERENT)
86
99
87
-
/*
88
-
* Permission masks used to generate the __P and __S table.
89
-
*
90
-
* Note:__pgprot is defined in arch/powerpc/include/asm/page.h
91
-
*
92
-
* Write permissions imply read permissions for now.
93
-
*/
94
-
#define PAGE_NONE __pgprot(_PAGE_BASE)
95
-
#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW)
96
-
#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW | _PAGE_EXEC)
97
-
#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_USER)
98
-
#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
99
-
#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_USER)
100
-
#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
100
+
#include <asm/pgtable-masks.h>
101
101
102
102
/* Permission masks used for kernel mappings */
103
103
#define PAGE_KERNEL __pgprot(_PAGE_BASE | _PAGE_KERNEL_RW)
···
144
170
* value (for now) on others, from where we can start layout kernel
145
171
* virtual space that goes below PKMAP and FIXMAP
146
172
*/
147
-
#include <asm/fixmap.h>
173
+
174
+
#define FIXADDR_SIZE 0
175
+
#ifdef CONFIG_KASAN
176
+
#include <asm/kasan.h>
177
+
#define FIXADDR_TOP (KASAN_SHADOW_START - PAGE_SIZE)
178
+
#else
179
+
#define FIXADDR_TOP ((unsigned long)(-PAGE_SIZE))
180
+
#endif
148
181
149
182
/*
150
183
* ioremap_bot starts at that address. Early ioremaps move down from there,
···
205
224
/* Bits to mask out from a PGD to get to the PUD page */
206
225
#define PGD_MASKED_BITS 0
207
226
208
-
#define pte_ERROR(e) \
209
-
pr_err("%s:%d: bad pte %llx.\n", __FILE__, __LINE__, \
210
-
(unsigned long long)pte_val(e))
211
227
#define pgd_ERROR(e) \
212
228
pr_err("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, pgd_val(e))
213
229
/*
···
321
343
static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr,
322
344
pte_t *ptep)
323
345
{
324
-
pte_update(mm, addr, ptep, _PAGE_RW, 0, 0);
346
+
pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 0);
325
347
}
326
348
327
349
static inline void __ptep_set_access_flags(struct vm_area_struct *vma,
···
380
402
}
381
403
382
404
/* Generic accessors to PTE bits */
383
-
static inline int pte_write(pte_t pte) { return !!(pte_val(pte) & _PAGE_RW);}
384
-
static inline int pte_read(pte_t pte) { return 1; }
405
+
static inline bool pte_read(pte_t pte)
406
+
{
407
+
return !!(pte_val(pte) & _PAGE_READ);
408
+
}
409
+
410
+
static inline bool pte_write(pte_t pte)
411
+
{
412
+
return !!(pte_val(pte) & _PAGE_WRITE);
413
+
}
414
+
385
415
static inline int pte_dirty(pte_t pte) { return !!(pte_val(pte) & _PAGE_DIRTY); }
386
416
static inline int pte_young(pte_t pte) { return !!(pte_val(pte) & _PAGE_ACCESSED); }
387
417
static inline int pte_special(pte_t pte) { return !!(pte_val(pte) & _PAGE_SPECIAL); }
···
424
438
static inline bool pte_access_permitted(pte_t pte, bool write)
425
439
{
426
440
/*
427
-
* A read-only access is controlled by _PAGE_USER bit.
428
-
* We have _PAGE_READ set for WRITE and EXECUTE
441
+
* A read-only access is controlled by _PAGE_READ bit.
442
+
* We have _PAGE_READ set for WRITE
429
443
*/
430
-
if (!pte_present(pte) || !pte_user(pte) || !pte_read(pte))
444
+
if (!pte_present(pte) || !pte_read(pte))
431
445
return false;
432
446
433
447
if (write && !pte_write(pte))
···
451
465
/* Generic modifiers for PTE bits */
452
466
static inline pte_t pte_wrprotect(pte_t pte)
453
467
{
454
-
return __pte(pte_val(pte) & ~_PAGE_RW);
468
+
return __pte(pte_val(pte) & ~_PAGE_WRITE);
455
469
}
456
470
457
471
static inline pte_t pte_exprotect(pte_t pte)
···
481
495
482
496
static inline pte_t pte_mkwrite_novma(pte_t pte)
483
497
{
498
+
/*
499
+
* write implies read, hence set both
500
+
*/
484
501
return __pte(pte_val(pte) | _PAGE_RW);
485
502
}
486
503
···
505
516
static inline pte_t pte_mkhuge(pte_t pte)
506
517
{
507
518
return pte;
508
-
}
509
-
510
-
static inline pte_t pte_mkprivileged(pte_t pte)
511
-
{
512
-
return __pte(pte_val(pte) & ~_PAGE_USER);
513
-
}
514
-
515
-
static inline pte_t pte_mkuser(pte_t pte)
516
-
{
517
-
return __pte(pte_val(pte) | _PAGE_USER);
518
519
}
519
520
520
521
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+1
-1
arch/powerpc/include/asm/book3s/32/tlbflush.h
+1
-1
arch/powerpc/include/asm/book3s/32/tlbflush.h
+7
-30
arch/powerpc/include/asm/book3s/64/pgtable.h
+7
-30
arch/powerpc/include/asm/book3s/64/pgtable.h
···
17
17
#define _PAGE_EXEC 0x00001 /* execute permission */
18
18
#define _PAGE_WRITE 0x00002 /* write access allowed */
19
19
#define _PAGE_READ 0x00004 /* read access allowed */
20
+
#define _PAGE_NA _PAGE_PRIVILEGED
21
+
#define _PAGE_NAX _PAGE_EXEC
22
+
#define _PAGE_RO _PAGE_READ
23
+
#define _PAGE_ROX (_PAGE_READ | _PAGE_EXEC)
20
24
#define _PAGE_RW (_PAGE_READ | _PAGE_WRITE)
21
25
#define _PAGE_RWX (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC)
22
26
#define _PAGE_PRIVILEGED 0x00008 /* kernel access only */
···
140
136
#define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED)
141
137
#define _PAGE_BASE (_PAGE_BASE_NC)
142
138
143
-
/* Permission masks used to generate the __P and __S table,
144
-
*
145
-
* Note:__pgprot is defined in arch/powerpc/include/asm/page.h
146
-
*
147
-
* Write permissions imply read permissions for now (we could make write-only
148
-
* pages on BookE but we don't bother for now). Execute permission control is
149
-
* possible on platforms that define _PAGE_EXEC
150
-
*/
151
-
#define PAGE_NONE __pgprot(_PAGE_BASE | _PAGE_PRIVILEGED)
152
-
#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_RW)
153
-
#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_RW | _PAGE_EXEC)
154
-
#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_READ)
155
-
#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_EXEC)
156
-
#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_READ)
157
-
#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_EXEC)
158
-
/* Radix only, Hash uses PAGE_READONLY_X + execute-only pkey instead */
159
-
#define PAGE_EXECONLY __pgprot(_PAGE_BASE | _PAGE_EXEC)
139
+
#include <asm/pgtable-masks.h>
160
140
161
141
/* Permission masks used for kernel mappings */
162
142
#define PAGE_KERNEL __pgprot(_PAGE_BASE | _PAGE_KERNEL_RW)
···
304
316
#define IOREMAP_START (ioremap_bot)
305
317
#define IOREMAP_END (KERN_IO_END - FIXADDR_SIZE)
306
318
#define FIXADDR_SIZE SZ_32M
319
+
#define FIXADDR_TOP (IOREMAP_END + FIXADDR_SIZE)
307
320
308
321
#ifndef __ASSEMBLY__
309
322
···
618
629
return __pte_raw(pte_raw(pte) | cpu_to_be64(_PAGE_SPECIAL | _PAGE_DEVMAP));
619
630
}
620
631
621
-
static inline pte_t pte_mkprivileged(pte_t pte)
622
-
{
623
-
return __pte_raw(pte_raw(pte) | cpu_to_be64(_PAGE_PRIVILEGED));
624
-
}
625
-
626
-
static inline pte_t pte_mkuser(pte_t pte)
627
-
{
628
-
return __pte_raw(pte_raw(pte) & cpu_to_be64(~_PAGE_PRIVILEGED));
629
-
}
630
-
631
632
/*
632
633
* This is potentially called with a pmd as the argument, in which case it's not
633
634
* safe to check _PAGE_DEVMAP unless we also confirm that _PAGE_PTE is set.
···
626
647
*/
627
648
static inline int pte_devmap(pte_t pte)
628
649
{
629
-
u64 mask = cpu_to_be64(_PAGE_DEVMAP | _PAGE_PTE);
650
+
__be64 mask = cpu_to_be64(_PAGE_DEVMAP | _PAGE_PTE);
630
651
631
652
return (pte_raw(pte) & mask) == mask;
632
653
}
···
993
1014
return (pmd_t *)__va(pud_val(pud) & ~PUD_MASKED_BITS);
994
1015
}
995
1016
996
-
#define pte_ERROR(e) \
997
-
pr_err("%s:%d: bad pte %08lx.\n", __FILE__, __LINE__, pte_val(e))
998
1017
#define pmd_ERROR(e) \
999
1018
pr_err("%s:%d: bad pmd %08lx.\n", __FILE__, __LINE__, pmd_val(e))
1000
1019
#define pud_ERROR(e) \
-33
arch/powerpc/include/asm/book3s/pgtable.h
-33
arch/powerpc/include/asm/book3s/pgtable.h
···
8
8
#include <asm/book3s/32/pgtable.h>
9
9
#endif
10
10
11
-
#ifndef __ASSEMBLY__
12
-
#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS
13
-
extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address,
14
-
pte_t *ptep, pte_t entry, int dirty);
15
-
16
-
struct file;
17
-
extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
18
-
unsigned long size, pgprot_t vma_prot);
19
-
#define __HAVE_PHYS_MEM_ACCESS_PROT
20
-
21
-
void __update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep);
22
-
23
-
/*
24
-
* This gets called at the end of handling a page fault, when
25
-
* the kernel has put a new PTE into the page table for the process.
26
-
* We use it to ensure coherency between the i-cache and d-cache
27
-
* for the page which has just been mapped in.
28
-
* On machines which use an MMU hash table, we use this to put a
29
-
* corresponding HPTE into the hash table ahead of time, instead of
30
-
* waiting for the inevitable extra hash-table miss exception.
31
-
*/
32
-
static inline void update_mmu_cache_range(struct vm_fault *vmf,
33
-
struct vm_area_struct *vma, unsigned long address,
34
-
pte_t *ptep, unsigned int nr)
35
-
{
36
-
if (IS_ENABLED(CONFIG_PPC32) && !mmu_has_feature(MMU_FTR_HPTE_TABLE))
37
-
return;
38
-
if (radix_enabled())
39
-
return;
40
-
__update_mmu_cache(vma, address, ptep);
41
-
}
42
-
43
-
#endif /* __ASSEMBLY__ */
44
11
#endif
+1
arch/powerpc/include/asm/code-patching.h
+1
arch/powerpc/include/asm/code-patching.h
···
74
74
int patch_branch(u32 *addr, unsigned long target, int flags);
75
75
int patch_instruction(u32 *addr, ppc_inst_t instr);
76
76
int raw_patch_instruction(u32 *addr, ppc_inst_t instr);
77
+
int patch_instructions(u32 *addr, u32 *code, size_t len, bool repeat_instr);
77
78
78
79
static inline unsigned long patch_site_addr(s32 *site)
79
80
{
-5
arch/powerpc/include/asm/cpm1.h
-5
arch/powerpc/include/asm/cpm1.h
···
49
49
*/
50
50
extern cpm8xx_t __iomem *cpmp; /* Pointer to comm processor */
51
51
52
-
#define cpm_dpalloc cpm_muram_alloc
53
-
#define cpm_dpfree cpm_muram_free
54
-
#define cpm_dpram_addr cpm_muram_addr
55
-
#define cpm_dpram_phys cpm_muram_dma
56
-
57
52
extern void cpm_setbrg(uint brg, uint rate);
58
53
59
54
extern void __init cpm_load_patch(cpm8xx_t *cp);
-4
arch/powerpc/include/asm/cpm2.h
-4
arch/powerpc/include/asm/cpm2.h
+4
-12
arch/powerpc/include/asm/fixmap.h
+4
-12
arch/powerpc/include/asm/fixmap.h
···
23
23
#include <asm/kmap_size.h>
24
24
#endif
25
25
26
-
#ifdef CONFIG_PPC64
27
-
#define FIXADDR_TOP (IOREMAP_END + FIXADDR_SIZE)
28
-
#else
29
-
#define FIXADDR_SIZE 0
30
-
#ifdef CONFIG_KASAN
31
-
#include <asm/kasan.h>
32
-
#define FIXADDR_TOP (KASAN_SHADOW_START - PAGE_SIZE)
33
-
#else
34
-
#define FIXADDR_TOP ((unsigned long)(-PAGE_SIZE))
35
-
#endif
36
-
#endif
37
-
38
26
/*
39
27
* Here we define all the compile-time 'special' virtual
40
28
* addresses. The point is to have a constant address at
···
106
118
}
107
119
108
120
#define __early_set_fixmap __set_fixmap
121
+
122
+
#ifdef CONFIG_PPC_8xx
123
+
#define VIRT_IMMR_BASE (__fix_to_virt(FIX_IMMR_BASE))
124
+
#endif
109
125
110
126
#endif /* !__ASSEMBLY__ */
111
127
#endif
+995
arch/powerpc/include/asm/guest-state-buffer.h
+995
arch/powerpc/include/asm/guest-state-buffer.h
···
1
+
/* SPDX-License-Identifier: GPL-2.0 */
2
+
/*
3
+
* Interface based on include/net/netlink.h
4
+
*/
5
+
#ifndef _ASM_POWERPC_GUEST_STATE_BUFFER_H
6
+
#define _ASM_POWERPC_GUEST_STATE_BUFFER_H
7
+
8
+
#include "asm/hvcall.h"
9
+
#include <linux/gfp.h>
10
+
#include <linux/bitmap.h>
11
+
#include <asm/plpar_wrappers.h>
12
+
13
+
/**************************************************************************
14
+
* Guest State Buffer Constants
15
+
**************************************************************************/
16
+
/* Element without a value and any length */
17
+
#define KVMPPC_GSID_BLANK 0x0000
18
+
/* Size required for the L0's internal VCPU representation */
19
+
#define KVMPPC_GSID_HOST_STATE_SIZE 0x0001
20
+
/* Minimum size for the H_GUEST_RUN_VCPU output buffer */
21
+
#define KVMPPC_GSID_RUN_OUTPUT_MIN_SIZE 0x0002
22
+
/* "Logical" PVR value as defined in the PAPR */
23
+
#define KVMPPC_GSID_LOGICAL_PVR 0x0003
24
+
/* L0 relative timebase offset */
25
+
#define KVMPPC_GSID_TB_OFFSET 0x0004
26
+
/* Partition Scoped Page Table Info */
27
+
#define KVMPPC_GSID_PARTITION_TABLE 0x0005
28
+
/* Process Table Info */
29
+
#define KVMPPC_GSID_PROCESS_TABLE 0x0006
30
+
31
+
/* H_GUEST_RUN_VCPU input buffer Info */
32
+
#define KVMPPC_GSID_RUN_INPUT 0x0C00
33
+
/* H_GUEST_RUN_VCPU output buffer Info */
34
+
#define KVMPPC_GSID_RUN_OUTPUT 0x0C01
35
+
#define KVMPPC_GSID_VPA 0x0C02
36
+
37
+
#define KVMPPC_GSID_GPR(x) (0x1000 + (x))
38
+
#define KVMPPC_GSID_HDEC_EXPIRY_TB 0x1020
39
+
#define KVMPPC_GSID_NIA 0x1021
40
+
#define KVMPPC_GSID_MSR 0x1022
41
+
#define KVMPPC_GSID_LR 0x1023
42
+
#define KVMPPC_GSID_XER 0x1024
43
+
#define KVMPPC_GSID_CTR 0x1025
44
+
#define KVMPPC_GSID_CFAR 0x1026
45
+
#define KVMPPC_GSID_SRR0 0x1027
46
+
#define KVMPPC_GSID_SRR1 0x1028
47
+
#define KVMPPC_GSID_DAR 0x1029
48
+
#define KVMPPC_GSID_DEC_EXPIRY_TB 0x102A
49
+
#define KVMPPC_GSID_VTB 0x102B
50
+
#define KVMPPC_GSID_LPCR 0x102C
51
+
#define KVMPPC_GSID_HFSCR 0x102D
52
+
#define KVMPPC_GSID_FSCR 0x102E
53
+
#define KVMPPC_GSID_FPSCR 0x102F
54
+
#define KVMPPC_GSID_DAWR0 0x1030
55
+
#define KVMPPC_GSID_DAWR1 0x1031
56
+
#define KVMPPC_GSID_CIABR 0x1032
57
+
#define KVMPPC_GSID_PURR 0x1033
58
+
#define KVMPPC_GSID_SPURR 0x1034
59
+
#define KVMPPC_GSID_IC 0x1035
60
+
#define KVMPPC_GSID_SPRG0 0x1036
61
+
#define KVMPPC_GSID_SPRG1 0x1037
62
+
#define KVMPPC_GSID_SPRG2 0x1038
63
+
#define KVMPPC_GSID_SPRG3 0x1039
64
+
#define KVMPPC_GSID_PPR 0x103A
65
+
#define KVMPPC_GSID_MMCR(x) (0x103B + (x))
66
+
#define KVMPPC_GSID_MMCRA 0x103F
67
+
#define KVMPPC_GSID_SIER(x) (0x1040 + (x))
68
+
#define KVMPPC_GSID_BESCR 0x1043
69
+
#define KVMPPC_GSID_EBBHR 0x1044
70
+
#define KVMPPC_GSID_EBBRR 0x1045
71
+
#define KVMPPC_GSID_AMR 0x1046
72
+
#define KVMPPC_GSID_IAMR 0x1047
73
+
#define KVMPPC_GSID_AMOR 0x1048
74
+
#define KVMPPC_GSID_UAMOR 0x1049
75
+
#define KVMPPC_GSID_SDAR 0x104A
76
+
#define KVMPPC_GSID_SIAR 0x104B
77
+
#define KVMPPC_GSID_DSCR 0x104C
78
+
#define KVMPPC_GSID_TAR 0x104D
79
+
#define KVMPPC_GSID_DEXCR 0x104E
80
+
#define KVMPPC_GSID_HDEXCR 0x104F
81
+
#define KVMPPC_GSID_HASHKEYR 0x1050
82
+
#define KVMPPC_GSID_HASHPKEYR 0x1051
83
+
#define KVMPPC_GSID_CTRL 0x1052
84
+
85
+
#define KVMPPC_GSID_CR 0x2000
86
+
#define KVMPPC_GSID_PIDR 0x2001
87
+
#define KVMPPC_GSID_DSISR 0x2002
88
+
#define KVMPPC_GSID_VSCR 0x2003
89
+
#define KVMPPC_GSID_VRSAVE 0x2004
90
+
#define KVMPPC_GSID_DAWRX0 0x2005
91
+
#define KVMPPC_GSID_DAWRX1 0x2006
92
+
#define KVMPPC_GSID_PMC(x) (0x2007 + (x))
93
+
#define KVMPPC_GSID_WORT 0x200D
94
+
#define KVMPPC_GSID_PSPB 0x200E
95
+
96
+
#define KVMPPC_GSID_VSRS(x) (0x3000 + (x))
97
+
98
+
#define KVMPPC_GSID_HDAR 0xF000
99
+
#define KVMPPC_GSID_HDSISR 0xF001
100
+
#define KVMPPC_GSID_HEIR 0xF002
101
+
#define KVMPPC_GSID_ASDR 0xF003
102
+
103
+
#define KVMPPC_GSE_GUESTWIDE_START KVMPPC_GSID_BLANK
104
+
#define KVMPPC_GSE_GUESTWIDE_END KVMPPC_GSID_PROCESS_TABLE
105
+
#define KVMPPC_GSE_GUESTWIDE_COUNT \
106
+
(KVMPPC_GSE_GUESTWIDE_END - KVMPPC_GSE_GUESTWIDE_START + 1)
107
+
108
+
#define KVMPPC_GSE_META_START KVMPPC_GSID_RUN_INPUT
109
+
#define KVMPPC_GSE_META_END KVMPPC_GSID_VPA
110
+
#define KVMPPC_GSE_META_COUNT (KVMPPC_GSE_META_END - KVMPPC_GSE_META_START + 1)
111
+
112
+
#define KVMPPC_GSE_DW_REGS_START KVMPPC_GSID_GPR(0)
113
+
#define KVMPPC_GSE_DW_REGS_END KVMPPC_GSID_CTRL
114
+
#define KVMPPC_GSE_DW_REGS_COUNT \
115
+
(KVMPPC_GSE_DW_REGS_END - KVMPPC_GSE_DW_REGS_START + 1)
116
+
117
+
#define KVMPPC_GSE_W_REGS_START KVMPPC_GSID_CR
118
+
#define KVMPPC_GSE_W_REGS_END KVMPPC_GSID_PSPB
119
+
#define KVMPPC_GSE_W_REGS_COUNT \
120
+
(KVMPPC_GSE_W_REGS_END - KVMPPC_GSE_W_REGS_START + 1)
121
+
122
+
#define KVMPPC_GSE_VSRS_START KVMPPC_GSID_VSRS(0)
123
+
#define KVMPPC_GSE_VSRS_END KVMPPC_GSID_VSRS(63)
124
+
#define KVMPPC_GSE_VSRS_COUNT (KVMPPC_GSE_VSRS_END - KVMPPC_GSE_VSRS_START + 1)
125
+
126
+
#define KVMPPC_GSE_INTR_REGS_START KVMPPC_GSID_HDAR
127
+
#define KVMPPC_GSE_INTR_REGS_END KVMPPC_GSID_ASDR
128
+
#define KVMPPC_GSE_INTR_REGS_COUNT \
129
+
(KVMPPC_GSE_INTR_REGS_END - KVMPPC_GSE_INTR_REGS_START + 1)
130
+
131
+
#define KVMPPC_GSE_IDEN_COUNT \
132
+
(KVMPPC_GSE_GUESTWIDE_COUNT + KVMPPC_GSE_META_COUNT + \
133
+
KVMPPC_GSE_DW_REGS_COUNT + KVMPPC_GSE_W_REGS_COUNT + \
134
+
KVMPPC_GSE_VSRS_COUNT + KVMPPC_GSE_INTR_REGS_COUNT)
135
+
136
+
/**
137
+
* Ranges of guest state buffer elements
138
+
*/
139
+
enum {
140
+
KVMPPC_GS_CLASS_GUESTWIDE = 0x01,
141
+
KVMPPC_GS_CLASS_META = 0x02,
142
+
KVMPPC_GS_CLASS_DWORD_REG = 0x04,
143
+
KVMPPC_GS_CLASS_WORD_REG = 0x08,
144
+
KVMPPC_GS_CLASS_VECTOR = 0x10,
145
+
KVMPPC_GS_CLASS_INTR = 0x20,
146
+
};
147
+
148
+
/**
149
+
* Types of guest state buffer elements
150
+
*/
151
+
enum {
152
+
KVMPPC_GSE_BE32,
153
+
KVMPPC_GSE_BE64,
154
+
KVMPPC_GSE_VEC128,
155
+
KVMPPC_GSE_PARTITION_TABLE,
156
+
KVMPPC_GSE_PROCESS_TABLE,
157
+
KVMPPC_GSE_BUFFER,
158
+
__KVMPPC_GSE_TYPE_MAX,
159
+
};
160
+
161
+
/**
162
+
* Flags for guest state elements
163
+
*/
164
+
enum {
165
+
KVMPPC_GS_FLAGS_WIDE = 0x01,
166
+
};
167
+
168
+
/**
169
+
* struct kvmppc_gs_part_table - deserialized partition table information
170
+
* element
171
+
* @address: start of the partition table
172
+
* @ea_bits: number of bits in the effective address
173
+
* @gpd_size: root page directory size
174
+
*/
175
+
struct kvmppc_gs_part_table {
176
+
u64 address;
177
+
u64 ea_bits;
178
+
u64 gpd_size;
179
+
};
180
+
181
+
/**
182
+
* struct kvmppc_gs_proc_table - deserialized process table information element
183
+
* @address: start of the process table
184
+
* @gpd_size: process table size
185
+
*/
186
+
struct kvmppc_gs_proc_table {
187
+
u64 address;
188
+
u64 gpd_size;
189
+
};
190
+
191
+
/**
192
+
* struct kvmppc_gs_buff_info - deserialized meta guest state buffer information
193
+
* @address: start of the guest state buffer
194
+
* @size: size of the guest state buffer
195
+
*/
196
+
struct kvmppc_gs_buff_info {
197
+
u64 address;
198
+
u64 size;
199
+
};
200
+
201
+
/**
202
+
* struct kvmppc_gs_header - serialized guest state buffer header
203
+
* @nelem: count of guest state elements in the buffer
204
+
* @data: start of the stream of elements in the buffer
205
+
*/
206
+
struct kvmppc_gs_header {
207
+
__be32 nelems;
208
+
char data[];
209
+
} __packed;
210
+
211
+
/**
212
+
* struct kvmppc_gs_elem - serialized guest state buffer element
213
+
* @iden: Guest State ID
214
+
* @len: length of data
215
+
* @data: the guest state buffer element's value
216
+
*/
217
+
struct kvmppc_gs_elem {
218
+
__be16 iden;
219
+
__be16 len;
220
+
char data[];
221
+
} __packed;
222
+
223
+
/**
224
+
* struct kvmppc_gs_buff - a guest state buffer with metadata.
225
+
* @capacity: total length of the buffer
226
+
* @len: current length of the elements and header
227
+
* @guest_id: guest id associated with the buffer
228
+
* @vcpu_id: vcpu_id associated with the buffer
229
+
* @hdr: the serialised guest state buffer
230
+
*/
231
+
struct kvmppc_gs_buff {
232
+
size_t capacity;
233
+
size_t len;
234
+
unsigned long guest_id;
235
+
unsigned long vcpu_id;
236
+
struct kvmppc_gs_header *hdr;
237
+
};
238
+
239
+
/**
240
+
* struct kvmppc_gs_bitmap - a bitmap for element ids
241
+
* @bitmap: a bitmap large enough for all Guest State IDs
242
+
*/
243
+
struct kvmppc_gs_bitmap {
244
+
/* private: */
245
+
DECLARE_BITMAP(bitmap, KVMPPC_GSE_IDEN_COUNT);
246
+
};
247
+
248
+
/**
249
+
* struct kvmppc_gs_parser - a map of element ids to locations in a buffer
250
+
* @iterator: bitmap used for iterating
251
+
* @gses: contains the pointers to elements
252
+
*
253
+
* A guest state parser is used for deserialising a guest state buffer.
254
+
* Given a buffer, it then allows looking up guest state elements using
255
+
* a guest state id.
256
+
*/
257
+
struct kvmppc_gs_parser {
258
+
/* private: */
259
+
struct kvmppc_gs_bitmap iterator;
260
+
struct kvmppc_gs_elem *gses[KVMPPC_GSE_IDEN_COUNT];
261
+
};
262
+
263
+
enum {
264
+
GSM_GUEST_WIDE = 0x1,
265
+
GSM_SEND = 0x2,
266
+
GSM_RECEIVE = 0x4,
267
+
GSM_GSB_OWNER = 0x8,
268
+
};
269
+
270
+
struct kvmppc_gs_msg;
271
+
272
+
/**
273
+
* struct kvmppc_gs_msg_ops - guest state message behavior
274
+
* @get_size: maximum size required for the message data
275
+
* @fill_info: serializes to the guest state buffer format
276
+
* @refresh_info: dserializes from the guest state buffer format
277
+
*/
278
+
struct kvmppc_gs_msg_ops {
279
+
size_t (*get_size)(struct kvmppc_gs_msg *gsm);
280
+
int (*fill_info)(struct kvmppc_gs_buff *gsb, struct kvmppc_gs_msg *gsm);
281
+
int (*refresh_info)(struct kvmppc_gs_msg *gsm,
282
+
struct kvmppc_gs_buff *gsb);
283
+
};
284
+
285
+
/**
286
+
* struct kvmppc_gs_msg - a guest state message
287
+
* @bitmap: the guest state ids that should be included
288
+
* @ops: modify message behavior for reading and writing to buffers
289
+
* @flags: guest wide or thread wide
290
+
* @data: location where buffer data will be written to or from.
291
+
*
292
+
* A guest state message is allows flexibility in sending in receiving data
293
+
* in a guest state buffer format.
294
+
*/
295
+
struct kvmppc_gs_msg {
296
+
struct kvmppc_gs_bitmap bitmap;
297
+
struct kvmppc_gs_msg_ops *ops;
298
+
unsigned long flags;
299
+
void *data;
300
+
};
301
+
302
+
/**************************************************************************
303
+
* Guest State IDs
304
+
**************************************************************************/
305
+
306
+
u16 kvmppc_gsid_size(u16 iden);
307
+
unsigned long kvmppc_gsid_flags(u16 iden);
308
+
u64 kvmppc_gsid_mask(u16 iden);
309
+
310
+
/**************************************************************************
311
+
* Guest State Buffers
312
+
**************************************************************************/
313
+
struct kvmppc_gs_buff *kvmppc_gsb_new(size_t size, unsigned long guest_id,
314
+
unsigned long vcpu_id, gfp_t flags);
315
+
void kvmppc_gsb_free(struct kvmppc_gs_buff *gsb);
316
+
void *kvmppc_gsb_put(struct kvmppc_gs_buff *gsb, size_t size);
317
+
int kvmppc_gsb_send(struct kvmppc_gs_buff *gsb, unsigned long flags);
318
+
int kvmppc_gsb_recv(struct kvmppc_gs_buff *gsb, unsigned long flags);
319
+
320
+
/**
321
+
* kvmppc_gsb_header() - the header of a guest state buffer
322
+
* @gsb: guest state buffer
323
+
*
324
+
* Returns a pointer to the buffer header.
325
+
*/
326
+
static inline struct kvmppc_gs_header *
327
+
kvmppc_gsb_header(struct kvmppc_gs_buff *gsb)
328
+
{
329
+
return gsb->hdr;
330
+
}
331
+
332
+
/**
333
+
* kvmppc_gsb_data() - the elements of a guest state buffer
334
+
* @gsb: guest state buffer
335
+
*
336
+
* Returns a pointer to the first element of the buffer data.
337
+
*/
338
+
static inline struct kvmppc_gs_elem *kvmppc_gsb_data(struct kvmppc_gs_buff *gsb)
339
+
{
340
+
return (struct kvmppc_gs_elem *)kvmppc_gsb_header(gsb)->data;
341
+
}
342
+
343
+
/**
344
+
* kvmppc_gsb_len() - the current length of a guest state buffer
345
+
* @gsb: guest state buffer
346
+
*
347
+
* Returns the length including the header of a buffer.
348
+
*/
349
+
static inline size_t kvmppc_gsb_len(struct kvmppc_gs_buff *gsb)
350
+
{
351
+
return gsb->len;
352
+
}
353
+
354
+
/**
355
+
* kvmppc_gsb_capacity() - the capacity of a guest state buffer
356
+
* @gsb: guest state buffer
357
+
*
358
+
* Returns the capacity of a buffer.
359
+
*/
360
+
static inline size_t kvmppc_gsb_capacity(struct kvmppc_gs_buff *gsb)
361
+
{
362
+
return gsb->capacity;
363
+
}
364
+
365
+
/**
366
+
* kvmppc_gsb_paddress() - the physical address of buffer
367
+
* @gsb: guest state buffer
368
+
*
369
+
* Returns the physical address of the buffer.
370
+
*/
371
+
static inline u64 kvmppc_gsb_paddress(struct kvmppc_gs_buff *gsb)
372
+
{
373
+
return __pa(kvmppc_gsb_header(gsb));
374
+
}
375
+
376
+
/**
377
+
* kvmppc_gsb_nelems() - the number of elements in a buffer
378
+
* @gsb: guest state buffer
379
+
*
380
+
* Returns the number of elements in a buffer
381
+
*/
382
+
static inline u32 kvmppc_gsb_nelems(struct kvmppc_gs_buff *gsb)
383
+
{
384
+
return be32_to_cpu(kvmppc_gsb_header(gsb)->nelems);
385
+
}
386
+
387
+
/**
388
+
* kvmppc_gsb_reset() - empty a guest state buffer
389
+
* @gsb: guest state buffer
390
+
*
391
+
* Reset the number of elements and length of buffer to empty.
392
+
*/
393
+
static inline void kvmppc_gsb_reset(struct kvmppc_gs_buff *gsb)
394
+
{
395
+
kvmppc_gsb_header(gsb)->nelems = cpu_to_be32(0);
396
+
gsb->len = sizeof(struct kvmppc_gs_header);
397
+
}
398
+
399
+
/**
400
+
* kvmppc_gsb_data_len() - the length of a buffer excluding the header
401
+
* @gsb: guest state buffer
402
+
*
403
+
* Returns the length of a buffer excluding the header
404
+
*/
405
+
static inline size_t kvmppc_gsb_data_len(struct kvmppc_gs_buff *gsb)
406
+
{
407
+
return gsb->len - sizeof(struct kvmppc_gs_header);
408
+
}
409
+
410
+
/**
411
+
* kvmppc_gsb_data_cap() - the capacity of a buffer excluding the header
412
+
* @gsb: guest state buffer
413
+
*
414
+
* Returns the capacity of a buffer excluding the header
415
+
*/
416
+
static inline size_t kvmppc_gsb_data_cap(struct kvmppc_gs_buff *gsb)
417
+
{
418
+
return gsb->capacity - sizeof(struct kvmppc_gs_header);
419
+
}
420
+
421
+
/**
422
+
* kvmppc_gsb_for_each_elem - iterate over the elements in a buffer
423
+
* @i: loop counter
424
+
* @pos: set to current element
425
+
* @gsb: guest state buffer
426
+
* @rem: initialized to buffer capacity, holds bytes currently remaining in
427
+
* stream
428
+
*/
429
+
#define kvmppc_gsb_for_each_elem(i, pos, gsb, rem) \
430
+
kvmppc_gse_for_each_elem(i, kvmppc_gsb_nelems(gsb), pos, \
431
+
kvmppc_gsb_data(gsb), \
432
+
kvmppc_gsb_data_cap(gsb), rem)
433
+
434
+
/**************************************************************************
435
+
* Guest State Elements
436
+
**************************************************************************/
437
+
438
+
/**
439
+
* kvmppc_gse_iden() - guest state ID of element
440
+
* @gse: guest state element
441
+
*
442
+
* Return the guest state ID in host endianness.
443
+
*/
444
+
static inline u16 kvmppc_gse_iden(const struct kvmppc_gs_elem *gse)
445
+
{
446
+
return be16_to_cpu(gse->iden);
447
+
}
448
+
449
+
/**
450
+
* kvmppc_gse_len() - length of guest state element data
451
+
* @gse: guest state element
452
+
*
453
+
* Returns the length of guest state element data
454
+
*/
455
+
static inline u16 kvmppc_gse_len(const struct kvmppc_gs_elem *gse)
456
+
{
457
+
return be16_to_cpu(gse->len);
458
+
}
459
+
460
+
/**
461
+
* kvmppc_gse_total_len() - total length of guest state element
462
+
* @gse: guest state element
463
+
*
464
+
* Returns the length of the data plus the ID and size header.
465
+
*/
466
+
static inline u16 kvmppc_gse_total_len(const struct kvmppc_gs_elem *gse)
467
+
{
468
+
return be16_to_cpu(gse->len) + sizeof(*gse);
469
+
}
470
+
471
+
/**
472
+
* kvmppc_gse_total_size() - space needed for a given data length
473
+
* @size: data length
474
+
*
475
+
* Returns size plus the space needed for the ID and size header.
476
+
*/
477
+
static inline u16 kvmppc_gse_total_size(u16 size)
478
+
{
479
+
return sizeof(struct kvmppc_gs_elem) + size;
480
+
}
481
+
482
+
/**
483
+
* kvmppc_gse_data() - pointer to data of a guest state element
484
+
* @gse: guest state element
485
+
*
486
+
* Returns a pointer to the beginning of guest state element data.
487
+
*/
488
+
static inline void *kvmppc_gse_data(const struct kvmppc_gs_elem *gse)
489
+
{
490
+
return (void *)gse->data;
491
+
}
492
+
493
+
/**
494
+
* kvmppc_gse_ok() - checks space exists for guest state element
495
+
* @gse: guest state element
496
+
* @remaining: bytes of space remaining
497
+
*
498
+
* Returns true if the guest state element can fit in remaining space.
499
+
*/
500
+
static inline bool kvmppc_gse_ok(const struct kvmppc_gs_elem *gse,
501
+
int remaining)
502
+
{
503
+
return remaining >= kvmppc_gse_total_len(gse);
504
+
}
505
+
506
+
/**
507
+
* kvmppc_gse_next() - iterate to the next guest state element in a stream
508
+
* @gse: stream of guest state elements
509
+
* @remaining: length of the guest element stream
510
+
*
511
+
* Returns the next guest state element in a stream of elements. The length of
512
+
* the stream is updated in remaining.
513
+
*/
514
+
static inline struct kvmppc_gs_elem *
515
+
kvmppc_gse_next(const struct kvmppc_gs_elem *gse, int *remaining)
516
+
{
517
+
int len = sizeof(*gse) + kvmppc_gse_len(gse);
518
+
519
+
*remaining -= len;
520
+
return (struct kvmppc_gs_elem *)(gse->data + kvmppc_gse_len(gse));
521
+
}
522
+
523
+
/**
524
+
* kvmppc_gse_for_each_elem - iterate over a stream of guest state elements
525
+
* @i: loop counter
526
+
* @max: number of elements
527
+
* @pos: set to current element
528
+
* @head: head of elements
529
+
* @len: length of the stream
530
+
* @rem: initialized to len, holds bytes currently remaining elements
531
+
*/
532
+
#define kvmppc_gse_for_each_elem(i, max, pos, head, len, rem) \
533
+
for (i = 0, pos = head, rem = len; kvmppc_gse_ok(pos, rem) && i < max; \
534
+
pos = kvmppc_gse_next(pos, &(rem)), i++)
535
+
536
+
int __kvmppc_gse_put(struct kvmppc_gs_buff *gsb, u16 iden, u16 size,
537
+
const void *data);
538
+
int kvmppc_gse_parse(struct kvmppc_gs_parser *gsp, struct kvmppc_gs_buff *gsb);
539
+
540
+
/**
541
+
* kvmppc_gse_put_be32() - add a be32 guest state element to a buffer
542
+
* @gsb: guest state buffer to add element to
543
+
* @iden: guest state ID
544
+
* @val: big endian value
545
+
*/
546
+
static inline int kvmppc_gse_put_be32(struct kvmppc_gs_buff *gsb, u16 iden,
547
+
__be32 val)
548
+
{
549
+
__be32 tmp;
550
+
551
+
tmp = val;
552
+
return __kvmppc_gse_put(gsb, iden, sizeof(__be32), &tmp);
553
+
}
554
+
555
+
/**
556
+
* kvmppc_gse_put_u32() - add a host endian 32bit int guest state element to a
557
+
* buffer
558
+
* @gsb: guest state buffer to add element to
559
+
* @iden: guest state ID
560
+
* @val: host endian value
561
+
*/
562
+
static inline int kvmppc_gse_put_u32(struct kvmppc_gs_buff *gsb, u16 iden,
563
+
u32 val)
564
+
{
565
+
__be32 tmp;
566
+
567
+
val &= kvmppc_gsid_mask(iden);
568
+
tmp = cpu_to_be32(val);
569
+
return kvmppc_gse_put_be32(gsb, iden, tmp);
570
+
}
571
+
572
+
/**
573
+
* kvmppc_gse_put_be64() - add a be64 guest state element to a buffer
574
+
* @gsb: guest state buffer to add element to
575
+
* @iden: guest state ID
576
+
* @val: big endian value
577
+
*/
578
+
static inline int kvmppc_gse_put_be64(struct kvmppc_gs_buff *gsb, u16 iden,
579
+
__be64 val)
580
+
{
581
+
__be64 tmp;
582
+
583
+
tmp = val;
584
+
return __kvmppc_gse_put(gsb, iden, sizeof(__be64), &tmp);
585
+
}
586
+
587
+
/**
588
+
* kvmppc_gse_put_u64() - add a host endian 64bit guest state element to a
589
+
* buffer
590
+
* @gsb: guest state buffer to add element to
591
+
* @iden: guest state ID
592
+
* @val: host endian value
593
+
*/
594
+
static inline int kvmppc_gse_put_u64(struct kvmppc_gs_buff *gsb, u16 iden,
595
+
u64 val)
596
+
{
597
+
__be64 tmp;
598
+
599
+
val &= kvmppc_gsid_mask(iden);
600
+
tmp = cpu_to_be64(val);
601
+
return kvmppc_gse_put_be64(gsb, iden, tmp);
602
+
}
603
+
604
+
/**
605
+
* __kvmppc_gse_put_reg() - add a register type guest state element to a buffer
606
+
* @gsb: guest state buffer to add element to
607
+
* @iden: guest state ID
608
+
* @val: host endian value
609
+
*
610
+
* Adds a register type guest state element. Uses the guest state ID for
611
+
* determining the length of the guest element. If the guest state ID has
612
+
* bits that can not be set they will be cleared.
613
+
*/
614
+
static inline int __kvmppc_gse_put_reg(struct kvmppc_gs_buff *gsb, u16 iden,
615
+
u64 val)
616
+
{
617
+
val &= kvmppc_gsid_mask(iden);
618
+
if (kvmppc_gsid_size(iden) == sizeof(u64))
619
+
return kvmppc_gse_put_u64(gsb, iden, val);
620
+
621
+
if (kvmppc_gsid_size(iden) == sizeof(u32)) {
622
+
u32 tmp;
623
+
624
+
tmp = (u32)val;
625
+
if (tmp != val)
626
+
return -EINVAL;
627
+
628
+
return kvmppc_gse_put_u32(gsb, iden, tmp);
629
+
}
630
+
return -EINVAL;
631
+
}
632
+
633
+
/**
634
+
* kvmppc_gse_put_vector128() - add a vector guest state element to a buffer
635
+
* @gsb: guest state buffer to add element to
636
+
* @iden: guest state ID
637
+
* @val: 16 byte vector value
638
+
*/
639
+
static inline int kvmppc_gse_put_vector128(struct kvmppc_gs_buff *gsb, u16 iden,
640
+
vector128 *val)
641
+
{
642
+
__be64 tmp[2] = { 0 };
643
+
union {
644
+
__vector128 v;
645
+
u64 dw[2];
646
+
} u;
647
+
648
+
u.v = *val;
649
+
tmp[0] = cpu_to_be64(u.dw[TS_FPROFFSET]);
650
+
#ifdef CONFIG_VSX
651
+
tmp[1] = cpu_to_be64(u.dw[TS_VSRLOWOFFSET]);
652
+
#endif
653
+
return __kvmppc_gse_put(gsb, iden, sizeof(tmp), &tmp);
654
+
}
655
+
656
+
/**
657
+
* kvmppc_gse_put_part_table() - add a partition table guest state element to a
658
+
* buffer
659
+
* @gsb: guest state buffer to add element to
660
+
* @iden: guest state ID
661
+
* @val: partition table value
662
+
*/
663
+
static inline int kvmppc_gse_put_part_table(struct kvmppc_gs_buff *gsb,
664
+
u16 iden,
665
+
struct kvmppc_gs_part_table val)
666
+
{
667
+
__be64 tmp[3];
668
+
669
+
tmp[0] = cpu_to_be64(val.address);
670
+
tmp[1] = cpu_to_be64(val.ea_bits);
671
+
tmp[2] = cpu_to_be64(val.gpd_size);
672
+
return __kvmppc_gse_put(gsb, KVMPPC_GSID_PARTITION_TABLE, sizeof(tmp),
673
+
&tmp);
674
+
}
675
+
676
+
/**
677
+
* kvmppc_gse_put_proc_table() - add a process table guest state element to a
678
+
* buffer
679
+
* @gsb: guest state buffer to add element to
680
+
* @iden: guest state ID
681
+
* @val: process table value
682
+
*/
683
+
static inline int kvmppc_gse_put_proc_table(struct kvmppc_gs_buff *gsb,
684
+
u16 iden,
685
+
struct kvmppc_gs_proc_table val)
686
+
{
687
+
__be64 tmp[2];
688
+
689
+
tmp[0] = cpu_to_be64(val.address);
690
+
tmp[1] = cpu_to_be64(val.gpd_size);
691
+
return __kvmppc_gse_put(gsb, KVMPPC_GSID_PROCESS_TABLE, sizeof(tmp),
692
+
&tmp);
693
+
}
694
+
695
+
/**
696
+
* kvmppc_gse_put_buff_info() - adds a GSB description guest state element to a
697
+
* buffer
698
+
* @gsb: guest state buffer to add element to
699
+
* @iden: guest state ID
700
+
* @val: guest state buffer description value
701
+
*/
702
+
static inline int kvmppc_gse_put_buff_info(struct kvmppc_gs_buff *gsb, u16 iden,
703
+
struct kvmppc_gs_buff_info val)
704
+
{
705
+
__be64 tmp[2];
706
+
707
+
tmp[0] = cpu_to_be64(val.address);
708
+
tmp[1] = cpu_to_be64(val.size);
709
+
return __kvmppc_gse_put(gsb, iden, sizeof(tmp), &tmp);
710
+
}
711
+
712
+
int __kvmppc_gse_put(struct kvmppc_gs_buff *gsb, u16 iden, u16 size,
713
+
const void *data);
714
+
715
+
/**
716
+
* kvmppc_gse_get_be32() - return the data of a be32 element
717
+
* @gse: guest state element
718
+
*/
719
+
static inline __be32 kvmppc_gse_get_be32(const struct kvmppc_gs_elem *gse)
720
+
{
721
+
if (WARN_ON(kvmppc_gse_len(gse) != sizeof(__be32)))
722
+
return 0;
723
+
return *(__be32 *)kvmppc_gse_data(gse);
724
+
}
725
+
726
+
/**
727
+
* kvmppc_gse_get_u32() - return the data of a be32 element in host endianness
728
+
* @gse: guest state element
729
+
*/
730
+
static inline u32 kvmppc_gse_get_u32(const struct kvmppc_gs_elem *gse)
731
+
{
732
+
return be32_to_cpu(kvmppc_gse_get_be32(gse));
733
+
}
734
+
735
+
/**
736
+
* kvmppc_gse_get_be64() - return the data of a be64 element
737
+
* @gse: guest state element
738
+
*/
739
+
static inline __be64 kvmppc_gse_get_be64(const struct kvmppc_gs_elem *gse)
740
+
{
741
+
if (WARN_ON(kvmppc_gse_len(gse) != sizeof(__be64)))
742
+
return 0;
743
+
return *(__be64 *)kvmppc_gse_data(gse);
744
+
}
745
+
746
+
/**
747
+
* kvmppc_gse_get_u64() - return the data of a be64 element in host endianness
748
+
* @gse: guest state element
749
+
*/
750
+
static inline u64 kvmppc_gse_get_u64(const struct kvmppc_gs_elem *gse)
751
+
{
752
+
return be64_to_cpu(kvmppc_gse_get_be64(gse));
753
+
}
754
+
755
+
/**
756
+
* kvmppc_gse_get_vector128() - return the data of a vector element
757
+
* @gse: guest state element
758
+
*/
759
+
static inline void kvmppc_gse_get_vector128(const struct kvmppc_gs_elem *gse,
760
+
vector128 *v)
761
+
{
762
+
union {
763
+
__vector128 v;
764
+
u64 dw[2];
765
+
} u = { 0 };
766
+
__be64 *src;
767
+
768
+
if (WARN_ON(kvmppc_gse_len(gse) != sizeof(__vector128)))
769
+
*v = u.v;
770
+
771
+
src = (__be64 *)kvmppc_gse_data(gse);
772
+
u.dw[TS_FPROFFSET] = be64_to_cpu(src[0]);
773
+
#ifdef CONFIG_VSX
774
+
u.dw[TS_VSRLOWOFFSET] = be64_to_cpu(src[1]);
775
+
#endif
776
+
*v = u.v;
777
+
}
778
+
779
+
/**************************************************************************
780
+
* Guest State Bitmap
781
+
**************************************************************************/
782
+
783
+
bool kvmppc_gsbm_test(struct kvmppc_gs_bitmap *gsbm, u16 iden);
784
+
void kvmppc_gsbm_set(struct kvmppc_gs_bitmap *gsbm, u16 iden);
785
+
void kvmppc_gsbm_clear(struct kvmppc_gs_bitmap *gsbm, u16 iden);
786
+
u16 kvmppc_gsbm_next(struct kvmppc_gs_bitmap *gsbm, u16 prev);
787
+
788
+
/**
789
+
* kvmppc_gsbm_zero - zero the entire bitmap
790
+
* @gsbm: guest state buffer bitmap
791
+
*/
792
+
static inline void kvmppc_gsbm_zero(struct kvmppc_gs_bitmap *gsbm)
793
+
{
794
+
bitmap_zero(gsbm->bitmap, KVMPPC_GSE_IDEN_COUNT);
795
+
}
796
+
797
+
/**
798
+
* kvmppc_gsbm_fill - fill the entire bitmap
799
+
* @gsbm: guest state buffer bitmap
800
+
*/
801
+
static inline void kvmppc_gsbm_fill(struct kvmppc_gs_bitmap *gsbm)
802
+
{
803
+
bitmap_fill(gsbm->bitmap, KVMPPC_GSE_IDEN_COUNT);
804
+
clear_bit(0, gsbm->bitmap);
805
+
}
806
+
807
+
/**
808
+
* kvmppc_gsbm_for_each - iterate the present guest state IDs
809
+
* @gsbm: guest state buffer bitmap
810
+
* @iden: current guest state ID
811
+
*/
812
+
#define kvmppc_gsbm_for_each(gsbm, iden) \
813
+
for (iden = kvmppc_gsbm_next(gsbm, 0); iden != 0; \
814
+
iden = kvmppc_gsbm_next(gsbm, iden))
815
+
816
+
/**************************************************************************
817
+
* Guest State Parser
818
+
**************************************************************************/
819
+
820
+
void kvmppc_gsp_insert(struct kvmppc_gs_parser *gsp, u16 iden,
821
+
struct kvmppc_gs_elem *gse);
822
+
struct kvmppc_gs_elem *kvmppc_gsp_lookup(struct kvmppc_gs_parser *gsp,
823
+
u16 iden);
824
+
825
+
/**
826
+
* kvmppc_gsp_for_each - iterate the <guest state IDs, guest state element>
827
+
* pairs
828
+
* @gsp: guest state buffer bitmap
829
+
* @iden: current guest state ID
830
+
* @gse: guest state element
831
+
*/
832
+
#define kvmppc_gsp_for_each(gsp, iden, gse) \
833
+
for (iden = kvmppc_gsbm_next(&(gsp)->iterator, 0), \
834
+
gse = kvmppc_gsp_lookup((gsp), iden); \
835
+
iden != 0; iden = kvmppc_gsbm_next(&(gsp)->iterator, iden), \
836
+
gse = kvmppc_gsp_lookup((gsp), iden))
837
+
838
+
/**************************************************************************
839
+
* Guest State Message
840
+
**************************************************************************/
841
+
842
+
/**
843
+
* kvmppc_gsm_for_each - iterate the guest state IDs included in a guest state
844
+
* message
845
+
* @gsp: guest state buffer bitmap
846
+
* @iden: current guest state ID
847
+
* @gse: guest state element
848
+
*/
849
+
#define kvmppc_gsm_for_each(gsm, iden) \
850
+
for (iden = kvmppc_gsbm_next(&gsm->bitmap, 0); iden != 0; \
851
+
iden = kvmppc_gsbm_next(&gsm->bitmap, iden))
852
+
853
+
int kvmppc_gsm_init(struct kvmppc_gs_msg *mgs, struct kvmppc_gs_msg_ops *ops,
854
+
void *data, unsigned long flags);
855
+
856
+
struct kvmppc_gs_msg *kvmppc_gsm_new(struct kvmppc_gs_msg_ops *ops, void *data,
857
+
unsigned long flags, gfp_t gfp_flags);
858
+
void kvmppc_gsm_free(struct kvmppc_gs_msg *gsm);
859
+
size_t kvmppc_gsm_size(struct kvmppc_gs_msg *gsm);
860
+
int kvmppc_gsm_fill_info(struct kvmppc_gs_msg *gsm, struct kvmppc_gs_buff *gsb);
861
+
int kvmppc_gsm_refresh_info(struct kvmppc_gs_msg *gsm,
862
+
struct kvmppc_gs_buff *gsb);
863
+
864
+
/**
865
+
* kvmppc_gsm_include - indicate a guest state ID should be included when
866
+
* serializing
867
+
* @gsm: guest state message
868
+
* @iden: guest state ID
869
+
*/
870
+
static inline void kvmppc_gsm_include(struct kvmppc_gs_msg *gsm, u16 iden)
871
+
{
872
+
kvmppc_gsbm_set(&gsm->bitmap, iden);
873
+
}
874
+
875
+
/**
876
+
* kvmppc_gsm_includes - check if a guest state ID will be included when
877
+
* serializing
878
+
* @gsm: guest state message
879
+
* @iden: guest state ID
880
+
*/
881
+
static inline bool kvmppc_gsm_includes(struct kvmppc_gs_msg *gsm, u16 iden)
882
+
{
883
+
return kvmppc_gsbm_test(&gsm->bitmap, iden);
884
+
}
885
+
886
+
/**
887
+
* kvmppc_gsm_includes - indicate all guest state IDs should be included when
888
+
* serializing
889
+
* @gsm: guest state message
890
+
* @iden: guest state ID
891
+
*/
892
+
static inline void kvmppc_gsm_include_all(struct kvmppc_gs_msg *gsm)
893
+
{
894
+
kvmppc_gsbm_fill(&gsm->bitmap);
895
+
}
896
+
897
+
/**
898
+
* kvmppc_gsm_include - clear the guest state IDs that should be included when
899
+
* serializing
900
+
* @gsm: guest state message
901
+
*/
902
+
static inline void kvmppc_gsm_reset(struct kvmppc_gs_msg *gsm)
903
+
{
904
+
kvmppc_gsbm_zero(&gsm->bitmap);
905
+
}
906
+
907
+
/**
908
+
* kvmppc_gsb_receive_data - flexibly update values from a guest state buffer
909
+
* @gsb: guest state buffer
910
+
* @gsm: guest state message
911
+
*
912
+
* Requests updated values for the guest state values included in the guest
913
+
* state message. The guest state message will then deserialize the guest state
914
+
* buffer.
915
+
*/
916
+
static inline int kvmppc_gsb_receive_data(struct kvmppc_gs_buff *gsb,
917
+
struct kvmppc_gs_msg *gsm)
918
+
{
919
+
int rc;
920
+
921
+
kvmppc_gsb_reset(gsb);
922
+
rc = kvmppc_gsm_fill_info(gsm, gsb);
923
+
if (rc < 0)
924
+
return rc;
925
+
926
+
rc = kvmppc_gsb_recv(gsb, gsm->flags);
927
+
if (rc < 0)
928
+
return rc;
929
+
930
+
rc = kvmppc_gsm_refresh_info(gsm, gsb);
931
+
if (rc < 0)
932
+
return rc;
933
+
return 0;
934
+
}
935
+
936
+
/**
937
+
* kvmppc_gsb_recv - receive a single guest state ID
938
+
* @gsb: guest state buffer
939
+
* @gsm: guest state message
940
+
* @iden: guest state identity
941
+
*/
942
+
static inline int kvmppc_gsb_receive_datum(struct kvmppc_gs_buff *gsb,
943
+
struct kvmppc_gs_msg *gsm, u16 iden)
944
+
{
945
+
int rc;
946
+
947
+
kvmppc_gsm_include(gsm, iden);
948
+
rc = kvmppc_gsb_receive_data(gsb, gsm);
949
+
if (rc < 0)
950
+
return rc;
951
+
kvmppc_gsm_reset(gsm);
952
+
return 0;
953
+
}
954
+
955
+
/**
956
+
* kvmppc_gsb_send_data - flexibly send values from a guest state buffer
957
+
* @gsb: guest state buffer
958
+
* @gsm: guest state message
959
+
*
960
+
* Sends the guest state values included in the guest state message.
961
+
*/
962
+
static inline int kvmppc_gsb_send_data(struct kvmppc_gs_buff *gsb,
963
+
struct kvmppc_gs_msg *gsm)
964
+
{
965
+
int rc;
966
+
967
+
kvmppc_gsb_reset(gsb);
968
+
rc = kvmppc_gsm_fill_info(gsm, gsb);
969
+
if (rc < 0)
970
+
return rc;
971
+
rc = kvmppc_gsb_send(gsb, gsm->flags);
972
+
973
+
return rc;
974
+
}
975
+
976
+
/**
977
+
* kvmppc_gsb_recv - send a single guest state ID
978
+
* @gsb: guest state buffer
979
+
* @gsm: guest state message
980
+
* @iden: guest state identity
981
+
*/
982
+
static inline int kvmppc_gsb_send_datum(struct kvmppc_gs_buff *gsb,
983
+
struct kvmppc_gs_msg *gsm, u16 iden)
984
+
{
985
+
int rc;
986
+
987
+
kvmppc_gsm_include(gsm, iden);
988
+
rc = kvmppc_gsb_send_data(gsb, gsm);
989
+
if (rc < 0)
990
+
return rc;
991
+
kvmppc_gsm_reset(gsm);
992
+
return 0;
993
+
}
994
+
995
+
#endif /* _ASM_POWERPC_GUEST_STATE_BUFFER_H */
+30
arch/powerpc/include/asm/hvcall.h
+30
arch/powerpc/include/asm/hvcall.h
···
100
100
#define H_COP_HW -74
101
101
#define H_STATE -75
102
102
#define H_IN_USE -77
103
+
104
+
#define H_INVALID_ELEMENT_ID -79
105
+
#define H_INVALID_ELEMENT_SIZE -80
106
+
#define H_INVALID_ELEMENT_VALUE -81
107
+
#define H_INPUT_BUFFER_NOT_DEFINED -82
108
+
#define H_INPUT_BUFFER_TOO_SMALL -83
109
+
#define H_OUTPUT_BUFFER_NOT_DEFINED -84
110
+
#define H_OUTPUT_BUFFER_TOO_SMALL -85
111
+
#define H_PARTITION_PAGE_TABLE_NOT_DEFINED -86
112
+
#define H_GUEST_VCPU_STATE_NOT_HV_OWNED -87
113
+
114
+
103
115
#define H_UNSUPPORTED_FLAG_START -256
104
116
#define H_UNSUPPORTED_FLAG_END -511
105
117
#define H_MULTI_THREADS_ACTIVE -9005
···
393
381
#define H_ENTER_NESTED 0xF804
394
382
#define H_TLB_INVALIDATE 0xF808
395
383
#define H_COPY_TOFROM_GUEST 0xF80C
384
+
#define H_GUEST_GET_CAPABILITIES 0x460
385
+
#define H_GUEST_SET_CAPABILITIES 0x464
386
+
#define H_GUEST_CREATE 0x470
387
+
#define H_GUEST_CREATE_VCPU 0x474
388
+
#define H_GUEST_GET_STATE 0x478
389
+
#define H_GUEST_SET_STATE 0x47C
390
+
#define H_GUEST_RUN_VCPU 0x480
391
+
#define H_GUEST_COPY_MEMORY 0x484
392
+
#define H_GUEST_DELETE 0x488
396
393
397
394
/* Flags for H_SVM_PAGE_IN */
398
395
#define H_PAGE_IN_SHARED 0x1
···
487
466
#define H_RPTI_PAGE_2M 0x04
488
467
#define H_RPTI_PAGE_1G 0x08
489
468
#define H_RPTI_PAGE_ALL (-1UL)
469
+
470
+
/* Flags for H_GUEST_{S,G}_STATE */
471
+
#define H_GUEST_FLAGS_WIDE (1UL<<(63-0))
472
+
473
+
/* Flag values used for H_{S,G}SET_GUEST_CAPABILITIES */
474
+
#define H_GUEST_CAP_COPY_MEM (1UL<<(63-0))
475
+
#define H_GUEST_CAP_POWER9 (1UL<<(63-1))
476
+
#define H_GUEST_CAP_POWER10 (1UL<<(63-2))
477
+
#define H_GUEST_CAP_BITMAP2 (1UL<<(63-63))
490
478
491
479
#ifndef __ASSEMBLY__
492
480
#include <linux/types.h>
+8
-8
arch/powerpc/include/asm/imc-pmu.h
+8
-8
arch/powerpc/include/asm/imc-pmu.h
···
74
74
* The following is the data structure to hold trace imc data.
75
75
*/
76
76
struct trace_imc_data {
77
-
u64 tb1;
78
-
u64 ip;
79
-
u64 val;
80
-
u64 cpmc1;
81
-
u64 cpmc2;
82
-
u64 cpmc3;
83
-
u64 cpmc4;
84
-
u64 tb2;
77
+
__be64 tb1;
78
+
__be64 ip;
79
+
__be64 val;
80
+
__be64 cpmc1;
81
+
__be64 cpmc2;
82
+
__be64 cpmc3;
83
+
__be64 cpmc4;
84
+
__be64 tb2;
85
85
};
86
86
87
87
/* Event attribute array index */
+1
-1
arch/powerpc/include/asm/io.h
+1
-1
arch/powerpc/include/asm/io.h
···
950
950
* almost all conceivable cases a device driver should not be using
951
951
* this function
952
952
*/
953
-
static inline unsigned long virt_to_phys(volatile void * address)
953
+
static inline unsigned long virt_to_phys(const volatile void * address)
954
954
{
955
955
WARN_ON(IS_ENABLED(CONFIG_DEBUG_VIRTUAL) && !virt_addr_valid(address));
956
956
+6
-2
arch/powerpc/include/asm/kexec.h
+6
-2
arch/powerpc/include/asm/kexec.h
···
99
99
100
100
void kexec_copy_flush(struct kimage *image);
101
101
102
-
#if defined(CONFIG_CRASH_DUMP) && defined(CONFIG_PPC_RTAS)
102
+
#if defined(CONFIG_CRASH_DUMP)
103
+
bool is_kdump_kernel(void);
104
+
#define is_kdump_kernel is_kdump_kernel
105
+
#if defined(CONFIG_PPC_RTAS)
103
106
void crash_free_reserved_phys_range(unsigned long begin, unsigned long end);
104
107
#define crash_free_reserved_phys_range crash_free_reserved_phys_range
105
-
#endif
108
+
#endif /* CONFIG_PPC_RTAS */
109
+
#endif /* CONFIG_CRASH_DUMP */
106
110
107
111
#ifdef CONFIG_KEXEC_FILE
108
112
extern const struct kexec_file_ops kexec_elf64_ops;
+214
-6
arch/powerpc/include/asm/kvm_book3s.h
+214
-6
arch/powerpc/include/asm/kvm_book3s.h
···
12
12
#include <linux/types.h>
13
13
#include <linux/kvm_host.h>
14
14
#include <asm/kvm_book3s_asm.h>
15
+
#include <asm/guest-state-buffer.h>
15
16
16
17
struct kvmppc_bat {
17
18
u64 raw;
···
192
191
extern int kvmppc_mmu_radix_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
193
192
struct kvmppc_pte *gpte, bool data, bool iswrite);
194
193
extern void kvmppc_radix_tlbie_page(struct kvm *kvm, unsigned long addr,
195
-
unsigned int pshift, unsigned int lpid);
194
+
unsigned int pshift, u64 lpid);
196
195
extern void kvmppc_unmap_pte(struct kvm *kvm, pte_t *pte, unsigned long gpa,
197
196
unsigned int shift,
198
197
const struct kvm_memory_slot *memslot,
199
-
unsigned int lpid);
198
+
u64 lpid);
200
199
extern bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nested,
201
200
bool writing, unsigned long gpa,
202
-
unsigned int lpid);
201
+
u64 lpid);
203
202
extern int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu,
204
203
unsigned long gpa,
205
204
struct kvm_memory_slot *memslot,
···
208
207
extern int kvmppc_init_vm_radix(struct kvm *kvm);
209
208
extern void kvmppc_free_radix(struct kvm *kvm);
210
209
extern void kvmppc_free_pgtable_radix(struct kvm *kvm, pgd_t *pgd,
211
-
unsigned int lpid);
210
+
u64 lpid);
212
211
extern int kvmppc_radix_init(void);
213
212
extern void kvmppc_radix_exit(void);
214
213
extern void kvm_unmap_radix(struct kvm *kvm, struct kvm_memory_slot *memslot,
···
296
295
static inline void kvmppc_restore_tm_sprs(struct kvm_vcpu *vcpu) {}
297
296
#endif
298
297
298
+
extern unsigned long nested_capabilities;
299
299
long kvmhv_nested_init(void);
300
300
void kvmhv_nested_exit(void);
301
301
void kvmhv_vm_nested_init(struct kvm *kvm);
302
302
long kvmhv_set_partition_table(struct kvm_vcpu *vcpu);
303
303
long kvmhv_copy_tofrom_guest_nested(struct kvm_vcpu *vcpu);
304
-
void kvmhv_set_ptbl_entry(unsigned int lpid, u64 dw0, u64 dw1);
304
+
void kvmhv_set_ptbl_entry(u64 lpid, u64 dw0, u64 dw1);
305
305
void kvmhv_release_all_nested(struct kvm *kvm);
306
306
long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu);
307
307
long kvmhv_do_nested_tlbie(struct kvm_vcpu *vcpu);
···
317
315
long int kvmhv_nested_page_fault(struct kvm_vcpu *vcpu);
318
316
319
317
void kvmppc_giveup_fac(struct kvm_vcpu *vcpu, ulong fac);
318
+
319
+
320
+
#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
321
+
322
+
extern struct static_key_false __kvmhv_is_nestedv2;
323
+
324
+
static inline bool kvmhv_is_nestedv2(void)
325
+
{
326
+
return static_branch_unlikely(&__kvmhv_is_nestedv2);
327
+
}
328
+
329
+
static inline bool kvmhv_is_nestedv1(void)
330
+
{
331
+
return !static_branch_likely(&__kvmhv_is_nestedv2);
332
+
}
333
+
334
+
#else
335
+
336
+
static inline bool kvmhv_is_nestedv2(void)
337
+
{
338
+
return false;
339
+
}
340
+
341
+
static inline bool kvmhv_is_nestedv1(void)
342
+
{
343
+
return false;
344
+
}
345
+
346
+
#endif
347
+
348
+
int __kvmhv_nestedv2_reload_ptregs(struct kvm_vcpu *vcpu, struct pt_regs *regs);
349
+
int __kvmhv_nestedv2_mark_dirty_ptregs(struct kvm_vcpu *vcpu, struct pt_regs *regs);
350
+
int __kvmhv_nestedv2_mark_dirty(struct kvm_vcpu *vcpu, u16 iden);
351
+
int __kvmhv_nestedv2_cached_reload(struct kvm_vcpu *vcpu, u16 iden);
352
+
353
+
static inline int kvmhv_nestedv2_reload_ptregs(struct kvm_vcpu *vcpu,
354
+
struct pt_regs *regs)
355
+
{
356
+
if (kvmhv_is_nestedv2())
357
+
return __kvmhv_nestedv2_reload_ptregs(vcpu, regs);
358
+
return 0;
359
+
}
360
+
static inline int kvmhv_nestedv2_mark_dirty_ptregs(struct kvm_vcpu *vcpu,
361
+
struct pt_regs *regs)
362
+
{
363
+
if (kvmhv_is_nestedv2())
364
+
return __kvmhv_nestedv2_mark_dirty_ptregs(vcpu, regs);
365
+
return 0;
366
+
}
367
+
368
+
static inline int kvmhv_nestedv2_mark_dirty(struct kvm_vcpu *vcpu, u16 iden)
369
+
{
370
+
if (kvmhv_is_nestedv2())
371
+
return __kvmhv_nestedv2_mark_dirty(vcpu, iden);
372
+
return 0;
373
+
}
374
+
375
+
static inline int kvmhv_nestedv2_cached_reload(struct kvm_vcpu *vcpu, u16 iden)
376
+
{
377
+
if (kvmhv_is_nestedv2())
378
+
return __kvmhv_nestedv2_cached_reload(vcpu, iden);
379
+
return 0;
380
+
}
320
381
321
382
extern int kvm_irq_bypass;
322
383
···
400
335
static inline void kvmppc_set_gpr(struct kvm_vcpu *vcpu, int num, ulong val)
401
336
{
402
337
vcpu->arch.regs.gpr[num] = val;
338
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_GPR(num));
403
339
}
404
340
405
341
static inline ulong kvmppc_get_gpr(struct kvm_vcpu *vcpu, int num)
406
342
{
343
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_GPR(num)) < 0);
407
344
return vcpu->arch.regs.gpr[num];
408
345
}
409
346
410
347
static inline void kvmppc_set_cr(struct kvm_vcpu *vcpu, u32 val)
411
348
{
412
349
vcpu->arch.regs.ccr = val;
350
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_CR);
413
351
}
414
352
415
353
static inline u32 kvmppc_get_cr(struct kvm_vcpu *vcpu)
416
354
{
355
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_CR) < 0);
417
356
return vcpu->arch.regs.ccr;
418
357
}
419
358
420
359
static inline void kvmppc_set_xer(struct kvm_vcpu *vcpu, ulong val)
421
360
{
422
361
vcpu->arch.regs.xer = val;
362
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_XER);
423
363
}
424
364
425
365
static inline ulong kvmppc_get_xer(struct kvm_vcpu *vcpu)
426
366
{
367
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_XER) < 0);
427
368
return vcpu->arch.regs.xer;
428
369
}
429
370
430
371
static inline void kvmppc_set_ctr(struct kvm_vcpu *vcpu, ulong val)
431
372
{
432
373
vcpu->arch.regs.ctr = val;
374
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_CTR);
433
375
}
434
376
435
377
static inline ulong kvmppc_get_ctr(struct kvm_vcpu *vcpu)
436
378
{
379
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_CTR) < 0);
437
380
return vcpu->arch.regs.ctr;
438
381
}
439
382
440
383
static inline void kvmppc_set_lr(struct kvm_vcpu *vcpu, ulong val)
441
384
{
442
385
vcpu->arch.regs.link = val;
386
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_LR);
443
387
}
444
388
445
389
static inline ulong kvmppc_get_lr(struct kvm_vcpu *vcpu)
446
390
{
391
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_LR) < 0);
447
392
return vcpu->arch.regs.link;
448
393
}
449
394
450
395
static inline void kvmppc_set_pc(struct kvm_vcpu *vcpu, ulong val)
451
396
{
452
397
vcpu->arch.regs.nip = val;
398
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_NIA);
453
399
}
454
400
455
401
static inline ulong kvmppc_get_pc(struct kvm_vcpu *vcpu)
456
402
{
403
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_NIA) < 0);
457
404
return vcpu->arch.regs.nip;
458
405
}
459
406
···
480
403
return vcpu->arch.fault_dar;
481
404
}
482
405
406
+
static inline u64 kvmppc_get_fpr(struct kvm_vcpu *vcpu, int i)
407
+
{
408
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_VSRS(i)) < 0);
409
+
return vcpu->arch.fp.fpr[i][TS_FPROFFSET];
410
+
}
411
+
412
+
static inline void kvmppc_set_fpr(struct kvm_vcpu *vcpu, int i, u64 val)
413
+
{
414
+
vcpu->arch.fp.fpr[i][TS_FPROFFSET] = val;
415
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_VSRS(i));
416
+
}
417
+
418
+
static inline u64 kvmppc_get_fpscr(struct kvm_vcpu *vcpu)
419
+
{
420
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_FPSCR) < 0);
421
+
return vcpu->arch.fp.fpscr;
422
+
}
423
+
424
+
static inline void kvmppc_set_fpscr(struct kvm_vcpu *vcpu, u64 val)
425
+
{
426
+
vcpu->arch.fp.fpscr = val;
427
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_FPSCR);
428
+
}
429
+
430
+
431
+
static inline u64 kvmppc_get_vsx_fpr(struct kvm_vcpu *vcpu, int i, int j)
432
+
{
433
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_VSRS(i)) < 0);
434
+
return vcpu->arch.fp.fpr[i][j];
435
+
}
436
+
437
+
static inline void kvmppc_set_vsx_fpr(struct kvm_vcpu *vcpu, int i, int j,
438
+
u64 val)
439
+
{
440
+
vcpu->arch.fp.fpr[i][j] = val;
441
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_VSRS(i));
442
+
}
443
+
444
+
#ifdef CONFIG_ALTIVEC
445
+
static inline void kvmppc_get_vsx_vr(struct kvm_vcpu *vcpu, int i, vector128 *v)
446
+
{
447
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_VSRS(32 + i)) < 0);
448
+
*v = vcpu->arch.vr.vr[i];
449
+
}
450
+
451
+
static inline void kvmppc_set_vsx_vr(struct kvm_vcpu *vcpu, int i,
452
+
vector128 *val)
453
+
{
454
+
vcpu->arch.vr.vr[i] = *val;
455
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_VSRS(32 + i));
456
+
}
457
+
458
+
static inline u32 kvmppc_get_vscr(struct kvm_vcpu *vcpu)
459
+
{
460
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_VSCR) < 0);
461
+
return vcpu->arch.vr.vscr.u[3];
462
+
}
463
+
464
+
static inline void kvmppc_set_vscr(struct kvm_vcpu *vcpu, u32 val)
465
+
{
466
+
vcpu->arch.vr.vscr.u[3] = val;
467
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_VSCR);
468
+
}
469
+
#endif
470
+
471
+
#define KVMPPC_BOOK3S_VCPU_ACCESSOR_SET(reg, size, iden) \
472
+
static inline void kvmppc_set_##reg(struct kvm_vcpu *vcpu, u##size val) \
473
+
{ \
474
+
\
475
+
vcpu->arch.reg = val; \
476
+
kvmhv_nestedv2_mark_dirty(vcpu, iden); \
477
+
}
478
+
479
+
#define KVMPPC_BOOK3S_VCPU_ACCESSOR_GET(reg, size, iden) \
480
+
static inline u##size kvmppc_get_##reg(struct kvm_vcpu *vcpu) \
481
+
{ \
482
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, iden) < 0); \
483
+
return vcpu->arch.reg; \
484
+
}
485
+
486
+
#define KVMPPC_BOOK3S_VCPU_ACCESSOR(reg, size, iden) \
487
+
KVMPPC_BOOK3S_VCPU_ACCESSOR_SET(reg, size, iden) \
488
+
KVMPPC_BOOK3S_VCPU_ACCESSOR_GET(reg, size, iden) \
489
+
490
+
KVMPPC_BOOK3S_VCPU_ACCESSOR(pid, 32, KVMPPC_GSID_PIDR)
491
+
KVMPPC_BOOK3S_VCPU_ACCESSOR(tar, 64, KVMPPC_GSID_TAR)
492
+
KVMPPC_BOOK3S_VCPU_ACCESSOR(ebbhr, 64, KVMPPC_GSID_EBBHR)
493
+
KVMPPC_BOOK3S_VCPU_ACCESSOR(ebbrr, 64, KVMPPC_GSID_EBBRR)
494
+
KVMPPC_BOOK3S_VCPU_ACCESSOR(bescr, 64, KVMPPC_GSID_BESCR)
495
+
KVMPPC_BOOK3S_VCPU_ACCESSOR(ic, 64, KVMPPC_GSID_IC)
496
+
KVMPPC_BOOK3S_VCPU_ACCESSOR(vrsave, 64, KVMPPC_GSID_VRSAVE)
497
+
498
+
499
+
#define KVMPPC_BOOK3S_VCORE_ACCESSOR_SET(reg, size, iden) \
500
+
static inline void kvmppc_set_##reg(struct kvm_vcpu *vcpu, u##size val) \
501
+
{ \
502
+
vcpu->arch.vcore->reg = val; \
503
+
kvmhv_nestedv2_mark_dirty(vcpu, iden); \
504
+
}
505
+
506
+
#define KVMPPC_BOOK3S_VCORE_ACCESSOR_GET(reg, size, iden) \
507
+
static inline u##size kvmppc_get_##reg(struct kvm_vcpu *vcpu) \
508
+
{ \
509
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, iden) < 0); \
510
+
return vcpu->arch.vcore->reg; \
511
+
}
512
+
513
+
#define KVMPPC_BOOK3S_VCORE_ACCESSOR(reg, size, iden) \
514
+
KVMPPC_BOOK3S_VCORE_ACCESSOR_SET(reg, size, iden) \
515
+
KVMPPC_BOOK3S_VCORE_ACCESSOR_GET(reg, size, iden) \
516
+
517
+
518
+
KVMPPC_BOOK3S_VCORE_ACCESSOR(vtb, 64, KVMPPC_GSID_VTB)
519
+
KVMPPC_BOOK3S_VCORE_ACCESSOR(tb_offset, 64, KVMPPC_GSID_TB_OFFSET)
520
+
KVMPPC_BOOK3S_VCORE_ACCESSOR_GET(arch_compat, 32, KVMPPC_GSID_LOGICAL_PVR)
521
+
KVMPPC_BOOK3S_VCORE_ACCESSOR_GET(lpcr, 64, KVMPPC_GSID_LPCR)
522
+
523
+
static inline u64 kvmppc_get_dec_expires(struct kvm_vcpu *vcpu)
524
+
{
525
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_TB_OFFSET) < 0);
526
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_DEC_EXPIRY_TB) < 0);
527
+
return vcpu->arch.dec_expires;
528
+
}
529
+
530
+
static inline void kvmppc_set_dec_expires(struct kvm_vcpu *vcpu, u64 val)
531
+
{
532
+
vcpu->arch.dec_expires = val;
533
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_TB_OFFSET) < 0);
534
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_DEC_EXPIRY_TB);
535
+
}
536
+
483
537
/* Expiry time of vcpu DEC relative to host TB */
484
538
static inline u64 kvmppc_dec_expires_host_tb(struct kvm_vcpu *vcpu)
485
539
{
486
-
return vcpu->arch.dec_expires - vcpu->arch.vcore->tb_offset;
540
+
return kvmppc_get_dec_expires(vcpu) - kvmppc_get_tb_offset(vcpu);
487
541
}
488
542
489
543
static inline bool is_kvmppc_resume_guest(int r)
+7
-1
arch/powerpc/include/asm/kvm_book3s_64.h
+7
-1
arch/powerpc/include/asm/kvm_book3s_64.h
···
624
624
625
625
extern int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
626
626
unsigned long gpa, unsigned int level,
627
-
unsigned long mmu_seq, unsigned int lpid,
627
+
unsigned long mmu_seq, u64 lpid,
628
628
unsigned long *rmapp, struct rmap_nested **n_rmap);
629
629
extern void kvmhv_insert_nest_rmap(struct kvm *kvm, unsigned long *rmapp,
630
630
struct rmap_nested **n_rmap);
···
676
676
677
677
extern pte_t *find_kvm_nested_guest_pte(struct kvm *kvm, unsigned long lpid,
678
678
unsigned long ea, unsigned *hshift);
679
+
680
+
int kvmhv_nestedv2_vcpu_create(struct kvm_vcpu *vcpu, struct kvmhv_nestedv2_io *io);
681
+
void kvmhv_nestedv2_vcpu_free(struct kvm_vcpu *vcpu, struct kvmhv_nestedv2_io *io);
682
+
int kvmhv_nestedv2_flush_vcpu(struct kvm_vcpu *vcpu, u64 time_limit);
683
+
int kvmhv_nestedv2_set_ptbl_entry(unsigned long lpid, u64 dw0, u64 dw1);
684
+
int kvmhv_nestedv2_parse_output(struct kvm_vcpu *vcpu);
679
685
680
686
#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */
681
687
+10
arch/powerpc/include/asm/kvm_booke.h
+10
arch/powerpc/include/asm/kvm_booke.h
···
89
89
return vcpu->arch.regs.nip;
90
90
}
91
91
92
+
static inline void kvmppc_set_fpr(struct kvm_vcpu *vcpu, int i, u64 val)
93
+
{
94
+
vcpu->arch.fp.fpr[i][TS_FPROFFSET] = val;
95
+
}
96
+
97
+
static inline u64 kvmppc_get_fpr(struct kvm_vcpu *vcpu, int i)
98
+
{
99
+
return vcpu->arch.fp.fpr[i][TS_FPROFFSET];
100
+
}
101
+
92
102
#ifdef CONFIG_BOOKE
93
103
static inline ulong kvmppc_get_fault_dar(struct kvm_vcpu *vcpu)
94
104
{
+21
-1
arch/powerpc/include/asm/kvm_host.h
+21
-1
arch/powerpc/include/asm/kvm_host.h
···
25
25
#include <asm/cacheflush.h>
26
26
#include <asm/hvcall.h>
27
27
#include <asm/mce.h>
28
+
#include <asm/guest-state-buffer.h>
28
29
29
30
#define __KVM_HAVE_ARCH_VCPU_DEBUGFS
30
31
···
277
276
#define KVMPPC_SECURE_INIT_ABORT 0x4 /* H_SVM_INIT_ABORT issued */
278
277
279
278
struct kvm_arch {
280
-
unsigned int lpid;
279
+
u64 lpid;
281
280
unsigned int smt_mode; /* # vcpus per virtual core */
282
281
unsigned int emul_smt_mode; /* emualted SMT mode, on P9 */
283
282
#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
···
508
507
u8 pipr;
509
508
};
510
509
__be64 w01;
510
+
};
511
+
512
+
/* Nestedv2 H_GUEST_RUN_VCPU configuration */
513
+
struct kvmhv_nestedv2_config {
514
+
struct kvmppc_gs_buff_info vcpu_run_output_cfg;
515
+
struct kvmppc_gs_buff_info vcpu_run_input_cfg;
516
+
u64 vcpu_run_output_size;
517
+
};
518
+
519
+
/* Nestedv2 L1<->L0 communication state */
520
+
struct kvmhv_nestedv2_io {
521
+
struct kvmhv_nestedv2_config cfg;
522
+
struct kvmppc_gs_buff *vcpu_run_output;
523
+
struct kvmppc_gs_buff *vcpu_run_input;
524
+
struct kvmppc_gs_msg *vcpu_message;
525
+
struct kvmppc_gs_msg *vcore_message;
526
+
struct kvmppc_gs_bitmap valids;
511
527
};
512
528
513
529
struct kvm_vcpu_arch {
···
847
829
u64 nested_hfscr; /* HFSCR that the L1 requested for the nested guest */
848
830
u32 nested_vcpu_id;
849
831
gpa_t nested_io_gpr;
832
+
/* For nested APIv2 guests*/
833
+
struct kvmhv_nestedv2_io nestedv2_io;
850
834
#endif
851
835
852
836
#ifdef CONFIG_KVM_BOOK3S_HV_EXIT_TIMING
+76
-34
arch/powerpc/include/asm/kvm_ppc.h
+76
-34
arch/powerpc/include/asm/kvm_ppc.h
···
615
615
{
616
616
return false;
617
617
}
618
+
619
+
#endif
620
+
621
+
#ifndef CONFIG_PPC_BOOK3S
622
+
623
+
static inline bool kvmhv_is_nestedv2(void)
624
+
{
625
+
return false;
626
+
}
627
+
628
+
static inline bool kvmhv_is_nestedv1(void)
629
+
{
630
+
return false;
631
+
}
632
+
633
+
static inline int kvmhv_nestedv2_reload_ptregs(struct kvm_vcpu *vcpu,
634
+
struct pt_regs *regs)
635
+
{
636
+
return 0;
637
+
}
638
+
static inline int kvmhv_nestedv2_mark_dirty_ptregs(struct kvm_vcpu *vcpu,
639
+
struct pt_regs *regs)
640
+
{
641
+
return 0;
642
+
}
643
+
644
+
static inline int kvmhv_nestedv2_mark_dirty(struct kvm_vcpu *vcpu, u16 iden)
645
+
{
646
+
return 0;
647
+
}
648
+
649
+
static inline int kvmhv_nestedv2_cached_reload(struct kvm_vcpu *vcpu, u16 iden)
650
+
{
651
+
return 0;
652
+
}
653
+
618
654
#endif
619
655
620
656
#ifdef CONFIG_KVM_XICS
···
963
927
#endif
964
928
}
965
929
966
-
#define SPRNG_WRAPPER_GET(reg, bookehv_spr) \
930
+
#define KVMPPC_BOOKE_HV_SPRNG_ACCESSOR_GET(reg, bookehv_spr) \
967
931
static inline ulong kvmppc_get_##reg(struct kvm_vcpu *vcpu) \
968
932
{ \
969
933
return mfspr(bookehv_spr); \
970
934
} \
971
935
972
-
#define SPRNG_WRAPPER_SET(reg, bookehv_spr) \
936
+
#define KVMPPC_BOOKE_HV_SPRNG_ACCESSOR_SET(reg, bookehv_spr) \
973
937
static inline void kvmppc_set_##reg(struct kvm_vcpu *vcpu, ulong val) \
974
938
{ \
975
939
mtspr(bookehv_spr, val); \
976
940
} \
977
941
978
-
#define SHARED_WRAPPER_GET(reg, size) \
942
+
#define KVMPPC_VCPU_SHARED_REGS_ACCESSOR_GET(reg, size, iden) \
979
943
static inline u##size kvmppc_get_##reg(struct kvm_vcpu *vcpu) \
980
944
{ \
945
+
if (iden) \
946
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, iden) < 0); \
981
947
if (kvmppc_shared_big_endian(vcpu)) \
982
-
return be##size##_to_cpu(vcpu->arch.shared->reg); \
948
+
return be##size##_to_cpu((__be##size __force)vcpu->arch.shared->reg); \
983
949
else \
984
-
return le##size##_to_cpu(vcpu->arch.shared->reg); \
950
+
return le##size##_to_cpu((__le##size __force)vcpu->arch.shared->reg); \
985
951
} \
986
952
987
-
#define SHARED_WRAPPER_SET(reg, size) \
953
+
#define KVMPPC_VCPU_SHARED_REGS_ACCESSOR_SET(reg, size, iden) \
988
954
static inline void kvmppc_set_##reg(struct kvm_vcpu *vcpu, u##size val) \
989
955
{ \
990
956
if (kvmppc_shared_big_endian(vcpu)) \
991
-
vcpu->arch.shared->reg = cpu_to_be##size(val); \
957
+
vcpu->arch.shared->reg = (u##size __force)cpu_to_be##size(val); \
992
958
else \
993
-
vcpu->arch.shared->reg = cpu_to_le##size(val); \
959
+
vcpu->arch.shared->reg = (u##size __force)cpu_to_le##size(val); \
960
+
\
961
+
if (iden) \
962
+
kvmhv_nestedv2_mark_dirty(vcpu, iden); \
994
963
} \
995
964
996
-
#define SHARED_WRAPPER(reg, size) \
997
-
SHARED_WRAPPER_GET(reg, size) \
998
-
SHARED_WRAPPER_SET(reg, size) \
965
+
#define KVMPPC_VCPU_SHARED_REGS_ACCESSOR(reg, size, iden) \
966
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR_GET(reg, size, iden) \
967
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR_SET(reg, size, iden) \
999
968
1000
-
#define SPRNG_WRAPPER(reg, bookehv_spr) \
1001
-
SPRNG_WRAPPER_GET(reg, bookehv_spr) \
1002
-
SPRNG_WRAPPER_SET(reg, bookehv_spr) \
969
+
#define KVMPPC_BOOKE_HV_SPRNG_ACCESSOR(reg, bookehv_spr) \
970
+
KVMPPC_BOOKE_HV_SPRNG_ACCESSOR_GET(reg, bookehv_spr) \
971
+
KVMPPC_BOOKE_HV_SPRNG_ACCESSOR_SET(reg, bookehv_spr) \
1003
972
1004
973
#ifdef CONFIG_KVM_BOOKE_HV
1005
974
1006
-
#define SHARED_SPRNG_WRAPPER(reg, size, bookehv_spr) \
1007
-
SPRNG_WRAPPER(reg, bookehv_spr) \
975
+
#define KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(reg, size, bookehv_spr, iden) \
976
+
KVMPPC_BOOKE_HV_SPRNG_ACCESSOR(reg, bookehv_spr) \
1008
977
1009
978
#else
1010
979
1011
-
#define SHARED_SPRNG_WRAPPER(reg, size, bookehv_spr) \
1012
-
SHARED_WRAPPER(reg, size) \
980
+
#define KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(reg, size, bookehv_spr, iden) \
981
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR(reg, size, iden) \
1013
982
1014
983
#endif
1015
984
1016
-
SHARED_WRAPPER(critical, 64)
1017
-
SHARED_SPRNG_WRAPPER(sprg0, 64, SPRN_GSPRG0)
1018
-
SHARED_SPRNG_WRAPPER(sprg1, 64, SPRN_GSPRG1)
1019
-
SHARED_SPRNG_WRAPPER(sprg2, 64, SPRN_GSPRG2)
1020
-
SHARED_SPRNG_WRAPPER(sprg3, 64, SPRN_GSPRG3)
1021
-
SHARED_SPRNG_WRAPPER(srr0, 64, SPRN_GSRR0)
1022
-
SHARED_SPRNG_WRAPPER(srr1, 64, SPRN_GSRR1)
1023
-
SHARED_SPRNG_WRAPPER(dar, 64, SPRN_GDEAR)
1024
-
SHARED_SPRNG_WRAPPER(esr, 64, SPRN_GESR)
1025
-
SHARED_WRAPPER_GET(msr, 64)
985
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR(critical, 64, 0)
986
+
KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(sprg0, 64, SPRN_GSPRG0, KVMPPC_GSID_SPRG0)
987
+
KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(sprg1, 64, SPRN_GSPRG1, KVMPPC_GSID_SPRG1)
988
+
KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(sprg2, 64, SPRN_GSPRG2, KVMPPC_GSID_SPRG2)
989
+
KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(sprg3, 64, SPRN_GSPRG3, KVMPPC_GSID_SPRG3)
990
+
KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(srr0, 64, SPRN_GSRR0, KVMPPC_GSID_SRR0)
991
+
KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(srr1, 64, SPRN_GSRR1, KVMPPC_GSID_SRR1)
992
+
KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(dar, 64, SPRN_GDEAR, KVMPPC_GSID_DAR)
993
+
KVMPPC_BOOKE_HV_SPRNG_OR_VCPU_SHARED_REGS_ACCESSOR(esr, 64, SPRN_GESR, 0)
994
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR_GET(msr, 64, KVMPPC_GSID_MSR)
1026
995
static inline void kvmppc_set_msr_fast(struct kvm_vcpu *vcpu, u64 val)
1027
996
{
1028
997
if (kvmppc_shared_big_endian(vcpu))
1029
998
vcpu->arch.shared->msr = cpu_to_be64(val);
1030
999
else
1031
1000
vcpu->arch.shared->msr = cpu_to_le64(val);
1001
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_MSR);
1032
1002
}
1033
-
SHARED_WRAPPER(dsisr, 32)
1034
-
SHARED_WRAPPER(int_pending, 32)
1035
-
SHARED_WRAPPER(sprg4, 64)
1036
-
SHARED_WRAPPER(sprg5, 64)
1037
-
SHARED_WRAPPER(sprg6, 64)
1038
-
SHARED_WRAPPER(sprg7, 64)
1003
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR(dsisr, 32, KVMPPC_GSID_DSISR)
1004
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR(int_pending, 32, 0)
1005
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR(sprg4, 64, 0)
1006
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR(sprg5, 64, 0)
1007
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR(sprg6, 64, 0)
1008
+
KVMPPC_VCPU_SHARED_REGS_ACCESSOR(sprg7, 64, 0)
1039
1009
1040
1010
static inline u32 kvmppc_get_sr(struct kvm_vcpu *vcpu, int nr)
1041
1011
{
-1
arch/powerpc/include/asm/nohash/32/mmu-8xx.h
-1
arch/powerpc/include/asm/nohash/32/mmu-8xx.h
+9
-192
arch/powerpc/include/asm/nohash/32/pgtable.h
+9
-192
arch/powerpc/include/asm/nohash/32/pgtable.h
···
9
9
#include <linux/threads.h>
10
10
#include <asm/mmu.h> /* For sub-arch specific PPC_PIN_SIZE */
11
11
12
-
#ifdef CONFIG_44x
13
-
extern int icache_44x_need_flush;
14
-
#endif
15
-
16
12
#endif /* __ASSEMBLY__ */
17
13
18
14
#define PTE_INDEX_SIZE PTE_SHIFT
···
51
55
52
56
#define USER_PTRS_PER_PGD (TASK_SIZE / PGDIR_SIZE)
53
57
54
-
#define pte_ERROR(e) \
55
-
pr_err("%s:%d: bad pte %llx.\n", __FILE__, __LINE__, \
56
-
(unsigned long long)pte_val(e))
57
58
#define pgd_ERROR(e) \
58
59
pr_err("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, pgd_val(e))
59
-
60
-
#ifndef __ASSEMBLY__
61
-
62
-
int map_kernel_page(unsigned long va, phys_addr_t pa, pgprot_t prot);
63
-
void unmap_kernel_page(unsigned long va);
64
-
65
-
#endif /* !__ASSEMBLY__ */
66
-
67
60
68
61
/*
69
62
* This is the bottom of the PKMAP area with HIGHMEM or an arbitrary
70
63
* value (for now) on others, from where we can start layout kernel
71
64
* virtual space that goes below PKMAP and FIXMAP
72
65
*/
73
-
#include <asm/fixmap.h>
66
+
67
+
#define FIXADDR_SIZE 0
68
+
#ifdef CONFIG_KASAN
69
+
#include <asm/kasan.h>
70
+
#define FIXADDR_TOP (KASAN_SHADOW_START - PAGE_SIZE)
71
+
#else
72
+
#define FIXADDR_TOP ((unsigned long)(-PAGE_SIZE))
73
+
#endif
74
74
75
75
/*
76
76
* ioremap_bot starts at that address. Early ioremaps move down from there,
···
143
151
* The mask covered by the RPN must be a ULL on 32-bit platforms with
144
152
* 64-bit PTEs.
145
153
*/
146
-
#if defined(CONFIG_PPC32) && defined(CONFIG_PTE_64BIT)
154
+
#ifdef CONFIG_PTE_64BIT
147
155
#define PTE_RPN_MASK (~((1ULL << PTE_RPN_SHIFT) - 1))
148
156
#define MAX_POSSIBLE_PHYSMEM_BITS 36
149
157
#else
···
151
159
#define MAX_POSSIBLE_PHYSMEM_BITS 32
152
160
#endif
153
161
154
-
/*
155
-
* _PAGE_CHG_MASK masks of bits that are to be preserved across
156
-
* pgprot changes.
157
-
*/
158
-
#define _PAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_SPECIAL)
159
-
160
162
#ifndef __ASSEMBLY__
161
-
162
-
#define pte_clear(mm, addr, ptep) \
163
-
do { pte_update(mm, addr, ptep, ~0, 0, 0); } while (0)
164
-
165
-
#ifndef pte_mkwrite_novma
166
-
static inline pte_t pte_mkwrite_novma(pte_t pte)
167
-
{
168
-
return __pte(pte_val(pte) | _PAGE_RW);
169
-
}
170
-
#endif
171
-
172
-
static inline pte_t pte_mkdirty(pte_t pte)
173
-
{
174
-
return __pte(pte_val(pte) | _PAGE_DIRTY);
175
-
}
176
-
177
-
static inline pte_t pte_mkyoung(pte_t pte)
178
-
{
179
-
return __pte(pte_val(pte) | _PAGE_ACCESSED);
180
-
}
181
-
182
-
#ifndef pte_wrprotect
183
-
static inline pte_t pte_wrprotect(pte_t pte)
184
-
{
185
-
return __pte(pte_val(pte) & ~_PAGE_RW);
186
-
}
187
-
#endif
188
-
189
-
#ifndef pte_mkexec
190
-
static inline pte_t pte_mkexec(pte_t pte)
191
-
{
192
-
return __pte(pte_val(pte) | _PAGE_EXEC);
193
-
}
194
-
#endif
195
163
196
164
#define pmd_none(pmd) (!pmd_val(pmd))
197
165
#define pmd_bad(pmd) (pmd_val(pmd) & _PMD_BAD)
···
159
207
static inline void pmd_clear(pmd_t *pmdp)
160
208
{
161
209
*pmdp = __pmd(0);
162
-
}
163
-
164
-
/*
165
-
* PTE updates. This function is called whenever an existing
166
-
* valid PTE is updated. This does -not- include set_pte_at()
167
-
* which nowadays only sets a new PTE.
168
-
*
169
-
* Depending on the type of MMU, we may need to use atomic updates
170
-
* and the PTE may be either 32 or 64 bit wide. In the later case,
171
-
* when using atomic updates, only the low part of the PTE is
172
-
* accessed atomically.
173
-
*
174
-
* In addition, on 44x, we also maintain a global flag indicating
175
-
* that an executable user mapping was modified, which is needed
176
-
* to properly flush the virtually tagged instruction cache of
177
-
* those implementations.
178
-
*
179
-
* On the 8xx, the page tables are a bit special. For 16k pages, we have
180
-
* 4 identical entries. For 512k pages, we have 128 entries as if it was
181
-
* 4k pages, but they are flagged as 512k pages for the hardware.
182
-
* For other page sizes, we have a single entry in the table.
183
-
*/
184
-
#ifdef CONFIG_PPC_8xx
185
-
static pmd_t *pmd_off(struct mm_struct *mm, unsigned long addr);
186
-
static int hugepd_ok(hugepd_t hpd);
187
-
188
-
static int number_of_cells_per_pte(pmd_t *pmd, pte_basic_t val, int huge)
189
-
{
190
-
if (!huge)
191
-
return PAGE_SIZE / SZ_4K;
192
-
else if (hugepd_ok(*((hugepd_t *)pmd)))
193
-
return 1;
194
-
else if (IS_ENABLED(CONFIG_PPC_4K_PAGES) && !(val & _PAGE_HUGE))
195
-
return SZ_16K / SZ_4K;
196
-
else
197
-
return SZ_512K / SZ_4K;
198
-
}
199
-
200
-
static inline pte_basic_t pte_update(struct mm_struct *mm, unsigned long addr, pte_t *p,
201
-
unsigned long clr, unsigned long set, int huge)
202
-
{
203
-
pte_basic_t *entry = (pte_basic_t *)p;
204
-
pte_basic_t old = pte_val(*p);
205
-
pte_basic_t new = (old & ~(pte_basic_t)clr) | set;
206
-
int num, i;
207
-
pmd_t *pmd = pmd_off(mm, addr);
208
-
209
-
num = number_of_cells_per_pte(pmd, new, huge);
210
-
211
-
for (i = 0; i < num; i += PAGE_SIZE / SZ_4K, new += PAGE_SIZE) {
212
-
*entry++ = new;
213
-
if (IS_ENABLED(CONFIG_PPC_16K_PAGES) && num != 1) {
214
-
*entry++ = new;
215
-
*entry++ = new;
216
-
*entry++ = new;
217
-
}
218
-
}
219
-
220
-
return old;
221
-
}
222
-
223
-
#ifdef CONFIG_PPC_16K_PAGES
224
-
#define ptep_get ptep_get
225
-
static inline pte_t ptep_get(pte_t *ptep)
226
-
{
227
-
pte_basic_t val = READ_ONCE(ptep->pte);
228
-
pte_t pte = {val, val, val, val};
229
-
230
-
return pte;
231
-
}
232
-
#endif /* CONFIG_PPC_16K_PAGES */
233
-
234
-
#else
235
-
static inline pte_basic_t pte_update(struct mm_struct *mm, unsigned long addr, pte_t *p,
236
-
unsigned long clr, unsigned long set, int huge)
237
-
{
238
-
pte_basic_t old = pte_val(*p);
239
-
pte_basic_t new = (old & ~(pte_basic_t)clr) | set;
240
-
241
-
*p = __pte(new);
242
-
243
-
#ifdef CONFIG_44x
244
-
if ((old & _PAGE_USER) && (old & _PAGE_EXEC))
245
-
icache_44x_need_flush = 1;
246
-
#endif
247
-
return old;
248
-
}
249
-
#endif
250
-
251
-
#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
252
-
static inline int __ptep_test_and_clear_young(struct mm_struct *mm,
253
-
unsigned long addr, pte_t *ptep)
254
-
{
255
-
unsigned long old;
256
-
old = pte_update(mm, addr, ptep, _PAGE_ACCESSED, 0, 0);
257
-
return (old & _PAGE_ACCESSED) != 0;
258
-
}
259
-
#define ptep_test_and_clear_young(__vma, __addr, __ptep) \
260
-
__ptep_test_and_clear_young((__vma)->vm_mm, __addr, __ptep)
261
-
262
-
#define __HAVE_ARCH_PTEP_GET_AND_CLEAR
263
-
static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long addr,
264
-
pte_t *ptep)
265
-
{
266
-
return __pte(pte_update(mm, addr, ptep, ~0, 0, 0));
267
-
}
268
-
269
-
#define __HAVE_ARCH_PTEP_SET_WRPROTECT
270
-
#ifndef ptep_set_wrprotect
271
-
static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr,
272
-
pte_t *ptep)
273
-
{
274
-
pte_update(mm, addr, ptep, _PAGE_RW, 0, 0);
275
-
}
276
-
#endif
277
-
278
-
#ifndef __ptep_set_access_flags
279
-
static inline void __ptep_set_access_flags(struct vm_area_struct *vma,
280
-
pte_t *ptep, pte_t entry,
281
-
unsigned long address,
282
-
int psize)
283
-
{
284
-
unsigned long set = pte_val(entry) &
285
-
(_PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_RW | _PAGE_EXEC);
286
-
int huge = psize > mmu_virtual_psize ? 1 : 0;
287
-
288
-
pte_update(vma->vm_mm, address, ptep, 0, set, huge);
289
-
290
-
flush_tlb_page(vma, address);
291
-
}
292
-
#endif
293
-
294
-
static inline int pte_young(pte_t pte)
295
-
{
296
-
return pte_val(pte) & _PAGE_ACCESSED;
297
210
}
298
211
299
212
/*
+3
-18
arch/powerpc/include/asm/nohash/32/pte-40x.h
+3
-18
arch/powerpc/include/asm/nohash/32/pte-40x.h
···
42
42
#define _PAGE_PRESENT 0x002 /* software: PTE contains a translation */
43
43
#define _PAGE_NO_CACHE 0x004 /* I: caching is inhibited */
44
44
#define _PAGE_WRITETHRU 0x008 /* W: caching is write-through */
45
-
#define _PAGE_USER 0x010 /* matches one of the zone permission bits */
45
+
#define _PAGE_READ 0x010 /* software: read permission */
46
46
#define _PAGE_SPECIAL 0x020 /* software: Special page */
47
47
#define _PAGE_DIRTY 0x080 /* software: dirty page */
48
-
#define _PAGE_RW 0x100 /* hardware: WR, anded with dirty in exception */
48
+
#define _PAGE_WRITE 0x100 /* hardware: WR, anded with dirty in exception */
49
49
#define _PAGE_EXEC 0x200 /* hardware: EX permission */
50
50
#define _PAGE_ACCESSED 0x400 /* software: R: page referenced */
51
51
···
54
54
55
55
/* cache related flags non existing on 40x */
56
56
#define _PAGE_COHERENT 0
57
-
58
-
#define _PAGE_KERNEL_RO 0
59
-
#define _PAGE_KERNEL_ROX _PAGE_EXEC
60
-
#define _PAGE_KERNEL_RW (_PAGE_DIRTY | _PAGE_RW)
61
-
#define _PAGE_KERNEL_RWX (_PAGE_DIRTY | _PAGE_RW | _PAGE_EXEC)
62
57
63
58
#define _PMD_PRESENT 0x400 /* PMD points to page of PTEs */
64
59
#define _PMD_PRESENT_MASK _PMD_PRESENT
···
64
69
65
70
#define _PTE_NONE_MASK 0
66
71
67
-
/* Until my rework is finished, 40x still needs atomic PTE updates */
68
-
#define PTE_ATOMIC_UPDATES 1
69
-
70
72
#define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED)
71
73
#define _PAGE_BASE (_PAGE_BASE_NC)
72
74
73
-
/* Permission masks used to generate the __P and __S table */
74
-
#define PAGE_NONE __pgprot(_PAGE_BASE)
75
-
#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW)
76
-
#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW | _PAGE_EXEC)
77
-
#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_USER)
78
-
#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
79
-
#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_USER)
80
-
#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
75
+
#include <asm/pgtable-masks.h>
81
76
82
77
#endif /* __KERNEL__ */
83
78
#endif /* _ASM_POWERPC_NOHASH_32_PTE_40x_H */
+4
-16
arch/powerpc/include/asm/nohash/32/pte-44x.h
+4
-16
arch/powerpc/include/asm/nohash/32/pte-44x.h
···
63
63
*/
64
64
65
65
#define _PAGE_PRESENT 0x00000001 /* S: PTE valid */
66
-
#define _PAGE_RW 0x00000002 /* S: Write permission */
66
+
#define _PAGE_WRITE 0x00000002 /* S: Write permission */
67
67
#define _PAGE_EXEC 0x00000004 /* H: Execute permission */
68
-
#define _PAGE_ACCESSED 0x00000008 /* S: Page referenced */
68
+
#define _PAGE_READ 0x00000008 /* S: Read permission */
69
69
#define _PAGE_DIRTY 0x00000010 /* S: Page dirty */
70
70
#define _PAGE_SPECIAL 0x00000020 /* S: Special page */
71
-
#define _PAGE_USER 0x00000040 /* S: User page */
71
+
#define _PAGE_ACCESSED 0x00000040 /* S: Page referenced */
72
72
#define _PAGE_ENDIAN 0x00000080 /* H: E bit */
73
73
#define _PAGE_GUARDED 0x00000100 /* H: G bit */
74
74
#define _PAGE_COHERENT 0x00000200 /* H: M bit */
···
77
77
78
78
/* No page size encoding in the linux PTE */
79
79
#define _PAGE_PSIZE 0
80
-
81
-
#define _PAGE_KERNEL_RO 0
82
-
#define _PAGE_KERNEL_ROX _PAGE_EXEC
83
-
#define _PAGE_KERNEL_RW (_PAGE_DIRTY | _PAGE_RW)
84
-
#define _PAGE_KERNEL_RWX (_PAGE_DIRTY | _PAGE_RW | _PAGE_EXEC)
85
80
86
81
/* TODO: Add large page lowmem mapping support */
87
82
#define _PMD_PRESENT 0
···
100
105
#define _PAGE_BASE (_PAGE_BASE_NC)
101
106
#endif
102
107
103
-
/* Permission masks used to generate the __P and __S table */
104
-
#define PAGE_NONE __pgprot(_PAGE_BASE)
105
-
#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW)
106
-
#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW | _PAGE_EXEC)
107
-
#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_USER)
108
-
#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
109
-
#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_USER)
110
-
#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
108
+
#include <asm/pgtable-masks.h>
111
109
112
110
#endif /* __KERNEL__ */
113
111
#endif /* _ASM_POWERPC_NOHASH_32_PTE_44x_H */
+4
-16
arch/powerpc/include/asm/nohash/32/pte-85xx.h
+4
-16
arch/powerpc/include/asm/nohash/32/pte-85xx.h
···
17
17
*/
18
18
19
19
/* Definitions for FSL Book-E Cores */
20
-
#define _PAGE_PRESENT 0x00001 /* S: PTE contains a translation */
21
-
#define _PAGE_USER 0x00002 /* S: User page (maps to UR) */
22
-
#define _PAGE_RW 0x00004 /* S: Write permission (SW) */
20
+
#define _PAGE_READ 0x00001 /* H: Read permission (SR) */
21
+
#define _PAGE_PRESENT 0x00002 /* S: PTE contains a translation */
22
+
#define _PAGE_WRITE 0x00004 /* S: Write permission (SW) */
23
23
#define _PAGE_DIRTY 0x00008 /* S: Page dirty */
24
24
#define _PAGE_EXEC 0x00010 /* H: SX permission */
25
25
#define _PAGE_ACCESSED 0x00020 /* S: Page referenced */
···
30
30
#define _PAGE_NO_CACHE 0x00200 /* H: I bit */
31
31
#define _PAGE_WRITETHRU 0x00400 /* H: W bit */
32
32
#define _PAGE_SPECIAL 0x00800 /* S: Special page */
33
-
34
-
#define _PAGE_KERNEL_RO 0
35
-
#define _PAGE_KERNEL_ROX _PAGE_EXEC
36
-
#define _PAGE_KERNEL_RW (_PAGE_DIRTY | _PAGE_RW)
37
-
#define _PAGE_KERNEL_RWX (_PAGE_DIRTY | _PAGE_RW | _PAGE_EXEC)
38
33
39
34
/* No page size encoding in the linux PTE */
40
35
#define _PAGE_PSIZE 0
···
56
61
#define _PAGE_BASE (_PAGE_BASE_NC)
57
62
#endif
58
63
59
-
/* Permission masks used to generate the __P and __S table */
60
-
#define PAGE_NONE __pgprot(_PAGE_BASE)
61
-
#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW)
62
-
#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW | _PAGE_EXEC)
63
-
#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_USER)
64
-
#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
65
-
#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_USER)
66
-
#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
64
+
#include <asm/pgtable-masks.h>
67
65
68
66
#endif /* __KERNEL__ */
69
67
#endif /* _ASM_POWERPC_NOHASH_32_PTE_FSL_85xx_H */
+63
-29
arch/powerpc/include/asm/nohash/32/pte-8xx.h
+63
-29
arch/powerpc/include/asm/nohash/32/pte-8xx.h
···
48
48
49
49
#define _PAGE_HUGE 0x0800 /* Copied to L1 PS bit 29 */
50
50
51
+
#define _PAGE_NAX (_PAGE_NA | _PAGE_EXEC)
52
+
#define _PAGE_ROX (_PAGE_RO | _PAGE_EXEC)
53
+
#define _PAGE_RW 0
54
+
#define _PAGE_RWX _PAGE_EXEC
55
+
51
56
/* cache related flags non existing on 8xx */
52
57
#define _PAGE_COHERENT 0
53
58
#define _PAGE_WRITETHRU 0
···
82
77
#define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_PSIZE)
83
78
#define _PAGE_BASE (_PAGE_BASE_NC)
84
79
85
-
/* Permission masks used to generate the __P and __S table */
86
-
#define PAGE_NONE __pgprot(_PAGE_BASE | _PAGE_NA)
87
-
#define PAGE_SHARED __pgprot(_PAGE_BASE)
88
-
#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_EXEC)
89
-
#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_RO)
90
-
#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_RO | _PAGE_EXEC)
91
-
#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_RO)
92
-
#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_RO | _PAGE_EXEC)
80
+
#include <asm/pgtable-masks.h>
93
81
94
82
#ifndef __ASSEMBLY__
95
83
static inline pte_t pte_wrprotect(pte_t pte)
···
112
114
}
113
115
114
116
#define pte_mkwrite_novma pte_mkwrite_novma
115
-
116
-
static inline bool pte_user(pte_t pte)
117
-
{
118
-
return !(pte_val(pte) & _PAGE_SH);
119
-
}
120
-
121
-
#define pte_user pte_user
122
-
123
-
static inline pte_t pte_mkprivileged(pte_t pte)
124
-
{
125
-
return __pte(pte_val(pte) | _PAGE_SH);
126
-
}
127
-
128
-
#define pte_mkprivileged pte_mkprivileged
129
-
130
-
static inline pte_t pte_mkuser(pte_t pte)
131
-
{
132
-
return __pte(pte_val(pte) & ~_PAGE_SH);
133
-
}
134
-
135
-
#define pte_mkuser pte_mkuser
136
117
137
118
static inline pte_t pte_mkhuge(pte_t pte)
138
119
{
···
163
186
}
164
187
165
188
#define pte_leaf_size pte_leaf_size
189
+
190
+
/*
191
+
* On the 8xx, the page tables are a bit special. For 16k pages, we have
192
+
* 4 identical entries. For 512k pages, we have 128 entries as if it was
193
+
* 4k pages, but they are flagged as 512k pages for the hardware.
194
+
* For other page sizes, we have a single entry in the table.
195
+
*/
196
+
static pmd_t *pmd_off(struct mm_struct *mm, unsigned long addr);
197
+
static int hugepd_ok(hugepd_t hpd);
198
+
199
+
static inline int number_of_cells_per_pte(pmd_t *pmd, pte_basic_t val, int huge)
200
+
{
201
+
if (!huge)
202
+
return PAGE_SIZE / SZ_4K;
203
+
else if (hugepd_ok(*((hugepd_t *)pmd)))
204
+
return 1;
205
+
else if (IS_ENABLED(CONFIG_PPC_4K_PAGES) && !(val & _PAGE_HUGE))
206
+
return SZ_16K / SZ_4K;
207
+
else
208
+
return SZ_512K / SZ_4K;
209
+
}
210
+
211
+
static inline pte_basic_t pte_update(struct mm_struct *mm, unsigned long addr, pte_t *p,
212
+
unsigned long clr, unsigned long set, int huge)
213
+
{
214
+
pte_basic_t *entry = (pte_basic_t *)p;
215
+
pte_basic_t old = pte_val(*p);
216
+
pte_basic_t new = (old & ~(pte_basic_t)clr) | set;
217
+
int num, i;
218
+
pmd_t *pmd = pmd_off(mm, addr);
219
+
220
+
num = number_of_cells_per_pte(pmd, new, huge);
221
+
222
+
for (i = 0; i < num; i += PAGE_SIZE / SZ_4K, new += PAGE_SIZE) {
223
+
*entry++ = new;
224
+
if (IS_ENABLED(CONFIG_PPC_16K_PAGES) && num != 1) {
225
+
*entry++ = new;
226
+
*entry++ = new;
227
+
*entry++ = new;
228
+
}
229
+
}
230
+
231
+
return old;
232
+
}
233
+
234
+
#define pte_update pte_update
235
+
236
+
#ifdef CONFIG_PPC_16K_PAGES
237
+
#define ptep_get ptep_get
238
+
static inline pte_t ptep_get(pte_t *ptep)
239
+
{
240
+
pte_basic_t val = READ_ONCE(ptep->pte);
241
+
pte_t pte = {val, val, val, val};
242
+
243
+
return pte;
244
+
}
245
+
#endif /* CONFIG_PPC_16K_PAGES */
166
246
167
247
#endif
168
248
+3
-117
arch/powerpc/include/asm/nohash/64/pgtable.h
+3
-117
arch/powerpc/include/asm/nohash/64/pgtable.h
···
57
57
#define IOREMAP_START (ioremap_bot)
58
58
#define IOREMAP_END (KERN_IO_START + KERN_IO_SIZE - FIXADDR_SIZE)
59
59
#define FIXADDR_SIZE SZ_32M
60
+
#define FIXADDR_TOP (IOREMAP_END + FIXADDR_SIZE)
60
61
61
62
/*
62
63
* Defines the address of the vmemap area, in its own region on
···
75
74
76
75
#define PTE_RPN_MASK (~((1UL << PTE_RPN_SHIFT) - 1))
77
76
78
-
/*
79
-
* _PAGE_CHG_MASK masks of bits that are to be preserved across
80
-
* pgprot changes.
81
-
*/
82
-
#define _PAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_SPECIAL)
83
-
84
77
#define H_PAGE_4K_PFN 0
85
78
86
79
#ifndef __ASSEMBLY__
87
80
/* pte_clear moved to later in this file */
88
-
89
-
static inline pte_t pte_mkwrite_novma(pte_t pte)
90
-
{
91
-
return __pte(pte_val(pte) | _PAGE_RW);
92
-
}
93
-
94
-
static inline pte_t pte_mkdirty(pte_t pte)
95
-
{
96
-
return __pte(pte_val(pte) | _PAGE_DIRTY);
97
-
}
98
-
99
-
static inline pte_t pte_mkyoung(pte_t pte)
100
-
{
101
-
return __pte(pte_val(pte) | _PAGE_ACCESSED);
102
-
}
103
-
104
-
static inline pte_t pte_wrprotect(pte_t pte)
105
-
{
106
-
return __pte(pte_val(pte) & ~_PAGE_RW);
107
-
}
108
81
109
82
#define PMD_BAD_BITS (PTE_TABLE_SIZE-1)
110
83
#define PUD_BAD_BITS (PMD_TABLE_SIZE-1)
···
145
170
*p4dp = __p4d(val);
146
171
}
147
172
148
-
/* Atomic PTE updates */
149
-
static inline unsigned long pte_update(struct mm_struct *mm,
150
-
unsigned long addr,
151
-
pte_t *ptep, unsigned long clr,
152
-
unsigned long set,
153
-
int huge)
154
-
{
155
-
unsigned long old = pte_val(*ptep);
156
-
*ptep = __pte((old & ~clr) | set);
157
-
158
-
/* huge pages use the old page table lock */
159
-
if (!huge)
160
-
assert_pte_locked(mm, addr);
161
-
162
-
return old;
163
-
}
164
-
165
-
static inline int pte_young(pte_t pte)
166
-
{
167
-
return pte_val(pte) & _PAGE_ACCESSED;
168
-
}
169
-
170
-
static inline int __ptep_test_and_clear_young(struct mm_struct *mm,
171
-
unsigned long addr, pte_t *ptep)
172
-
{
173
-
unsigned long old;
174
-
175
-
if (!pte_young(*ptep))
176
-
return 0;
177
-
old = pte_update(mm, addr, ptep, _PAGE_ACCESSED, 0, 0);
178
-
return (old & _PAGE_ACCESSED) != 0;
179
-
}
180
-
#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
181
-
#define ptep_test_and_clear_young(__vma, __addr, __ptep) \
182
-
({ \
183
-
int __r; \
184
-
__r = __ptep_test_and_clear_young((__vma)->vm_mm, __addr, __ptep); \
185
-
__r; \
186
-
})
187
-
188
-
#define __HAVE_ARCH_PTEP_SET_WRPROTECT
189
-
static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr,
190
-
pte_t *ptep)
191
-
{
192
-
193
-
if ((pte_val(*ptep) & _PAGE_RW) == 0)
194
-
return;
195
-
196
-
pte_update(mm, addr, ptep, _PAGE_RW, 0, 0);
197
-
}
198
-
199
173
#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT
200
174
static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,
201
175
unsigned long addr, pte_t *ptep)
202
176
{
203
-
if ((pte_val(*ptep) & _PAGE_RW) == 0)
204
-
return;
205
-
206
-
pte_update(mm, addr, ptep, _PAGE_RW, 0, 1);
177
+
pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 1);
207
178
}
208
179
209
180
#define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
210
181
#define ptep_clear_flush_young(__vma, __address, __ptep) \
211
182
({ \
212
-
int __young = __ptep_test_and_clear_young((__vma)->vm_mm, __address, \
213
-
__ptep); \
183
+
int __young = ptep_test_and_clear_young(__vma, __address, __ptep);\
214
184
__young; \
215
185
})
216
186
217
-
#define __HAVE_ARCH_PTEP_GET_AND_CLEAR
218
-
static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
219
-
unsigned long addr, pte_t *ptep)
220
-
{
221
-
unsigned long old = pte_update(mm, addr, ptep, ~0UL, 0, 0);
222
-
return __pte(old);
223
-
}
224
-
225
-
static inline void pte_clear(struct mm_struct *mm, unsigned long addr,
226
-
pte_t * ptep)
227
-
{
228
-
pte_update(mm, addr, ptep, ~0UL, 0, 0);
229
-
}
230
-
231
-
232
-
/* Set the dirty and/or accessed bits atomically in a linux PTE */
233
-
static inline void __ptep_set_access_flags(struct vm_area_struct *vma,
234
-
pte_t *ptep, pte_t entry,
235
-
unsigned long address,
236
-
int psize)
237
-
{
238
-
unsigned long bits = pte_val(entry) &
239
-
(_PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_RW | _PAGE_EXEC);
240
-
241
-
unsigned long old = pte_val(*ptep);
242
-
*ptep = __pte(old | bits);
243
-
244
-
flush_tlb_page(vma, address);
245
-
}
246
-
247
-
#define pte_ERROR(e) \
248
-
pr_err("%s:%d: bad pte %08lx.\n", __FILE__, __LINE__, pte_val(e))
249
187
#define pmd_ERROR(e) \
250
188
pr_err("%s:%d: bad pmd %08lx.\n", __FILE__, __LINE__, pmd_val(e))
251
189
#define pgd_ERROR(e) \
···
198
310
/* We borrow MSB 56 (LSB 7) to store the exclusive marker in swap PTEs. */
199
311
#define _PAGE_SWP_EXCLUSIVE 0x80
200
312
201
-
int map_kernel_page(unsigned long ea, unsigned long pa, pgprot_t prot);
202
-
void unmap_kernel_page(unsigned long va);
203
313
extern int __meminit vmemmap_create_mapping(unsigned long start,
204
314
unsigned long page_size,
205
315
unsigned long phys);
+153
-70
arch/powerpc/include/asm/nohash/pgtable.h
+153
-70
arch/powerpc/include/asm/nohash/pgtable.h
···
2
2
#ifndef _ASM_POWERPC_NOHASH_PGTABLE_H
3
3
#define _ASM_POWERPC_NOHASH_PGTABLE_H
4
4
5
+
#ifndef __ASSEMBLY__
6
+
static inline pte_basic_t pte_update(struct mm_struct *mm, unsigned long addr, pte_t *p,
7
+
unsigned long clr, unsigned long set, int huge);
8
+
#endif
9
+
5
10
#if defined(CONFIG_PPC64)
6
11
#include <asm/nohash/64/pgtable.h>
7
12
#else
8
13
#include <asm/nohash/32/pgtable.h>
9
14
#endif
15
+
16
+
/*
17
+
* _PAGE_CHG_MASK masks of bits that are to be preserved across
18
+
* pgprot changes.
19
+
*/
20
+
#define _PAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_SPECIAL)
10
21
11
22
/* Permission masks used for kernel mappings */
12
23
#define PAGE_KERNEL __pgprot(_PAGE_BASE | _PAGE_KERNEL_RW)
···
29
18
30
19
#ifndef __ASSEMBLY__
31
20
21
+
extern int icache_44x_need_flush;
22
+
23
+
/*
24
+
* PTE updates. This function is called whenever an existing
25
+
* valid PTE is updated. This does -not- include set_pte_at()
26
+
* which nowadays only sets a new PTE.
27
+
*
28
+
* Depending on the type of MMU, we may need to use atomic updates
29
+
* and the PTE may be either 32 or 64 bit wide. In the later case,
30
+
* when using atomic updates, only the low part of the PTE is
31
+
* accessed atomically.
32
+
*
33
+
* In addition, on 44x, we also maintain a global flag indicating
34
+
* that an executable user mapping was modified, which is needed
35
+
* to properly flush the virtually tagged instruction cache of
36
+
* those implementations.
37
+
*/
38
+
#ifndef pte_update
39
+
static inline pte_basic_t pte_update(struct mm_struct *mm, unsigned long addr, pte_t *p,
40
+
unsigned long clr, unsigned long set, int huge)
41
+
{
42
+
pte_basic_t old = pte_val(*p);
43
+
pte_basic_t new = (old & ~(pte_basic_t)clr) | set;
44
+
45
+
if (new == old)
46
+
return old;
47
+
48
+
*p = __pte(new);
49
+
50
+
if (IS_ENABLED(CONFIG_44x) && !is_kernel_addr(addr) && (old & _PAGE_EXEC))
51
+
icache_44x_need_flush = 1;
52
+
53
+
/* huge pages use the old page table lock */
54
+
if (!huge)
55
+
assert_pte_locked(mm, addr);
56
+
57
+
return old;
58
+
}
59
+
#endif
60
+
61
+
static inline int ptep_test_and_clear_young(struct vm_area_struct *vma,
62
+
unsigned long addr, pte_t *ptep)
63
+
{
64
+
unsigned long old;
65
+
66
+
old = pte_update(vma->vm_mm, addr, ptep, _PAGE_ACCESSED, 0, 0);
67
+
68
+
return (old & _PAGE_ACCESSED) != 0;
69
+
}
70
+
#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
71
+
72
+
#ifndef ptep_set_wrprotect
73
+
static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr,
74
+
pte_t *ptep)
75
+
{
76
+
pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 0);
77
+
}
78
+
#endif
79
+
#define __HAVE_ARCH_PTEP_SET_WRPROTECT
80
+
81
+
static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long addr,
82
+
pte_t *ptep)
83
+
{
84
+
return __pte(pte_update(mm, addr, ptep, ~0UL, 0, 0));
85
+
}
86
+
#define __HAVE_ARCH_PTEP_GET_AND_CLEAR
87
+
88
+
static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
89
+
{
90
+
pte_update(mm, addr, ptep, ~0UL, 0, 0);
91
+
}
92
+
93
+
/* Set the dirty and/or accessed bits atomically in a linux PTE */
94
+
#ifndef __ptep_set_access_flags
95
+
static inline void __ptep_set_access_flags(struct vm_area_struct *vma,
96
+
pte_t *ptep, pte_t entry,
97
+
unsigned long address,
98
+
int psize)
99
+
{
100
+
unsigned long set = pte_val(entry) &
101
+
(_PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_RW | _PAGE_EXEC);
102
+
int huge = psize > mmu_virtual_psize ? 1 : 0;
103
+
104
+
pte_update(vma->vm_mm, address, ptep, 0, set, huge);
105
+
106
+
flush_tlb_page(vma, address);
107
+
}
108
+
#endif
109
+
32
110
/* Generic accessors to PTE bits */
111
+
#ifndef pte_mkwrite_novma
112
+
static inline pte_t pte_mkwrite_novma(pte_t pte)
113
+
{
114
+
/*
115
+
* write implies read, hence set both
116
+
*/
117
+
return __pte(pte_val(pte) | _PAGE_RW);
118
+
}
119
+
#endif
120
+
121
+
static inline pte_t pte_mkdirty(pte_t pte)
122
+
{
123
+
return __pte(pte_val(pte) | _PAGE_DIRTY);
124
+
}
125
+
126
+
static inline pte_t pte_mkyoung(pte_t pte)
127
+
{
128
+
return __pte(pte_val(pte) | _PAGE_ACCESSED);
129
+
}
130
+
131
+
#ifndef pte_wrprotect
132
+
static inline pte_t pte_wrprotect(pte_t pte)
133
+
{
134
+
return __pte(pte_val(pte) & ~_PAGE_WRITE);
135
+
}
136
+
#endif
137
+
138
+
#ifndef pte_mkexec
139
+
static inline pte_t pte_mkexec(pte_t pte)
140
+
{
141
+
return __pte(pte_val(pte) | _PAGE_EXEC);
142
+
}
143
+
#endif
144
+
33
145
#ifndef pte_write
34
146
static inline int pte_write(pte_t pte)
35
147
{
36
-
return pte_val(pte) & _PAGE_RW;
148
+
return pte_val(pte) & _PAGE_WRITE;
37
149
}
38
-
#endif
39
-
#ifndef pte_read
40
-
static inline int pte_read(pte_t pte) { return 1; }
41
150
#endif
42
151
static inline int pte_dirty(pte_t pte) { return pte_val(pte) & _PAGE_DIRTY; }
43
152
static inline int pte_special(pte_t pte) { return pte_val(pte) & _PAGE_SPECIAL; }
···
165
34
static inline bool pte_hashpte(pte_t pte) { return false; }
166
35
static inline bool pte_ci(pte_t pte) { return pte_val(pte) & _PAGE_NO_CACHE; }
167
36
static inline bool pte_exec(pte_t pte) { return pte_val(pte) & _PAGE_EXEC; }
168
-
169
-
#ifdef CONFIG_NUMA_BALANCING
170
-
/*
171
-
* These work without NUMA balancing but the kernel does not care. See the
172
-
* comment in include/linux/pgtable.h . On powerpc, this will only
173
-
* work for user pages and always return true for kernel pages.
174
-
*/
175
-
static inline int pte_protnone(pte_t pte)
176
-
{
177
-
return pte_present(pte) && !pte_user(pte);
178
-
}
179
-
180
-
static inline int pmd_protnone(pmd_t pmd)
181
-
{
182
-
return pte_protnone(pmd_pte(pmd));
183
-
}
184
-
#endif /* CONFIG_NUMA_BALANCING */
185
37
186
38
static inline int pte_present(pte_t pte)
187
39
{
···
176
62
return pte_val(pte) & _PAGE_PRESENT;
177
63
}
178
64
179
-
/*
180
-
* Don't just check for any non zero bits in __PAGE_USER, since for book3e
181
-
* and PTE_64BIT, PAGE_KERNEL_X contains _PAGE_BAP_SR which is also in
182
-
* _PAGE_USER. Need to explicitly match _PAGE_BAP_UR bit in that case too.
183
-
*/
184
-
#ifndef pte_user
185
-
static inline bool pte_user(pte_t pte)
65
+
static inline int pte_young(pte_t pte)
186
66
{
187
-
return (pte_val(pte) & _PAGE_USER) == _PAGE_USER;
67
+
return pte_val(pte) & _PAGE_ACCESSED;
68
+
}
69
+
70
+
/*
71
+
* Don't just check for any non zero bits in __PAGE_READ, since for book3e
72
+
* and PTE_64BIT, PAGE_KERNEL_X contains _PAGE_BAP_SR which is also in
73
+
* _PAGE_READ. Need to explicitly match _PAGE_BAP_UR bit in that case too.
74
+
*/
75
+
#ifndef pte_read
76
+
static inline bool pte_read(pte_t pte)
77
+
{
78
+
return (pte_val(pte) & _PAGE_READ) == _PAGE_READ;
188
79
}
189
80
#endif
190
81
···
201
82
static inline bool pte_access_permitted(pte_t pte, bool write)
202
83
{
203
84
/*
204
-
* A read-only access is controlled by _PAGE_USER bit.
205
-
* We have _PAGE_READ set for WRITE and EXECUTE
85
+
* A read-only access is controlled by _PAGE_READ bit.
86
+
* We have _PAGE_READ set for WRITE
206
87
*/
207
-
if (!pte_present(pte) || !pte_user(pte) || !pte_read(pte))
88
+
if (!pte_present(pte) || !pte_read(pte))
208
89
return false;
209
90
210
91
if (write && !pte_write(pte))
···
248
129
static inline pte_t pte_mkhuge(pte_t pte)
249
130
{
250
131
return __pte(pte_val(pte));
251
-
}
252
-
#endif
253
-
254
-
#ifndef pte_mkprivileged
255
-
static inline pte_t pte_mkprivileged(pte_t pte)
256
-
{
257
-
return __pte(pte_val(pte) & ~_PAGE_USER);
258
-
}
259
-
#endif
260
-
261
-
#ifndef pte_mkuser
262
-
static inline pte_t pte_mkuser(pte_t pte)
263
-
{
264
-
return __pte(pte_val(pte) | _PAGE_USER);
265
132
}
266
133
#endif
267
134
···
312
207
mb();
313
208
}
314
209
315
-
316
-
#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS
317
-
extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address,
318
-
pte_t *ptep, pte_t entry, int dirty);
319
-
320
210
/*
321
211
* Macro to mark a page protection value as "uncacheable".
322
212
*/
···
340
240
341
241
#define pgprot_writecombine pgprot_noncached_wc
342
242
343
-
struct file;
344
-
extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
345
-
unsigned long size, pgprot_t vma_prot);
346
-
#define __HAVE_PHYS_MEM_ACCESS_PROT
347
-
348
243
#ifdef CONFIG_HUGETLB_PAGE
349
244
static inline int hugepd_ok(hugepd_t hpd)
350
245
{
···
364
269
#define is_hugepd(hpd) (hugepd_ok(hpd))
365
270
#endif
366
271
367
-
/*
368
-
* This gets called at the end of handling a page fault, when
369
-
* the kernel has put a new PTE into the page table for the process.
370
-
* We use it to ensure coherency between the i-cache and d-cache
371
-
* for the page which has just been mapped in.
372
-
*/
373
-
#if defined(CONFIG_PPC_E500) && defined(CONFIG_HUGETLB_PAGE)
374
-
void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma,
375
-
unsigned long address, pte_t *ptep, unsigned int nr);
376
-
#else
377
-
static inline void update_mmu_cache_range(struct vm_fault *vmf,
378
-
struct vm_area_struct *vma, unsigned long address,
379
-
pte_t *ptep, unsigned int nr) {}
380
-
#endif
272
+
int map_kernel_page(unsigned long va, phys_addr_t pa, pgprot_t prot);
273
+
void unmap_kernel_page(unsigned long va);
381
274
382
275
#endif /* __ASSEMBLY__ */
383
276
#endif
+12
-29
arch/powerpc/include/asm/nohash/pte-e500.h
+12
-29
arch/powerpc/include/asm/nohash/pte-e500.h
···
48
48
49
49
/* "Higher level" linux bit combinations */
50
50
#define _PAGE_EXEC (_PAGE_BAP_SX | _PAGE_BAP_UX) /* .. and was cache cleaned */
51
-
#define _PAGE_RW (_PAGE_BAP_SW | _PAGE_BAP_UW) /* User write permission */
51
+
#define _PAGE_READ (_PAGE_BAP_SR | _PAGE_BAP_UR) /* User read permission */
52
+
#define _PAGE_WRITE (_PAGE_BAP_SW | _PAGE_BAP_UW) /* User write permission */
53
+
52
54
#define _PAGE_KERNEL_RW (_PAGE_BAP_SW | _PAGE_BAP_SR | _PAGE_DIRTY)
53
55
#define _PAGE_KERNEL_RO (_PAGE_BAP_SR)
54
56
#define _PAGE_KERNEL_RWX (_PAGE_BAP_SW | _PAGE_BAP_SR | _PAGE_DIRTY | _PAGE_BAP_SX)
55
57
#define _PAGE_KERNEL_ROX (_PAGE_BAP_SR | _PAGE_BAP_SX)
56
-
#define _PAGE_USER (_PAGE_BAP_UR | _PAGE_BAP_SR) /* Can be read */
57
-
#define _PAGE_PRIVILEGED (_PAGE_BAP_SR)
58
+
59
+
#define _PAGE_NA 0
60
+
#define _PAGE_NAX _PAGE_BAP_UX
61
+
#define _PAGE_RO _PAGE_READ
62
+
#define _PAGE_ROX (_PAGE_READ | _PAGE_BAP_UX)
63
+
#define _PAGE_RW (_PAGE_READ | _PAGE_WRITE)
64
+
#define _PAGE_RWX (_PAGE_READ | _PAGE_WRITE | _PAGE_BAP_UX)
58
65
59
66
#define _PAGE_SPECIAL _PAGE_SW0
60
67
···
96
89
#define _PAGE_BASE (_PAGE_BASE_NC)
97
90
#endif
98
91
99
-
/* Permission masks used to generate the __P and __S table */
100
-
#define PAGE_NONE __pgprot(_PAGE_BASE)
101
-
#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW)
102
-
#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW | _PAGE_BAP_UX)
103
-
#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_USER)
104
-
#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_BAP_UX)
105
-
#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_USER)
106
-
#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_BAP_UX)
92
+
#include <asm/pgtable-masks.h>
107
93
108
94
#ifndef __ASSEMBLY__
109
-
static inline pte_t pte_mkprivileged(pte_t pte)
110
-
{
111
-
return __pte((pte_val(pte) & ~_PAGE_USER) | _PAGE_PRIVILEGED);
112
-
}
113
-
114
-
#define pte_mkprivileged pte_mkprivileged
115
-
116
-
static inline pte_t pte_mkuser(pte_t pte)
117
-
{
118
-
return __pte((pte_val(pte) & ~_PAGE_PRIVILEGED) | _PAGE_USER);
119
-
}
120
-
121
-
#define pte_mkuser pte_mkuser
122
-
123
95
static inline pte_t pte_mkexec(pte_t pte)
124
96
{
125
-
if (pte_val(pte) & _PAGE_BAP_UR)
126
-
return __pte((pte_val(pte) & ~_PAGE_BAP_SX) | _PAGE_BAP_UX);
127
-
else
128
-
return __pte((pte_val(pte) & ~_PAGE_BAP_UX) | _PAGE_BAP_SX);
97
+
return __pte((pte_val(pte) & ~_PAGE_BAP_SX) | _PAGE_BAP_UX);
129
98
}
130
99
#define pte_mkexec pte_mkexec
131
100
+1
-1
arch/powerpc/include/asm/opal.h
+1
-1
arch/powerpc/include/asm/opal.h
···
227
227
uint64_t data);
228
228
int64_t opal_pci_poll2(uint64_t id, uint64_t data);
229
229
230
-
int64_t opal_int_get_xirr(uint32_t *out_xirr, bool just_poll);
230
+
int64_t opal_int_get_xirr(__be32 *out_xirr, bool just_poll);
231
231
int64_t opal_int_set_cppr(uint8_t cppr);
232
232
int64_t opal_int_eoi(uint32_t xirr);
233
233
int64_t opal_int_set_mfrr(uint32_t cpu, uint8_t mfrr);
+44
-3
arch/powerpc/include/asm/paravirt.h
+44
-3
arch/powerpc/include/asm/paravirt.h
···
71
71
{
72
72
plpar_hcall_norets_notrace(H_CONFER, -1, 0);
73
73
}
74
+
75
+
static inline bool is_vcpu_idle(int vcpu)
76
+
{
77
+
return lppaca_of(vcpu).idle;
78
+
}
74
79
#else
75
80
static inline bool is_shared_processor(void)
76
81
{
···
105
100
___bad_prod_cpu(); /* This would be a bug */
106
101
}
107
102
103
+
static inline bool is_vcpu_idle(int vcpu)
104
+
{
105
+
return false;
106
+
}
108
107
#endif
109
108
110
109
#define vcpu_is_preempted vcpu_is_preempted
···
130
121
if (!is_shared_processor())
131
122
return false;
132
123
124
+
/*
125
+
* If the hypervisor has dispatched the target CPU on a physical
126
+
* processor, then the target CPU is definitely not preempted.
127
+
*/
128
+
if (!(yield_count_of(cpu) & 1))
129
+
return false;
130
+
131
+
/*
132
+
* If the target CPU has yielded to Hypervisor but OS has not
133
+
* requested idle then the target CPU is definitely preempted.
134
+
*/
135
+
if (!is_vcpu_idle(cpu))
136
+
return true;
137
+
133
138
#ifdef CONFIG_PPC_SPLPAR
134
139
if (!is_kvm_guest()) {
135
-
int first_cpu;
140
+
int first_cpu, i;
136
141
137
142
/*
138
143
* The result of vcpu_is_preempted() is used in a
···
172
149
*/
173
150
if (cpu_first_thread_sibling(cpu) == first_cpu)
174
151
return false;
152
+
153
+
/*
154
+
* If any of the threads of the target CPU's core are not
155
+
* preempted or ceded, then consider target CPU to be
156
+
* non-preempted.
157
+
*/
158
+
first_cpu = cpu_first_thread_sibling(cpu);
159
+
for (i = first_cpu; i < first_cpu + threads_per_core; i++) {
160
+
if (i == cpu)
161
+
continue;
162
+
if (!(yield_count_of(i) & 1))
163
+
return false;
164
+
if (!is_vcpu_idle(i))
165
+
return true;
166
+
}
175
167
}
176
168
#endif
177
169
178
-
if (yield_count_of(cpu) & 1)
179
-
return true;
170
+
/*
171
+
* None of the threads in target CPU's core are running but none of
172
+
* them were preempted too. Hence assume the target CPU to be
173
+
* non-preempted.
174
+
*/
180
175
return false;
181
176
}
182
177
+32
arch/powerpc/include/asm/pgtable-masks.h
+32
arch/powerpc/include/asm/pgtable-masks.h
···
1
+
/* SPDX-License-Identifier: GPL-2.0 */
2
+
#ifndef _ASM_POWERPC_PGTABLE_MASKS_H
3
+
#define _ASM_POWERPC_PGTABLE_MASKS_H
4
+
5
+
#ifndef _PAGE_NA
6
+
#define _PAGE_NA 0
7
+
#define _PAGE_NAX _PAGE_EXEC
8
+
#define _PAGE_RO _PAGE_READ
9
+
#define _PAGE_ROX (_PAGE_READ | _PAGE_EXEC)
10
+
#define _PAGE_RW (_PAGE_READ | _PAGE_WRITE)
11
+
#define _PAGE_RWX (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC)
12
+
#endif
13
+
14
+
/* Permission flags for kernel mappings */
15
+
#ifndef _PAGE_KERNEL_RO
16
+
#define _PAGE_KERNEL_RO _PAGE_RO
17
+
#define _PAGE_KERNEL_ROX _PAGE_ROX
18
+
#define _PAGE_KERNEL_RW (_PAGE_RW | _PAGE_DIRTY)
19
+
#define _PAGE_KERNEL_RWX (_PAGE_RWX | _PAGE_DIRTY)
20
+
#endif
21
+
22
+
/* Permission masks used to generate the __P and __S table */
23
+
#define PAGE_NONE __pgprot(_PAGE_BASE | _PAGE_NA)
24
+
#define PAGE_EXECONLY_X __pgprot(_PAGE_BASE | _PAGE_NAX)
25
+
#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_RW)
26
+
#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_RWX)
27
+
#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_RO)
28
+
#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_ROX)
29
+
#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_RO)
30
+
#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_ROX)
31
+
32
+
#endif /* _ASM_POWERPC_PGTABLE_MASKS_H */
+35
arch/powerpc/include/asm/pgtable.h
+35
arch/powerpc/include/asm/pgtable.h
···
71
71
return __pgprot(pte_flags);
72
72
}
73
73
74
+
static inline pgprot_t pgprot_nx(pgprot_t prot)
75
+
{
76
+
return pte_pgprot(pte_exprotect(__pte(pgprot_val(prot))));
77
+
}
78
+
#define pgprot_nx pgprot_nx
79
+
74
80
#ifndef pmd_page_vaddr
75
81
static inline const void *pmd_page_vaddr(pmd_t pmd)
76
82
{
···
115
109
#else
116
110
static inline void mark_initmem_nx(void) { }
117
111
#endif
112
+
113
+
#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS
114
+
int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address,
115
+
pte_t *ptep, pte_t entry, int dirty);
116
+
117
+
struct file;
118
+
pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
119
+
unsigned long size, pgprot_t vma_prot);
120
+
#define __HAVE_PHYS_MEM_ACCESS_PROT
121
+
122
+
void __update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep);
123
+
124
+
/*
125
+
* This gets called at the end of handling a page fault, when
126
+
* the kernel has put a new PTE into the page table for the process.
127
+
* We use it to ensure coherency between the i-cache and d-cache
128
+
* for the page which has just been mapped in.
129
+
* On machines which use an MMU hash table, we use this to put a
130
+
* corresponding HPTE into the hash table ahead of time, instead of
131
+
* waiting for the inevitable extra hash-table miss exception.
132
+
*/
133
+
static inline void update_mmu_cache_range(struct vm_fault *vmf,
134
+
struct vm_area_struct *vma, unsigned long address,
135
+
pte_t *ptep, unsigned int nr)
136
+
{
137
+
if ((mmu_has_feature(MMU_FTR_HPTE_TABLE) && !radix_enabled()) ||
138
+
(IS_ENABLED(CONFIG_PPC_E500) && IS_ENABLED(CONFIG_HUGETLB_PAGE)))
139
+
__update_mmu_cache(vma, address, ptep);
140
+
}
118
141
119
142
/*
120
143
* When used, PTE_FRAG_NR is defined in subarch pgtable.h
+265
-2
arch/powerpc/include/asm/plpar_wrappers.h
+265
-2
arch/powerpc/include/asm/plpar_wrappers.h
···
6
6
7
7
#include <linux/string.h>
8
8
#include <linux/irqflags.h>
9
+
#include <linux/delay.h>
9
10
10
11
#include <asm/hvcall.h>
11
12
#include <asm/paca.h>
···
344
343
return rc;
345
344
}
346
345
346
+
static inline long plpar_guest_create(unsigned long flags, unsigned long *guest_id)
347
+
{
348
+
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
349
+
unsigned long token;
350
+
long rc;
351
+
352
+
token = -1UL;
353
+
do {
354
+
rc = plpar_hcall(H_GUEST_CREATE, retbuf, flags, token);
355
+
if (rc == H_SUCCESS)
356
+
*guest_id = retbuf[0];
357
+
358
+
if (rc == H_BUSY) {
359
+
token = retbuf[0];
360
+
cond_resched();
361
+
}
362
+
363
+
if (H_IS_LONG_BUSY(rc)) {
364
+
token = retbuf[0];
365
+
msleep(get_longbusy_msecs(rc));
366
+
rc = H_BUSY;
367
+
}
368
+
369
+
} while (rc == H_BUSY);
370
+
371
+
return rc;
372
+
}
373
+
374
+
static inline long plpar_guest_create_vcpu(unsigned long flags,
375
+
unsigned long guest_id,
376
+
unsigned long vcpu_id)
377
+
{
378
+
long rc;
379
+
380
+
do {
381
+
rc = plpar_hcall_norets(H_GUEST_CREATE_VCPU, 0, guest_id, vcpu_id);
382
+
383
+
if (rc == H_BUSY)
384
+
cond_resched();
385
+
386
+
if (H_IS_LONG_BUSY(rc)) {
387
+
msleep(get_longbusy_msecs(rc));
388
+
rc = H_BUSY;
389
+
}
390
+
391
+
} while (rc == H_BUSY);
392
+
393
+
return rc;
394
+
}
395
+
396
+
static inline long plpar_guest_set_state(unsigned long flags,
397
+
unsigned long guest_id,
398
+
unsigned long vcpu_id,
399
+
unsigned long data_buffer,
400
+
unsigned long data_size,
401
+
unsigned long *failed_index)
402
+
{
403
+
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
404
+
long rc;
405
+
406
+
while (true) {
407
+
rc = plpar_hcall(H_GUEST_SET_STATE, retbuf, flags, guest_id,
408
+
vcpu_id, data_buffer, data_size);
409
+
410
+
if (rc == H_BUSY) {
411
+
cpu_relax();
412
+
continue;
413
+
}
414
+
415
+
if (H_IS_LONG_BUSY(rc)) {
416
+
mdelay(get_longbusy_msecs(rc));
417
+
continue;
418
+
}
419
+
420
+
if (rc == H_INVALID_ELEMENT_ID)
421
+
*failed_index = retbuf[0];
422
+
else if (rc == H_INVALID_ELEMENT_SIZE)
423
+
*failed_index = retbuf[0];
424
+
else if (rc == H_INVALID_ELEMENT_VALUE)
425
+
*failed_index = retbuf[0];
426
+
427
+
break;
428
+
}
429
+
430
+
return rc;
431
+
}
432
+
433
+
static inline long plpar_guest_get_state(unsigned long flags,
434
+
unsigned long guest_id,
435
+
unsigned long vcpu_id,
436
+
unsigned long data_buffer,
437
+
unsigned long data_size,
438
+
unsigned long *failed_index)
439
+
{
440
+
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
441
+
long rc;
442
+
443
+
while (true) {
444
+
rc = plpar_hcall(H_GUEST_GET_STATE, retbuf, flags, guest_id,
445
+
vcpu_id, data_buffer, data_size);
446
+
447
+
if (rc == H_BUSY) {
448
+
cpu_relax();
449
+
continue;
450
+
}
451
+
452
+
if (H_IS_LONG_BUSY(rc)) {
453
+
mdelay(get_longbusy_msecs(rc));
454
+
continue;
455
+
}
456
+
457
+
if (rc == H_INVALID_ELEMENT_ID)
458
+
*failed_index = retbuf[0];
459
+
else if (rc == H_INVALID_ELEMENT_SIZE)
460
+
*failed_index = retbuf[0];
461
+
else if (rc == H_INVALID_ELEMENT_VALUE)
462
+
*failed_index = retbuf[0];
463
+
464
+
break;
465
+
}
466
+
467
+
return rc;
468
+
}
469
+
470
+
static inline long plpar_guest_run_vcpu(unsigned long flags, unsigned long guest_id,
471
+
unsigned long vcpu_id, int *trap,
472
+
unsigned long *failed_index)
473
+
{
474
+
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
475
+
long rc;
476
+
477
+
rc = plpar_hcall(H_GUEST_RUN_VCPU, retbuf, flags, guest_id, vcpu_id);
478
+
if (rc == H_SUCCESS)
479
+
*trap = retbuf[0];
480
+
else if (rc == H_INVALID_ELEMENT_ID)
481
+
*failed_index = retbuf[0];
482
+
else if (rc == H_INVALID_ELEMENT_SIZE)
483
+
*failed_index = retbuf[0];
484
+
else if (rc == H_INVALID_ELEMENT_VALUE)
485
+
*failed_index = retbuf[0];
486
+
487
+
return rc;
488
+
}
489
+
490
+
static inline long plpar_guest_delete(unsigned long flags, u64 guest_id)
491
+
{
492
+
long rc;
493
+
494
+
do {
495
+
rc = plpar_hcall_norets(H_GUEST_DELETE, flags, guest_id);
496
+
if (rc == H_BUSY)
497
+
cond_resched();
498
+
499
+
if (H_IS_LONG_BUSY(rc)) {
500
+
msleep(get_longbusy_msecs(rc));
501
+
rc = H_BUSY;
502
+
}
503
+
504
+
} while (rc == H_BUSY);
505
+
506
+
return rc;
507
+
}
508
+
509
+
static inline long plpar_guest_set_capabilities(unsigned long flags,
510
+
unsigned long capabilities)
511
+
{
512
+
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
513
+
long rc;
514
+
515
+
do {
516
+
rc = plpar_hcall(H_GUEST_SET_CAPABILITIES, retbuf, flags, capabilities);
517
+
if (rc == H_BUSY)
518
+
cond_resched();
519
+
520
+
if (H_IS_LONG_BUSY(rc)) {
521
+
msleep(get_longbusy_msecs(rc));
522
+
rc = H_BUSY;
523
+
}
524
+
} while (rc == H_BUSY);
525
+
526
+
return rc;
527
+
}
528
+
529
+
static inline long plpar_guest_get_capabilities(unsigned long flags,
530
+
unsigned long *capabilities)
531
+
{
532
+
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
533
+
long rc;
534
+
535
+
do {
536
+
rc = plpar_hcall(H_GUEST_GET_CAPABILITIES, retbuf, flags);
537
+
if (rc == H_BUSY)
538
+
cond_resched();
539
+
540
+
if (H_IS_LONG_BUSY(rc)) {
541
+
msleep(get_longbusy_msecs(rc));
542
+
rc = H_BUSY;
543
+
}
544
+
} while (rc == H_BUSY);
545
+
546
+
if (rc == H_SUCCESS)
547
+
*capabilities = retbuf[0];
548
+
549
+
return rc;
550
+
}
551
+
347
552
/*
348
553
* Wrapper to H_RPT_INVALIDATE hcall that handles return values appropriately
349
554
*
···
562
355
* error recovery of killing the process/guest will be eventually
563
356
* needed.
564
357
*/
565
-
static inline long pseries_rpt_invalidate(u32 pid, u64 target, u64 type,
358
+
static inline long pseries_rpt_invalidate(u64 pid, u64 target, u64 type,
566
359
u64 page_sizes, u64 start, u64 end)
567
360
{
568
361
long rc;
···
608
401
return 0;
609
402
}
610
403
611
-
static inline long pseries_rpt_invalidate(u32 pid, u64 target, u64 type,
404
+
static inline long pseries_rpt_invalidate(u64 pid, u64 target, u64 type,
612
405
u64 page_sizes, u64 start, u64 end)
406
+
{
407
+
return 0;
408
+
}
409
+
410
+
static inline long plpar_guest_create_vcpu(unsigned long flags,
411
+
unsigned long guest_id,
412
+
unsigned long vcpu_id)
413
+
{
414
+
return 0;
415
+
}
416
+
417
+
static inline long plpar_guest_get_state(unsigned long flags,
418
+
unsigned long guest_id,
419
+
unsigned long vcpu_id,
420
+
unsigned long data_buffer,
421
+
unsigned long data_size,
422
+
unsigned long *failed_index)
423
+
{
424
+
return 0;
425
+
}
426
+
427
+
static inline long plpar_guest_set_state(unsigned long flags,
428
+
unsigned long guest_id,
429
+
unsigned long vcpu_id,
430
+
unsigned long data_buffer,
431
+
unsigned long data_size,
432
+
unsigned long *failed_index)
433
+
{
434
+
return 0;
435
+
}
436
+
437
+
static inline long plpar_guest_run_vcpu(unsigned long flags, unsigned long guest_id,
438
+
unsigned long vcpu_id, int *trap,
439
+
unsigned long *failed_index)
440
+
{
441
+
return 0;
442
+
}
443
+
444
+
static inline long plpar_guest_create(unsigned long flags, unsigned long *guest_id)
445
+
{
446
+
return 0;
447
+
}
448
+
449
+
static inline long plpar_guest_delete(unsigned long flags, u64 guest_id)
450
+
{
451
+
return 0;
452
+
}
453
+
454
+
static inline long plpar_guest_get_capabilities(unsigned long flags,
455
+
unsigned long *capabilities)
456
+
{
457
+
return 0;
458
+
}
459
+
460
+
static inline long plpar_guest_set_capabilities(unsigned long flags,
461
+
unsigned long capabilities)
613
462
{
614
463
return 0;
615
464
}
+17
arch/powerpc/include/asm/ptrace.h
+17
arch/powerpc/include/asm/ptrace.h
···
397
397
return 0;
398
398
}
399
399
400
+
/**
401
+
* regs_get_kernel_argument() - get Nth function argument in kernel
402
+
* @regs: pt_regs of that context
403
+
* @n: function argument number (start from 0)
404
+
*
405
+
* We support up to 8 arguments and assume they are sent in through the GPRs.
406
+
* This will fail for fp/vector arguments, but those aren't usually found in
407
+
* kernel code. This is expected to be called from kprobes or ftrace with regs.
408
+
*/
409
+
static inline unsigned long regs_get_kernel_argument(struct pt_regs *regs, unsigned int n)
410
+
{
411
+
#define NR_REG_ARGUMENTS 8
412
+
if (n < NR_REG_ARGUMENTS)
413
+
return regs_get_register(regs, offsetof(struct pt_regs, gpr[3 + n]));
414
+
return 0;
415
+
}
416
+
400
417
#endif /* __ASSEMBLY__ */
401
418
402
419
#ifndef __powerpc64__
+1
-1
arch/powerpc/include/asm/uaccess.h
+1
-1
arch/powerpc/include/asm/uaccess.h
+6
-354
arch/powerpc/kernel/btext.c
+6
-354
arch/powerpc/kernel/btext.c
···
8
8
#include <linux/string.h>
9
9
#include <linux/init.h>
10
10
#include <linux/export.h>
11
+
#include <linux/font.h>
11
12
#include <linux/memblock.h>
12
13
#include <linux/pgtable.h>
13
14
#include <linux/of.h>
···
41
40
static unsigned char *logicalDisplayBase __force_data;
42
41
43
42
unsigned long disp_BAT[2] __initdata = {0, 0};
44
-
45
-
#define cmapsz (16*256)
46
-
47
-
static unsigned char vga_font[cmapsz];
48
43
49
44
static int boot_text_mapped __force_data;
50
45
···
404
407
};
405
408
406
409
407
-
static void draw_byte_32(unsigned char *font, unsigned int *base, int rb)
410
+
static void draw_byte_32(const unsigned char *font, unsigned int *base, int rb)
408
411
{
409
412
int l, bits;
410
413
int fg = 0xFFFFFFFFUL;
···
425
428
}
426
429
}
427
430
428
-
static inline void draw_byte_16(unsigned char *font, unsigned int *base, int rb)
431
+
static inline void draw_byte_16(const unsigned char *font, unsigned int *base, int rb)
429
432
{
430
433
int l, bits;
431
434
int fg = 0xFFFFFFFFUL;
···
443
446
}
444
447
}
445
448
446
-
static inline void draw_byte_8(unsigned char *font, unsigned int *base, int rb)
449
+
static inline void draw_byte_8(const unsigned char *font, unsigned int *base, int rb)
447
450
{
448
451
int l, bits;
449
452
int fg = 0x0F0F0F0FUL;
···
462
465
static noinline void draw_byte(unsigned char c, long locX, long locY)
463
466
{
464
467
unsigned char *base = calc_base(locX << 3, locY << 4);
465
-
unsigned char *font = &vga_font[((unsigned int)c) * 16];
468
+
unsigned int font_index = c * 16;
469
+
const unsigned char *font = font_sun_8x16.data + font_index;
466
470
int rb = dispDeviceRowBytes;
467
471
468
472
rmci_maybe_on();
···
581
583
*/
582
584
udbg_putc = btext_drawchar;
583
585
}
584
-
585
-
static unsigned char vga_font[cmapsz] = {
586
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
587
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0x81, 0xa5, 0x81, 0x81, 0xbd,
588
-
0x99, 0x81, 0x81, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0xff,
589
-
0xdb, 0xff, 0xff, 0xc3, 0xe7, 0xff, 0xff, 0x7e, 0x00, 0x00, 0x00, 0x00,
590
-
0x00, 0x00, 0x00, 0x00, 0x6c, 0xfe, 0xfe, 0xfe, 0xfe, 0x7c, 0x38, 0x10,
591
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x38, 0x7c, 0xfe,
592
-
0x7c, 0x38, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18,
593
-
0x3c, 0x3c, 0xe7, 0xe7, 0xe7, 0x18, 0x18, 0x3c, 0x00, 0x00, 0x00, 0x00,
594
-
0x00, 0x00, 0x00, 0x18, 0x3c, 0x7e, 0xff, 0xff, 0x7e, 0x18, 0x18, 0x3c,
595
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x3c,
596
-
0x3c, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff,
597
-
0xff, 0xff, 0xe7, 0xc3, 0xc3, 0xe7, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
598
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x66, 0x42, 0x42, 0x66, 0x3c, 0x00,
599
-
0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xc3, 0x99, 0xbd,
600
-
0xbd, 0x99, 0xc3, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x1e, 0x0e,
601
-
0x1a, 0x32, 0x78, 0xcc, 0xcc, 0xcc, 0xcc, 0x78, 0x00, 0x00, 0x00, 0x00,
602
-
0x00, 0x00, 0x3c, 0x66, 0x66, 0x66, 0x66, 0x3c, 0x18, 0x7e, 0x18, 0x18,
603
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3f, 0x33, 0x3f, 0x30, 0x30, 0x30,
604
-
0x30, 0x70, 0xf0, 0xe0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7f, 0x63,
605
-
0x7f, 0x63, 0x63, 0x63, 0x63, 0x67, 0xe7, 0xe6, 0xc0, 0x00, 0x00, 0x00,
606
-
0x00, 0x00, 0x00, 0x18, 0x18, 0xdb, 0x3c, 0xe7, 0x3c, 0xdb, 0x18, 0x18,
607
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0xc0, 0xe0, 0xf0, 0xf8, 0xfe, 0xf8,
608
-
0xf0, 0xe0, 0xc0, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x06, 0x0e,
609
-
0x1e, 0x3e, 0xfe, 0x3e, 0x1e, 0x0e, 0x06, 0x02, 0x00, 0x00, 0x00, 0x00,
610
-
0x00, 0x00, 0x18, 0x3c, 0x7e, 0x18, 0x18, 0x18, 0x7e, 0x3c, 0x18, 0x00,
611
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66,
612
-
0x66, 0x00, 0x66, 0x66, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7f, 0xdb,
613
-
0xdb, 0xdb, 0x7b, 0x1b, 0x1b, 0x1b, 0x1b, 0x1b, 0x00, 0x00, 0x00, 0x00,
614
-
0x00, 0x7c, 0xc6, 0x60, 0x38, 0x6c, 0xc6, 0xc6, 0x6c, 0x38, 0x0c, 0xc6,
615
-
0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
616
-
0xfe, 0xfe, 0xfe, 0xfe, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x3c,
617
-
0x7e, 0x18, 0x18, 0x18, 0x7e, 0x3c, 0x18, 0x7e, 0x00, 0x00, 0x00, 0x00,
618
-
0x00, 0x00, 0x18, 0x3c, 0x7e, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
619
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
620
-
0x18, 0x7e, 0x3c, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
621
-
0x00, 0x18, 0x0c, 0xfe, 0x0c, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
622
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x30, 0x60, 0xfe, 0x60, 0x30, 0x00, 0x00,
623
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0xc0,
624
-
0xc0, 0xfe, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
625
-
0x00, 0x24, 0x66, 0xff, 0x66, 0x24, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
626
-
0x00, 0x00, 0x00, 0x00, 0x10, 0x38, 0x38, 0x7c, 0x7c, 0xfe, 0xfe, 0x00,
627
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfe, 0xfe, 0x7c, 0x7c,
628
-
0x38, 0x38, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
629
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
630
-
0x00, 0x00, 0x18, 0x3c, 0x3c, 0x3c, 0x18, 0x18, 0x18, 0x00, 0x18, 0x18,
631
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x66, 0x66, 0x66, 0x24, 0x00, 0x00, 0x00,
632
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x6c,
633
-
0x6c, 0xfe, 0x6c, 0x6c, 0x6c, 0xfe, 0x6c, 0x6c, 0x00, 0x00, 0x00, 0x00,
634
-
0x18, 0x18, 0x7c, 0xc6, 0xc2, 0xc0, 0x7c, 0x06, 0x06, 0x86, 0xc6, 0x7c,
635
-
0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc2, 0xc6, 0x0c, 0x18,
636
-
0x30, 0x60, 0xc6, 0x86, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x38, 0x6c,
637
-
0x6c, 0x38, 0x76, 0xdc, 0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00,
638
-
0x00, 0x30, 0x30, 0x30, 0x60, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
639
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0c, 0x18, 0x30, 0x30, 0x30, 0x30,
640
-
0x30, 0x30, 0x18, 0x0c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30, 0x18,
641
-
0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x18, 0x30, 0x00, 0x00, 0x00, 0x00,
642
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x66, 0x3c, 0xff, 0x3c, 0x66, 0x00, 0x00,
643
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x7e,
644
-
0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
645
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x18, 0x30, 0x00, 0x00, 0x00,
646
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0x00, 0x00, 0x00, 0x00,
647
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
648
-
0x00, 0x00, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
649
-
0x02, 0x06, 0x0c, 0x18, 0x30, 0x60, 0xc0, 0x80, 0x00, 0x00, 0x00, 0x00,
650
-
0x00, 0x00, 0x7c, 0xc6, 0xc6, 0xce, 0xde, 0xf6, 0xe6, 0xc6, 0xc6, 0x7c,
651
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x38, 0x78, 0x18, 0x18, 0x18,
652
-
0x18, 0x18, 0x18, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0xc6,
653
-
0x06, 0x0c, 0x18, 0x30, 0x60, 0xc0, 0xc6, 0xfe, 0x00, 0x00, 0x00, 0x00,
654
-
0x00, 0x00, 0x7c, 0xc6, 0x06, 0x06, 0x3c, 0x06, 0x06, 0x06, 0xc6, 0x7c,
655
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0c, 0x1c, 0x3c, 0x6c, 0xcc, 0xfe,
656
-
0x0c, 0x0c, 0x0c, 0x1e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfe, 0xc0,
657
-
0xc0, 0xc0, 0xfc, 0x06, 0x06, 0x06, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00,
658
-
0x00, 0x00, 0x38, 0x60, 0xc0, 0xc0, 0xfc, 0xc6, 0xc6, 0xc6, 0xc6, 0x7c,
659
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfe, 0xc6, 0x06, 0x06, 0x0c, 0x18,
660
-
0x30, 0x30, 0x30, 0x30, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0xc6,
661
-
0xc6, 0xc6, 0x7c, 0xc6, 0xc6, 0xc6, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00,
662
-
0x00, 0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0x7e, 0x06, 0x06, 0x06, 0x0c, 0x78,
663
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x00, 0x00,
664
-
0x00, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
665
-
0x18, 0x18, 0x00, 0x00, 0x00, 0x18, 0x18, 0x30, 0x00, 0x00, 0x00, 0x00,
666
-
0x00, 0x00, 0x00, 0x06, 0x0c, 0x18, 0x30, 0x60, 0x30, 0x18, 0x0c, 0x06,
667
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0x00, 0x00,
668
-
0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x60,
669
-
0x30, 0x18, 0x0c, 0x06, 0x0c, 0x18, 0x30, 0x60, 0x00, 0x00, 0x00, 0x00,
670
-
0x00, 0x00, 0x7c, 0xc6, 0xc6, 0x0c, 0x18, 0x18, 0x18, 0x00, 0x18, 0x18,
671
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0xde, 0xde,
672
-
0xde, 0xdc, 0xc0, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x38,
673
-
0x6c, 0xc6, 0xc6, 0xfe, 0xc6, 0xc6, 0xc6, 0xc6, 0x00, 0x00, 0x00, 0x00,
674
-
0x00, 0x00, 0xfc, 0x66, 0x66, 0x66, 0x7c, 0x66, 0x66, 0x66, 0x66, 0xfc,
675
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x66, 0xc2, 0xc0, 0xc0, 0xc0,
676
-
0xc0, 0xc2, 0x66, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf8, 0x6c,
677
-
0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x6c, 0xf8, 0x00, 0x00, 0x00, 0x00,
678
-
0x00, 0x00, 0xfe, 0x66, 0x62, 0x68, 0x78, 0x68, 0x60, 0x62, 0x66, 0xfe,
679
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfe, 0x66, 0x62, 0x68, 0x78, 0x68,
680
-
0x60, 0x60, 0x60, 0xf0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x66,
681
-
0xc2, 0xc0, 0xc0, 0xde, 0xc6, 0xc6, 0x66, 0x3a, 0x00, 0x00, 0x00, 0x00,
682
-
0x00, 0x00, 0xc6, 0xc6, 0xc6, 0xc6, 0xfe, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6,
683
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x18, 0x18, 0x18, 0x18, 0x18,
684
-
0x18, 0x18, 0x18, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0x0c,
685
-
0x0c, 0x0c, 0x0c, 0x0c, 0xcc, 0xcc, 0xcc, 0x78, 0x00, 0x00, 0x00, 0x00,
686
-
0x00, 0x00, 0xe6, 0x66, 0x66, 0x6c, 0x78, 0x78, 0x6c, 0x66, 0x66, 0xe6,
687
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf0, 0x60, 0x60, 0x60, 0x60, 0x60,
688
-
0x60, 0x62, 0x66, 0xfe, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc3, 0xe7,
689
-
0xff, 0xff, 0xdb, 0xc3, 0xc3, 0xc3, 0xc3, 0xc3, 0x00, 0x00, 0x00, 0x00,
690
-
0x00, 0x00, 0xc6, 0xe6, 0xf6, 0xfe, 0xde, 0xce, 0xc6, 0xc6, 0xc6, 0xc6,
691
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6,
692
-
0xc6, 0xc6, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfc, 0x66,
693
-
0x66, 0x66, 0x7c, 0x60, 0x60, 0x60, 0x60, 0xf0, 0x00, 0x00, 0x00, 0x00,
694
-
0x00, 0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xd6, 0xde, 0x7c,
695
-
0x0c, 0x0e, 0x00, 0x00, 0x00, 0x00, 0xfc, 0x66, 0x66, 0x66, 0x7c, 0x6c,
696
-
0x66, 0x66, 0x66, 0xe6, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0xc6,
697
-
0xc6, 0x60, 0x38, 0x0c, 0x06, 0xc6, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00,
698
-
0x00, 0x00, 0xff, 0xdb, 0x99, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x3c,
699
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6,
700
-
0xc6, 0xc6, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc3, 0xc3,
701
-
0xc3, 0xc3, 0xc3, 0xc3, 0xc3, 0x66, 0x3c, 0x18, 0x00, 0x00, 0x00, 0x00,
702
-
0x00, 0x00, 0xc3, 0xc3, 0xc3, 0xc3, 0xc3, 0xdb, 0xdb, 0xff, 0x66, 0x66,
703
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc3, 0xc3, 0x66, 0x3c, 0x18, 0x18,
704
-
0x3c, 0x66, 0xc3, 0xc3, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc3, 0xc3,
705
-
0xc3, 0x66, 0x3c, 0x18, 0x18, 0x18, 0x18, 0x3c, 0x00, 0x00, 0x00, 0x00,
706
-
0x00, 0x00, 0xff, 0xc3, 0x86, 0x0c, 0x18, 0x30, 0x60, 0xc1, 0xc3, 0xff,
707
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x30, 0x30, 0x30, 0x30, 0x30,
708
-
0x30, 0x30, 0x30, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80,
709
-
0xc0, 0xe0, 0x70, 0x38, 0x1c, 0x0e, 0x06, 0x02, 0x00, 0x00, 0x00, 0x00,
710
-
0x00, 0x00, 0x3c, 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x3c,
711
-
0x00, 0x00, 0x00, 0x00, 0x10, 0x38, 0x6c, 0xc6, 0x00, 0x00, 0x00, 0x00,
712
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
713
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0x00, 0x00,
714
-
0x30, 0x30, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
715
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x78, 0x0c, 0x7c,
716
-
0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe0, 0x60,
717
-
0x60, 0x78, 0x6c, 0x66, 0x66, 0x66, 0x66, 0x7c, 0x00, 0x00, 0x00, 0x00,
718
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0xc6, 0xc0, 0xc0, 0xc0, 0xc6, 0x7c,
719
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1c, 0x0c, 0x0c, 0x3c, 0x6c, 0xcc,
720
-
0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
721
-
0x00, 0x7c, 0xc6, 0xfe, 0xc0, 0xc0, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00,
722
-
0x00, 0x00, 0x38, 0x6c, 0x64, 0x60, 0xf0, 0x60, 0x60, 0x60, 0x60, 0xf0,
723
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x76, 0xcc, 0xcc,
724
-
0xcc, 0xcc, 0xcc, 0x7c, 0x0c, 0xcc, 0x78, 0x00, 0x00, 0x00, 0xe0, 0x60,
725
-
0x60, 0x6c, 0x76, 0x66, 0x66, 0x66, 0x66, 0xe6, 0x00, 0x00, 0x00, 0x00,
726
-
0x00, 0x00, 0x18, 0x18, 0x00, 0x38, 0x18, 0x18, 0x18, 0x18, 0x18, 0x3c,
727
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x06, 0x00, 0x0e, 0x06, 0x06,
728
-
0x06, 0x06, 0x06, 0x06, 0x66, 0x66, 0x3c, 0x00, 0x00, 0x00, 0xe0, 0x60,
729
-
0x60, 0x66, 0x6c, 0x78, 0x78, 0x6c, 0x66, 0xe6, 0x00, 0x00, 0x00, 0x00,
730
-
0x00, 0x00, 0x38, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x3c,
731
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe6, 0xff, 0xdb,
732
-
0xdb, 0xdb, 0xdb, 0xdb, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
733
-
0x00, 0xdc, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x00, 0x00, 0x00, 0x00,
734
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0x7c,
735
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xdc, 0x66, 0x66,
736
-
0x66, 0x66, 0x66, 0x7c, 0x60, 0x60, 0xf0, 0x00, 0x00, 0x00, 0x00, 0x00,
737
-
0x00, 0x76, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0x7c, 0x0c, 0x0c, 0x1e, 0x00,
738
-
0x00, 0x00, 0x00, 0x00, 0x00, 0xdc, 0x76, 0x66, 0x60, 0x60, 0x60, 0xf0,
739
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0xc6, 0x60,
740
-
0x38, 0x0c, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x30,
741
-
0x30, 0xfc, 0x30, 0x30, 0x30, 0x30, 0x36, 0x1c, 0x00, 0x00, 0x00, 0x00,
742
-
0x00, 0x00, 0x00, 0x00, 0x00, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0x76,
743
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc3, 0xc3, 0xc3,
744
-
0xc3, 0x66, 0x3c, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
745
-
0x00, 0xc3, 0xc3, 0xc3, 0xdb, 0xdb, 0xff, 0x66, 0x00, 0x00, 0x00, 0x00,
746
-
0x00, 0x00, 0x00, 0x00, 0x00, 0xc3, 0x66, 0x3c, 0x18, 0x3c, 0x66, 0xc3,
747
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc6, 0xc6, 0xc6,
748
-
0xc6, 0xc6, 0xc6, 0x7e, 0x06, 0x0c, 0xf8, 0x00, 0x00, 0x00, 0x00, 0x00,
749
-
0x00, 0xfe, 0xcc, 0x18, 0x30, 0x60, 0xc6, 0xfe, 0x00, 0x00, 0x00, 0x00,
750
-
0x00, 0x00, 0x0e, 0x18, 0x18, 0x18, 0x70, 0x18, 0x18, 0x18, 0x18, 0x0e,
751
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x18, 0x18, 0x00, 0x18,
752
-
0x18, 0x18, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x70, 0x18,
753
-
0x18, 0x18, 0x0e, 0x18, 0x18, 0x18, 0x18, 0x70, 0x00, 0x00, 0x00, 0x00,
754
-
0x00, 0x00, 0x76, 0xdc, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
755
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x38, 0x6c, 0xc6,
756
-
0xc6, 0xc6, 0xfe, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x66,
757
-
0xc2, 0xc0, 0xc0, 0xc0, 0xc2, 0x66, 0x3c, 0x0c, 0x06, 0x7c, 0x00, 0x00,
758
-
0x00, 0x00, 0xcc, 0x00, 0x00, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0x76,
759
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x0c, 0x18, 0x30, 0x00, 0x7c, 0xc6, 0xfe,
760
-
0xc0, 0xc0, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x38, 0x6c,
761
-
0x00, 0x78, 0x0c, 0x7c, 0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00,
762
-
0x00, 0x00, 0xcc, 0x00, 0x00, 0x78, 0x0c, 0x7c, 0xcc, 0xcc, 0xcc, 0x76,
763
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x60, 0x30, 0x18, 0x00, 0x78, 0x0c, 0x7c,
764
-
0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00, 0x00, 0x38, 0x6c, 0x38,
765
-
0x00, 0x78, 0x0c, 0x7c, 0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00,
766
-
0x00, 0x00, 0x00, 0x00, 0x3c, 0x66, 0x60, 0x60, 0x66, 0x3c, 0x0c, 0x06,
767
-
0x3c, 0x00, 0x00, 0x00, 0x00, 0x10, 0x38, 0x6c, 0x00, 0x7c, 0xc6, 0xfe,
768
-
0xc0, 0xc0, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc6, 0x00,
769
-
0x00, 0x7c, 0xc6, 0xfe, 0xc0, 0xc0, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00,
770
-
0x00, 0x60, 0x30, 0x18, 0x00, 0x7c, 0xc6, 0xfe, 0xc0, 0xc0, 0xc6, 0x7c,
771
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x66, 0x00, 0x00, 0x38, 0x18, 0x18,
772
-
0x18, 0x18, 0x18, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x3c, 0x66,
773
-
0x00, 0x38, 0x18, 0x18, 0x18, 0x18, 0x18, 0x3c, 0x00, 0x00, 0x00, 0x00,
774
-
0x00, 0x60, 0x30, 0x18, 0x00, 0x38, 0x18, 0x18, 0x18, 0x18, 0x18, 0x3c,
775
-
0x00, 0x00, 0x00, 0x00, 0x00, 0xc6, 0x00, 0x10, 0x38, 0x6c, 0xc6, 0xc6,
776
-
0xfe, 0xc6, 0xc6, 0xc6, 0x00, 0x00, 0x00, 0x00, 0x38, 0x6c, 0x38, 0x00,
777
-
0x38, 0x6c, 0xc6, 0xc6, 0xfe, 0xc6, 0xc6, 0xc6, 0x00, 0x00, 0x00, 0x00,
778
-
0x18, 0x30, 0x60, 0x00, 0xfe, 0x66, 0x60, 0x7c, 0x60, 0x60, 0x66, 0xfe,
779
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x6e, 0x3b, 0x1b,
780
-
0x7e, 0xd8, 0xdc, 0x77, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3e, 0x6c,
781
-
0xcc, 0xcc, 0xfe, 0xcc, 0xcc, 0xcc, 0xcc, 0xce, 0x00, 0x00, 0x00, 0x00,
782
-
0x00, 0x10, 0x38, 0x6c, 0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0x7c,
783
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc6, 0x00, 0x00, 0x7c, 0xc6, 0xc6,
784
-
0xc6, 0xc6, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x60, 0x30, 0x18,
785
-
0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00,
786
-
0x00, 0x30, 0x78, 0xcc, 0x00, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0x76,
787
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x60, 0x30, 0x18, 0x00, 0xcc, 0xcc, 0xcc,
788
-
0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc6, 0x00,
789
-
0x00, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0x7e, 0x06, 0x0c, 0x78, 0x00,
790
-
0x00, 0xc6, 0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0x7c,
791
-
0x00, 0x00, 0x00, 0x00, 0x00, 0xc6, 0x00, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6,
792
-
0xc6, 0xc6, 0xc6, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x7e,
793
-
0xc3, 0xc0, 0xc0, 0xc0, 0xc3, 0x7e, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00,
794
-
0x00, 0x38, 0x6c, 0x64, 0x60, 0xf0, 0x60, 0x60, 0x60, 0x60, 0xe6, 0xfc,
795
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc3, 0x66, 0x3c, 0x18, 0xff, 0x18,
796
-
0xff, 0x18, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfc, 0x66, 0x66,
797
-
0x7c, 0x62, 0x66, 0x6f, 0x66, 0x66, 0x66, 0xf3, 0x00, 0x00, 0x00, 0x00,
798
-
0x00, 0x0e, 0x1b, 0x18, 0x18, 0x18, 0x7e, 0x18, 0x18, 0x18, 0x18, 0x18,
799
-
0xd8, 0x70, 0x00, 0x00, 0x00, 0x18, 0x30, 0x60, 0x00, 0x78, 0x0c, 0x7c,
800
-
0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0c, 0x18, 0x30,
801
-
0x00, 0x38, 0x18, 0x18, 0x18, 0x18, 0x18, 0x3c, 0x00, 0x00, 0x00, 0x00,
802
-
0x00, 0x18, 0x30, 0x60, 0x00, 0x7c, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0x7c,
803
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x30, 0x60, 0x00, 0xcc, 0xcc, 0xcc,
804
-
0xcc, 0xcc, 0xcc, 0x76, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x76, 0xdc,
805
-
0x00, 0xdc, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0x00, 0x00, 0x00, 0x00,
806
-
0x76, 0xdc, 0x00, 0xc6, 0xe6, 0xf6, 0xfe, 0xde, 0xce, 0xc6, 0xc6, 0xc6,
807
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x6c, 0x6c, 0x3e, 0x00, 0x7e, 0x00,
808
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x38, 0x6c, 0x6c,
809
-
0x38, 0x00, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
810
-
0x00, 0x00, 0x30, 0x30, 0x00, 0x30, 0x30, 0x60, 0xc0, 0xc6, 0xc6, 0x7c,
811
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfe, 0xc0,
812
-
0xc0, 0xc0, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
813
-
0x00, 0x00, 0xfe, 0x06, 0x06, 0x06, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00,
814
-
0x00, 0xc0, 0xc0, 0xc2, 0xc6, 0xcc, 0x18, 0x30, 0x60, 0xce, 0x9b, 0x06,
815
-
0x0c, 0x1f, 0x00, 0x00, 0x00, 0xc0, 0xc0, 0xc2, 0xc6, 0xcc, 0x18, 0x30,
816
-
0x66, 0xce, 0x96, 0x3e, 0x06, 0x06, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18,
817
-
0x00, 0x18, 0x18, 0x18, 0x3c, 0x3c, 0x3c, 0x18, 0x00, 0x00, 0x00, 0x00,
818
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x36, 0x6c, 0xd8, 0x6c, 0x36, 0x00, 0x00,
819
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xd8, 0x6c, 0x36,
820
-
0x6c, 0xd8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x11, 0x44, 0x11, 0x44,
821
-
0x11, 0x44, 0x11, 0x44, 0x11, 0x44, 0x11, 0x44, 0x11, 0x44, 0x11, 0x44,
822
-
0x55, 0xaa, 0x55, 0xaa, 0x55, 0xaa, 0x55, 0xaa, 0x55, 0xaa, 0x55, 0xaa,
823
-
0x55, 0xaa, 0x55, 0xaa, 0xdd, 0x77, 0xdd, 0x77, 0xdd, 0x77, 0xdd, 0x77,
824
-
0xdd, 0x77, 0xdd, 0x77, 0xdd, 0x77, 0xdd, 0x77, 0x18, 0x18, 0x18, 0x18,
825
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
826
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0xf8, 0x18, 0x18, 0x18, 0x18,
827
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0xf8, 0x18, 0xf8,
828
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x36, 0x36, 0x36, 0x36,
829
-
0x36, 0x36, 0x36, 0xf6, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
830
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfe, 0x36, 0x36, 0x36, 0x36,
831
-
0x36, 0x36, 0x36, 0x36, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf8, 0x18, 0xf8,
832
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x36, 0x36, 0x36, 0x36,
833
-
0x36, 0xf6, 0x06, 0xf6, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
834
-
0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
835
-
0x36, 0x36, 0x36, 0x36, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfe, 0x06, 0xf6,
836
-
0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
837
-
0x36, 0xf6, 0x06, 0xfe, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
838
-
0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0xfe, 0x00, 0x00, 0x00, 0x00,
839
-
0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x18, 0x18, 0x18, 0xf8, 0x18, 0xf8,
840
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
841
-
0x00, 0x00, 0x00, 0xf8, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
842
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x1f, 0x00, 0x00, 0x00, 0x00,
843
-
0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0xff,
844
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
845
-
0x00, 0x00, 0x00, 0xff, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
846
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x1f, 0x18, 0x18, 0x18, 0x18,
847
-
0x18, 0x18, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff,
848
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x18, 0x18,
849
-
0x18, 0x18, 0x18, 0xff, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
850
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x1f, 0x18, 0x1f, 0x18, 0x18, 0x18, 0x18,
851
-
0x18, 0x18, 0x18, 0x18, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x37,
852
-
0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
853
-
0x36, 0x37, 0x30, 0x3f, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
854
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x3f, 0x30, 0x37, 0x36, 0x36, 0x36, 0x36,
855
-
0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0xf7, 0x00, 0xff,
856
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
857
-
0x00, 0xff, 0x00, 0xf7, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
858
-
0x36, 0x36, 0x36, 0x36, 0x36, 0x37, 0x30, 0x37, 0x36, 0x36, 0x36, 0x36,
859
-
0x36, 0x36, 0x36, 0x36, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0x00, 0xff,
860
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x36, 0x36, 0x36, 0x36,
861
-
0x36, 0xf7, 0x00, 0xf7, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
862
-
0x18, 0x18, 0x18, 0x18, 0x18, 0xff, 0x00, 0xff, 0x00, 0x00, 0x00, 0x00,
863
-
0x00, 0x00, 0x00, 0x00, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0xff,
864
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
865
-
0x00, 0xff, 0x00, 0xff, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
866
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0x36, 0x36, 0x36, 0x36,
867
-
0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x3f,
868
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x18, 0x18,
869
-
0x18, 0x1f, 0x18, 0x1f, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
870
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x1f, 0x18, 0x1f, 0x18, 0x18, 0x18, 0x18,
871
-
0x18, 0x18, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3f,
872
-
0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
873
-
0x36, 0x36, 0x36, 0xff, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36, 0x36,
874
-
0x18, 0x18, 0x18, 0x18, 0x18, 0xff, 0x18, 0xff, 0x18, 0x18, 0x18, 0x18,
875
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0xf8,
876
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
877
-
0x00, 0x00, 0x00, 0x1f, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
878
-
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
879
-
0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff,
880
-
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xf0, 0xf0, 0xf0, 0xf0,
881
-
0xf0, 0xf0, 0xf0, 0xf0, 0xf0, 0xf0, 0xf0, 0xf0, 0xf0, 0xf0, 0xf0, 0xf0,
882
-
0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f,
883
-
0x0f, 0x0f, 0x0f, 0x0f, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00,
884
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
885
-
0x00, 0x76, 0xdc, 0xd8, 0xd8, 0xd8, 0xdc, 0x76, 0x00, 0x00, 0x00, 0x00,
886
-
0x00, 0x00, 0x78, 0xcc, 0xcc, 0xcc, 0xd8, 0xcc, 0xc6, 0xc6, 0xc6, 0xcc,
887
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xfe, 0xc6, 0xc6, 0xc0, 0xc0, 0xc0,
888
-
0xc0, 0xc0, 0xc0, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
889
-
0xfe, 0x6c, 0x6c, 0x6c, 0x6c, 0x6c, 0x6c, 0x6c, 0x00, 0x00, 0x00, 0x00,
890
-
0x00, 0x00, 0x00, 0xfe, 0xc6, 0x60, 0x30, 0x18, 0x30, 0x60, 0xc6, 0xfe,
891
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0xd8, 0xd8,
892
-
0xd8, 0xd8, 0xd8, 0x70, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
893
-
0x66, 0x66, 0x66, 0x66, 0x66, 0x7c, 0x60, 0x60, 0xc0, 0x00, 0x00, 0x00,
894
-
0x00, 0x00, 0x00, 0x00, 0x76, 0xdc, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
895
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0x18, 0x3c, 0x66, 0x66,
896
-
0x66, 0x3c, 0x18, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x38,
897
-
0x6c, 0xc6, 0xc6, 0xfe, 0xc6, 0xc6, 0x6c, 0x38, 0x00, 0x00, 0x00, 0x00,
898
-
0x00, 0x00, 0x38, 0x6c, 0xc6, 0xc6, 0xc6, 0x6c, 0x6c, 0x6c, 0x6c, 0xee,
899
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0x30, 0x18, 0x0c, 0x3e, 0x66,
900
-
0x66, 0x66, 0x66, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
901
-
0x00, 0x7e, 0xdb, 0xdb, 0xdb, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
902
-
0x00, 0x00, 0x00, 0x03, 0x06, 0x7e, 0xdb, 0xdb, 0xf3, 0x7e, 0x60, 0xc0,
903
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1c, 0x30, 0x60, 0x60, 0x7c, 0x60,
904
-
0x60, 0x60, 0x30, 0x1c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c,
905
-
0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0xc6, 0x00, 0x00, 0x00, 0x00,
906
-
0x00, 0x00, 0x00, 0x00, 0xfe, 0x00, 0x00, 0xfe, 0x00, 0x00, 0xfe, 0x00,
907
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x7e, 0x18,
908
-
0x18, 0x00, 0x00, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30,
909
-
0x18, 0x0c, 0x06, 0x0c, 0x18, 0x30, 0x00, 0x7e, 0x00, 0x00, 0x00, 0x00,
910
-
0x00, 0x00, 0x00, 0x0c, 0x18, 0x30, 0x60, 0x30, 0x18, 0x0c, 0x00, 0x7e,
911
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0e, 0x1b, 0x1b, 0x1b, 0x18, 0x18,
912
-
0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18,
913
-
0x18, 0x18, 0x18, 0x18, 0xd8, 0xd8, 0xd8, 0x70, 0x00, 0x00, 0x00, 0x00,
914
-
0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x00, 0x7e, 0x00, 0x18, 0x18, 0x00,
915
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x76, 0xdc, 0x00,
916
-
0x76, 0xdc, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x38, 0x6c, 0x6c,
917
-
0x38, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
918
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x00, 0x00, 0x00,
919
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
920
-
0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0f, 0x0c, 0x0c,
921
-
0x0c, 0x0c, 0x0c, 0xec, 0x6c, 0x6c, 0x3c, 0x1c, 0x00, 0x00, 0x00, 0x00,
922
-
0x00, 0xd8, 0x6c, 0x6c, 0x6c, 0x6c, 0x6c, 0x00, 0x00, 0x00, 0x00, 0x00,
923
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x70, 0xd8, 0x30, 0x60, 0xc8, 0xf8, 0x00,
924
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
925
-
0x7c, 0x7c, 0x7c, 0x7c, 0x7c, 0x7c, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00,
926
-
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
927
-
0x00, 0x00, 0x00, 0x00,
928
-
};
929
-
+12
arch/powerpc/kernel/crash_dump.c
+12
arch/powerpc/kernel/crash_dump.c
···
19
19
#include <linux/uio.h>
20
20
#include <asm/rtas.h>
21
21
#include <asm/inst.h>
22
+
#include <asm/fadump.h>
22
23
23
24
#ifdef DEBUG
24
25
#include <asm/udbg.h>
···
92
91
93
92
return csize;
94
93
}
94
+
95
+
/*
96
+
* Return true only when kexec based kernel dump capturing method is used.
97
+
* This ensures all restritions applied for kdump case are not automatically
98
+
* applied for fadump case.
99
+
*/
100
+
bool is_kdump_kernel(void)
101
+
{
102
+
return !is_fadump_active() && elfcorehdr_addr != ELFCORE_ADDR_MAX;
103
+
}
104
+
EXPORT_SYMBOL_GPL(is_kdump_kernel);
95
105
96
106
#ifdef CONFIG_PPC_RTAS
97
107
/*
+2
-2
arch/powerpc/kernel/eeh_driver.c
+2
-2
arch/powerpc/kernel/eeh_driver.c
···
39
39
case PCI_ERS_RESULT_NEED_RESET:
40
40
return 6;
41
41
default:
42
-
WARN_ONCE(1, "Unknown pci_ers_result value: %d\n", (int)result);
42
+
WARN_ONCE(1, "Unknown pci_ers_result value: %d\n", result);
43
43
return 0;
44
44
}
45
45
};
···
60
60
case PCI_ERS_RESULT_NO_AER_DRIVER:
61
61
return "no AER driver";
62
62
default:
63
-
WARN_ONCE(1, "Unknown result type: %d\n", (int)result);
63
+
WARN_ONCE(1, "Unknown result type: %d\n", result);
64
64
return "unknown";
65
65
}
66
66
};
+10
-9
arch/powerpc/kernel/head_40x.S
+10
-9
arch/powerpc/kernel/head_40x.S
···
312
312
313
313
rlwimi r11, r10, 22, 20, 29 /* Compute PTE address */
314
314
lwz r11, 0(r11) /* Get Linux PTE */
315
-
li r9, _PAGE_PRESENT | _PAGE_ACCESSED
315
+
li r9, _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_READ
316
316
andc. r9, r9, r11 /* Check permission */
317
317
bne 5f
318
318
319
-
rlwinm r9, r11, 1, _PAGE_RW /* dirty => rw */
320
-
and r9, r9, r11 /* hwwrite = dirty & rw */
321
-
rlwimi r11, r9, 0, _PAGE_RW /* replace rw by hwwrite */
319
+
rlwinm r9, r11, 1, _PAGE_WRITE /* dirty => w */
320
+
and r9, r9, r11 /* hwwrite = dirty & w */
321
+
rlwimi r11, r9, 0, _PAGE_WRITE /* replace w by hwwrite */
322
322
323
323
/* Create TLB tag. This is the faulting address plus a static
324
324
* set of bits. These are size, valid, E, U0.
···
400
400
andc. r9, r9, r11 /* Check permission */
401
401
bne 5f
402
402
403
-
rlwinm r9, r11, 1, _PAGE_RW /* dirty => rw */
404
-
and r9, r9, r11 /* hwwrite = dirty & rw */
405
-
rlwimi r11, r9, 0, _PAGE_RW /* replace rw by hwwrite */
403
+
rlwinm r9, r11, 1, _PAGE_WRITE /* dirty => w */
404
+
and r9, r9, r11 /* hwwrite = dirty & w */
405
+
rlwimi r11, r9, 0, _PAGE_WRITE /* replace w by hwwrite */
406
406
407
407
/* Create TLB tag. This is the faulting address plus a static
408
408
* set of bits. These are size, valid, E, U0.
···
561
561
/*
562
562
* Clear out the software-only bits in the PTE to generate the
563
563
* TLB_DATA value. These are the bottom 2 bits of the RPM, the
564
-
* top 3 bits of the zone field, and M.
564
+
* 4 bits of the zone field, and M.
565
565
*/
566
-
li r9, 0x0ce2
566
+
li r9, 0x0cf2
567
567
andc r11, r11, r9
568
+
rlwimi r11, r10, 8, 24, 27 /* Copy 4 upper address bit into zone */
568
569
569
570
/* load the next available TLB index. */
570
571
lwz r9, tlb_4xx_index@l(0)
+20
-20
arch/powerpc/kernel/head_44x.S
+20
-20
arch/powerpc/kernel/head_44x.S
···
314
314
* kernel page tables.
315
315
*/
316
316
lis r11, PAGE_OFFSET@h
317
-
cmplw r10, r11
318
-
blt+ 3f
317
+
cmplw cr7, r10, r11
318
+
blt+ cr7, 3f
319
319
lis r11, swapper_pg_dir@h
320
320
ori r11, r11, swapper_pg_dir@l
321
321
···
342
342
mtspr SPRN_MMUCR,r12
343
343
344
344
/* Mask of required permission bits. Note that while we
345
-
* do copy ESR:ST to _PAGE_RW position as trying to write
345
+
* do copy ESR:ST to _PAGE_WRITE position as trying to write
346
346
* to an RO page is pretty common, we don't do it with
347
347
* _PAGE_DIRTY. We could do it, but it's a fairly rare
348
348
* event so I'd rather take the overhead when it happens
···
355
355
* place or can we save a couple of instructions here ?
356
356
*/
357
357
mfspr r12,SPRN_ESR
358
-
li r13,_PAGE_PRESENT|_PAGE_ACCESSED
358
+
li r13,_PAGE_PRESENT|_PAGE_ACCESSED|_PAGE_READ
359
359
rlwimi r13,r12,10,30,30
360
360
361
361
/* Load the PTE */
···
428
428
* kernel page tables.
429
429
*/
430
430
lis r11, PAGE_OFFSET@h
431
-
cmplw r10, r11
432
-
blt+ 3f
431
+
cmplw cr7, r10, r11
432
+
blt+ cr7, 3f
433
433
lis r11, swapper_pg_dir@h
434
434
ori r11, r11, swapper_pg_dir@l
435
435
···
515
515
* r11 - PTE high word value
516
516
* r12 - PTE low word value
517
517
* r13 - TLB index
518
+
* cr7 - Result of comparison with PAGE_OFFSET
518
519
* MMUCR - loaded with proper value when we get here
519
520
* Upon exit, we reload everything and RFI.
520
521
*/
···
534
533
tlbwe r10,r13,PPC44x_TLB_PAGEID /* Write PAGEID */
535
534
536
535
/* And WS 2 */
537
-
li r10,0xf85 /* Mask to apply from PTE */
538
-
rlwimi r10,r12,29,30,30 /* DIRTY -> SW position */
536
+
li r10,0xf84 /* Mask to apply from PTE */
537
+
rlwimi r10,r12,29,30,31 /* DIRTY,READ -> SW,SR position */
539
538
and r11,r12,r10 /* Mask PTE bits to keep */
540
-
andi. r10,r12,_PAGE_USER /* User page ? */
541
-
beq 1f /* nope, leave U bits empty */
539
+
bge cr7,1f /* User page ? no, leave U bits empty */
542
540
rlwimi r11,r11,3,26,28 /* yes, copy S bits to U */
543
541
rlwinm r11,r11,0,~PPC44x_TLB_SX /* Clear SX if User page */
544
542
1: tlbwe r11,r13,PPC44x_TLB_ATTRIB /* Write ATTRIB */
···
568
568
* kernel page tables.
569
569
*/
570
570
lis r11,PAGE_OFFSET@h
571
-
cmplw cr0,r10,r11
572
-
blt+ 3f
571
+
cmplw cr7,r10,r11
572
+
blt+ cr7,3f
573
573
lis r11,swapper_pg_dir@h
574
574
ori r11,r11, swapper_pg_dir@l
575
575
li r12,0 /* MMUCR = 0 */
···
586
586
4: mtspr SPRN_MMUCR,r12 /* Set MMUCR */
587
587
588
588
/* Mask of required permission bits. Note that while we
589
-
* do copy ESR:ST to _PAGE_RW position as trying to write
589
+
* do copy ESR:ST to _PAGE_WRITE position as trying to write
590
590
* to an RO page is pretty common, we don't do it with
591
591
* _PAGE_DIRTY. We could do it, but it's a fairly rare
592
592
* event so I'd rather take the overhead when it happens
···
599
599
* place or can we save a couple of instructions here ?
600
600
*/
601
601
mfspr r12,SPRN_ESR
602
-
li r13,_PAGE_PRESENT|_PAGE_ACCESSED
602
+
li r13,_PAGE_PRESENT|_PAGE_ACCESSED|_PAGE_READ
603
603
rlwimi r13,r12,10,30,30
604
604
605
605
/* Load the PTE */
···
669
669
* kernel page tables.
670
670
*/
671
671
lis r11,PAGE_OFFSET@h
672
-
cmplw cr0,r10,r11
673
-
blt+ 3f
672
+
cmplw cr7,r10,r11
673
+
blt+ cr7,3f
674
674
lis r11,swapper_pg_dir@h
675
675
ori r11,r11, swapper_pg_dir@l
676
676
li r12,0 /* MMUCR = 0 */
···
744
744
* r11 - PTE high word value
745
745
* r12 - PTE low word value
746
746
* r13 - free to use
747
+
* cr7 - Result of comparison with PAGE_OFFSET
747
748
* MMUCR - loaded with proper value when we get here
748
749
* Upon exit, we reload everything and RFI.
749
750
*/
···
754
753
tlbwe r11,r13,1
755
754
756
755
/* And make up word 2 */
757
-
li r10,0xf85 /* Mask to apply from PTE */
758
-
rlwimi r10,r12,29,30,30 /* DIRTY -> SW position */
756
+
li r10,0xf84 /* Mask to apply from PTE */
757
+
rlwimi r10,r12,29,30,31 /* DIRTY,READ -> SW,SR position */
759
758
and r11,r12,r10 /* Mask PTE bits to keep */
760
-
andi. r10,r12,_PAGE_USER /* User page ? */
761
-
beq 1f /* nope, leave U bits empty */
759
+
bge cr7,1f /* User page ? no, leave U bits empty */
762
760
rlwimi r11,r11,3,26,28 /* yes, copy S bits to U */
763
761
rlwinm r11,r11,0,~PPC47x_TLB2_SX /* Clear SX if User page */
764
762
1: tlbwe r11,r13,2
+6
-6
arch/powerpc/kernel/head_85xx.S
+6
-6
arch/powerpc/kernel/head_85xx.S
···
471
471
472
472
4:
473
473
/* Mask of required permission bits. Note that while we
474
-
* do copy ESR:ST to _PAGE_RW position as trying to write
474
+
* do copy ESR:ST to _PAGE_WRITE position as trying to write
475
475
* to an RO page is pretty common, we don't do it with
476
476
* _PAGE_DIRTY. We could do it, but it's a fairly rare
477
477
* event so I'd rather take the overhead when it happens
···
485
485
*/
486
486
mfspr r12,SPRN_ESR
487
487
#ifdef CONFIG_PTE_64BIT
488
-
li r13,_PAGE_PRESENT
488
+
li r13,_PAGE_PRESENT|_PAGE_BAP_SR
489
489
oris r13,r13,_PAGE_ACCESSED@h
490
490
#else
491
-
li r13,_PAGE_PRESENT|_PAGE_ACCESSED
491
+
li r13,_PAGE_PRESENT|_PAGE_READ|_PAGE_ACCESSED
492
492
#endif
493
493
rlwimi r13,r12,11,29,29
494
494
···
783
783
mtspr SPRN_MAS7, r10
784
784
END_MMU_FTR_SECTION_IFSET(MMU_FTR_BIG_PHYS)
785
785
#else
786
-
li r10, (_PAGE_EXEC | _PAGE_PRESENT)
786
+
li r10, (_PAGE_EXEC | _PAGE_READ)
787
787
mr r13, r11
788
788
rlwimi r10, r11, 31, 29, 29 /* extract _PAGE_DIRTY into SW */
789
789
and r12, r11, r10
790
-
andi. r10, r11, _PAGE_USER /* Test for _PAGE_USER */
790
+
mcrf cr0, cr5 /* Test for user page */
791
791
slwi r10, r12, 1
792
792
or r10, r10, r12
793
793
rlwinm r10, r10, 0, ~_PAGE_EXEC /* Clear SX on user pages */
794
-
iseleq r12, r12, r10
794
+
isellt r12, r10, r12
795
795
rlwimi r13, r12, 0, 20, 31 /* Get RPN from PTE, merge w/ perms */
796
796
mtspr SPRN_MAS3, r13
797
797
#endif
+34
-29
arch/powerpc/kernel/head_book3s_32.S
+34
-29
arch/powerpc/kernel/head_book3s_32.S
···
412
412
. = INTERRUPT_INST_TLB_MISS_603
413
413
InstructionTLBMiss:
414
414
/*
415
-
* r0: scratch
415
+
* r0: userspace flag (later scratch)
416
416
* r1: linux style pte ( later becomes ppc hardware pte )
417
417
* r2: ptr to linux-style pte
418
-
* r3: scratch
418
+
* r3: fault address
419
419
*/
420
420
/* Get PTE (linux-style) and check access */
421
421
mfspr r3,SPRN_IMISS
···
424
424
cmplw 0,r1,r3
425
425
#endif
426
426
mfspr r2, SPRN_SDR1
427
-
li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | _PAGE_USER
427
+
li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC
428
428
rlwinm r2, r2, 28, 0xfffff000
429
429
#ifdef CONFIG_MODULES
430
+
li r0, 3
430
431
bgt- 112f
431
432
lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */
432
-
li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC
433
+
li r0, 0
433
434
addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */
434
435
#endif
435
436
112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */
···
438
437
rlwinm. r2,r2,0,0,19 /* extract address of pte page */
439
438
beq- InstructionAddressInvalid /* return if no mapping */
440
439
rlwimi r2,r3,22,20,29 /* insert next 10 bits of address */
441
-
lwz r0,0(r2) /* get linux-style pte */
442
-
andc. r1,r1,r0 /* check access & ~permission */
440
+
lwz r2,0(r2) /* get linux-style pte */
441
+
andc. r1,r1,r2 /* check access & ~permission */
443
442
bne- InstructionAddressInvalid /* return if access not permitted */
444
443
/* Convert linux-style PTE to low word of PPC-style PTE */
445
-
rlwimi r0,r0,32-2,31,31 /* _PAGE_USER -> PP lsb */
444
+
#ifdef CONFIG_MODULES
445
+
rlwimi r2, r0, 0, 31, 31 /* userspace ? -> PP lsb */
446
+
#endif
446
447
ori r1, r1, 0xe06 /* clear out reserved bits */
447
-
andc r1, r0, r1 /* PP = user? 1 : 0 */
448
+
andc r1, r2, r1 /* PP = user? 1 : 0 */
448
449
BEGIN_FTR_SECTION
449
450
rlwinm r1,r1,0,~_PAGE_COHERENT /* clear M (coherence not required) */
450
451
END_FTR_SECTION_IFCLR(CPU_FTR_NEED_COHERENT)
···
481
478
. = INTERRUPT_DATA_LOAD_TLB_MISS_603
482
479
DataLoadTLBMiss:
483
480
/*
484
-
* r0: scratch
481
+
* r0: userspace flag (later scratch)
485
482
* r1: linux style pte ( later becomes ppc hardware pte )
486
483
* r2: ptr to linux-style pte
487
-
* r3: scratch
484
+
* r3: fault address
488
485
*/
489
486
/* Get PTE (linux-style) and check access */
490
487
mfspr r3,SPRN_DMISS
491
488
lis r1, TASK_SIZE@h /* check if kernel address */
492
489
cmplw 0,r1,r3
493
490
mfspr r2, SPRN_SDR1
494
-
li r1, _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_USER
491
+
li r1, _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_READ
495
492
rlwinm r2, r2, 28, 0xfffff000
493
+
li r0, 3
496
494
bgt- 112f
497
495
lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */
498
-
li r1, _PAGE_PRESENT | _PAGE_ACCESSED
496
+
li r0, 0
499
497
addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */
500
498
112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */
501
499
lwz r2,0(r2) /* get pmd entry */
502
500
rlwinm. r2,r2,0,0,19 /* extract address of pte page */
503
501
beq- DataAddressInvalid /* return if no mapping */
504
502
rlwimi r2,r3,22,20,29 /* insert next 10 bits of address */
505
-
lwz r0,0(r2) /* get linux-style pte */
506
-
andc. r1,r1,r0 /* check access & ~permission */
503
+
lwz r2,0(r2) /* get linux-style pte */
504
+
andc. r1,r1,r2 /* check access & ~permission */
507
505
bne- DataAddressInvalid /* return if access not permitted */
508
506
/* Convert linux-style PTE to low word of PPC-style PTE */
509
-
rlwinm r1,r0,32-9,30,30 /* _PAGE_RW -> PP msb */
510
-
rlwimi r0,r0,32-1,30,30 /* _PAGE_USER -> PP msb */
511
-
rlwimi r1,r0,32-3,24,24 /* _PAGE_RW -> _PAGE_DIRTY */
512
-
rlwimi r0,r0,32-1,31,31 /* _PAGE_USER -> PP lsb */
507
+
rlwinm r1,r2,32-9,30,30 /* _PAGE_WRITE -> PP msb */
508
+
rlwimi r2,r0,0,30,31 /* userspace ? -> PP */
509
+
rlwimi r1,r2,32-3,24,24 /* _PAGE_WRITE -> _PAGE_DIRTY */
513
510
xori r1,r1,_PAGE_DIRTY /* clear dirty when not rw */
514
511
ori r1,r1,0xe04 /* clear out reserved bits */
515
-
andc r1,r0,r1 /* PP = user? rw? 1: 3: 0 */
512
+
andc r1,r2,r1 /* PP = user? rw? 1: 3: 0 */
516
513
BEGIN_FTR_SECTION
517
514
rlwinm r1,r1,0,~_PAGE_COHERENT /* clear M (coherence not required) */
518
515
END_FTR_SECTION_IFCLR(CPU_FTR_NEED_COHERENT)
···
561
558
. = INTERRUPT_DATA_STORE_TLB_MISS_603
562
559
DataStoreTLBMiss:
563
560
/*
564
-
* r0: scratch
561
+
* r0: userspace flag (later scratch)
565
562
* r1: linux style pte ( later becomes ppc hardware pte )
566
563
* r2: ptr to linux-style pte
567
-
* r3: scratch
564
+
* r3: fault address
568
565
*/
569
566
/* Get PTE (linux-style) and check access */
570
567
mfspr r3,SPRN_DMISS
571
568
lis r1, TASK_SIZE@h /* check if kernel address */
572
569
cmplw 0,r1,r3
573
570
mfspr r2, SPRN_SDR1
574
-
li r1, _PAGE_RW | _PAGE_DIRTY | _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_USER
571
+
li r1, _PAGE_RW | _PAGE_DIRTY | _PAGE_PRESENT | _PAGE_ACCESSED
575
572
rlwinm r2, r2, 28, 0xfffff000
573
+
li r0, 3
576
574
bgt- 112f
577
575
lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */
578
-
li r1, _PAGE_RW | _PAGE_DIRTY | _PAGE_PRESENT | _PAGE_ACCESSED
576
+
li r0, 0
579
577
addi r2, r2, (swapper_pg_dir - PAGE_OFFSET)@l /* kernel page table */
580
578
112: rlwimi r2,r3,12,20,29 /* insert top 10 bits of address */
581
579
lwz r2,0(r2) /* get pmd entry */
582
580
rlwinm. r2,r2,0,0,19 /* extract address of pte page */
583
581
beq- DataAddressInvalid /* return if no mapping */
584
582
rlwimi r2,r3,22,20,29 /* insert next 10 bits of address */
585
-
lwz r0,0(r2) /* get linux-style pte */
586
-
andc. r1,r1,r0 /* check access & ~permission */
583
+
lwz r2,0(r2) /* get linux-style pte */
584
+
andc. r1,r1,r2 /* check access & ~permission */
587
585
bne- DataAddressInvalid /* return if access not permitted */
588
586
/* Convert linux-style PTE to low word of PPC-style PTE */
589
-
rlwimi r0,r0,32-2,31,31 /* _PAGE_USER -> PP lsb */
587
+
rlwimi r2,r0,0,31,31 /* userspace ? -> PP lsb */
590
588
li r1,0xe06 /* clear out reserved bits & PP msb */
591
-
andc r1,r0,r1 /* PP = user? 1: 0 */
589
+
andc r1,r2,r1 /* PP = user? 1: 0 */
592
590
BEGIN_FTR_SECTION
593
591
rlwinm r1,r1,0,~_PAGE_COHERENT /* clear M (coherence not required) */
594
592
END_FTR_SECTION_IFCLR(CPU_FTR_NEED_COHERENT)
···
693
689
mfdar r4
694
690
mfsrr0 r5
695
691
mfsrr1 r9
696
-
rlwinm r3, r3, 32 - 15, _PAGE_RW /* DSISR_STORE -> _PAGE_RW */
692
+
rlwinm r3, r3, 32 - 15, _PAGE_WRITE /* DSISR_STORE -> _PAGE_WRITE */
693
+
ori r3, r3, _PAGE_PRESENT | _PAGE_READ
697
694
bl hash_page
698
695
mfspr r10, SPRN_SPRG_THREAD
699
696
restore_regs_thread r10
···
704
699
mr r11, r10
705
700
mfspr r10, SPRN_SPRG_THREAD
706
701
save_regs_thread r10
707
-
li r3, 0
702
+
li r3, _PAGE_PRESENT | _PAGE_EXEC
708
703
lwz r4, SRR0(r10)
709
704
lwz r9, SRR1(r10)
710
705
bl hash_page
+6
-6
arch/powerpc/kernel/io.c
+6
-6
arch/powerpc/kernel/io.c
···
33
33
return;
34
34
asm volatile("sync");
35
35
do {
36
-
tmp = *port;
36
+
tmp = *(const volatile u8 __force *)port;
37
37
eieio();
38
38
*tbuf++ = tmp;
39
39
} while (--count != 0);
···
49
49
return;
50
50
asm volatile("sync");
51
51
do {
52
-
*port = *tbuf++;
52
+
*(volatile u8 __force *)port = *tbuf++;
53
53
} while (--count != 0);
54
54
asm volatile("sync");
55
55
}
···
64
64
return;
65
65
asm volatile("sync");
66
66
do {
67
-
tmp = *port;
67
+
tmp = *(const volatile u16 __force *)port;
68
68
eieio();
69
69
*tbuf++ = tmp;
70
70
} while (--count != 0);
···
80
80
return;
81
81
asm volatile("sync");
82
82
do {
83
-
*port = *tbuf++;
83
+
*(volatile u16 __force *)port = *tbuf++;
84
84
} while (--count != 0);
85
85
asm volatile("sync");
86
86
}
···
95
95
return;
96
96
asm volatile("sync");
97
97
do {
98
-
tmp = *port;
98
+
tmp = *(const volatile u32 __force *)port;
99
99
eieio();
100
100
*tbuf++ = tmp;
101
101
} while (--count != 0);
···
111
111
return;
112
112
asm volatile("sync");
113
113
do {
114
-
*port = *tbuf++;
114
+
*(volatile u32 __force *)port = *tbuf++;
115
115
} while (--count != 0);
116
116
asm volatile("sync");
117
117
}
+4
-4
arch/powerpc/kernel/iommu.c
+4
-4
arch/powerpc/kernel/iommu.c
···
1074
1074
}
1075
1075
EXPORT_SYMBOL_GPL(iommu_tce_check_gpa);
1076
1076
1077
-
extern long iommu_tce_xchg_no_kill(struct mm_struct *mm,
1078
-
struct iommu_table *tbl,
1079
-
unsigned long entry, unsigned long *hpa,
1080
-
enum dma_data_direction *direction)
1077
+
long iommu_tce_xchg_no_kill(struct mm_struct *mm,
1078
+
struct iommu_table *tbl,
1079
+
unsigned long entry, unsigned long *hpa,
1080
+
enum dma_data_direction *direction)
1081
1081
{
1082
1082
long ret;
1083
1083
unsigned long size = 0;
+23
-3
arch/powerpc/kernel/process.c
+23
-3
arch/powerpc/kernel/process.c
···
2258
2258
return ret;
2259
2259
}
2260
2260
2261
+
static bool empty_user_regs(struct pt_regs *regs, struct task_struct *tsk)
2262
+
{
2263
+
unsigned long stack_page;
2264
+
2265
+
// A non-empty pt_regs should never have a zero MSR or TRAP value.
2266
+
if (regs->msr || regs->trap)
2267
+
return false;
2268
+
2269
+
// Check it sits at the very base of the stack
2270
+
stack_page = (unsigned long)task_stack_page(tsk);
2271
+
if ((unsigned long)(regs + 1) != stack_page + THREAD_SIZE)
2272
+
return false;
2273
+
2274
+
return true;
2275
+
}
2276
+
2261
2277
static int kstack_depth_to_print = CONFIG_PRINT_STACK_DEPTH;
2262
2278
2263
2279
void __no_sanitize_address show_stack(struct task_struct *tsk,
···
2338
2322
lr = regs->link;
2339
2323
printk("%s--- interrupt: %lx at %pS\n",
2340
2324
loglvl, regs->trap, (void *)regs->nip);
2341
-
__show_regs(regs);
2342
-
printk("%s--- interrupt: %lx\n",
2343
-
loglvl, regs->trap);
2325
+
2326
+
// Detect the case of an empty pt_regs at the very base
2327
+
// of the stack and suppress showing it in full.
2328
+
if (!empty_user_regs(regs, tsk)) {
2329
+
__show_regs(regs);
2330
+
printk("%s--- interrupt: %lx\n", loglvl, regs->trap);
2331
+
}
2344
2332
2345
2333
firstframe = 1;
2346
2334
}
+1
-1
arch/powerpc/kernel/prom_init.c
+1
-1
arch/powerpc/kernel/prom_init.c
+1
-4
arch/powerpc/kernel/ptrace/ptrace.c
+1
-4
arch/powerpc/kernel/ptrace/ptrace.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-or-later
1
2
/*
2
3
* PowerPC version
3
4
* Copyright (C) 1995-1996 Gary Thomas (gdt@linuxppc.org)
···
10
9
*
11
10
* Modified by Cort Dougan (cort@hq.fsmlabs.com)
12
11
* and Paul Mackerras (paulus@samba.org).
13
-
*
14
-
* This file is subject to the terms and conditions of the GNU General
15
-
* Public License. See the file README.legal in the main directory of
16
-
* this archive for more details.
17
12
*/
18
13
19
14
#include <linux/regset.h>
+1
-1
arch/powerpc/kernel/setup_64.c
+1
-1
arch/powerpc/kernel/setup_64.c
+1
-4
arch/powerpc/kernel/signal.c
+1
-4
arch/powerpc/kernel/signal.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-or-later
1
2
/*
2
3
* Common signal handling code for both 32 and 64 bits
3
4
*
4
5
* Copyright (c) 2007 Benjamin Herrenschmidt, IBM Corporation
5
6
* Extracted from signal_32.c and signal_64.c
6
-
*
7
-
* This file is subject to the terms and conditions of the GNU General
8
-
* Public License. See the file README.legal in the main directory of
9
-
* this archive for more details.
10
7
*/
11
8
12
9
#include <linux/resume_user_mode.h>
+2
-5
arch/powerpc/kernel/signal.h
+2
-5
arch/powerpc/kernel/signal.h
···
1
-
/*
1
+
/* SPDX-License-Identifier: GPL-2.0-or-later
2
+
*
2
3
* Copyright (c) 2007 Benjamin Herrenschmidt, IBM Corporation
3
4
* Extracted from signal_32.c and signal_64.c
4
-
*
5
-
* This file is subject to the terms and conditions of the GNU General
6
-
* Public License. See the file README.legal in the main directory of
7
-
* this archive for more details.
8
5
*/
9
6
10
7
#ifndef _POWERPC_ARCH_SIGNAL_H
+4
-2
arch/powerpc/kernel/traps.c
+4
-2
arch/powerpc/kernel/traps.c
···
157
157
static unsigned int die_nest_count;
158
158
static int die_counter;
159
159
160
-
extern void panic_flush_kmsg_start(void)
160
+
void panic_flush_kmsg_start(void)
161
161
{
162
162
/*
163
163
* These are mostly taken from kernel/panic.c, but tries to do
···
170
170
bust_spinlocks(1);
171
171
}
172
172
173
-
extern void panic_flush_kmsg_end(void)
173
+
void panic_flush_kmsg_end(void)
174
174
{
175
175
kmsg_dump(KMSG_DUMP_PANIC);
176
176
bust_spinlocks(0);
···
1164
1164
__single_step_exception(regs);
1165
1165
}
1166
1166
1167
+
#ifdef CONFIG_PPC_FPU_REGS
1167
1168
static inline int __parse_fpscr(unsigned long fpscr)
1168
1169
{
1169
1170
int ret = FPE_FLTUNK;
···
1191
1190
1192
1191
return ret;
1193
1192
}
1193
+
#endif
1194
1194
1195
1195
static void parse_fpe(struct pt_regs *regs)
1196
1196
{
+3
arch/powerpc/kexec/core.c
+3
arch/powerpc/kexec/core.c
···
74
74
VMCOREINFO_STRUCT_SIZE(mmu_psize_def);
75
75
VMCOREINFO_OFFSET(mmu_psize_def, shift);
76
76
#endif
77
+
VMCOREINFO_SYMBOL(cur_cpu_spec);
78
+
VMCOREINFO_OFFSET(cpu_spec, mmu_features);
79
+
vmcoreinfo_append_str("NUMBER(RADIX_MMU)=%d\n", early_radix_enabled());
77
80
vmcoreinfo_append_str("KERNELOFFSET=%lx\n", kaslr_offset());
78
81
}
79
82
+2
-2
arch/powerpc/kexec/core_64.c
+2
-2
arch/powerpc/kexec/core_64.c
···
379
379
380
380
#ifdef CONFIG_PPC_64S_HASH_MMU
381
381
/* Values we need to export to the second kernel via the device tree. */
382
-
static unsigned long htab_base;
383
-
static unsigned long htab_size;
382
+
static __be64 htab_base;
383
+
static __be64 htab_size;
384
384
385
385
static struct property htab_base_prop = {
386
386
.name = "linux,htab-base",
+9
-5
arch/powerpc/kexec/file_load_64.c
+9
-5
arch/powerpc/kexec/file_load_64.c
···
32
32
#include <asm/plpks.h>
33
33
34
34
struct umem_info {
35
-
u64 *buf; /* data buffer for usable-memory property */
35
+
__be64 *buf; /* data buffer for usable-memory property */
36
36
u32 size; /* size allocated for the data buffer */
37
37
u32 max_entries; /* maximum no. of entries */
38
38
u32 idx; /* index of current entry */
···
443
443
*
444
444
* Returns buffer on success, NULL on error.
445
445
*/
446
-
static u64 *check_realloc_usable_mem(struct umem_info *um_info, int cnt)
446
+
static __be64 *check_realloc_usable_mem(struct umem_info *um_info, int cnt)
447
447
{
448
448
u32 new_size;
449
-
u64 *tbuf;
449
+
__be64 *tbuf;
450
450
451
451
if ((um_info->idx + cnt) <= um_info->max_entries)
452
452
return um_info->buf;
···
1138
1138
continue;
1139
1139
1140
1140
ret = copy_property(fdt, pci_offset, dn, "ibm,dma-window");
1141
-
if (ret < 0)
1141
+
if (ret < 0) {
1142
+
of_node_put(dn);
1142
1143
break;
1144
+
}
1143
1145
ret = copy_property(fdt, pci_offset, dn, dmapropname);
1144
-
if (ret < 0)
1146
+
if (ret < 0) {
1147
+
of_node_put(dn);
1145
1148
break;
1149
+
}
1146
1150
}
1147
1151
1148
1152
return ret;
+4
arch/powerpc/kvm/Makefile
+4
arch/powerpc/kvm/Makefile
···
87
87
book3s_hv_ras.o \
88
88
book3s_hv_builtin.o \
89
89
book3s_hv_p9_perf.o \
90
+
book3s_hv_nestedv2.o \
91
+
guest-state-buffer.o \
90
92
$(kvm-book3s_64-builtin-tm-objs-y) \
91
93
$(kvm-book3s_64-builtin-xics-objs-y)
94
+
95
+
obj-$(CONFIG_GUEST_STATE_BUFFER_TEST) += test-guest-state-buffer.o
92
96
endif
93
97
94
98
kvm-book3s_64-objs-$(CONFIG_KVM_XICS) += \
+19
-19
arch/powerpc/kvm/book3s.c
+19
-19
arch/powerpc/kvm/book3s.c
···
565
565
regs->msr = kvmppc_get_msr(vcpu);
566
566
regs->srr0 = kvmppc_get_srr0(vcpu);
567
567
regs->srr1 = kvmppc_get_srr1(vcpu);
568
-
regs->pid = vcpu->arch.pid;
568
+
regs->pid = kvmppc_get_pid(vcpu);
569
569
regs->sprg0 = kvmppc_get_sprg0(vcpu);
570
570
regs->sprg1 = kvmppc_get_sprg1(vcpu);
571
571
regs->sprg2 = kvmppc_get_sprg2(vcpu);
···
636
636
break;
637
637
case KVM_REG_PPC_FPR0 ... KVM_REG_PPC_FPR31:
638
638
i = id - KVM_REG_PPC_FPR0;
639
-
*val = get_reg_val(id, VCPU_FPR(vcpu, i));
639
+
*val = get_reg_val(id, kvmppc_get_fpr(vcpu, i));
640
640
break;
641
641
case KVM_REG_PPC_FPSCR:
642
-
*val = get_reg_val(id, vcpu->arch.fp.fpscr);
642
+
*val = get_reg_val(id, kvmppc_get_fpscr(vcpu));
643
643
break;
644
644
#ifdef CONFIG_VSX
645
645
case KVM_REG_PPC_VSR0 ... KVM_REG_PPC_VSR31:
646
646
if (cpu_has_feature(CPU_FTR_VSX)) {
647
647
i = id - KVM_REG_PPC_VSR0;
648
-
val->vsxval[0] = vcpu->arch.fp.fpr[i][0];
649
-
val->vsxval[1] = vcpu->arch.fp.fpr[i][1];
648
+
val->vsxval[0] = kvmppc_get_vsx_fpr(vcpu, i, 0);
649
+
val->vsxval[1] = kvmppc_get_vsx_fpr(vcpu, i, 1);
650
650
} else {
651
651
r = -ENXIO;
652
652
}
···
683
683
*val = get_reg_val(id, vcpu->arch.fscr);
684
684
break;
685
685
case KVM_REG_PPC_TAR:
686
-
*val = get_reg_val(id, vcpu->arch.tar);
686
+
*val = get_reg_val(id, kvmppc_get_tar(vcpu));
687
687
break;
688
688
case KVM_REG_PPC_EBBHR:
689
-
*val = get_reg_val(id, vcpu->arch.ebbhr);
689
+
*val = get_reg_val(id, kvmppc_get_ebbhr(vcpu));
690
690
break;
691
691
case KVM_REG_PPC_EBBRR:
692
-
*val = get_reg_val(id, vcpu->arch.ebbrr);
692
+
*val = get_reg_val(id, kvmppc_get_ebbrr(vcpu));
693
693
break;
694
694
case KVM_REG_PPC_BESCR:
695
-
*val = get_reg_val(id, vcpu->arch.bescr);
695
+
*val = get_reg_val(id, kvmppc_get_bescr(vcpu));
696
696
break;
697
697
case KVM_REG_PPC_IC:
698
-
*val = get_reg_val(id, vcpu->arch.ic);
698
+
*val = get_reg_val(id, kvmppc_get_ic(vcpu));
699
699
break;
700
700
default:
701
701
r = -EINVAL;
···
724
724
break;
725
725
case KVM_REG_PPC_FPR0 ... KVM_REG_PPC_FPR31:
726
726
i = id - KVM_REG_PPC_FPR0;
727
-
VCPU_FPR(vcpu, i) = set_reg_val(id, *val);
727
+
kvmppc_set_fpr(vcpu, i, set_reg_val(id, *val));
728
728
break;
729
729
case KVM_REG_PPC_FPSCR:
730
730
vcpu->arch.fp.fpscr = set_reg_val(id, *val);
···
733
733
case KVM_REG_PPC_VSR0 ... KVM_REG_PPC_VSR31:
734
734
if (cpu_has_feature(CPU_FTR_VSX)) {
735
735
i = id - KVM_REG_PPC_VSR0;
736
-
vcpu->arch.fp.fpr[i][0] = val->vsxval[0];
737
-
vcpu->arch.fp.fpr[i][1] = val->vsxval[1];
736
+
kvmppc_set_vsx_fpr(vcpu, i, 0, val->vsxval[0]);
737
+
kvmppc_set_vsx_fpr(vcpu, i, 1, val->vsxval[1]);
738
738
} else {
739
739
r = -ENXIO;
740
740
}
···
765
765
break;
766
766
#endif /* CONFIG_KVM_XIVE */
767
767
case KVM_REG_PPC_FSCR:
768
-
vcpu->arch.fscr = set_reg_val(id, *val);
768
+
kvmppc_set_fpscr(vcpu, set_reg_val(id, *val));
769
769
break;
770
770
case KVM_REG_PPC_TAR:
771
-
vcpu->arch.tar = set_reg_val(id, *val);
771
+
kvmppc_set_tar(vcpu, set_reg_val(id, *val));
772
772
break;
773
773
case KVM_REG_PPC_EBBHR:
774
-
vcpu->arch.ebbhr = set_reg_val(id, *val);
774
+
kvmppc_set_ebbhr(vcpu, set_reg_val(id, *val));
775
775
break;
776
776
case KVM_REG_PPC_EBBRR:
777
-
vcpu->arch.ebbrr = set_reg_val(id, *val);
777
+
kvmppc_set_ebbrr(vcpu, set_reg_val(id, *val));
778
778
break;
779
779
case KVM_REG_PPC_BESCR:
780
-
vcpu->arch.bescr = set_reg_val(id, *val);
780
+
kvmppc_set_bescr(vcpu, set_reg_val(id, *val));
781
781
break;
782
782
case KVM_REG_PPC_IC:
783
-
vcpu->arch.ic = set_reg_val(id, *val);
783
+
kvmppc_set_ic(vcpu, set_reg_val(id, *val));
784
784
break;
785
785
default:
786
786
r = -EINVAL;
+4
-3
arch/powerpc/kvm/book3s_64_mmu_hv.c
+4
-3
arch/powerpc/kvm/book3s_64_mmu_hv.c
···
28
28
#include <asm/pte-walk.h>
29
29
30
30
#include "book3s.h"
31
+
#include "book3s_hv.h"
31
32
#include "trace_hv.h"
32
33
33
34
//#define DEBUG_RESIZE_HPT 1
···
121
120
kvm->arch.hpt = *info;
122
121
kvm->arch.sdr1 = __pa(info->virt) | (info->order - 18);
123
122
124
-
pr_debug("KVM guest htab at %lx (order %ld), LPID %x\n",
123
+
pr_debug("KVM guest htab at %lx (order %ld), LPID %llx\n",
125
124
info->virt, (long)info->order, kvm->arch.lpid);
126
125
}
127
126
···
348
347
unsigned long v, orig_v, gr;
349
348
__be64 *hptep;
350
349
long int index;
351
-
int virtmode = vcpu->arch.shregs.msr & (data ? MSR_DR : MSR_IR);
350
+
int virtmode = __kvmppc_get_msr_hv(vcpu) & (data ? MSR_DR : MSR_IR);
352
351
353
352
if (kvm_is_radix(vcpu->kvm))
354
353
return kvmppc_mmu_radix_xlate(vcpu, eaddr, gpte, data, iswrite);
···
386
385
387
386
/* Get PP bits and key for permission check */
388
387
pp = gr & (HPTE_R_PP0 | HPTE_R_PP);
389
-
key = (vcpu->arch.shregs.msr & MSR_PR) ? SLB_VSID_KP : SLB_VSID_KS;
388
+
key = (__kvmppc_get_msr_hv(vcpu) & MSR_PR) ? SLB_VSID_KP : SLB_VSID_KS;
390
389
key &= slb_v;
391
390
392
391
/* Calculate permissions */
+16
-15
arch/powerpc/kvm/book3s_64_mmu_radix.c
+16
-15
arch/powerpc/kvm/book3s_64_mmu_radix.c
···
15
15
16
16
#include <asm/kvm_ppc.h>
17
17
#include <asm/kvm_book3s.h>
18
+
#include "book3s_hv.h"
18
19
#include <asm/page.h>
19
20
#include <asm/mmu.h>
20
21
#include <asm/pgalloc.h>
···
97
96
void *to, void *from, unsigned long n)
98
97
{
99
98
int lpid = vcpu->kvm->arch.lpid;
100
-
int pid = vcpu->arch.pid;
99
+
int pid = kvmppc_get_pid(vcpu);
101
100
102
101
/* This would cause a data segment intr so don't allow the access */
103
102
if (eaddr & (0x3FFUL << 52))
···
271
270
/* Work out effective PID */
272
271
switch (eaddr >> 62) {
273
272
case 0:
274
-
pid = vcpu->arch.pid;
273
+
pid = kvmppc_get_pid(vcpu);
275
274
break;
276
275
case 3:
277
276
pid = 0;
···
295
294
} else {
296
295
if (!(pte & _PAGE_PRIVILEGED)) {
297
296
/* Check AMR/IAMR to see if strict mode is in force */
298
-
if (vcpu->arch.amr & (1ul << 62))
297
+
if (kvmppc_get_amr_hv(vcpu) & (1ul << 62))
299
298
gpte->may_read = 0;
300
-
if (vcpu->arch.amr & (1ul << 63))
299
+
if (kvmppc_get_amr_hv(vcpu) & (1ul << 63))
301
300
gpte->may_write = 0;
302
301
if (vcpu->arch.iamr & (1ul << 62))
303
302
gpte->may_execute = 0;
···
308
307
}
309
308
310
309
void kvmppc_radix_tlbie_page(struct kvm *kvm, unsigned long addr,
311
-
unsigned int pshift, unsigned int lpid)
310
+
unsigned int pshift, u64 lpid)
312
311
{
313
312
unsigned long psize = PAGE_SIZE;
314
313
int psi;
···
345
344
pr_err("KVM: TLB page invalidation hcall failed, rc=%ld\n", rc);
346
345
}
347
346
348
-
static void kvmppc_radix_flush_pwc(struct kvm *kvm, unsigned int lpid)
347
+
static void kvmppc_radix_flush_pwc(struct kvm *kvm, u64 lpid)
349
348
{
350
349
long rc;
351
350
···
418
417
void kvmppc_unmap_pte(struct kvm *kvm, pte_t *pte, unsigned long gpa,
419
418
unsigned int shift,
420
419
const struct kvm_memory_slot *memslot,
421
-
unsigned int lpid)
420
+
u64 lpid)
422
421
423
422
{
424
423
unsigned long old;
···
469
468
* (or 4kB) mappings (of sub-pages of the same 2MB page).
470
469
*/
471
470
static void kvmppc_unmap_free_pte(struct kvm *kvm, pte_t *pte, bool full,
472
-
unsigned int lpid)
471
+
u64 lpid)
473
472
{
474
473
if (full) {
475
474
memset(pte, 0, sizeof(long) << RADIX_PTE_INDEX_SIZE);
···
490
489
}
491
490
492
491
static void kvmppc_unmap_free_pmd(struct kvm *kvm, pmd_t *pmd, bool full,
493
-
unsigned int lpid)
492
+
u64 lpid)
494
493
{
495
494
unsigned long im;
496
495
pmd_t *p = pmd;
···
519
518
}
520
519
521
520
static void kvmppc_unmap_free_pud(struct kvm *kvm, pud_t *pud,
522
-
unsigned int lpid)
521
+
u64 lpid)
523
522
{
524
523
unsigned long iu;
525
524
pud_t *p = pud;
···
540
539
pud_free(kvm->mm, pud);
541
540
}
542
541
543
-
void kvmppc_free_pgtable_radix(struct kvm *kvm, pgd_t *pgd, unsigned int lpid)
542
+
void kvmppc_free_pgtable_radix(struct kvm *kvm, pgd_t *pgd, u64 lpid)
544
543
{
545
544
unsigned long ig;
546
545
···
567
566
}
568
567
569
568
static void kvmppc_unmap_free_pmd_entry_table(struct kvm *kvm, pmd_t *pmd,
570
-
unsigned long gpa, unsigned int lpid)
569
+
unsigned long gpa, u64 lpid)
571
570
{
572
571
pte_t *pte = pte_offset_kernel(pmd, 0);
573
572
···
583
582
}
584
583
585
584
static void kvmppc_unmap_free_pud_entry_table(struct kvm *kvm, pud_t *pud,
586
-
unsigned long gpa, unsigned int lpid)
585
+
unsigned long gpa, u64 lpid)
587
586
{
588
587
pmd_t *pmd = pmd_offset(pud, 0);
589
588
···
609
608
610
609
int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
611
610
unsigned long gpa, unsigned int level,
612
-
unsigned long mmu_seq, unsigned int lpid,
611
+
unsigned long mmu_seq, u64 lpid,
613
612
unsigned long *rmapp, struct rmap_nested **n_rmap)
614
613
{
615
614
pgd_t *pgd;
···
786
785
}
787
786
788
787
bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nested, bool writing,
789
-
unsigned long gpa, unsigned int lpid)
788
+
unsigned long gpa, u64 lpid)
790
789
{
791
790
unsigned long pgflags;
792
791
unsigned int shift;
+6
-6
arch/powerpc/kvm/book3s_64_vio.c
+6
-6
arch/powerpc/kvm/book3s_64_vio.c
···
77
77
call_rcu(&stit->rcu, kvm_spapr_tce_iommu_table_free);
78
78
}
79
79
80
-
extern void kvm_spapr_tce_release_iommu_group(struct kvm *kvm,
81
-
struct iommu_group *grp)
80
+
void kvm_spapr_tce_release_iommu_group(struct kvm *kvm,
81
+
struct iommu_group *grp)
82
82
{
83
83
int i;
84
84
struct kvmppc_spapr_tce_table *stt;
···
105
105
rcu_read_unlock();
106
106
}
107
107
108
-
extern long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
109
-
struct iommu_group *grp)
108
+
long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
109
+
struct iommu_group *grp)
110
110
{
111
111
struct kvmppc_spapr_tce_table *stt = NULL;
112
112
bool found = false;
···
786
786
idx = (ioba >> stt->page_shift) - stt->offset;
787
787
page = stt->pages[idx / TCES_PER_PAGE];
788
788
if (!page) {
789
-
vcpu->arch.regs.gpr[4] = 0;
789
+
kvmppc_set_gpr(vcpu, 4, 0);
790
790
return H_SUCCESS;
791
791
}
792
792
tbl = (u64 *)page_address(page);
793
793
794
-
vcpu->arch.regs.gpr[4] = tbl[idx % TCES_PER_PAGE];
794
+
kvmppc_set_gpr(vcpu, 4, tbl[idx % TCES_PER_PAGE]);
795
795
796
796
return H_SUCCESS;
797
797
}
+243
-117
arch/powerpc/kvm/book3s_hv.c
+243
-117
arch/powerpc/kvm/book3s_hv.c
···
393
393
394
394
static int kvmppc_set_arch_compat(struct kvm_vcpu *vcpu, u32 arch_compat)
395
395
{
396
-
unsigned long host_pcr_bit = 0, guest_pcr_bit = 0;
396
+
unsigned long host_pcr_bit = 0, guest_pcr_bit = 0, cap = 0;
397
397
struct kvmppc_vcore *vc = vcpu->arch.vcore;
398
398
399
399
/* We can (emulate) our own architecture version and anything older */
···
424
424
break;
425
425
case PVR_ARCH_300:
426
426
guest_pcr_bit = PCR_ARCH_300;
427
+
cap = H_GUEST_CAP_POWER9;
427
428
break;
428
429
case PVR_ARCH_31:
429
430
guest_pcr_bit = PCR_ARCH_31;
431
+
cap = H_GUEST_CAP_POWER10;
430
432
break;
431
433
default:
432
434
return -EINVAL;
···
439
437
if (guest_pcr_bit > host_pcr_bit)
440
438
return -EINVAL;
441
439
440
+
if (kvmhv_on_pseries() && kvmhv_is_nestedv2()) {
441
+
if (!(cap & nested_capabilities))
442
+
return -EINVAL;
443
+
}
444
+
442
445
spin_lock(&vc->lock);
443
446
vc->arch_compat = arch_compat;
447
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_LOGICAL_PVR);
444
448
/*
445
449
* Set all PCR bits for which guest_pcr_bit <= bit < host_pcr_bit
446
450
* Also set all reserved PCR bits
···
802
794
803
795
vpa->enqueue_dispatch_tb = cpu_to_be64(be64_to_cpu(vpa->enqueue_dispatch_tb) + stolen);
804
796
805
-
__kvmppc_create_dtl_entry(vcpu, vpa, vc->pcpu, now + vc->tb_offset, stolen);
797
+
__kvmppc_create_dtl_entry(vcpu, vpa, vc->pcpu, now + kvmppc_get_tb_offset(vcpu), stolen);
806
798
807
799
vcpu->arch.vpa.dirty = true;
808
800
}
···
853
845
854
846
static bool kvmppc_power8_compatible(struct kvm_vcpu *vcpu)
855
847
{
856
-
if (vcpu->arch.vcore->arch_compat >= PVR_ARCH_207)
848
+
if (kvmppc_get_arch_compat(vcpu) >= PVR_ARCH_207)
857
849
return true;
858
-
if ((!vcpu->arch.vcore->arch_compat) &&
850
+
if ((!kvmppc_get_arch_compat(vcpu)) &&
859
851
cpu_has_feature(CPU_FTR_ARCH_207S))
860
852
return true;
861
853
return false;
···
876
868
/* Guests can't breakpoint the hypervisor */
877
869
if ((value1 & CIABR_PRIV) == CIABR_PRIV_HYPER)
878
870
return H_P3;
879
-
vcpu->arch.ciabr = value1;
871
+
kvmppc_set_ciabr_hv(vcpu, value1);
880
872
return H_SUCCESS;
881
873
case H_SET_MODE_RESOURCE_SET_DAWR0:
882
874
if (!kvmppc_power8_compatible(vcpu))
···
887
879
return H_UNSUPPORTED_FLAG_START;
888
880
if (value2 & DABRX_HYP)
889
881
return H_P4;
890
-
vcpu->arch.dawr0 = value1;
891
-
vcpu->arch.dawrx0 = value2;
882
+
kvmppc_set_dawr0_hv(vcpu, value1);
883
+
kvmppc_set_dawrx0_hv(vcpu, value2);
892
884
return H_SUCCESS;
893
885
case H_SET_MODE_RESOURCE_SET_DAWR1:
894
886
if (!kvmppc_power8_compatible(vcpu))
···
903
895
return H_UNSUPPORTED_FLAG_START;
904
896
if (value2 & DABRX_HYP)
905
897
return H_P4;
906
-
vcpu->arch.dawr1 = value1;
907
-
vcpu->arch.dawrx1 = value2;
898
+
kvmppc_set_dawr1_hv(vcpu, value1);
899
+
kvmppc_set_dawrx1_hv(vcpu, value2);
908
900
return H_SUCCESS;
909
901
case H_SET_MODE_RESOURCE_ADDR_TRANS_MODE:
910
902
/*
···
1275
1267
return RESUME_HOST;
1276
1268
break;
1277
1269
#endif
1278
-
case H_RANDOM:
1279
-
if (!arch_get_random_seed_longs(&vcpu->arch.regs.gpr[4], 1))
1270
+
case H_RANDOM: {
1271
+
unsigned long rand;
1272
+
1273
+
if (!arch_get_random_seed_longs(&rand, 1))
1280
1274
ret = H_HARDWARE;
1275
+
kvmppc_set_gpr(vcpu, 4, rand);
1281
1276
break;
1277
+
}
1282
1278
case H_RPT_INVALIDATE:
1283
1279
ret = kvmppc_h_rpt_invalidate(vcpu, kvmppc_get_gpr(vcpu, 4),
1284
1280
kvmppc_get_gpr(vcpu, 5),
···
1382
1370
*/
1383
1371
static void kvmppc_cede(struct kvm_vcpu *vcpu)
1384
1372
{
1385
-
vcpu->arch.shregs.msr |= MSR_EE;
1373
+
__kvmppc_set_msr_hv(vcpu, __kvmppc_get_msr_hv(vcpu) | MSR_EE);
1386
1374
vcpu->arch.ceded = 1;
1387
1375
smp_mb();
1388
1376
if (vcpu->arch.prodded) {
···
1556
1544
if (!(vcpu->arch.hfscr_permitted & HFSCR_PM))
1557
1545
return EMULATE_FAIL;
1558
1546
1559
-
vcpu->arch.hfscr |= HFSCR_PM;
1547
+
kvmppc_set_hfscr_hv(vcpu, kvmppc_get_hfscr_hv(vcpu) | HFSCR_PM);
1560
1548
1561
1549
return RESUME_GUEST;
1562
1550
}
···
1566
1554
if (!(vcpu->arch.hfscr_permitted & HFSCR_EBB))
1567
1555
return EMULATE_FAIL;
1568
1556
1569
-
vcpu->arch.hfscr |= HFSCR_EBB;
1557
+
kvmppc_set_hfscr_hv(vcpu, kvmppc_get_hfscr_hv(vcpu) | HFSCR_EBB);
1570
1558
1571
1559
return RESUME_GUEST;
1572
1560
}
···
1576
1564
if (!(vcpu->arch.hfscr_permitted & HFSCR_TM))
1577
1565
return EMULATE_FAIL;
1578
1566
1579
-
vcpu->arch.hfscr |= HFSCR_TM;
1567
+
kvmppc_set_hfscr_hv(vcpu, kvmppc_get_hfscr_hv(vcpu) | HFSCR_TM);
1580
1568
1581
1569
return RESUME_GUEST;
1582
1570
}
···
1597
1585
* That can happen due to a bug, or due to a machine check
1598
1586
* occurring at just the wrong time.
1599
1587
*/
1600
-
if (vcpu->arch.shregs.msr & MSR_HV) {
1588
+
if (__kvmppc_get_msr_hv(vcpu) & MSR_HV) {
1601
1589
printk(KERN_EMERG "KVM trap in HV mode!\n");
1602
1590
printk(KERN_EMERG "trap=0x%x | pc=0x%lx | msr=0x%llx\n",
1603
1591
vcpu->arch.trap, kvmppc_get_pc(vcpu),
···
1648
1636
* so that it knows that the machine check occurred.
1649
1637
*/
1650
1638
if (!vcpu->kvm->arch.fwnmi_enabled) {
1651
-
ulong flags = (vcpu->arch.shregs.msr & 0x083c0000) |
1639
+
ulong flags = (__kvmppc_get_msr_hv(vcpu) & 0x083c0000) |
1652
1640
(kvmppc_get_msr(vcpu) & SRR1_PREFIXED);
1653
1641
kvmppc_core_queue_machine_check(vcpu, flags);
1654
1642
r = RESUME_GUEST;
···
1678
1666
* as a result of a hypervisor emulation interrupt
1679
1667
* (e40) getting turned into a 700 by BML RTAS.
1680
1668
*/
1681
-
flags = (vcpu->arch.shregs.msr & 0x1f0000ull) |
1669
+
flags = (__kvmppc_get_msr_hv(vcpu) & 0x1f0000ull) |
1682
1670
(kvmppc_get_msr(vcpu) & SRR1_PREFIXED);
1683
1671
kvmppc_core_queue_program(vcpu, flags);
1684
1672
r = RESUME_GUEST;
···
1688
1676
{
1689
1677
int i;
1690
1678
1691
-
if (unlikely(vcpu->arch.shregs.msr & MSR_PR)) {
1679
+
if (unlikely(__kvmppc_get_msr_hv(vcpu) & MSR_PR)) {
1692
1680
/*
1693
1681
* Guest userspace executed sc 1. This can only be
1694
1682
* reached by the P9 path because the old path
···
1766
1754
break;
1767
1755
}
1768
1756
1769
-
if (!(vcpu->arch.shregs.msr & MSR_DR))
1757
+
if (!(__kvmppc_get_msr_hv(vcpu) & MSR_DR))
1770
1758
vsid = vcpu->kvm->arch.vrma_slb_v;
1771
1759
else
1772
1760
vsid = vcpu->arch.fault_gpa;
···
1790
1778
long err;
1791
1779
1792
1780
vcpu->arch.fault_dar = kvmppc_get_pc(vcpu);
1793
-
vcpu->arch.fault_dsisr = vcpu->arch.shregs.msr &
1781
+
vcpu->arch.fault_dsisr = __kvmppc_get_msr_hv(vcpu) &
1794
1782
DSISR_SRR1_MATCH_64S;
1795
1783
if (kvm_is_radix(vcpu->kvm) || !cpu_has_feature(CPU_FTR_ARCH_300)) {
1796
1784
/*
···
1799
1787
* hash fault handling below is v3 only (it uses ASDR
1800
1788
* via fault_gpa).
1801
1789
*/
1802
-
if (vcpu->arch.shregs.msr & HSRR1_HISI_WRITE)
1790
+
if (__kvmppc_get_msr_hv(vcpu) & HSRR1_HISI_WRITE)
1803
1791
vcpu->arch.fault_dsisr |= DSISR_ISSTORE;
1804
1792
r = RESUME_PAGE_FAULT;
1805
1793
break;
···
1813
1801
break;
1814
1802
}
1815
1803
1816
-
if (!(vcpu->arch.shregs.msr & MSR_IR))
1804
+
if (!(__kvmppc_get_msr_hv(vcpu) & MSR_IR))
1817
1805
vsid = vcpu->kvm->arch.vrma_slb_v;
1818
1806
else
1819
1807
vsid = vcpu->arch.fault_gpa;
···
1875
1863
* Otherwise, we just generate a program interrupt to the guest.
1876
1864
*/
1877
1865
case BOOK3S_INTERRUPT_H_FAC_UNAVAIL: {
1878
-
u64 cause = vcpu->arch.hfscr >> 56;
1866
+
u64 cause = kvmppc_get_hfscr_hv(vcpu) >> 56;
1879
1867
1880
1868
r = EMULATE_FAIL;
1881
1869
if (cpu_has_feature(CPU_FTR_ARCH_300)) {
···
1903
1891
kvmppc_dump_regs(vcpu);
1904
1892
printk(KERN_EMERG "trap=0x%x | pc=0x%lx | msr=0x%llx\n",
1905
1893
vcpu->arch.trap, kvmppc_get_pc(vcpu),
1906
-
vcpu->arch.shregs.msr);
1894
+
__kvmppc_get_msr_hv(vcpu));
1907
1895
run->hw.hardware_exit_reason = vcpu->arch.trap;
1908
1896
r = RESUME_HOST;
1909
1897
break;
···
1927
1915
* That can happen due to a bug, or due to a machine check
1928
1916
* occurring at just the wrong time.
1929
1917
*/
1930
-
if (vcpu->arch.shregs.msr & MSR_HV) {
1918
+
if (__kvmppc_get_msr_hv(vcpu) & MSR_HV) {
1931
1919
pr_emerg("KVM trap in HV mode while nested!\n");
1932
1920
pr_emerg("trap=0x%x | pc=0x%lx | msr=0x%llx\n",
1933
1921
vcpu->arch.trap, kvmppc_get_pc(vcpu),
1934
-
vcpu->arch.shregs.msr);
1922
+
__kvmppc_get_msr_hv(vcpu));
1935
1923
kvmppc_dump_regs(vcpu);
1936
1924
return RESUME_HOST;
1937
1925
}
···
1988
1976
vcpu->arch.fault_dar = kvmppc_get_pc(vcpu);
1989
1977
vcpu->arch.fault_dsisr = kvmppc_get_msr(vcpu) &
1990
1978
DSISR_SRR1_MATCH_64S;
1991
-
if (vcpu->arch.shregs.msr & HSRR1_HISI_WRITE)
1979
+
if (__kvmppc_get_msr_hv(vcpu) & HSRR1_HISI_WRITE)
1992
1980
vcpu->arch.fault_dsisr |= DSISR_ISSTORE;
1993
1981
srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
1994
1982
r = kvmhv_nested_page_fault(vcpu);
···
2195
2183
}
2196
2184
2197
2185
vc->lpcr = new_lpcr;
2186
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_LPCR);
2198
2187
2199
2188
spin_unlock(&vc->lock);
2200
2189
}
···
2220
2207
*val = get_reg_val(id, vcpu->arch.dabrx);
2221
2208
break;
2222
2209
case KVM_REG_PPC_DSCR:
2223
-
*val = get_reg_val(id, vcpu->arch.dscr);
2210
+
*val = get_reg_val(id, kvmppc_get_dscr_hv(vcpu));
2224
2211
break;
2225
2212
case KVM_REG_PPC_PURR:
2226
-
*val = get_reg_val(id, vcpu->arch.purr);
2213
+
*val = get_reg_val(id, kvmppc_get_purr_hv(vcpu));
2227
2214
break;
2228
2215
case KVM_REG_PPC_SPURR:
2229
-
*val = get_reg_val(id, vcpu->arch.spurr);
2216
+
*val = get_reg_val(id, kvmppc_get_spurr_hv(vcpu));
2230
2217
break;
2231
2218
case KVM_REG_PPC_AMR:
2232
-
*val = get_reg_val(id, vcpu->arch.amr);
2219
+
*val = get_reg_val(id, kvmppc_get_amr_hv(vcpu));
2233
2220
break;
2234
2221
case KVM_REG_PPC_UAMOR:
2235
-
*val = get_reg_val(id, vcpu->arch.uamor);
2222
+
*val = get_reg_val(id, kvmppc_get_uamor_hv(vcpu));
2236
2223
break;
2237
2224
case KVM_REG_PPC_MMCR0 ... KVM_REG_PPC_MMCR1:
2238
2225
i = id - KVM_REG_PPC_MMCR0;
2239
-
*val = get_reg_val(id, vcpu->arch.mmcr[i]);
2226
+
*val = get_reg_val(id, kvmppc_get_mmcr_hv(vcpu, i));
2240
2227
break;
2241
2228
case KVM_REG_PPC_MMCR2:
2242
-
*val = get_reg_val(id, vcpu->arch.mmcr[2]);
2229
+
*val = get_reg_val(id, kvmppc_get_mmcr_hv(vcpu, 2));
2243
2230
break;
2244
2231
case KVM_REG_PPC_MMCRA:
2245
-
*val = get_reg_val(id, vcpu->arch.mmcra);
2232
+
*val = get_reg_val(id, kvmppc_get_mmcra_hv(vcpu));
2246
2233
break;
2247
2234
case KVM_REG_PPC_MMCRS:
2248
2235
*val = get_reg_val(id, vcpu->arch.mmcrs);
2249
2236
break;
2250
2237
case KVM_REG_PPC_MMCR3:
2251
-
*val = get_reg_val(id, vcpu->arch.mmcr[3]);
2238
+
*val = get_reg_val(id, kvmppc_get_mmcr_hv(vcpu, 3));
2252
2239
break;
2253
2240
case KVM_REG_PPC_PMC1 ... KVM_REG_PPC_PMC8:
2254
2241
i = id - KVM_REG_PPC_PMC1;
2255
-
*val = get_reg_val(id, vcpu->arch.pmc[i]);
2242
+
*val = get_reg_val(id, kvmppc_get_pmc_hv(vcpu, i));
2256
2243
break;
2257
2244
case KVM_REG_PPC_SPMC1 ... KVM_REG_PPC_SPMC2:
2258
2245
i = id - KVM_REG_PPC_SPMC1;
2259
2246
*val = get_reg_val(id, vcpu->arch.spmc[i]);
2260
2247
break;
2261
2248
case KVM_REG_PPC_SIAR:
2262
-
*val = get_reg_val(id, vcpu->arch.siar);
2249
+
*val = get_reg_val(id, kvmppc_get_siar_hv(vcpu));
2263
2250
break;
2264
2251
case KVM_REG_PPC_SDAR:
2265
-
*val = get_reg_val(id, vcpu->arch.sdar);
2252
+
*val = get_reg_val(id, kvmppc_get_siar_hv(vcpu));
2266
2253
break;
2267
2254
case KVM_REG_PPC_SIER:
2268
-
*val = get_reg_val(id, vcpu->arch.sier[0]);
2255
+
*val = get_reg_val(id, kvmppc_get_sier_hv(vcpu, 0));
2269
2256
break;
2270
2257
case KVM_REG_PPC_SIER2:
2271
-
*val = get_reg_val(id, vcpu->arch.sier[1]);
2258
+
*val = get_reg_val(id, kvmppc_get_sier_hv(vcpu, 1));
2272
2259
break;
2273
2260
case KVM_REG_PPC_SIER3:
2274
-
*val = get_reg_val(id, vcpu->arch.sier[2]);
2261
+
*val = get_reg_val(id, kvmppc_get_sier_hv(vcpu, 2));
2275
2262
break;
2276
2263
case KVM_REG_PPC_IAMR:
2277
-
*val = get_reg_val(id, vcpu->arch.iamr);
2264
+
*val = get_reg_val(id, kvmppc_get_iamr_hv(vcpu));
2278
2265
break;
2279
2266
case KVM_REG_PPC_PSPB:
2280
-
*val = get_reg_val(id, vcpu->arch.pspb);
2267
+
*val = get_reg_val(id, kvmppc_get_pspb_hv(vcpu));
2281
2268
break;
2282
2269
case KVM_REG_PPC_DPDES:
2283
2270
/*
···
2292
2279
*val = get_reg_val(id, vcpu->arch.vcore->dpdes);
2293
2280
break;
2294
2281
case KVM_REG_PPC_VTB:
2295
-
*val = get_reg_val(id, vcpu->arch.vcore->vtb);
2282
+
*val = get_reg_val(id, kvmppc_get_vtb(vcpu));
2296
2283
break;
2297
2284
case KVM_REG_PPC_DAWR:
2298
-
*val = get_reg_val(id, vcpu->arch.dawr0);
2285
+
*val = get_reg_val(id, kvmppc_get_dawr0_hv(vcpu));
2299
2286
break;
2300
2287
case KVM_REG_PPC_DAWRX:
2301
-
*val = get_reg_val(id, vcpu->arch.dawrx0);
2288
+
*val = get_reg_val(id, kvmppc_get_dawrx0_hv(vcpu));
2302
2289
break;
2303
2290
case KVM_REG_PPC_DAWR1:
2304
-
*val = get_reg_val(id, vcpu->arch.dawr1);
2291
+
*val = get_reg_val(id, kvmppc_get_dawr1_hv(vcpu));
2305
2292
break;
2306
2293
case KVM_REG_PPC_DAWRX1:
2307
-
*val = get_reg_val(id, vcpu->arch.dawrx1);
2294
+
*val = get_reg_val(id, kvmppc_get_dawrx1_hv(vcpu));
2308
2295
break;
2309
2296
case KVM_REG_PPC_CIABR:
2310
-
*val = get_reg_val(id, vcpu->arch.ciabr);
2297
+
*val = get_reg_val(id, kvmppc_get_ciabr_hv(vcpu));
2311
2298
break;
2312
2299
case KVM_REG_PPC_CSIGR:
2313
2300
*val = get_reg_val(id, vcpu->arch.csigr);
···
2319
2306
*val = get_reg_val(id, vcpu->arch.tcscr);
2320
2307
break;
2321
2308
case KVM_REG_PPC_PID:
2322
-
*val = get_reg_val(id, vcpu->arch.pid);
2309
+
*val = get_reg_val(id, kvmppc_get_pid(vcpu));
2323
2310
break;
2324
2311
case KVM_REG_PPC_ACOP:
2325
2312
*val = get_reg_val(id, vcpu->arch.acop);
2326
2313
break;
2327
2314
case KVM_REG_PPC_WORT:
2328
-
*val = get_reg_val(id, vcpu->arch.wort);
2315
+
*val = get_reg_val(id, kvmppc_get_wort_hv(vcpu));
2329
2316
break;
2330
2317
case KVM_REG_PPC_TIDR:
2331
2318
*val = get_reg_val(id, vcpu->arch.tid);
···
2351
2338
spin_unlock(&vcpu->arch.vpa_update_lock);
2352
2339
break;
2353
2340
case KVM_REG_PPC_TB_OFFSET:
2354
-
*val = get_reg_val(id, vcpu->arch.vcore->tb_offset);
2341
+
*val = get_reg_val(id, kvmppc_get_tb_offset(vcpu));
2355
2342
break;
2356
2343
case KVM_REG_PPC_LPCR:
2357
2344
case KVM_REG_PPC_LPCR_64:
2358
-
*val = get_reg_val(id, vcpu->arch.vcore->lpcr);
2345
+
*val = get_reg_val(id, kvmppc_get_lpcr(vcpu));
2359
2346
break;
2360
2347
case KVM_REG_PPC_PPR:
2361
-
*val = get_reg_val(id, vcpu->arch.ppr);
2348
+
*val = get_reg_val(id, kvmppc_get_ppr_hv(vcpu));
2362
2349
break;
2363
2350
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2364
2351
case KVM_REG_PPC_TFHAR:
···
2427
2414
break;
2428
2415
#endif
2429
2416
case KVM_REG_PPC_ARCH_COMPAT:
2430
-
*val = get_reg_val(id, vcpu->arch.vcore->arch_compat);
2417
+
*val = get_reg_val(id, kvmppc_get_arch_compat(vcpu));
2431
2418
break;
2432
2419
case KVM_REG_PPC_DEC_EXPIRY:
2433
-
*val = get_reg_val(id, vcpu->arch.dec_expires);
2420
+
*val = get_reg_val(id, kvmppc_get_dec_expires(vcpu));
2434
2421
break;
2435
2422
case KVM_REG_PPC_ONLINE:
2436
2423
*val = get_reg_val(id, vcpu->arch.online);
2437
2424
break;
2438
2425
case KVM_REG_PPC_PTCR:
2439
2426
*val = get_reg_val(id, vcpu->kvm->arch.l1_ptcr);
2427
+
break;
2428
+
case KVM_REG_PPC_FSCR:
2429
+
*val = get_reg_val(id, kvmppc_get_fscr_hv(vcpu));
2440
2430
break;
2441
2431
default:
2442
2432
r = -EINVAL;
···
2469
2453
vcpu->arch.dabrx = set_reg_val(id, *val) & ~DABRX_HYP;
2470
2454
break;
2471
2455
case KVM_REG_PPC_DSCR:
2472
-
vcpu->arch.dscr = set_reg_val(id, *val);
2456
+
kvmppc_set_dscr_hv(vcpu, set_reg_val(id, *val));
2473
2457
break;
2474
2458
case KVM_REG_PPC_PURR:
2475
-
vcpu->arch.purr = set_reg_val(id, *val);
2459
+
kvmppc_set_purr_hv(vcpu, set_reg_val(id, *val));
2476
2460
break;
2477
2461
case KVM_REG_PPC_SPURR:
2478
-
vcpu->arch.spurr = set_reg_val(id, *val);
2462
+
kvmppc_set_spurr_hv(vcpu, set_reg_val(id, *val));
2479
2463
break;
2480
2464
case KVM_REG_PPC_AMR:
2481
-
vcpu->arch.amr = set_reg_val(id, *val);
2465
+
kvmppc_set_amr_hv(vcpu, set_reg_val(id, *val));
2482
2466
break;
2483
2467
case KVM_REG_PPC_UAMOR:
2484
-
vcpu->arch.uamor = set_reg_val(id, *val);
2468
+
kvmppc_set_uamor_hv(vcpu, set_reg_val(id, *val));
2485
2469
break;
2486
2470
case KVM_REG_PPC_MMCR0 ... KVM_REG_PPC_MMCR1:
2487
2471
i = id - KVM_REG_PPC_MMCR0;
2488
-
vcpu->arch.mmcr[i] = set_reg_val(id, *val);
2472
+
kvmppc_set_mmcr_hv(vcpu, i, set_reg_val(id, *val));
2489
2473
break;
2490
2474
case KVM_REG_PPC_MMCR2:
2491
-
vcpu->arch.mmcr[2] = set_reg_val(id, *val);
2475
+
kvmppc_set_mmcr_hv(vcpu, 2, set_reg_val(id, *val));
2492
2476
break;
2493
2477
case KVM_REG_PPC_MMCRA:
2494
-
vcpu->arch.mmcra = set_reg_val(id, *val);
2478
+
kvmppc_set_mmcra_hv(vcpu, set_reg_val(id, *val));
2495
2479
break;
2496
2480
case KVM_REG_PPC_MMCRS:
2497
2481
vcpu->arch.mmcrs = set_reg_val(id, *val);
···
2501
2485
break;
2502
2486
case KVM_REG_PPC_PMC1 ... KVM_REG_PPC_PMC8:
2503
2487
i = id - KVM_REG_PPC_PMC1;
2504
-
vcpu->arch.pmc[i] = set_reg_val(id, *val);
2488
+
kvmppc_set_pmc_hv(vcpu, i, set_reg_val(id, *val));
2505
2489
break;
2506
2490
case KVM_REG_PPC_SPMC1 ... KVM_REG_PPC_SPMC2:
2507
2491
i = id - KVM_REG_PPC_SPMC1;
2508
2492
vcpu->arch.spmc[i] = set_reg_val(id, *val);
2509
2493
break;
2510
2494
case KVM_REG_PPC_SIAR:
2511
-
vcpu->arch.siar = set_reg_val(id, *val);
2495
+
kvmppc_set_siar_hv(vcpu, set_reg_val(id, *val));
2512
2496
break;
2513
2497
case KVM_REG_PPC_SDAR:
2514
-
vcpu->arch.sdar = set_reg_val(id, *val);
2498
+
kvmppc_set_sdar_hv(vcpu, set_reg_val(id, *val));
2515
2499
break;
2516
2500
case KVM_REG_PPC_SIER:
2517
-
vcpu->arch.sier[0] = set_reg_val(id, *val);
2501
+
kvmppc_set_sier_hv(vcpu, 0, set_reg_val(id, *val));
2518
2502
break;
2519
2503
case KVM_REG_PPC_SIER2:
2520
-
vcpu->arch.sier[1] = set_reg_val(id, *val);
2504
+
kvmppc_set_sier_hv(vcpu, 1, set_reg_val(id, *val));
2521
2505
break;
2522
2506
case KVM_REG_PPC_SIER3:
2523
-
vcpu->arch.sier[2] = set_reg_val(id, *val);
2507
+
kvmppc_set_sier_hv(vcpu, 2, set_reg_val(id, *val));
2524
2508
break;
2525
2509
case KVM_REG_PPC_IAMR:
2526
-
vcpu->arch.iamr = set_reg_val(id, *val);
2510
+
kvmppc_set_iamr_hv(vcpu, set_reg_val(id, *val));
2527
2511
break;
2528
2512
case KVM_REG_PPC_PSPB:
2529
-
vcpu->arch.pspb = set_reg_val(id, *val);
2513
+
kvmppc_set_pspb_hv(vcpu, set_reg_val(id, *val));
2530
2514
break;
2531
2515
case KVM_REG_PPC_DPDES:
2532
2516
if (cpu_has_feature(CPU_FTR_ARCH_300))
···
2535
2519
vcpu->arch.vcore->dpdes = set_reg_val(id, *val);
2536
2520
break;
2537
2521
case KVM_REG_PPC_VTB:
2538
-
vcpu->arch.vcore->vtb = set_reg_val(id, *val);
2522
+
kvmppc_set_vtb(vcpu, set_reg_val(id, *val));
2539
2523
break;
2540
2524
case KVM_REG_PPC_DAWR:
2541
-
vcpu->arch.dawr0 = set_reg_val(id, *val);
2525
+
kvmppc_set_dawr0_hv(vcpu, set_reg_val(id, *val));
2542
2526
break;
2543
2527
case KVM_REG_PPC_DAWRX:
2544
-
vcpu->arch.dawrx0 = set_reg_val(id, *val) & ~DAWRX_HYP;
2528
+
kvmppc_set_dawrx0_hv(vcpu, set_reg_val(id, *val) & ~DAWRX_HYP);
2545
2529
break;
2546
2530
case KVM_REG_PPC_DAWR1:
2547
-
vcpu->arch.dawr1 = set_reg_val(id, *val);
2531
+
kvmppc_set_dawr1_hv(vcpu, set_reg_val(id, *val));
2548
2532
break;
2549
2533
case KVM_REG_PPC_DAWRX1:
2550
-
vcpu->arch.dawrx1 = set_reg_val(id, *val) & ~DAWRX_HYP;
2534
+
kvmppc_set_dawrx1_hv(vcpu, set_reg_val(id, *val) & ~DAWRX_HYP);
2551
2535
break;
2552
2536
case KVM_REG_PPC_CIABR:
2553
-
vcpu->arch.ciabr = set_reg_val(id, *val);
2537
+
kvmppc_set_ciabr_hv(vcpu, set_reg_val(id, *val));
2554
2538
/* Don't allow setting breakpoints in hypervisor code */
2555
-
if ((vcpu->arch.ciabr & CIABR_PRIV) == CIABR_PRIV_HYPER)
2556
-
vcpu->arch.ciabr &= ~CIABR_PRIV; /* disable */
2539
+
if ((kvmppc_get_ciabr_hv(vcpu) & CIABR_PRIV) == CIABR_PRIV_HYPER)
2540
+
kvmppc_set_ciabr_hv(vcpu, kvmppc_get_ciabr_hv(vcpu) & ~CIABR_PRIV);
2557
2541
break;
2558
2542
case KVM_REG_PPC_CSIGR:
2559
2543
vcpu->arch.csigr = set_reg_val(id, *val);
···
2565
2549
vcpu->arch.tcscr = set_reg_val(id, *val);
2566
2550
break;
2567
2551
case KVM_REG_PPC_PID:
2568
-
vcpu->arch.pid = set_reg_val(id, *val);
2552
+
kvmppc_set_pid(vcpu, set_reg_val(id, *val));
2569
2553
break;
2570
2554
case KVM_REG_PPC_ACOP:
2571
2555
vcpu->arch.acop = set_reg_val(id, *val);
2572
2556
break;
2573
2557
case KVM_REG_PPC_WORT:
2574
-
vcpu->arch.wort = set_reg_val(id, *val);
2558
+
kvmppc_set_wort_hv(vcpu, set_reg_val(id, *val));
2575
2559
break;
2576
2560
case KVM_REG_PPC_TIDR:
2577
2561
vcpu->arch.tid = set_reg_val(id, *val);
···
2618
2602
* decrementer, which is better than a large one that
2619
2603
* causes a hang.
2620
2604
*/
2621
-
if (!vcpu->arch.dec_expires && tb_offset)
2622
-
vcpu->arch.dec_expires = get_tb() + tb_offset;
2605
+
kvmppc_set_tb_offset(vcpu, tb_offset);
2606
+
if (!kvmppc_get_dec_expires(vcpu) && tb_offset)
2607
+
kvmppc_set_dec_expires(vcpu, get_tb() + tb_offset);
2623
2608
2624
-
vcpu->arch.vcore->tb_offset = tb_offset;
2609
+
kvmppc_set_tb_offset(vcpu, tb_offset);
2625
2610
break;
2626
2611
}
2627
2612
case KVM_REG_PPC_LPCR:
···
2632
2615
kvmppc_set_lpcr(vcpu, set_reg_val(id, *val), false);
2633
2616
break;
2634
2617
case KVM_REG_PPC_PPR:
2635
-
vcpu->arch.ppr = set_reg_val(id, *val);
2618
+
kvmppc_set_ppr_hv(vcpu, set_reg_val(id, *val));
2636
2619
break;
2637
2620
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2638
2621
case KVM_REG_PPC_TFHAR:
···
2703
2686
r = kvmppc_set_arch_compat(vcpu, set_reg_val(id, *val));
2704
2687
break;
2705
2688
case KVM_REG_PPC_DEC_EXPIRY:
2706
-
vcpu->arch.dec_expires = set_reg_val(id, *val);
2689
+
kvmppc_set_dec_expires(vcpu, set_reg_val(id, *val));
2707
2690
break;
2708
2691
case KVM_REG_PPC_ONLINE:
2709
2692
i = set_reg_val(id, *val);
···
2715
2698
break;
2716
2699
case KVM_REG_PPC_PTCR:
2717
2700
vcpu->kvm->arch.l1_ptcr = set_reg_val(id, *val);
2701
+
break;
2702
+
case KVM_REG_PPC_FSCR:
2703
+
kvmppc_set_fscr_hv(vcpu, set_reg_val(id, *val));
2718
2704
break;
2719
2705
default:
2720
2706
r = -EINVAL;
···
2936
2916
vcpu->arch.shared_big_endian = false;
2937
2917
#endif
2938
2918
#endif
2939
-
vcpu->arch.mmcr[0] = MMCR0_FC;
2940
-
if (cpu_has_feature(CPU_FTR_ARCH_31)) {
2941
-
vcpu->arch.mmcr[0] |= MMCR0_PMCCEXT;
2942
-
vcpu->arch.mmcra = MMCRA_BHRB_DISABLE;
2919
+
2920
+
if (kvmhv_is_nestedv2()) {
2921
+
err = kvmhv_nestedv2_vcpu_create(vcpu, &vcpu->arch.nestedv2_io);
2922
+
if (err < 0)
2923
+
return err;
2943
2924
}
2944
2925
2945
-
vcpu->arch.ctrl = CTRL_RUNLATCH;
2926
+
kvmppc_set_mmcr_hv(vcpu, 0, MMCR0_FC);
2927
+
if (cpu_has_feature(CPU_FTR_ARCH_31)) {
2928
+
kvmppc_set_mmcr_hv(vcpu, 0, kvmppc_get_mmcr_hv(vcpu, 0) | MMCR0_PMCCEXT);
2929
+
kvmppc_set_mmcra_hv(vcpu, MMCRA_BHRB_DISABLE);
2930
+
}
2931
+
2932
+
kvmppc_set_ctrl_hv(vcpu, CTRL_RUNLATCH);
2946
2933
/* default to host PVR, since we can't spoof it */
2947
2934
kvmppc_set_pvr_hv(vcpu, mfspr(SPRN_PVR));
2948
2935
spin_lock_init(&vcpu->arch.vpa_update_lock);
2949
2936
spin_lock_init(&vcpu->arch.tbacct_lock);
2950
2937
vcpu->arch.busy_preempt = TB_NIL;
2951
-
vcpu->arch.shregs.msr = MSR_ME;
2938
+
__kvmppc_set_msr_hv(vcpu, MSR_ME);
2952
2939
vcpu->arch.intr_msr = MSR_SF | MSR_ME;
2953
2940
2954
2941
/*
···
2965
2938
* don't set the HFSCR_MSGP bit, and that causes those instructions
2966
2939
* to trap and then we emulate them.
2967
2940
*/
2968
-
vcpu->arch.hfscr = HFSCR_TAR | HFSCR_EBB | HFSCR_PM | HFSCR_BHRB |
2969
-
HFSCR_DSCR | HFSCR_VECVSX | HFSCR_FP;
2941
+
kvmppc_set_hfscr_hv(vcpu, HFSCR_TAR | HFSCR_EBB | HFSCR_PM | HFSCR_BHRB |
2942
+
HFSCR_DSCR | HFSCR_VECVSX | HFSCR_FP);
2970
2943
2971
2944
/* On POWER10 and later, allow prefixed instructions */
2972
2945
if (cpu_has_feature(CPU_FTR_ARCH_31))
2973
-
vcpu->arch.hfscr |= HFSCR_PREFIX;
2946
+
kvmppc_set_hfscr_hv(vcpu, kvmppc_get_hfscr_hv(vcpu) | HFSCR_PREFIX);
2974
2947
2975
2948
if (cpu_has_feature(CPU_FTR_HVMODE)) {
2976
-
vcpu->arch.hfscr &= mfspr(SPRN_HFSCR);
2949
+
kvmppc_set_hfscr_hv(vcpu, kvmppc_get_hfscr_hv(vcpu) & mfspr(SPRN_HFSCR));
2950
+
2977
2951
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
2978
2952
if (cpu_has_feature(CPU_FTR_P9_TM_HV_ASSIST))
2979
-
vcpu->arch.hfscr |= HFSCR_TM;
2953
+
kvmppc_set_hfscr_hv(vcpu, kvmppc_get_hfscr_hv(vcpu) | HFSCR_TM);
2980
2954
#endif
2981
2955
}
2982
2956
if (cpu_has_feature(CPU_FTR_TM_COMP))
2983
2957
vcpu->arch.hfscr |= HFSCR_TM;
2984
2958
2985
-
vcpu->arch.hfscr_permitted = vcpu->arch.hfscr;
2959
+
vcpu->arch.hfscr_permitted = kvmppc_get_hfscr_hv(vcpu);
2986
2960
2987
2961
/*
2988
2962
* PM, EBB, TM are demand-faulted so start with it clear.
2989
2963
*/
2990
-
vcpu->arch.hfscr &= ~(HFSCR_PM | HFSCR_EBB | HFSCR_TM);
2964
+
kvmppc_set_hfscr_hv(vcpu, kvmppc_get_hfscr_hv(vcpu) & ~(HFSCR_PM | HFSCR_EBB | HFSCR_TM));
2991
2965
2992
2966
kvmppc_mmu_book3s_hv_init(vcpu);
2993
2967
···
3099
3071
unpin_vpa(vcpu->kvm, &vcpu->arch.slb_shadow);
3100
3072
unpin_vpa(vcpu->kvm, &vcpu->arch.vpa);
3101
3073
spin_unlock(&vcpu->arch.vpa_update_lock);
3074
+
if (kvmhv_is_nestedv2())
3075
+
kvmhv_nestedv2_vcpu_free(vcpu, &vcpu->arch.nestedv2_io);
3102
3076
}
3103
3077
3104
3078
static int kvmppc_core_check_requests_hv(struct kvm_vcpu *vcpu)
···
4065
4035
}
4066
4036
}
4067
4037
4038
+
static int kvmhv_vcpu_entry_nestedv2(struct kvm_vcpu *vcpu, u64 time_limit,
4039
+
unsigned long lpcr, u64 *tb)
4040
+
{
4041
+
struct kvmhv_nestedv2_io *io;
4042
+
unsigned long msr, i;
4043
+
int trap;
4044
+
long rc;
4045
+
4046
+
io = &vcpu->arch.nestedv2_io;
4047
+
4048
+
msr = mfmsr();
4049
+
kvmppc_msr_hard_disable_set_facilities(vcpu, msr);
4050
+
if (lazy_irq_pending())
4051
+
return 0;
4052
+
4053
+
rc = kvmhv_nestedv2_flush_vcpu(vcpu, time_limit);
4054
+
if (rc < 0)
4055
+
return -EINVAL;
4056
+
4057
+
accumulate_time(vcpu, &vcpu->arch.in_guest);
4058
+
rc = plpar_guest_run_vcpu(0, vcpu->kvm->arch.lpid, vcpu->vcpu_id,
4059
+
&trap, &i);
4060
+
4061
+
if (rc != H_SUCCESS) {
4062
+
pr_err("KVM Guest Run VCPU hcall failed\n");
4063
+
if (rc == H_INVALID_ELEMENT_ID)
4064
+
pr_err("KVM: Guest Run VCPU invalid element id at %ld\n", i);
4065
+
else if (rc == H_INVALID_ELEMENT_SIZE)
4066
+
pr_err("KVM: Guest Run VCPU invalid element size at %ld\n", i);
4067
+
else if (rc == H_INVALID_ELEMENT_VALUE)
4068
+
pr_err("KVM: Guest Run VCPU invalid element value at %ld\n", i);
4069
+
return -EINVAL;
4070
+
}
4071
+
accumulate_time(vcpu, &vcpu->arch.guest_exit);
4072
+
4073
+
*tb = mftb();
4074
+
kvmppc_gsm_reset(io->vcpu_message);
4075
+
kvmppc_gsm_reset(io->vcore_message);
4076
+
kvmppc_gsbm_zero(&io->valids);
4077
+
4078
+
rc = kvmhv_nestedv2_parse_output(vcpu);
4079
+
if (rc < 0)
4080
+
return -EINVAL;
4081
+
4082
+
timer_rearm_host_dec(*tb);
4083
+
4084
+
return trap;
4085
+
}
4086
+
4068
4087
/* call our hypervisor to load up HV regs and go */
4069
4088
static int kvmhv_vcpu_entry_p9_nested(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr, u64 *tb)
4070
4089
{
4071
-
struct kvmppc_vcore *vc = vcpu->arch.vcore;
4072
4090
unsigned long host_psscr;
4073
4091
unsigned long msr;
4074
4092
struct hv_guest_state hvregs;
···
4196
4118
if (!(lpcr & LPCR_LD)) /* Sign extend if not using large decrementer */
4197
4119
dec = (s32) dec;
4198
4120
*tb = mftb();
4199
-
vcpu->arch.dec_expires = dec + (*tb + vc->tb_offset);
4121
+
vcpu->arch.dec_expires = dec + (*tb + kvmppc_get_tb_offset(vcpu));
4200
4122
4201
4123
timer_rearm_host_dec(*tb);
4202
4124
···
4231
4153
vcpu_vpa_increment_dispatch(vcpu);
4232
4154
4233
4155
if (kvmhv_on_pseries()) {
4234
-
trap = kvmhv_vcpu_entry_p9_nested(vcpu, time_limit, lpcr, tb);
4156
+
if (kvmhv_is_nestedv1())
4157
+
trap = kvmhv_vcpu_entry_p9_nested(vcpu, time_limit, lpcr, tb);
4158
+
else
4159
+
trap = kvmhv_vcpu_entry_nestedv2(vcpu, time_limit, lpcr, tb);
4235
4160
4236
4161
/* H_CEDE has to be handled now, not later */
4237
4162
if (trap == BOOK3S_INTERRUPT_SYSCALL && !nested &&
···
4257
4176
__this_cpu_write(cpu_in_guest, NULL);
4258
4177
4259
4178
if (trap == BOOK3S_INTERRUPT_SYSCALL &&
4260
-
!(vcpu->arch.shregs.msr & MSR_PR)) {
4179
+
!(__kvmppc_get_msr_hv(vcpu) & MSR_PR)) {
4261
4180
unsigned long req = kvmppc_get_gpr(vcpu, 3);
4262
4181
4263
4182
/*
···
4736
4655
4737
4656
if (!nested) {
4738
4657
kvmppc_core_prepare_to_enter(vcpu);
4739
-
if (vcpu->arch.shregs.msr & MSR_EE) {
4658
+
if (__kvmppc_get_msr_hv(vcpu) & MSR_EE) {
4740
4659
if (xive_interrupt_pending(vcpu))
4741
4660
kvmppc_inject_interrupt_hv(vcpu,
4742
4661
BOOK3S_INTERRUPT_EXTERNAL, 0);
···
4758
4677
4759
4678
tb = mftb();
4760
4679
4761
-
kvmppc_update_vpa_dispatch_p9(vcpu, vc, tb + vc->tb_offset);
4680
+
kvmppc_update_vpa_dispatch_p9(vcpu, vc, tb + kvmppc_get_tb_offset(vcpu));
4762
4681
4763
4682
trace_kvm_guest_enter(vcpu);
4764
4683
···
4925
4844
msr |= MSR_VSX;
4926
4845
if ((cpu_has_feature(CPU_FTR_TM) ||
4927
4846
cpu_has_feature(CPU_FTR_P9_TM_HV_ASSIST)) &&
4928
-
(vcpu->arch.hfscr & HFSCR_TM))
4847
+
(kvmppc_get_hfscr_hv(vcpu) & HFSCR_TM))
4929
4848
msr |= MSR_TM;
4930
4849
msr = msr_check_and_set(msr);
4931
4850
···
4949
4868
if (run->exit_reason == KVM_EXIT_PAPR_HCALL) {
4950
4869
accumulate_time(vcpu, &vcpu->arch.hcall);
4951
4870
4952
-
if (WARN_ON_ONCE(vcpu->arch.shregs.msr & MSR_PR)) {
4871
+
if (WARN_ON_ONCE(__kvmppc_get_msr_hv(vcpu) & MSR_PR)) {
4953
4872
/*
4954
4873
* These should have been caught reflected
4955
4874
* into the guest by now. Final sanity check:
···
5213
5132
spin_unlock(&vc->lock);
5214
5133
if (++cores_done >= kvm->arch.online_vcores)
5215
5134
break;
5135
+
}
5136
+
5137
+
if (kvmhv_is_nestedv2()) {
5138
+
struct kvm_vcpu *vcpu;
5139
+
5140
+
kvm_for_each_vcpu(i, vcpu, kvm) {
5141
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_LPCR);
5142
+
}
5216
5143
}
5217
5144
}
5218
5145
···
5488
5399
5489
5400
/* Allocate the guest's logical partition ID */
5490
5401
5491
-
lpid = kvmppc_alloc_lpid();
5492
-
if ((long)lpid < 0)
5493
-
return -ENOMEM;
5494
-
kvm->arch.lpid = lpid;
5402
+
if (!kvmhv_is_nestedv2()) {
5403
+
lpid = kvmppc_alloc_lpid();
5404
+
if ((long)lpid < 0)
5405
+
return -ENOMEM;
5406
+
kvm->arch.lpid = lpid;
5407
+
}
5495
5408
5496
5409
kvmppc_alloc_host_rm_ops();
5497
5410
5498
5411
kvmhv_vm_nested_init(kvm);
5412
+
5413
+
if (kvmhv_is_nestedv2()) {
5414
+
long rc;
5415
+
unsigned long guest_id;
5416
+
5417
+
rc = plpar_guest_create(0, &guest_id);
5418
+
5419
+
if (rc != H_SUCCESS)
5420
+
pr_err("KVM: Create Guest hcall failed, rc=%ld\n", rc);
5421
+
5422
+
switch (rc) {
5423
+
case H_PARAMETER:
5424
+
case H_FUNCTION:
5425
+
case H_STATE:
5426
+
return -EINVAL;
5427
+
case H_NOT_ENOUGH_RESOURCES:
5428
+
case H_ABORTED:
5429
+
return -ENOMEM;
5430
+
case H_AUTHORITY:
5431
+
return -EPERM;
5432
+
case H_NOT_AVAILABLE:
5433
+
return -EBUSY;
5434
+
}
5435
+
kvm->arch.lpid = guest_id;
5436
+
}
5437
+
5499
5438
5500
5439
/*
5501
5440
* Since we don't flush the TLB when tearing down a VM,
···
5594
5477
lpcr |= LPCR_HAIL;
5595
5478
ret = kvmppc_init_vm_radix(kvm);
5596
5479
if (ret) {
5597
-
kvmppc_free_lpid(kvm->arch.lpid);
5480
+
if (kvmhv_is_nestedv2())
5481
+
plpar_guest_delete(0, kvm->arch.lpid);
5482
+
else
5483
+
kvmppc_free_lpid(kvm->arch.lpid);
5598
5484
return ret;
5599
5485
}
5600
5486
kvmppc_setup_partition_table(kvm);
···
5687
5567
kvm->arch.process_table = 0;
5688
5568
if (kvm->arch.secure_guest)
5689
5569
uv_svm_terminate(kvm->arch.lpid);
5690
-
kvmhv_set_ptbl_entry(kvm->arch.lpid, 0, 0);
5570
+
if (!kvmhv_is_nestedv2())
5571
+
kvmhv_set_ptbl_entry(kvm->arch.lpid, 0, 0);
5691
5572
}
5692
5573
5693
-
kvmppc_free_lpid(kvm->arch.lpid);
5574
+
if (kvmhv_is_nestedv2())
5575
+
plpar_guest_delete(0, kvm->arch.lpid);
5576
+
else
5577
+
kvmppc_free_lpid(kvm->arch.lpid);
5694
5578
5695
5579
kvmppc_free_pimap(kvm);
5696
5580
}
···
6105
5981
if (!cpu_has_feature(CPU_FTR_ARCH_300))
6106
5982
return -ENODEV;
6107
5983
if (!radix_enabled())
5984
+
return -ENODEV;
5985
+
if (kvmhv_is_nestedv2())
6108
5986
return -ENODEV;
6109
5987
6110
5988
/* kvm == NULL means the caller is testing if the capability exists */
+76
arch/powerpc/kvm/book3s_hv.h
+76
arch/powerpc/kvm/book3s_hv.h
···
3
3
/*
4
4
* Privileged (non-hypervisor) host registers to save.
5
5
*/
6
+
#include "asm/guest-state-buffer.h"
7
+
6
8
struct p9_host_os_sprs {
7
9
unsigned long iamr;
8
10
unsigned long amr;
···
52
50
#define start_timing(vcpu, next) do {} while (0)
53
51
#define end_timing(vcpu) do {} while (0)
54
52
#endif
53
+
54
+
static inline void __kvmppc_set_msr_hv(struct kvm_vcpu *vcpu, u64 val)
55
+
{
56
+
vcpu->arch.shregs.msr = val;
57
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_MSR);
58
+
}
59
+
60
+
static inline u64 __kvmppc_get_msr_hv(struct kvm_vcpu *vcpu)
61
+
{
62
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, KVMPPC_GSID_MSR) < 0);
63
+
return vcpu->arch.shregs.msr;
64
+
}
65
+
66
+
#define KVMPPC_BOOK3S_HV_VCPU_ACCESSOR_SET(reg, size, iden) \
67
+
static inline void kvmppc_set_##reg ##_hv(struct kvm_vcpu *vcpu, u##size val) \
68
+
{ \
69
+
vcpu->arch.reg = val; \
70
+
kvmhv_nestedv2_mark_dirty(vcpu, iden); \
71
+
}
72
+
73
+
#define KVMPPC_BOOK3S_HV_VCPU_ACCESSOR_GET(reg, size, iden) \
74
+
static inline u##size kvmppc_get_##reg ##_hv(struct kvm_vcpu *vcpu) \
75
+
{ \
76
+
kvmhv_nestedv2_cached_reload(vcpu, iden); \
77
+
return vcpu->arch.reg; \
78
+
}
79
+
80
+
#define KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(reg, size, iden) \
81
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR_SET(reg, size, iden) \
82
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR_GET(reg, size, iden) \
83
+
84
+
#define KVMPPC_BOOK3S_HV_VCPU_ARRAY_ACCESSOR_SET(reg, size, iden) \
85
+
static inline void kvmppc_set_##reg ##_hv(struct kvm_vcpu *vcpu, int i, u##size val) \
86
+
{ \
87
+
vcpu->arch.reg[i] = val; \
88
+
kvmhv_nestedv2_mark_dirty(vcpu, iden(i)); \
89
+
}
90
+
91
+
#define KVMPPC_BOOK3S_HV_VCPU_ARRAY_ACCESSOR_GET(reg, size, iden) \
92
+
static inline u##size kvmppc_get_##reg ##_hv(struct kvm_vcpu *vcpu, int i) \
93
+
{ \
94
+
WARN_ON(kvmhv_nestedv2_cached_reload(vcpu, iden(i)) < 0); \
95
+
return vcpu->arch.reg[i]; \
96
+
}
97
+
98
+
#define KVMPPC_BOOK3S_HV_VCPU_ARRAY_ACCESSOR(reg, size, iden) \
99
+
KVMPPC_BOOK3S_HV_VCPU_ARRAY_ACCESSOR_SET(reg, size, iden) \
100
+
KVMPPC_BOOK3S_HV_VCPU_ARRAY_ACCESSOR_GET(reg, size, iden) \
101
+
102
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(mmcra, 64, KVMPPC_GSID_MMCRA)
103
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(hfscr, 64, KVMPPC_GSID_HFSCR)
104
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(fscr, 64, KVMPPC_GSID_FSCR)
105
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dscr, 64, KVMPPC_GSID_DSCR)
106
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(purr, 64, KVMPPC_GSID_PURR)
107
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(spurr, 64, KVMPPC_GSID_SPURR)
108
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(amr, 64, KVMPPC_GSID_AMR)
109
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(uamor, 64, KVMPPC_GSID_UAMOR)
110
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(siar, 64, KVMPPC_GSID_SIAR)
111
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(sdar, 64, KVMPPC_GSID_SDAR)
112
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(iamr, 64, KVMPPC_GSID_IAMR)
113
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawr0, 64, KVMPPC_GSID_DAWR0)
114
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawr1, 64, KVMPPC_GSID_DAWR1)
115
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx0, 64, KVMPPC_GSID_DAWRX0)
116
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx1, 64, KVMPPC_GSID_DAWRX1)
117
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ciabr, 64, KVMPPC_GSID_CIABR)
118
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(wort, 64, KVMPPC_GSID_WORT)
119
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ppr, 64, KVMPPC_GSID_PPR)
120
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ctrl, 64, KVMPPC_GSID_CTRL);
121
+
122
+
KVMPPC_BOOK3S_HV_VCPU_ARRAY_ACCESSOR(mmcr, 64, KVMPPC_GSID_MMCR)
123
+
KVMPPC_BOOK3S_HV_VCPU_ARRAY_ACCESSOR(sier, 64, KVMPPC_GSID_SIER)
124
+
KVMPPC_BOOK3S_HV_VCPU_ARRAY_ACCESSOR(pmc, 32, KVMPPC_GSID_PMC)
125
+
126
+
KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(pspb, 32, KVMPPC_GSID_PSPB)
+8
-3
arch/powerpc/kvm/book3s_hv_builtin.c
+8
-3
arch/powerpc/kvm/book3s_hv_builtin.c
···
32
32
33
33
#include "book3s_xics.h"
34
34
#include "book3s_xive.h"
35
+
#include "book3s_hv.h"
35
36
36
37
/*
37
38
* Hash page table alignment on newer cpus(CPU_FTR_ARCH_206)
···
183
182
184
183
long kvmppc_rm_h_random(struct kvm_vcpu *vcpu)
185
184
{
185
+
unsigned long rand;
186
+
186
187
if (ppc_md.get_random_seed &&
187
-
ppc_md.get_random_seed(&vcpu->arch.regs.gpr[4]))
188
+
ppc_md.get_random_seed(&rand)) {
189
+
kvmppc_set_gpr(vcpu, 4, rand);
188
190
return H_SUCCESS;
191
+
}
189
192
190
193
return H_HARDWARE;
191
194
}
···
515
510
*/
516
511
if ((msr & MSR_TS_MASK) == MSR_TS_MASK)
517
512
msr &= ~MSR_TS_MASK;
518
-
vcpu->arch.shregs.msr = msr;
513
+
__kvmppc_set_msr_hv(vcpu, msr);
519
514
kvmppc_end_cede(vcpu);
520
515
}
521
516
EXPORT_SYMBOL_GPL(kvmppc_set_msr_hv);
···
553
548
kvmppc_set_srr0(vcpu, pc);
554
549
kvmppc_set_srr1(vcpu, (msr & SRR1_MSR_BITS) | srr1_flags);
555
550
kvmppc_set_pc(vcpu, new_pc);
556
-
vcpu->arch.shregs.msr = new_msr;
551
+
__kvmppc_set_msr_hv(vcpu, new_msr);
557
552
}
558
553
559
554
void kvmppc_inject_interrupt_hv(struct kvm_vcpu *vcpu, int vec, u64 srr1_flags)
+37
-7
arch/powerpc/kvm/book3s_hv_nested.c
+37
-7
arch/powerpc/kvm/book3s_hv_nested.c
···
428
428
return vcpu->arch.trap;
429
429
}
430
430
431
+
unsigned long nested_capabilities;
432
+
431
433
long kvmhv_nested_init(void)
432
434
{
433
435
long int ptb_order;
434
-
unsigned long ptcr;
436
+
unsigned long ptcr, host_capabilities;
435
437
long rc;
436
438
437
439
if (!kvmhv_on_pseries())
···
441
439
if (!radix_enabled())
442
440
return -ENODEV;
443
441
442
+
rc = plpar_guest_get_capabilities(0, &host_capabilities);
443
+
if (rc == H_SUCCESS) {
444
+
unsigned long capabilities = 0;
445
+
446
+
if (cpu_has_feature(CPU_FTR_ARCH_31))
447
+
capabilities |= H_GUEST_CAP_POWER10;
448
+
if (cpu_has_feature(CPU_FTR_ARCH_300))
449
+
capabilities |= H_GUEST_CAP_POWER9;
450
+
451
+
nested_capabilities = capabilities & host_capabilities;
452
+
rc = plpar_guest_set_capabilities(0, nested_capabilities);
453
+
if (rc != H_SUCCESS) {
454
+
pr_err("kvm-hv: Could not configure parent hypervisor capabilities (rc=%ld)",
455
+
rc);
456
+
return -ENODEV;
457
+
}
458
+
459
+
static_branch_enable(&__kvmhv_is_nestedv2);
460
+
return 0;
461
+
}
462
+
463
+
pr_info("kvm-hv: nestedv2 get capabilities hcall failed, falling back to nestedv1 (rc=%ld)\n",
464
+
rc);
444
465
/* Partition table entry is 1<<4 bytes in size, hence the 4. */
445
466
ptb_order = KVM_MAX_NESTED_GUESTS_SHIFT + 4;
446
467
/* Minimum partition table size is 1<<12 bytes */
···
503
478
}
504
479
}
505
480
506
-
static void kvmhv_flush_lpid(unsigned int lpid)
481
+
static void kvmhv_flush_lpid(u64 lpid)
507
482
{
508
483
long rc;
509
484
···
525
500
pr_err("KVM: TLB LPID invalidation hcall failed, rc=%ld\n", rc);
526
501
}
527
502
528
-
void kvmhv_set_ptbl_entry(unsigned int lpid, u64 dw0, u64 dw1)
503
+
void kvmhv_set_ptbl_entry(u64 lpid, u64 dw0, u64 dw1)
529
504
{
530
505
if (!kvmhv_on_pseries()) {
531
506
mmu_partition_table_set_entry(lpid, dw0, dw1, true);
532
507
return;
533
508
}
534
509
535
-
pseries_partition_tb[lpid].patb0 = cpu_to_be64(dw0);
536
-
pseries_partition_tb[lpid].patb1 = cpu_to_be64(dw1);
537
-
/* L0 will do the necessary barriers */
538
-
kvmhv_flush_lpid(lpid);
510
+
if (kvmhv_is_nestedv1()) {
511
+
pseries_partition_tb[lpid].patb0 = cpu_to_be64(dw0);
512
+
pseries_partition_tb[lpid].patb1 = cpu_to_be64(dw1);
513
+
/* L0 will do the necessary barriers */
514
+
kvmhv_flush_lpid(lpid);
515
+
}
516
+
517
+
if (kvmhv_is_nestedv2())
518
+
kvmhv_nestedv2_set_ptbl_entry(lpid, dw0, dw1);
539
519
}
540
520
541
521
static void kvmhv_set_nested_ptbl(struct kvm_nested_guest *gp)
+994
arch/powerpc/kvm/book3s_hv_nestedv2.c
+994
arch/powerpc/kvm/book3s_hv_nestedv2.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-only
2
+
/*
3
+
* Copyright 2023 Jordan Niethe, IBM Corp. <jniethe5@gmail.com>
4
+
*
5
+
* Authors:
6
+
* Jordan Niethe <jniethe5@gmail.com>
7
+
*
8
+
* Description: KVM functions specific to running on Book 3S
9
+
* processors as a NESTEDv2 guest.
10
+
*
11
+
*/
12
+
13
+
#include "linux/blk-mq.h"
14
+
#include "linux/console.h"
15
+
#include "linux/gfp_types.h"
16
+
#include "linux/signal.h"
17
+
#include <linux/kernel.h>
18
+
#include <linux/kvm_host.h>
19
+
#include <linux/pgtable.h>
20
+
21
+
#include <asm/kvm_ppc.h>
22
+
#include <asm/kvm_book3s.h>
23
+
#include <asm/hvcall.h>
24
+
#include <asm/pgalloc.h>
25
+
#include <asm/reg.h>
26
+
#include <asm/plpar_wrappers.h>
27
+
#include <asm/guest-state-buffer.h>
28
+
#include "trace_hv.h"
29
+
30
+
struct static_key_false __kvmhv_is_nestedv2 __read_mostly;
31
+
EXPORT_SYMBOL_GPL(__kvmhv_is_nestedv2);
32
+
33
+
34
+
static size_t
35
+
gs_msg_ops_kvmhv_nestedv2_config_get_size(struct kvmppc_gs_msg *gsm)
36
+
{
37
+
u16 ids[] = {
38
+
KVMPPC_GSID_RUN_OUTPUT_MIN_SIZE,
39
+
KVMPPC_GSID_RUN_INPUT,
40
+
KVMPPC_GSID_RUN_OUTPUT,
41
+
42
+
};
43
+
size_t size = 0;
44
+
45
+
for (int i = 0; i < ARRAY_SIZE(ids); i++)
46
+
size += kvmppc_gse_total_size(kvmppc_gsid_size(ids[i]));
47
+
return size;
48
+
}
49
+
50
+
static int
51
+
gs_msg_ops_kvmhv_nestedv2_config_fill_info(struct kvmppc_gs_buff *gsb,
52
+
struct kvmppc_gs_msg *gsm)
53
+
{
54
+
struct kvmhv_nestedv2_config *cfg;
55
+
int rc;
56
+
57
+
cfg = gsm->data;
58
+
59
+
if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_RUN_OUTPUT_MIN_SIZE)) {
60
+
rc = kvmppc_gse_put_u64(gsb, KVMPPC_GSID_RUN_OUTPUT_MIN_SIZE,
61
+
cfg->vcpu_run_output_size);
62
+
if (rc < 0)
63
+
return rc;
64
+
}
65
+
66
+
if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_RUN_INPUT)) {
67
+
rc = kvmppc_gse_put_buff_info(gsb, KVMPPC_GSID_RUN_INPUT,
68
+
cfg->vcpu_run_input_cfg);
69
+
if (rc < 0)
70
+
return rc;
71
+
}
72
+
73
+
if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_RUN_OUTPUT)) {
74
+
kvmppc_gse_put_buff_info(gsb, KVMPPC_GSID_RUN_OUTPUT,
75
+
cfg->vcpu_run_output_cfg);
76
+
if (rc < 0)
77
+
return rc;
78
+
}
79
+
80
+
return 0;
81
+
}
82
+
83
+
static int
84
+
gs_msg_ops_kvmhv_nestedv2_config_refresh_info(struct kvmppc_gs_msg *gsm,
85
+
struct kvmppc_gs_buff *gsb)
86
+
{
87
+
struct kvmhv_nestedv2_config *cfg;
88
+
struct kvmppc_gs_parser gsp = { 0 };
89
+
struct kvmppc_gs_elem *gse;
90
+
int rc;
91
+
92
+
cfg = gsm->data;
93
+
94
+
rc = kvmppc_gse_parse(&gsp, gsb);
95
+
if (rc < 0)
96
+
return rc;
97
+
98
+
gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_RUN_OUTPUT_MIN_SIZE);
99
+
if (gse)
100
+
cfg->vcpu_run_output_size = kvmppc_gse_get_u64(gse);
101
+
return 0;
102
+
}
103
+
104
+
static struct kvmppc_gs_msg_ops config_msg_ops = {
105
+
.get_size = gs_msg_ops_kvmhv_nestedv2_config_get_size,
106
+
.fill_info = gs_msg_ops_kvmhv_nestedv2_config_fill_info,
107
+
.refresh_info = gs_msg_ops_kvmhv_nestedv2_config_refresh_info,
108
+
};
109
+
110
+
static size_t gs_msg_ops_vcpu_get_size(struct kvmppc_gs_msg *gsm)
111
+
{
112
+
struct kvmppc_gs_bitmap gsbm = { 0 };
113
+
size_t size = 0;
114
+
u16 iden;
115
+
116
+
kvmppc_gsbm_fill(&gsbm);
117
+
kvmppc_gsbm_for_each(&gsbm, iden)
118
+
{
119
+
switch (iden) {
120
+
case KVMPPC_GSID_HOST_STATE_SIZE:
121
+
case KVMPPC_GSID_RUN_OUTPUT_MIN_SIZE:
122
+
case KVMPPC_GSID_PARTITION_TABLE:
123
+
case KVMPPC_GSID_PROCESS_TABLE:
124
+
case KVMPPC_GSID_RUN_INPUT:
125
+
case KVMPPC_GSID_RUN_OUTPUT:
126
+
break;
127
+
default:
128
+
size += kvmppc_gse_total_size(kvmppc_gsid_size(iden));
129
+
}
130
+
}
131
+
return size;
132
+
}
133
+
134
+
static int gs_msg_ops_vcpu_fill_info(struct kvmppc_gs_buff *gsb,
135
+
struct kvmppc_gs_msg *gsm)
136
+
{
137
+
struct kvm_vcpu *vcpu;
138
+
vector128 v;
139
+
int rc, i;
140
+
u16 iden;
141
+
142
+
vcpu = gsm->data;
143
+
144
+
kvmppc_gsm_for_each(gsm, iden)
145
+
{
146
+
rc = 0;
147
+
148
+
if ((gsm->flags & KVMPPC_GS_FLAGS_WIDE) !=
149
+
(kvmppc_gsid_flags(iden) & KVMPPC_GS_FLAGS_WIDE))
150
+
continue;
151
+
152
+
switch (iden) {
153
+
case KVMPPC_GSID_DSCR:
154
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.dscr);
155
+
break;
156
+
case KVMPPC_GSID_MMCRA:
157
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.mmcra);
158
+
break;
159
+
case KVMPPC_GSID_HFSCR:
160
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.hfscr);
161
+
break;
162
+
case KVMPPC_GSID_PURR:
163
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.purr);
164
+
break;
165
+
case KVMPPC_GSID_SPURR:
166
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.spurr);
167
+
break;
168
+
case KVMPPC_GSID_AMR:
169
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.amr);
170
+
break;
171
+
case KVMPPC_GSID_UAMOR:
172
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.uamor);
173
+
break;
174
+
case KVMPPC_GSID_SIAR:
175
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.siar);
176
+
break;
177
+
case KVMPPC_GSID_SDAR:
178
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.sdar);
179
+
break;
180
+
case KVMPPC_GSID_IAMR:
181
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.iamr);
182
+
break;
183
+
case KVMPPC_GSID_DAWR0:
184
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.dawr0);
185
+
break;
186
+
case KVMPPC_GSID_DAWR1:
187
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.dawr1);
188
+
break;
189
+
case KVMPPC_GSID_DAWRX0:
190
+
rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.dawrx0);
191
+
break;
192
+
case KVMPPC_GSID_DAWRX1:
193
+
rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.dawrx1);
194
+
break;
195
+
case KVMPPC_GSID_CIABR:
196
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ciabr);
197
+
break;
198
+
case KVMPPC_GSID_WORT:
199
+
rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.wort);
200
+
break;
201
+
case KVMPPC_GSID_PPR:
202
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ppr);
203
+
break;
204
+
case KVMPPC_GSID_PSPB:
205
+
rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.pspb);
206
+
break;
207
+
case KVMPPC_GSID_TAR:
208
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.tar);
209
+
break;
210
+
case KVMPPC_GSID_FSCR:
211
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.fscr);
212
+
break;
213
+
case KVMPPC_GSID_EBBHR:
214
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ebbhr);
215
+
break;
216
+
case KVMPPC_GSID_EBBRR:
217
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ebbrr);
218
+
break;
219
+
case KVMPPC_GSID_BESCR:
220
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.bescr);
221
+
break;
222
+
case KVMPPC_GSID_IC:
223
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ic);
224
+
break;
225
+
case KVMPPC_GSID_CTRL:
226
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ctrl);
227
+
break;
228
+
case KVMPPC_GSID_PIDR:
229
+
rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.pid);
230
+
break;
231
+
case KVMPPC_GSID_AMOR: {
232
+
u64 amor = ~0;
233
+
234
+
rc = kvmppc_gse_put_u64(gsb, iden, amor);
235
+
break;
236
+
}
237
+
case KVMPPC_GSID_VRSAVE:
238
+
rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.vrsave);
239
+
break;
240
+
case KVMPPC_GSID_MMCR(0)... KVMPPC_GSID_MMCR(3):
241
+
i = iden - KVMPPC_GSID_MMCR(0);
242
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.mmcr[i]);
243
+
break;
244
+
case KVMPPC_GSID_SIER(0)... KVMPPC_GSID_SIER(2):
245
+
i = iden - KVMPPC_GSID_SIER(0);
246
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.sier[i]);
247
+
break;
248
+
case KVMPPC_GSID_PMC(0)... KVMPPC_GSID_PMC(5):
249
+
i = iden - KVMPPC_GSID_PMC(0);
250
+
rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.pmc[i]);
251
+
break;
252
+
case KVMPPC_GSID_GPR(0)... KVMPPC_GSID_GPR(31):
253
+
i = iden - KVMPPC_GSID_GPR(0);
254
+
rc = kvmppc_gse_put_u64(gsb, iden,
255
+
vcpu->arch.regs.gpr[i]);
256
+
break;
257
+
case KVMPPC_GSID_CR:
258
+
rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.regs.ccr);
259
+
break;
260
+
case KVMPPC_GSID_XER:
261
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.regs.xer);
262
+
break;
263
+
case KVMPPC_GSID_CTR:
264
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.regs.ctr);
265
+
break;
266
+
case KVMPPC_GSID_LR:
267
+
rc = kvmppc_gse_put_u64(gsb, iden,
268
+
vcpu->arch.regs.link);
269
+
break;
270
+
case KVMPPC_GSID_NIA:
271
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.regs.nip);
272
+
break;
273
+
case KVMPPC_GSID_SRR0:
274
+
rc = kvmppc_gse_put_u64(gsb, iden,
275
+
vcpu->arch.shregs.srr0);
276
+
break;
277
+
case KVMPPC_GSID_SRR1:
278
+
rc = kvmppc_gse_put_u64(gsb, iden,
279
+
vcpu->arch.shregs.srr1);
280
+
break;
281
+
case KVMPPC_GSID_SPRG0:
282
+
rc = kvmppc_gse_put_u64(gsb, iden,
283
+
vcpu->arch.shregs.sprg0);
284
+
break;
285
+
case KVMPPC_GSID_SPRG1:
286
+
rc = kvmppc_gse_put_u64(gsb, iden,
287
+
vcpu->arch.shregs.sprg1);
288
+
break;
289
+
case KVMPPC_GSID_SPRG2:
290
+
rc = kvmppc_gse_put_u64(gsb, iden,
291
+
vcpu->arch.shregs.sprg2);
292
+
break;
293
+
case KVMPPC_GSID_SPRG3:
294
+
rc = kvmppc_gse_put_u64(gsb, iden,
295
+
vcpu->arch.shregs.sprg3);
296
+
break;
297
+
case KVMPPC_GSID_DAR:
298
+
rc = kvmppc_gse_put_u64(gsb, iden,
299
+
vcpu->arch.shregs.dar);
300
+
break;
301
+
case KVMPPC_GSID_DSISR:
302
+
rc = kvmppc_gse_put_u32(gsb, iden,
303
+
vcpu->arch.shregs.dsisr);
304
+
break;
305
+
case KVMPPC_GSID_MSR:
306
+
rc = kvmppc_gse_put_u64(gsb, iden,
307
+
vcpu->arch.shregs.msr);
308
+
break;
309
+
case KVMPPC_GSID_VTB:
310
+
rc = kvmppc_gse_put_u64(gsb, iden,
311
+
vcpu->arch.vcore->vtb);
312
+
break;
313
+
case KVMPPC_GSID_LPCR:
314
+
rc = kvmppc_gse_put_u64(gsb, iden,
315
+
vcpu->arch.vcore->lpcr);
316
+
break;
317
+
case KVMPPC_GSID_TB_OFFSET:
318
+
rc = kvmppc_gse_put_u64(gsb, iden,
319
+
vcpu->arch.vcore->tb_offset);
320
+
break;
321
+
case KVMPPC_GSID_FPSCR:
322
+
rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.fp.fpscr);
323
+
break;
324
+
case KVMPPC_GSID_VSRS(0)... KVMPPC_GSID_VSRS(31):
325
+
i = iden - KVMPPC_GSID_VSRS(0);
326
+
memcpy(&v, &vcpu->arch.fp.fpr[i],
327
+
sizeof(vcpu->arch.fp.fpr[i]));
328
+
rc = kvmppc_gse_put_vector128(gsb, iden, &v);
329
+
break;
330
+
#ifdef CONFIG_VSX
331
+
case KVMPPC_GSID_VSCR:
332
+
rc = kvmppc_gse_put_u32(gsb, iden,
333
+
vcpu->arch.vr.vscr.u[3]);
334
+
break;
335
+
case KVMPPC_GSID_VSRS(32)... KVMPPC_GSID_VSRS(63):
336
+
i = iden - KVMPPC_GSID_VSRS(32);
337
+
rc = kvmppc_gse_put_vector128(gsb, iden,
338
+
&vcpu->arch.vr.vr[i]);
339
+
break;
340
+
#endif
341
+
case KVMPPC_GSID_DEC_EXPIRY_TB: {
342
+
u64 dw;
343
+
344
+
dw = vcpu->arch.dec_expires -
345
+
vcpu->arch.vcore->tb_offset;
346
+
rc = kvmppc_gse_put_u64(gsb, iden, dw);
347
+
break;
348
+
}
349
+
case KVMPPC_GSID_LOGICAL_PVR:
350
+
rc = kvmppc_gse_put_u32(gsb, iden,
351
+
vcpu->arch.vcore->arch_compat);
352
+
break;
353
+
}
354
+
355
+
if (rc < 0)
356
+
return rc;
357
+
}
358
+
359
+
return 0;
360
+
}
361
+
362
+
static int gs_msg_ops_vcpu_refresh_info(struct kvmppc_gs_msg *gsm,
363
+
struct kvmppc_gs_buff *gsb)
364
+
{
365
+
struct kvmppc_gs_parser gsp = { 0 };
366
+
struct kvmhv_nestedv2_io *io;
367
+
struct kvmppc_gs_bitmap *valids;
368
+
struct kvm_vcpu *vcpu;
369
+
struct kvmppc_gs_elem *gse;
370
+
vector128 v;
371
+
int rc, i;
372
+
u16 iden;
373
+
374
+
vcpu = gsm->data;
375
+
376
+
rc = kvmppc_gse_parse(&gsp, gsb);
377
+
if (rc < 0)
378
+
return rc;
379
+
380
+
io = &vcpu->arch.nestedv2_io;
381
+
valids = &io->valids;
382
+
383
+
kvmppc_gsp_for_each(&gsp, iden, gse)
384
+
{
385
+
switch (iden) {
386
+
case KVMPPC_GSID_DSCR:
387
+
vcpu->arch.dscr = kvmppc_gse_get_u64(gse);
388
+
break;
389
+
case KVMPPC_GSID_MMCRA:
390
+
vcpu->arch.mmcra = kvmppc_gse_get_u64(gse);
391
+
break;
392
+
case KVMPPC_GSID_HFSCR:
393
+
vcpu->arch.hfscr = kvmppc_gse_get_u64(gse);
394
+
break;
395
+
case KVMPPC_GSID_PURR:
396
+
vcpu->arch.purr = kvmppc_gse_get_u64(gse);
397
+
break;
398
+
case KVMPPC_GSID_SPURR:
399
+
vcpu->arch.spurr = kvmppc_gse_get_u64(gse);
400
+
break;
401
+
case KVMPPC_GSID_AMR:
402
+
vcpu->arch.amr = kvmppc_gse_get_u64(gse);
403
+
break;
404
+
case KVMPPC_GSID_UAMOR:
405
+
vcpu->arch.uamor = kvmppc_gse_get_u64(gse);
406
+
break;
407
+
case KVMPPC_GSID_SIAR:
408
+
vcpu->arch.siar = kvmppc_gse_get_u64(gse);
409
+
break;
410
+
case KVMPPC_GSID_SDAR:
411
+
vcpu->arch.sdar = kvmppc_gse_get_u64(gse);
412
+
break;
413
+
case KVMPPC_GSID_IAMR:
414
+
vcpu->arch.iamr = kvmppc_gse_get_u64(gse);
415
+
break;
416
+
case KVMPPC_GSID_DAWR0:
417
+
vcpu->arch.dawr0 = kvmppc_gse_get_u64(gse);
418
+
break;
419
+
case KVMPPC_GSID_DAWR1:
420
+
vcpu->arch.dawr1 = kvmppc_gse_get_u64(gse);
421
+
break;
422
+
case KVMPPC_GSID_DAWRX0:
423
+
vcpu->arch.dawrx0 = kvmppc_gse_get_u32(gse);
424
+
break;
425
+
case KVMPPC_GSID_DAWRX1:
426
+
vcpu->arch.dawrx1 = kvmppc_gse_get_u32(gse);
427
+
break;
428
+
case KVMPPC_GSID_CIABR:
429
+
vcpu->arch.ciabr = kvmppc_gse_get_u64(gse);
430
+
break;
431
+
case KVMPPC_GSID_WORT:
432
+
vcpu->arch.wort = kvmppc_gse_get_u32(gse);
433
+
break;
434
+
case KVMPPC_GSID_PPR:
435
+
vcpu->arch.ppr = kvmppc_gse_get_u64(gse);
436
+
break;
437
+
case KVMPPC_GSID_PSPB:
438
+
vcpu->arch.pspb = kvmppc_gse_get_u32(gse);
439
+
break;
440
+
case KVMPPC_GSID_TAR:
441
+
vcpu->arch.tar = kvmppc_gse_get_u64(gse);
442
+
break;
443
+
case KVMPPC_GSID_FSCR:
444
+
vcpu->arch.fscr = kvmppc_gse_get_u64(gse);
445
+
break;
446
+
case KVMPPC_GSID_EBBHR:
447
+
vcpu->arch.ebbhr = kvmppc_gse_get_u64(gse);
448
+
break;
449
+
case KVMPPC_GSID_EBBRR:
450
+
vcpu->arch.ebbrr = kvmppc_gse_get_u64(gse);
451
+
break;
452
+
case KVMPPC_GSID_BESCR:
453
+
vcpu->arch.bescr = kvmppc_gse_get_u64(gse);
454
+
break;
455
+
case KVMPPC_GSID_IC:
456
+
vcpu->arch.ic = kvmppc_gse_get_u64(gse);
457
+
break;
458
+
case KVMPPC_GSID_CTRL:
459
+
vcpu->arch.ctrl = kvmppc_gse_get_u64(gse);
460
+
break;
461
+
case KVMPPC_GSID_PIDR:
462
+
vcpu->arch.pid = kvmppc_gse_get_u32(gse);
463
+
break;
464
+
case KVMPPC_GSID_AMOR:
465
+
break;
466
+
case KVMPPC_GSID_VRSAVE:
467
+
vcpu->arch.vrsave = kvmppc_gse_get_u32(gse);
468
+
break;
469
+
case KVMPPC_GSID_MMCR(0)... KVMPPC_GSID_MMCR(3):
470
+
i = iden - KVMPPC_GSID_MMCR(0);
471
+
vcpu->arch.mmcr[i] = kvmppc_gse_get_u64(gse);
472
+
break;
473
+
case KVMPPC_GSID_SIER(0)... KVMPPC_GSID_SIER(2):
474
+
i = iden - KVMPPC_GSID_SIER(0);
475
+
vcpu->arch.sier[i] = kvmppc_gse_get_u64(gse);
476
+
break;
477
+
case KVMPPC_GSID_PMC(0)... KVMPPC_GSID_PMC(5):
478
+
i = iden - KVMPPC_GSID_PMC(0);
479
+
vcpu->arch.pmc[i] = kvmppc_gse_get_u32(gse);
480
+
break;
481
+
case KVMPPC_GSID_GPR(0)... KVMPPC_GSID_GPR(31):
482
+
i = iden - KVMPPC_GSID_GPR(0);
483
+
vcpu->arch.regs.gpr[i] = kvmppc_gse_get_u64(gse);
484
+
break;
485
+
case KVMPPC_GSID_CR:
486
+
vcpu->arch.regs.ccr = kvmppc_gse_get_u32(gse);
487
+
break;
488
+
case KVMPPC_GSID_XER:
489
+
vcpu->arch.regs.xer = kvmppc_gse_get_u64(gse);
490
+
break;
491
+
case KVMPPC_GSID_CTR:
492
+
vcpu->arch.regs.ctr = kvmppc_gse_get_u64(gse);
493
+
break;
494
+
case KVMPPC_GSID_LR:
495
+
vcpu->arch.regs.link = kvmppc_gse_get_u64(gse);
496
+
break;
497
+
case KVMPPC_GSID_NIA:
498
+
vcpu->arch.regs.nip = kvmppc_gse_get_u64(gse);
499
+
break;
500
+
case KVMPPC_GSID_SRR0:
501
+
vcpu->arch.shregs.srr0 = kvmppc_gse_get_u64(gse);
502
+
break;
503
+
case KVMPPC_GSID_SRR1:
504
+
vcpu->arch.shregs.srr1 = kvmppc_gse_get_u64(gse);
505
+
break;
506
+
case KVMPPC_GSID_SPRG0:
507
+
vcpu->arch.shregs.sprg0 = kvmppc_gse_get_u64(gse);
508
+
break;
509
+
case KVMPPC_GSID_SPRG1:
510
+
vcpu->arch.shregs.sprg1 = kvmppc_gse_get_u64(gse);
511
+
break;
512
+
case KVMPPC_GSID_SPRG2:
513
+
vcpu->arch.shregs.sprg2 = kvmppc_gse_get_u64(gse);
514
+
break;
515
+
case KVMPPC_GSID_SPRG3:
516
+
vcpu->arch.shregs.sprg3 = kvmppc_gse_get_u64(gse);
517
+
break;
518
+
case KVMPPC_GSID_DAR:
519
+
vcpu->arch.shregs.dar = kvmppc_gse_get_u64(gse);
520
+
break;
521
+
case KVMPPC_GSID_DSISR:
522
+
vcpu->arch.shregs.dsisr = kvmppc_gse_get_u32(gse);
523
+
break;
524
+
case KVMPPC_GSID_MSR:
525
+
vcpu->arch.shregs.msr = kvmppc_gse_get_u64(gse);
526
+
break;
527
+
case KVMPPC_GSID_VTB:
528
+
vcpu->arch.vcore->vtb = kvmppc_gse_get_u64(gse);
529
+
break;
530
+
case KVMPPC_GSID_LPCR:
531
+
vcpu->arch.vcore->lpcr = kvmppc_gse_get_u64(gse);
532
+
break;
533
+
case KVMPPC_GSID_TB_OFFSET:
534
+
vcpu->arch.vcore->tb_offset = kvmppc_gse_get_u64(gse);
535
+
break;
536
+
case KVMPPC_GSID_FPSCR:
537
+
vcpu->arch.fp.fpscr = kvmppc_gse_get_u64(gse);
538
+
break;
539
+
case KVMPPC_GSID_VSRS(0)... KVMPPC_GSID_VSRS(31):
540
+
kvmppc_gse_get_vector128(gse, &v);
541
+
i = iden - KVMPPC_GSID_VSRS(0);
542
+
memcpy(&vcpu->arch.fp.fpr[i], &v,
543
+
sizeof(vcpu->arch.fp.fpr[i]));
544
+
break;
545
+
#ifdef CONFIG_VSX
546
+
case KVMPPC_GSID_VSCR:
547
+
vcpu->arch.vr.vscr.u[3] = kvmppc_gse_get_u32(gse);
548
+
break;
549
+
case KVMPPC_GSID_VSRS(32)... KVMPPC_GSID_VSRS(63):
550
+
i = iden - KVMPPC_GSID_VSRS(32);
551
+
kvmppc_gse_get_vector128(gse, &vcpu->arch.vr.vr[i]);
552
+
break;
553
+
#endif
554
+
case KVMPPC_GSID_HDAR:
555
+
vcpu->arch.fault_dar = kvmppc_gse_get_u64(gse);
556
+
break;
557
+
case KVMPPC_GSID_HDSISR:
558
+
vcpu->arch.fault_dsisr = kvmppc_gse_get_u32(gse);
559
+
break;
560
+
case KVMPPC_GSID_ASDR:
561
+
vcpu->arch.fault_gpa = kvmppc_gse_get_u64(gse);
562
+
break;
563
+
case KVMPPC_GSID_HEIR:
564
+
vcpu->arch.emul_inst = kvmppc_gse_get_u64(gse);
565
+
break;
566
+
case KVMPPC_GSID_DEC_EXPIRY_TB: {
567
+
u64 dw;
568
+
569
+
dw = kvmppc_gse_get_u64(gse);
570
+
vcpu->arch.dec_expires =
571
+
dw + vcpu->arch.vcore->tb_offset;
572
+
break;
573
+
}
574
+
case KVMPPC_GSID_LOGICAL_PVR:
575
+
vcpu->arch.vcore->arch_compat = kvmppc_gse_get_u32(gse);
576
+
break;
577
+
default:
578
+
continue;
579
+
}
580
+
kvmppc_gsbm_set(valids, iden);
581
+
}
582
+
583
+
return 0;
584
+
}
585
+
586
+
static struct kvmppc_gs_msg_ops vcpu_message_ops = {
587
+
.get_size = gs_msg_ops_vcpu_get_size,
588
+
.fill_info = gs_msg_ops_vcpu_fill_info,
589
+
.refresh_info = gs_msg_ops_vcpu_refresh_info,
590
+
};
591
+
592
+
static int kvmhv_nestedv2_host_create(struct kvm_vcpu *vcpu,
593
+
struct kvmhv_nestedv2_io *io)
594
+
{
595
+
struct kvmhv_nestedv2_config *cfg;
596
+
struct kvmppc_gs_buff *gsb, *vcpu_run_output, *vcpu_run_input;
597
+
unsigned long guest_id, vcpu_id;
598
+
struct kvmppc_gs_msg *gsm, *vcpu_message, *vcore_message;
599
+
int rc;
600
+
601
+
cfg = &io->cfg;
602
+
guest_id = vcpu->kvm->arch.lpid;
603
+
vcpu_id = vcpu->vcpu_id;
604
+
605
+
gsm = kvmppc_gsm_new(&config_msg_ops, cfg, KVMPPC_GS_FLAGS_WIDE,
606
+
GFP_KERNEL);
607
+
if (!gsm) {
608
+
rc = -ENOMEM;
609
+
goto err;
610
+
}
611
+
612
+
gsb = kvmppc_gsb_new(kvmppc_gsm_size(gsm), guest_id, vcpu_id,
613
+
GFP_KERNEL);
614
+
if (!gsb) {
615
+
rc = -ENOMEM;
616
+
goto free_gsm;
617
+
}
618
+
619
+
rc = kvmppc_gsb_receive_datum(gsb, gsm,
620
+
KVMPPC_GSID_RUN_OUTPUT_MIN_SIZE);
621
+
if (rc < 0) {
622
+
pr_err("KVM-NESTEDv2: couldn't get vcpu run output buffer minimum size\n");
623
+
goto free_gsb;
624
+
}
625
+
626
+
vcpu_run_output = kvmppc_gsb_new(cfg->vcpu_run_output_size, guest_id,
627
+
vcpu_id, GFP_KERNEL);
628
+
if (!vcpu_run_output) {
629
+
rc = -ENOMEM;
630
+
goto free_gsb;
631
+
}
632
+
633
+
cfg->vcpu_run_output_cfg.address = kvmppc_gsb_paddress(vcpu_run_output);
634
+
cfg->vcpu_run_output_cfg.size = kvmppc_gsb_capacity(vcpu_run_output);
635
+
io->vcpu_run_output = vcpu_run_output;
636
+
637
+
gsm->flags = 0;
638
+
rc = kvmppc_gsb_send_datum(gsb, gsm, KVMPPC_GSID_RUN_OUTPUT);
639
+
if (rc < 0) {
640
+
pr_err("KVM-NESTEDv2: couldn't set vcpu run output buffer\n");
641
+
goto free_gs_out;
642
+
}
643
+
644
+
vcpu_message = kvmppc_gsm_new(&vcpu_message_ops, vcpu, 0, GFP_KERNEL);
645
+
if (!vcpu_message) {
646
+
rc = -ENOMEM;
647
+
goto free_gs_out;
648
+
}
649
+
kvmppc_gsm_include_all(vcpu_message);
650
+
651
+
io->vcpu_message = vcpu_message;
652
+
653
+
vcpu_run_input = kvmppc_gsb_new(kvmppc_gsm_size(vcpu_message), guest_id,
654
+
vcpu_id, GFP_KERNEL);
655
+
if (!vcpu_run_input) {
656
+
rc = -ENOMEM;
657
+
goto free_vcpu_message;
658
+
}
659
+
660
+
io->vcpu_run_input = vcpu_run_input;
661
+
cfg->vcpu_run_input_cfg.address = kvmppc_gsb_paddress(vcpu_run_input);
662
+
cfg->vcpu_run_input_cfg.size = kvmppc_gsb_capacity(vcpu_run_input);
663
+
rc = kvmppc_gsb_send_datum(gsb, gsm, KVMPPC_GSID_RUN_INPUT);
664
+
if (rc < 0) {
665
+
pr_err("KVM-NESTEDv2: couldn't set vcpu run input buffer\n");
666
+
goto free_vcpu_run_input;
667
+
}
668
+
669
+
vcore_message = kvmppc_gsm_new(&vcpu_message_ops, vcpu,
670
+
KVMPPC_GS_FLAGS_WIDE, GFP_KERNEL);
671
+
if (!vcore_message) {
672
+
rc = -ENOMEM;
673
+
goto free_vcpu_run_input;
674
+
}
675
+
676
+
kvmppc_gsm_include_all(vcore_message);
677
+
kvmppc_gsbm_clear(&vcore_message->bitmap, KVMPPC_GSID_LOGICAL_PVR);
678
+
io->vcore_message = vcore_message;
679
+
680
+
kvmppc_gsbm_fill(&io->valids);
681
+
kvmppc_gsm_free(gsm);
682
+
kvmppc_gsb_free(gsb);
683
+
return 0;
684
+
685
+
free_vcpu_run_input:
686
+
kvmppc_gsb_free(vcpu_run_input);
687
+
free_vcpu_message:
688
+
kvmppc_gsm_free(vcpu_message);
689
+
free_gs_out:
690
+
kvmppc_gsb_free(vcpu_run_output);
691
+
free_gsb:
692
+
kvmppc_gsb_free(gsb);
693
+
free_gsm:
694
+
kvmppc_gsm_free(gsm);
695
+
err:
696
+
return rc;
697
+
}
698
+
699
+
/**
700
+
* __kvmhv_nestedv2_mark_dirty() - mark a Guest State ID to be sent to the host
701
+
* @vcpu: vcpu
702
+
* @iden: guest state ID
703
+
*
704
+
* Mark a guest state ID as having been changed by the L1 host and thus
705
+
* the new value must be sent to the L0 hypervisor. See kvmhv_nestedv2_flush_vcpu()
706
+
*/
707
+
int __kvmhv_nestedv2_mark_dirty(struct kvm_vcpu *vcpu, u16 iden)
708
+
{
709
+
struct kvmhv_nestedv2_io *io;
710
+
struct kvmppc_gs_bitmap *valids;
711
+
struct kvmppc_gs_msg *gsm;
712
+
713
+
if (!iden)
714
+
return 0;
715
+
716
+
io = &vcpu->arch.nestedv2_io;
717
+
valids = &io->valids;
718
+
gsm = io->vcpu_message;
719
+
kvmppc_gsm_include(gsm, iden);
720
+
gsm = io->vcore_message;
721
+
kvmppc_gsm_include(gsm, iden);
722
+
kvmppc_gsbm_set(valids, iden);
723
+
return 0;
724
+
}
725
+
EXPORT_SYMBOL_GPL(__kvmhv_nestedv2_mark_dirty);
726
+
727
+
/**
728
+
* __kvmhv_nestedv2_cached_reload() - reload a Guest State ID from the host
729
+
* @vcpu: vcpu
730
+
* @iden: guest state ID
731
+
*
732
+
* Reload the value for the guest state ID from the L0 host into the L1 host.
733
+
* This is cached so that going out to the L0 host only happens if necessary.
734
+
*/
735
+
int __kvmhv_nestedv2_cached_reload(struct kvm_vcpu *vcpu, u16 iden)
736
+
{
737
+
struct kvmhv_nestedv2_io *io;
738
+
struct kvmppc_gs_bitmap *valids;
739
+
struct kvmppc_gs_buff *gsb;
740
+
struct kvmppc_gs_msg gsm;
741
+
int rc;
742
+
743
+
if (!iden)
744
+
return 0;
745
+
746
+
io = &vcpu->arch.nestedv2_io;
747
+
valids = &io->valids;
748
+
if (kvmppc_gsbm_test(valids, iden))
749
+
return 0;
750
+
751
+
gsb = io->vcpu_run_input;
752
+
kvmppc_gsm_init(&gsm, &vcpu_message_ops, vcpu, kvmppc_gsid_flags(iden));
753
+
rc = kvmppc_gsb_receive_datum(gsb, &gsm, iden);
754
+
if (rc < 0) {
755
+
pr_err("KVM-NESTEDv2: couldn't get GSID: 0x%x\n", iden);
756
+
return rc;
757
+
}
758
+
return 0;
759
+
}
760
+
EXPORT_SYMBOL_GPL(__kvmhv_nestedv2_cached_reload);
761
+
762
+
/**
763
+
* kvmhv_nestedv2_flush_vcpu() - send modified Guest State IDs to the host
764
+
* @vcpu: vcpu
765
+
* @time_limit: hdec expiry tb
766
+
*
767
+
* Send the values marked by __kvmhv_nestedv2_mark_dirty() to the L0 host.
768
+
* Thread wide values are copied to the H_GUEST_RUN_VCPU input buffer. Guest
769
+
* wide values need to be sent with H_GUEST_SET first.
770
+
*
771
+
* The hdec tb offset is always sent to L0 host.
772
+
*/
773
+
int kvmhv_nestedv2_flush_vcpu(struct kvm_vcpu *vcpu, u64 time_limit)
774
+
{
775
+
struct kvmhv_nestedv2_io *io;
776
+
struct kvmppc_gs_buff *gsb;
777
+
struct kvmppc_gs_msg *gsm;
778
+
int rc;
779
+
780
+
io = &vcpu->arch.nestedv2_io;
781
+
gsb = io->vcpu_run_input;
782
+
gsm = io->vcore_message;
783
+
rc = kvmppc_gsb_send_data(gsb, gsm);
784
+
if (rc < 0) {
785
+
pr_err("KVM-NESTEDv2: couldn't set guest wide elements\n");
786
+
return rc;
787
+
}
788
+
789
+
gsm = io->vcpu_message;
790
+
kvmppc_gsb_reset(gsb);
791
+
rc = kvmppc_gsm_fill_info(gsm, gsb);
792
+
if (rc < 0) {
793
+
pr_err("KVM-NESTEDv2: couldn't fill vcpu run input buffer\n");
794
+
return rc;
795
+
}
796
+
797
+
rc = kvmppc_gse_put_u64(gsb, KVMPPC_GSID_HDEC_EXPIRY_TB, time_limit);
798
+
if (rc < 0)
799
+
return rc;
800
+
return 0;
801
+
}
802
+
EXPORT_SYMBOL_GPL(kvmhv_nestedv2_flush_vcpu);
803
+
804
+
/**
805
+
* kvmhv_nestedv2_set_ptbl_entry() - send partition and process table state to
806
+
* L0 host
807
+
* @lpid: guest id
808
+
* @dw0: partition table double word
809
+
* @dw1: process table double word
810
+
*/
811
+
int kvmhv_nestedv2_set_ptbl_entry(unsigned long lpid, u64 dw0, u64 dw1)
812
+
{
813
+
struct kvmppc_gs_part_table patbl;
814
+
struct kvmppc_gs_proc_table prtbl;
815
+
struct kvmppc_gs_buff *gsb;
816
+
size_t size;
817
+
int rc;
818
+
819
+
size = kvmppc_gse_total_size(
820
+
kvmppc_gsid_size(KVMPPC_GSID_PARTITION_TABLE)) +
821
+
kvmppc_gse_total_size(
822
+
kvmppc_gsid_size(KVMPPC_GSID_PROCESS_TABLE)) +
823
+
sizeof(struct kvmppc_gs_header);
824
+
gsb = kvmppc_gsb_new(size, lpid, 0, GFP_KERNEL);
825
+
if (!gsb)
826
+
return -ENOMEM;
827
+
828
+
patbl.address = dw0 & RPDB_MASK;
829
+
patbl.ea_bits = ((((dw0 & RTS1_MASK) >> (RTS1_SHIFT - 3)) |
830
+
((dw0 & RTS2_MASK) >> RTS2_SHIFT)) +
831
+
31);
832
+
patbl.gpd_size = 1ul << ((dw0 & RPDS_MASK) + 3);
833
+
rc = kvmppc_gse_put_part_table(gsb, KVMPPC_GSID_PARTITION_TABLE, patbl);
834
+
if (rc < 0)
835
+
goto free_gsb;
836
+
837
+
prtbl.address = dw1 & PRTB_MASK;
838
+
prtbl.gpd_size = 1ul << ((dw1 & PRTS_MASK) + 12);
839
+
rc = kvmppc_gse_put_proc_table(gsb, KVMPPC_GSID_PROCESS_TABLE, prtbl);
840
+
if (rc < 0)
841
+
goto free_gsb;
842
+
843
+
rc = kvmppc_gsb_send(gsb, KVMPPC_GS_FLAGS_WIDE);
844
+
if (rc < 0) {
845
+
pr_err("KVM-NESTEDv2: couldn't set the PATE\n");
846
+
goto free_gsb;
847
+
}
848
+
849
+
kvmppc_gsb_free(gsb);
850
+
return 0;
851
+
852
+
free_gsb:
853
+
kvmppc_gsb_free(gsb);
854
+
return rc;
855
+
}
856
+
EXPORT_SYMBOL_GPL(kvmhv_nestedv2_set_ptbl_entry);
857
+
858
+
/**
859
+
* kvmhv_nestedv2_parse_output() - receive values from H_GUEST_RUN_VCPU output
860
+
* @vcpu: vcpu
861
+
*
862
+
* Parse the output buffer from H_GUEST_RUN_VCPU to update vcpu.
863
+
*/
864
+
int kvmhv_nestedv2_parse_output(struct kvm_vcpu *vcpu)
865
+
{
866
+
struct kvmhv_nestedv2_io *io;
867
+
struct kvmppc_gs_buff *gsb;
868
+
struct kvmppc_gs_msg gsm;
869
+
870
+
io = &vcpu->arch.nestedv2_io;
871
+
gsb = io->vcpu_run_output;
872
+
873
+
vcpu->arch.fault_dar = 0;
874
+
vcpu->arch.fault_dsisr = 0;
875
+
vcpu->arch.fault_gpa = 0;
876
+
vcpu->arch.emul_inst = KVM_INST_FETCH_FAILED;
877
+
878
+
kvmppc_gsm_init(&gsm, &vcpu_message_ops, vcpu, 0);
879
+
return kvmppc_gsm_refresh_info(&gsm, gsb);
880
+
}
881
+
EXPORT_SYMBOL_GPL(kvmhv_nestedv2_parse_output);
882
+
883
+
static void kvmhv_nestedv2_host_free(struct kvm_vcpu *vcpu,
884
+
struct kvmhv_nestedv2_io *io)
885
+
{
886
+
kvmppc_gsm_free(io->vcpu_message);
887
+
kvmppc_gsm_free(io->vcore_message);
888
+
kvmppc_gsb_free(io->vcpu_run_input);
889
+
kvmppc_gsb_free(io->vcpu_run_output);
890
+
}
891
+
892
+
int __kvmhv_nestedv2_reload_ptregs(struct kvm_vcpu *vcpu, struct pt_regs *regs)
893
+
{
894
+
struct kvmhv_nestedv2_io *io;
895
+
struct kvmppc_gs_bitmap *valids;
896
+
struct kvmppc_gs_buff *gsb;
897
+
struct kvmppc_gs_msg gsm;
898
+
int rc = 0;
899
+
900
+
901
+
io = &vcpu->arch.nestedv2_io;
902
+
valids = &io->valids;
903
+
904
+
gsb = io->vcpu_run_input;
905
+
kvmppc_gsm_init(&gsm, &vcpu_message_ops, vcpu, 0);
906
+
907
+
for (int i = 0; i < 32; i++) {
908
+
if (!kvmppc_gsbm_test(valids, KVMPPC_GSID_GPR(i)))
909
+
kvmppc_gsm_include(&gsm, KVMPPC_GSID_GPR(i));
910
+
}
911
+
912
+
if (!kvmppc_gsbm_test(valids, KVMPPC_GSID_CR))
913
+
kvmppc_gsm_include(&gsm, KVMPPC_GSID_CR);
914
+
915
+
if (!kvmppc_gsbm_test(valids, KVMPPC_GSID_XER))
916
+
kvmppc_gsm_include(&gsm, KVMPPC_GSID_XER);
917
+
918
+
if (!kvmppc_gsbm_test(valids, KVMPPC_GSID_CTR))
919
+
kvmppc_gsm_include(&gsm, KVMPPC_GSID_CTR);
920
+
921
+
if (!kvmppc_gsbm_test(valids, KVMPPC_GSID_LR))
922
+
kvmppc_gsm_include(&gsm, KVMPPC_GSID_LR);
923
+
924
+
if (!kvmppc_gsbm_test(valids, KVMPPC_GSID_NIA))
925
+
kvmppc_gsm_include(&gsm, KVMPPC_GSID_NIA);
926
+
927
+
rc = kvmppc_gsb_receive_data(gsb, &gsm);
928
+
if (rc < 0)
929
+
pr_err("KVM-NESTEDv2: couldn't reload ptregs\n");
930
+
931
+
return rc;
932
+
}
933
+
EXPORT_SYMBOL_GPL(__kvmhv_nestedv2_reload_ptregs);
934
+
935
+
int __kvmhv_nestedv2_mark_dirty_ptregs(struct kvm_vcpu *vcpu,
936
+
struct pt_regs *regs)
937
+
{
938
+
for (int i = 0; i < 32; i++)
939
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_GPR(i));
940
+
941
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_CR);
942
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_XER);
943
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_CTR);
944
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_LR);
945
+
kvmhv_nestedv2_mark_dirty(vcpu, KVMPPC_GSID_NIA);
946
+
947
+
return 0;
948
+
}
949
+
EXPORT_SYMBOL_GPL(__kvmhv_nestedv2_mark_dirty_ptregs);
950
+
951
+
/**
952
+
* kvmhv_nestedv2_vcpu_create() - create nested vcpu for the NESTEDv2 API
953
+
* @vcpu: vcpu
954
+
* @io: NESTEDv2 nested io state
955
+
*
956
+
* Parse the output buffer from H_GUEST_RUN_VCPU to update vcpu.
957
+
*/
958
+
int kvmhv_nestedv2_vcpu_create(struct kvm_vcpu *vcpu,
959
+
struct kvmhv_nestedv2_io *io)
960
+
{
961
+
long rc;
962
+
963
+
rc = plpar_guest_create_vcpu(0, vcpu->kvm->arch.lpid, vcpu->vcpu_id);
964
+
965
+
if (rc != H_SUCCESS) {
966
+
pr_err("KVM: Create Guest vcpu hcall failed, rc=%ld\n", rc);
967
+
switch (rc) {
968
+
case H_NOT_ENOUGH_RESOURCES:
969
+
case H_ABORTED:
970
+
return -ENOMEM;
971
+
case H_AUTHORITY:
972
+
return -EPERM;
973
+
default:
974
+
return -EINVAL;
975
+
}
976
+
}
977
+
978
+
rc = kvmhv_nestedv2_host_create(vcpu, io);
979
+
980
+
return rc;
981
+
}
982
+
EXPORT_SYMBOL_GPL(kvmhv_nestedv2_vcpu_create);
983
+
984
+
/**
985
+
* kvmhv_nestedv2_vcpu_free() - free the NESTEDv2 state
986
+
* @vcpu: vcpu
987
+
* @io: NESTEDv2 nested io state
988
+
*/
989
+
void kvmhv_nestedv2_vcpu_free(struct kvm_vcpu *vcpu,
990
+
struct kvmhv_nestedv2_io *io)
991
+
{
992
+
kvmhv_nestedv2_host_free(vcpu, io);
993
+
}
994
+
EXPORT_SYMBOL_GPL(kvmhv_nestedv2_vcpu_free);
+2
-2
arch/powerpc/kvm/book3s_hv_p9_entry.c
+2
-2
arch/powerpc/kvm/book3s_hv_p9_entry.c
···
305
305
u32 pid;
306
306
307
307
lpid = nested ? nested->shadow_lpid : kvm->arch.lpid;
308
-
pid = vcpu->arch.pid;
308
+
pid = kvmppc_get_pid(vcpu);
309
309
310
310
/*
311
311
* Prior memory accesses to host PID Q3 must be completed before we
···
330
330
int i;
331
331
332
332
lpid = kvm->arch.lpid;
333
-
pid = vcpu->arch.pid;
333
+
pid = kvmppc_get_pid(vcpu);
334
334
335
335
/*
336
336
* See switch_mmu_to_guest_radix. ptesync should not be required here
+2
-2
arch/powerpc/kvm/book3s_hv_ras.c
+2
-2
arch/powerpc/kvm/book3s_hv_ras.c
···
174
174
ppc_md.hmi_exception_early(NULL);
175
175
176
176
out:
177
-
if (vc->tb_offset) {
177
+
if (kvmppc_get_tb_offset(vcpu)) {
178
178
u64 new_tb = mftb() + vc->tb_offset;
179
179
mtspr(SPRN_TBU40, new_tb);
180
180
if ((mftb() & 0xffffff) < (new_tb & 0xffffff)) {
181
181
new_tb += 0x1000000;
182
182
mtspr(SPRN_TBU40, new_tb);
183
183
}
184
-
vc->tb_offset_applied = vc->tb_offset;
184
+
vc->tb_offset_applied = kvmppc_get_tb_offset(vcpu);
185
185
}
186
186
187
187
return ret;
+4
-4
arch/powerpc/kvm/book3s_hv_rm_mmu.c
+4
-4
arch/powerpc/kvm/book3s_hv_rm_mmu.c
···
776
776
r = rev[i].guest_rpte | (r & (HPTE_R_R | HPTE_R_C));
777
777
r &= ~HPTE_GR_RESERVED;
778
778
}
779
-
vcpu->arch.regs.gpr[4 + i * 2] = v;
780
-
vcpu->arch.regs.gpr[5 + i * 2] = r;
779
+
kvmppc_set_gpr(vcpu, 4 + i * 2, v);
780
+
kvmppc_set_gpr(vcpu, 5 + i * 2, r);
781
781
}
782
782
return H_SUCCESS;
783
783
}
···
824
824
}
825
825
}
826
826
}
827
-
vcpu->arch.regs.gpr[4] = gr;
827
+
kvmppc_set_gpr(vcpu, 4, gr);
828
828
ret = H_SUCCESS;
829
829
out:
830
830
unlock_hpte(hpte, v & ~HPTE_V_HVLOCK);
···
872
872
kvmppc_set_dirty_from_hpte(kvm, v, gr);
873
873
}
874
874
}
875
-
vcpu->arch.regs.gpr[4] = gr;
875
+
kvmppc_set_gpr(vcpu, 4, gr);
876
876
ret = H_SUCCESS;
877
877
out:
878
878
unlock_hpte(hpte, v & ~HPTE_V_HVLOCK);
+2
-2
arch/powerpc/kvm/book3s_hv_rm_xics.c
+2
-2
arch/powerpc/kvm/book3s_hv_rm_xics.c
···
481
481
482
482
unsigned long xics_rm_h_xirr_x(struct kvm_vcpu *vcpu)
483
483
{
484
-
vcpu->arch.regs.gpr[5] = get_tb();
484
+
kvmppc_set_gpr(vcpu, 5, get_tb());
485
485
return xics_rm_h_xirr(vcpu);
486
486
}
487
487
···
518
518
} while (!icp_rm_try_update(icp, old_state, new_state));
519
519
520
520
/* Return the result in GPR4 */
521
-
vcpu->arch.regs.gpr[4] = xirr;
521
+
kvmppc_set_gpr(vcpu, 4, xirr);
522
522
523
523
return check_too_hard(xics, icp);
524
524
}
+1
-1
arch/powerpc/kvm/book3s_hv_uvmem.c
+1
-1
arch/powerpc/kvm/book3s_hv_uvmem.c
+5
-7
arch/powerpc/kvm/book3s_xive.c
+5
-7
arch/powerpc/kvm/book3s_xive.c
···
328
328
*/
329
329
330
330
/* Return interrupt and old CPPR in GPR4 */
331
-
vcpu->arch.regs.gpr[4] = hirq | (old_cppr << 24);
331
+
kvmppc_set_gpr(vcpu, 4, hirq | (old_cppr << 24));
332
332
333
333
return H_SUCCESS;
334
334
}
···
364
364
hirq = xive_vm_scan_interrupts(xc, pending, scan_poll);
365
365
366
366
/* Return interrupt and old CPPR in GPR4 */
367
-
vcpu->arch.regs.gpr[4] = hirq | (xc->cppr << 24);
367
+
kvmppc_set_gpr(vcpu, 4, hirq | (xc->cppr << 24));
368
368
369
369
return H_SUCCESS;
370
370
}
···
884
884
}
885
885
886
886
if (single_escalation)
887
-
name = kasprintf(GFP_KERNEL, "kvm-%d-%d",
887
+
name = kasprintf(GFP_KERNEL, "kvm-%lld-%d",
888
888
vcpu->kvm->arch.lpid, xc->server_num);
889
889
else
890
-
name = kasprintf(GFP_KERNEL, "kvm-%d-%d-%d",
890
+
name = kasprintf(GFP_KERNEL, "kvm-%lld-%d-%d",
891
891
vcpu->kvm->arch.lpid, xc->server_num, prio);
892
892
if (!name) {
893
893
pr_err("Failed to allocate escalation irq name for queue %d of VCPU %d\n",
···
2779
2779
2780
2780
int kvmppc_xive_xics_hcall(struct kvm_vcpu *vcpu, u32 req)
2781
2781
{
2782
-
struct kvmppc_vcore *vc = vcpu->arch.vcore;
2783
-
2784
2782
/* The VM should have configured XICS mode before doing XICS hcalls. */
2785
2783
if (!kvmppc_xics_enabled(vcpu))
2786
2784
return H_TOO_HARD;
···
2797
2799
return xive_vm_h_ipoll(vcpu, kvmppc_get_gpr(vcpu, 4));
2798
2800
case H_XIRR_X:
2799
2801
xive_vm_h_xirr(vcpu);
2800
-
kvmppc_set_gpr(vcpu, 5, get_tb() + vc->tb_offset);
2802
+
kvmppc_set_gpr(vcpu, 5, get_tb() + kvmppc_get_tb_offset(vcpu));
2801
2803
return H_SUCCESS;
2802
2804
}
2803
2805
+1
-1
arch/powerpc/kvm/book3s_xive_native.c
+1
-1
arch/powerpc/kvm/book3s_xive_native.c
+4
-2
arch/powerpc/kvm/emulate_loadstore.c
+4
-2
arch/powerpc/kvm/emulate_loadstore.c
···
92
92
vcpu->arch.mmio_host_swabbed = 0;
93
93
94
94
emulated = EMULATE_FAIL;
95
-
vcpu->arch.regs.msr = vcpu->arch.shared->msr;
95
+
vcpu->arch.regs.msr = kvmppc_get_msr(vcpu);
96
+
kvmhv_nestedv2_reload_ptregs(vcpu, &vcpu->arch.regs);
96
97
if (analyse_instr(&op, &vcpu->arch.regs, inst) == 0) {
97
98
int type = op.type & INSTR_TYPE_MASK;
98
99
int size = GETSIZE(op.type);
···
251
250
vcpu->arch.mmio_sp64_extend = 1;
252
251
253
252
emulated = kvmppc_handle_store(vcpu,
254
-
VCPU_FPR(vcpu, op.reg), size, 1);
253
+
kvmppc_get_fpr(vcpu, op.reg), size, 1);
255
254
256
255
if ((op.type & UPDATE) && (emulated != EMULATE_FAIL))
257
256
kvmppc_set_gpr(vcpu, op.update_reg, op.ea);
···
358
357
}
359
358
360
359
trace_kvm_ppc_instr(ppc_inst_val(inst), kvmppc_get_pc(vcpu), emulated);
360
+
kvmhv_nestedv2_mark_dirty_ptregs(vcpu, &vcpu->arch.regs);
361
361
362
362
/* Advance past emulated instruction. */
363
363
if (emulated != EMULATE_FAIL)
+621
arch/powerpc/kvm/guest-state-buffer.c
+621
arch/powerpc/kvm/guest-state-buffer.c
···
1
+
// SPDX-License-Identifier: GPL-2.0
2
+
3
+
#include "asm/hvcall.h"
4
+
#include <linux/log2.h>
5
+
#include <asm/pgalloc.h>
6
+
#include <asm/guest-state-buffer.h>
7
+
8
+
static const u16 kvmppc_gse_iden_len[__KVMPPC_GSE_TYPE_MAX] = {
9
+
[KVMPPC_GSE_BE32] = sizeof(__be32),
10
+
[KVMPPC_GSE_BE64] = sizeof(__be64),
11
+
[KVMPPC_GSE_VEC128] = sizeof(vector128),
12
+
[KVMPPC_GSE_PARTITION_TABLE] = sizeof(struct kvmppc_gs_part_table),
13
+
[KVMPPC_GSE_PROCESS_TABLE] = sizeof(struct kvmppc_gs_proc_table),
14
+
[KVMPPC_GSE_BUFFER] = sizeof(struct kvmppc_gs_buff_info),
15
+
};
16
+
17
+
/**
18
+
* kvmppc_gsb_new() - create a new guest state buffer
19
+
* @size: total size of the guest state buffer (includes header)
20
+
* @guest_id: guest_id
21
+
* @vcpu_id: vcpu_id
22
+
* @flags: GFP flags
23
+
*
24
+
* Returns a guest state buffer.
25
+
*/
26
+
struct kvmppc_gs_buff *kvmppc_gsb_new(size_t size, unsigned long guest_id,
27
+
unsigned long vcpu_id, gfp_t flags)
28
+
{
29
+
struct kvmppc_gs_buff *gsb;
30
+
31
+
gsb = kzalloc(sizeof(*gsb), flags);
32
+
if (!gsb)
33
+
return NULL;
34
+
35
+
size = roundup_pow_of_two(size);
36
+
gsb->hdr = kzalloc(size, GFP_KERNEL);
37
+
if (!gsb->hdr)
38
+
goto free;
39
+
40
+
gsb->capacity = size;
41
+
gsb->len = sizeof(struct kvmppc_gs_header);
42
+
gsb->vcpu_id = vcpu_id;
43
+
gsb->guest_id = guest_id;
44
+
45
+
gsb->hdr->nelems = cpu_to_be32(0);
46
+
47
+
return gsb;
48
+
49
+
free:
50
+
kfree(gsb);
51
+
return NULL;
52
+
}
53
+
EXPORT_SYMBOL_GPL(kvmppc_gsb_new);
54
+
55
+
/**
56
+
* kvmppc_gsb_free() - free a guest state buffer
57
+
* @gsb: guest state buffer
58
+
*/
59
+
void kvmppc_gsb_free(struct kvmppc_gs_buff *gsb)
60
+
{
61
+
kfree(gsb->hdr);
62
+
kfree(gsb);
63
+
}
64
+
EXPORT_SYMBOL_GPL(kvmppc_gsb_free);
65
+
66
+
/**
67
+
* kvmppc_gsb_put() - allocate space in a guest state buffer
68
+
* @gsb: buffer to allocate in
69
+
* @size: amount of space to allocate
70
+
*
71
+
* Returns a pointer to the amount of space requested within the buffer and
72
+
* increments the count of elements in the buffer.
73
+
*
74
+
* Does not check if there is enough space in the buffer.
75
+
*/
76
+
void *kvmppc_gsb_put(struct kvmppc_gs_buff *gsb, size_t size)
77
+
{
78
+
u32 nelems = kvmppc_gsb_nelems(gsb);
79
+
void *p;
80
+
81
+
p = (void *)kvmppc_gsb_header(gsb) + kvmppc_gsb_len(gsb);
82
+
gsb->len += size;
83
+
84
+
kvmppc_gsb_header(gsb)->nelems = cpu_to_be32(nelems + 1);
85
+
return p;
86
+
}
87
+
EXPORT_SYMBOL_GPL(kvmppc_gsb_put);
88
+
89
+
static int kvmppc_gsid_class(u16 iden)
90
+
{
91
+
if ((iden >= KVMPPC_GSE_GUESTWIDE_START) &&
92
+
(iden <= KVMPPC_GSE_GUESTWIDE_END))
93
+
return KVMPPC_GS_CLASS_GUESTWIDE;
94
+
95
+
if ((iden >= KVMPPC_GSE_META_START) && (iden <= KVMPPC_GSE_META_END))
96
+
return KVMPPC_GS_CLASS_META;
97
+
98
+
if ((iden >= KVMPPC_GSE_DW_REGS_START) &&
99
+
(iden <= KVMPPC_GSE_DW_REGS_END))
100
+
return KVMPPC_GS_CLASS_DWORD_REG;
101
+
102
+
if ((iden >= KVMPPC_GSE_W_REGS_START) &&
103
+
(iden <= KVMPPC_GSE_W_REGS_END))
104
+
return KVMPPC_GS_CLASS_WORD_REG;
105
+
106
+
if ((iden >= KVMPPC_GSE_VSRS_START) && (iden <= KVMPPC_GSE_VSRS_END))
107
+
return KVMPPC_GS_CLASS_VECTOR;
108
+
109
+
if ((iden >= KVMPPC_GSE_INTR_REGS_START) &&
110
+
(iden <= KVMPPC_GSE_INTR_REGS_END))
111
+
return KVMPPC_GS_CLASS_INTR;
112
+
113
+
return -1;
114
+
}
115
+
116
+
static int kvmppc_gsid_type(u16 iden)
117
+
{
118
+
int type = -1;
119
+
120
+
switch (kvmppc_gsid_class(iden)) {
121
+
case KVMPPC_GS_CLASS_GUESTWIDE:
122
+
switch (iden) {
123
+
case KVMPPC_GSID_HOST_STATE_SIZE:
124
+
case KVMPPC_GSID_RUN_OUTPUT_MIN_SIZE:
125
+
case KVMPPC_GSID_TB_OFFSET:
126
+
type = KVMPPC_GSE_BE64;
127
+
break;
128
+
case KVMPPC_GSID_PARTITION_TABLE:
129
+
type = KVMPPC_GSE_PARTITION_TABLE;
130
+
break;
131
+
case KVMPPC_GSID_PROCESS_TABLE:
132
+
type = KVMPPC_GSE_PROCESS_TABLE;
133
+
break;
134
+
case KVMPPC_GSID_LOGICAL_PVR:
135
+
type = KVMPPC_GSE_BE32;
136
+
break;
137
+
}
138
+
break;
139
+
case KVMPPC_GS_CLASS_META:
140
+
switch (iden) {
141
+
case KVMPPC_GSID_RUN_INPUT:
142
+
case KVMPPC_GSID_RUN_OUTPUT:
143
+
type = KVMPPC_GSE_BUFFER;
144
+
break;
145
+
case KVMPPC_GSID_VPA:
146
+
type = KVMPPC_GSE_BE64;
147
+
break;
148
+
}
149
+
break;
150
+
case KVMPPC_GS_CLASS_DWORD_REG:
151
+
type = KVMPPC_GSE_BE64;
152
+
break;
153
+
case KVMPPC_GS_CLASS_WORD_REG:
154
+
type = KVMPPC_GSE_BE32;
155
+
break;
156
+
case KVMPPC_GS_CLASS_VECTOR:
157
+
type = KVMPPC_GSE_VEC128;
158
+
break;
159
+
case KVMPPC_GS_CLASS_INTR:
160
+
switch (iden) {
161
+
case KVMPPC_GSID_HDAR:
162
+
case KVMPPC_GSID_ASDR:
163
+
case KVMPPC_GSID_HEIR:
164
+
type = KVMPPC_GSE_BE64;
165
+
break;
166
+
case KVMPPC_GSID_HDSISR:
167
+
type = KVMPPC_GSE_BE32;
168
+
break;
169
+
}
170
+
break;
171
+
}
172
+
173
+
return type;
174
+
}
175
+
176
+
/**
177
+
* kvmppc_gsid_flags() - the flags for a guest state ID
178
+
* @iden: guest state ID
179
+
*
180
+
* Returns any flags for the guest state ID.
181
+
*/
182
+
unsigned long kvmppc_gsid_flags(u16 iden)
183
+
{
184
+
unsigned long flags = 0;
185
+
186
+
switch (kvmppc_gsid_class(iden)) {
187
+
case KVMPPC_GS_CLASS_GUESTWIDE:
188
+
flags = KVMPPC_GS_FLAGS_WIDE;
189
+
break;
190
+
case KVMPPC_GS_CLASS_META:
191
+
case KVMPPC_GS_CLASS_DWORD_REG:
192
+
case KVMPPC_GS_CLASS_WORD_REG:
193
+
case KVMPPC_GS_CLASS_VECTOR:
194
+
case KVMPPC_GS_CLASS_INTR:
195
+
break;
196
+
}
197
+
198
+
return flags;
199
+
}
200
+
EXPORT_SYMBOL_GPL(kvmppc_gsid_flags);
201
+
202
+
/**
203
+
* kvmppc_gsid_size() - the size of a guest state ID
204
+
* @iden: guest state ID
205
+
*
206
+
* Returns the size of guest state ID.
207
+
*/
208
+
u16 kvmppc_gsid_size(u16 iden)
209
+
{
210
+
int type;
211
+
212
+
type = kvmppc_gsid_type(iden);
213
+
if (type == -1)
214
+
return 0;
215
+
216
+
if (type >= __KVMPPC_GSE_TYPE_MAX)
217
+
return 0;
218
+
219
+
return kvmppc_gse_iden_len[type];
220
+
}
221
+
EXPORT_SYMBOL_GPL(kvmppc_gsid_size);
222
+
223
+
/**
224
+
* kvmppc_gsid_mask() - the settable bits of a guest state ID
225
+
* @iden: guest state ID
226
+
*
227
+
* Returns a mask of settable bits for a guest state ID.
228
+
*/
229
+
u64 kvmppc_gsid_mask(u16 iden)
230
+
{
231
+
u64 mask = ~0ull;
232
+
233
+
switch (iden) {
234
+
case KVMPPC_GSID_LPCR:
235
+
mask = LPCR_DPFD | LPCR_ILE | LPCR_AIL | LPCR_LD | LPCR_MER |
236
+
LPCR_GTSE;
237
+
break;
238
+
case KVMPPC_GSID_MSR:
239
+
mask = ~(MSR_HV | MSR_S | MSR_ME);
240
+
break;
241
+
}
242
+
243
+
return mask;
244
+
}
245
+
EXPORT_SYMBOL_GPL(kvmppc_gsid_mask);
246
+
247
+
/**
248
+
* __kvmppc_gse_put() - add a guest state element to a buffer
249
+
* @gsb: buffer to the element to
250
+
* @iden: guest state ID
251
+
* @size: length of data
252
+
* @data: pointer to data
253
+
*/
254
+
int __kvmppc_gse_put(struct kvmppc_gs_buff *gsb, u16 iden, u16 size,
255
+
const void *data)
256
+
{
257
+
struct kvmppc_gs_elem *gse;
258
+
u16 total_size;
259
+
260
+
total_size = sizeof(*gse) + size;
261
+
if (total_size + kvmppc_gsb_len(gsb) > kvmppc_gsb_capacity(gsb))
262
+
return -ENOMEM;
263
+
264
+
if (kvmppc_gsid_size(iden) != size)
265
+
return -EINVAL;
266
+
267
+
gse = kvmppc_gsb_put(gsb, total_size);
268
+
gse->iden = cpu_to_be16(iden);
269
+
gse->len = cpu_to_be16(size);
270
+
memcpy(gse->data, data, size);
271
+
272
+
return 0;
273
+
}
274
+
EXPORT_SYMBOL_GPL(__kvmppc_gse_put);
275
+
276
+
/**
277
+
* kvmppc_gse_parse() - create a parse map from a guest state buffer
278
+
* @gsp: guest state parser
279
+
* @gsb: guest state buffer
280
+
*/
281
+
int kvmppc_gse_parse(struct kvmppc_gs_parser *gsp, struct kvmppc_gs_buff *gsb)
282
+
{
283
+
struct kvmppc_gs_elem *curr;
284
+
int rem, i;
285
+
286
+
kvmppc_gsb_for_each_elem(i, curr, gsb, rem) {
287
+
if (kvmppc_gse_len(curr) !=
288
+
kvmppc_gsid_size(kvmppc_gse_iden(curr)))
289
+
return -EINVAL;
290
+
kvmppc_gsp_insert(gsp, kvmppc_gse_iden(curr), curr);
291
+
}
292
+
293
+
if (kvmppc_gsb_nelems(gsb) != i)
294
+
return -EINVAL;
295
+
return 0;
296
+
}
297
+
EXPORT_SYMBOL_GPL(kvmppc_gse_parse);
298
+
299
+
static inline int kvmppc_gse_flatten_iden(u16 iden)
300
+
{
301
+
int bit = 0;
302
+
int class;
303
+
304
+
class = kvmppc_gsid_class(iden);
305
+
306
+
if (class == KVMPPC_GS_CLASS_GUESTWIDE) {
307
+
bit += iden - KVMPPC_GSE_GUESTWIDE_START;
308
+
return bit;
309
+
}
310
+
311
+
bit += KVMPPC_GSE_GUESTWIDE_COUNT;
312
+
313
+
if (class == KVMPPC_GS_CLASS_META) {
314
+
bit += iden - KVMPPC_GSE_META_START;
315
+
return bit;
316
+
}
317
+
318
+
bit += KVMPPC_GSE_META_COUNT;
319
+
320
+
if (class == KVMPPC_GS_CLASS_DWORD_REG) {
321
+
bit += iden - KVMPPC_GSE_DW_REGS_START;
322
+
return bit;
323
+
}
324
+
325
+
bit += KVMPPC_GSE_DW_REGS_COUNT;
326
+
327
+
if (class == KVMPPC_GS_CLASS_WORD_REG) {
328
+
bit += iden - KVMPPC_GSE_W_REGS_START;
329
+
return bit;
330
+
}
331
+
332
+
bit += KVMPPC_GSE_W_REGS_COUNT;
333
+
334
+
if (class == KVMPPC_GS_CLASS_VECTOR) {
335
+
bit += iden - KVMPPC_GSE_VSRS_START;
336
+
return bit;
337
+
}
338
+
339
+
bit += KVMPPC_GSE_VSRS_COUNT;
340
+
341
+
if (class == KVMPPC_GS_CLASS_INTR) {
342
+
bit += iden - KVMPPC_GSE_INTR_REGS_START;
343
+
return bit;
344
+
}
345
+
346
+
return 0;
347
+
}
348
+
349
+
static inline u16 kvmppc_gse_unflatten_iden(int bit)
350
+
{
351
+
u16 iden;
352
+
353
+
if (bit < KVMPPC_GSE_GUESTWIDE_COUNT) {
354
+
iden = KVMPPC_GSE_GUESTWIDE_START + bit;
355
+
return iden;
356
+
}
357
+
bit -= KVMPPC_GSE_GUESTWIDE_COUNT;
358
+
359
+
if (bit < KVMPPC_GSE_META_COUNT) {
360
+
iden = KVMPPC_GSE_META_START + bit;
361
+
return iden;
362
+
}
363
+
bit -= KVMPPC_GSE_META_COUNT;
364
+
365
+
if (bit < KVMPPC_GSE_DW_REGS_COUNT) {
366
+
iden = KVMPPC_GSE_DW_REGS_START + bit;
367
+
return iden;
368
+
}
369
+
bit -= KVMPPC_GSE_DW_REGS_COUNT;
370
+
371
+
if (bit < KVMPPC_GSE_W_REGS_COUNT) {
372
+
iden = KVMPPC_GSE_W_REGS_START + bit;
373
+
return iden;
374
+
}
375
+
bit -= KVMPPC_GSE_W_REGS_COUNT;
376
+
377
+
if (bit < KVMPPC_GSE_VSRS_COUNT) {
378
+
iden = KVMPPC_GSE_VSRS_START + bit;
379
+
return iden;
380
+
}
381
+
bit -= KVMPPC_GSE_VSRS_COUNT;
382
+
383
+
if (bit < KVMPPC_GSE_IDEN_COUNT) {
384
+
iden = KVMPPC_GSE_INTR_REGS_START + bit;
385
+
return iden;
386
+
}
387
+
388
+
return 0;
389
+
}
390
+
391
+
/**
392
+
* kvmppc_gsp_insert() - add a mapping from an guest state ID to an element
393
+
* @gsp: guest state parser
394
+
* @iden: guest state id (key)
395
+
* @gse: guest state element (value)
396
+
*/
397
+
void kvmppc_gsp_insert(struct kvmppc_gs_parser *gsp, u16 iden,
398
+
struct kvmppc_gs_elem *gse)
399
+
{
400
+
int i;
401
+
402
+
i = kvmppc_gse_flatten_iden(iden);
403
+
kvmppc_gsbm_set(&gsp->iterator, iden);
404
+
gsp->gses[i] = gse;
405
+
}
406
+
EXPORT_SYMBOL_GPL(kvmppc_gsp_insert);
407
+
408
+
/**
409
+
* kvmppc_gsp_lookup() - lookup an element from a guest state ID
410
+
* @gsp: guest state parser
411
+
* @iden: guest state ID (key)
412
+
*
413
+
* Returns the guest state element if present.
414
+
*/
415
+
struct kvmppc_gs_elem *kvmppc_gsp_lookup(struct kvmppc_gs_parser *gsp, u16 iden)
416
+
{
417
+
int i;
418
+
419
+
i = kvmppc_gse_flatten_iden(iden);
420
+
return gsp->gses[i];
421
+
}
422
+
EXPORT_SYMBOL_GPL(kvmppc_gsp_lookup);
423
+
424
+
/**
425
+
* kvmppc_gsbm_set() - set the guest state ID
426
+
* @gsbm: guest state bitmap
427
+
* @iden: guest state ID
428
+
*/
429
+
void kvmppc_gsbm_set(struct kvmppc_gs_bitmap *gsbm, u16 iden)
430
+
{
431
+
set_bit(kvmppc_gse_flatten_iden(iden), gsbm->bitmap);
432
+
}
433
+
EXPORT_SYMBOL_GPL(kvmppc_gsbm_set);
434
+
435
+
/**
436
+
* kvmppc_gsbm_clear() - clear the guest state ID
437
+
* @gsbm: guest state bitmap
438
+
* @iden: guest state ID
439
+
*/
440
+
void kvmppc_gsbm_clear(struct kvmppc_gs_bitmap *gsbm, u16 iden)
441
+
{
442
+
clear_bit(kvmppc_gse_flatten_iden(iden), gsbm->bitmap);
443
+
}
444
+
EXPORT_SYMBOL_GPL(kvmppc_gsbm_clear);
445
+
446
+
/**
447
+
* kvmppc_gsbm_test() - test the guest state ID
448
+
* @gsbm: guest state bitmap
449
+
* @iden: guest state ID
450
+
*/
451
+
bool kvmppc_gsbm_test(struct kvmppc_gs_bitmap *gsbm, u16 iden)
452
+
{
453
+
return test_bit(kvmppc_gse_flatten_iden(iden), gsbm->bitmap);
454
+
}
455
+
EXPORT_SYMBOL_GPL(kvmppc_gsbm_test);
456
+
457
+
/**
458
+
* kvmppc_gsbm_next() - return the next set guest state ID
459
+
* @gsbm: guest state bitmap
460
+
* @prev: last guest state ID
461
+
*/
462
+
u16 kvmppc_gsbm_next(struct kvmppc_gs_bitmap *gsbm, u16 prev)
463
+
{
464
+
int bit, pbit;
465
+
466
+
pbit = prev ? kvmppc_gse_flatten_iden(prev) + 1 : 0;
467
+
bit = find_next_bit(gsbm->bitmap, KVMPPC_GSE_IDEN_COUNT, pbit);
468
+
469
+
if (bit < KVMPPC_GSE_IDEN_COUNT)
470
+
return kvmppc_gse_unflatten_iden(bit);
471
+
return 0;
472
+
}
473
+
EXPORT_SYMBOL_GPL(kvmppc_gsbm_next);
474
+
475
+
/**
476
+
* kvmppc_gsm_init() - initialize a guest state message
477
+
* @gsm: guest state message
478
+
* @ops: callbacks
479
+
* @data: private data
480
+
* @flags: guest wide or thread wide
481
+
*/
482
+
int kvmppc_gsm_init(struct kvmppc_gs_msg *gsm, struct kvmppc_gs_msg_ops *ops,
483
+
void *data, unsigned long flags)
484
+
{
485
+
memset(gsm, 0, sizeof(*gsm));
486
+
gsm->ops = ops;
487
+
gsm->data = data;
488
+
gsm->flags = flags;
489
+
490
+
return 0;
491
+
}
492
+
EXPORT_SYMBOL_GPL(kvmppc_gsm_init);
493
+
494
+
/**
495
+
* kvmppc_gsm_new() - creates a new guest state message
496
+
* @ops: callbacks
497
+
* @data: private data
498
+
* @flags: guest wide or thread wide
499
+
* @gfp_flags: GFP allocation flags
500
+
*
501
+
* Returns an initialized guest state message.
502
+
*/
503
+
struct kvmppc_gs_msg *kvmppc_gsm_new(struct kvmppc_gs_msg_ops *ops, void *data,
504
+
unsigned long flags, gfp_t gfp_flags)
505
+
{
506
+
struct kvmppc_gs_msg *gsm;
507
+
508
+
gsm = kzalloc(sizeof(*gsm), gfp_flags);
509
+
if (!gsm)
510
+
return NULL;
511
+
512
+
kvmppc_gsm_init(gsm, ops, data, flags);
513
+
514
+
return gsm;
515
+
}
516
+
EXPORT_SYMBOL_GPL(kvmppc_gsm_new);
517
+
518
+
/**
519
+
* kvmppc_gsm_size() - creates a new guest state message
520
+
* @gsm: self
521
+
*
522
+
* Returns the size required for the message.
523
+
*/
524
+
size_t kvmppc_gsm_size(struct kvmppc_gs_msg *gsm)
525
+
{
526
+
if (gsm->ops->get_size)
527
+
return gsm->ops->get_size(gsm);
528
+
return 0;
529
+
}
530
+
EXPORT_SYMBOL_GPL(kvmppc_gsm_size);
531
+
532
+
/**
533
+
* kvmppc_gsm_free() - free guest state message
534
+
* @gsm: guest state message
535
+
*
536
+
* Returns the size required for the message.
537
+
*/
538
+
void kvmppc_gsm_free(struct kvmppc_gs_msg *gsm)
539
+
{
540
+
kfree(gsm);
541
+
}
542
+
EXPORT_SYMBOL_GPL(kvmppc_gsm_free);
543
+
544
+
/**
545
+
* kvmppc_gsm_fill_info() - serialises message to guest state buffer format
546
+
* @gsm: self
547
+
* @gsb: buffer to serialise into
548
+
*/
549
+
int kvmppc_gsm_fill_info(struct kvmppc_gs_msg *gsm, struct kvmppc_gs_buff *gsb)
550
+
{
551
+
if (!gsm->ops->fill_info)
552
+
return -EINVAL;
553
+
554
+
return gsm->ops->fill_info(gsb, gsm);
555
+
}
556
+
EXPORT_SYMBOL_GPL(kvmppc_gsm_fill_info);
557
+
558
+
/**
559
+
* kvmppc_gsm_refresh_info() - deserialises from guest state buffer
560
+
* @gsm: self
561
+
* @gsb: buffer to serialise from
562
+
*/
563
+
int kvmppc_gsm_refresh_info(struct kvmppc_gs_msg *gsm,
564
+
struct kvmppc_gs_buff *gsb)
565
+
{
566
+
if (!gsm->ops->fill_info)
567
+
return -EINVAL;
568
+
569
+
return gsm->ops->refresh_info(gsm, gsb);
570
+
}
571
+
EXPORT_SYMBOL_GPL(kvmppc_gsm_refresh_info);
572
+
573
+
/**
574
+
* kvmppc_gsb_send - send all elements in the buffer to the hypervisor.
575
+
* @gsb: guest state buffer
576
+
* @flags: guest wide or thread wide
577
+
*
578
+
* Performs the H_GUEST_SET_STATE hcall for the guest state buffer.
579
+
*/
580
+
int kvmppc_gsb_send(struct kvmppc_gs_buff *gsb, unsigned long flags)
581
+
{
582
+
unsigned long hflags = 0;
583
+
unsigned long i;
584
+
int rc;
585
+
586
+
if (kvmppc_gsb_nelems(gsb) == 0)
587
+
return 0;
588
+
589
+
if (flags & KVMPPC_GS_FLAGS_WIDE)
590
+
hflags |= H_GUEST_FLAGS_WIDE;
591
+
592
+
rc = plpar_guest_set_state(hflags, gsb->guest_id, gsb->vcpu_id,
593
+
__pa(gsb->hdr), gsb->capacity, &i);
594
+
return rc;
595
+
}
596
+
EXPORT_SYMBOL_GPL(kvmppc_gsb_send);
597
+
598
+
/**
599
+
* kvmppc_gsb_recv - request all elements in the buffer have their value
600
+
* updated.
601
+
* @gsb: guest state buffer
602
+
* @flags: guest wide or thread wide
603
+
*
604
+
* Performs the H_GUEST_GET_STATE hcall for the guest state buffer.
605
+
* After returning from the hcall the guest state elements that were
606
+
* present in the buffer will have updated values from the hypervisor.
607
+
*/
608
+
int kvmppc_gsb_recv(struct kvmppc_gs_buff *gsb, unsigned long flags)
609
+
{
610
+
unsigned long hflags = 0;
611
+
unsigned long i;
612
+
int rc;
613
+
614
+
if (flags & KVMPPC_GS_FLAGS_WIDE)
615
+
hflags |= H_GUEST_FLAGS_WIDE;
616
+
617
+
rc = plpar_guest_get_state(hflags, gsb->guest_id, gsb->vcpu_id,
618
+
__pa(gsb->hdr), gsb->capacity, &i);
619
+
return rc;
620
+
}
621
+
EXPORT_SYMBOL_GPL(kvmppc_gsb_recv);
+38
-38
arch/powerpc/kvm/powerpc.c
+38
-38
arch/powerpc/kvm/powerpc.c
···
934
934
return;
935
935
936
936
if (index >= 32) {
937
-
val.vval = VCPU_VSX_VR(vcpu, index - 32);
937
+
kvmppc_get_vsx_vr(vcpu, index - 32, &val.vval);
938
938
val.vsxval[offset] = gpr;
939
-
VCPU_VSX_VR(vcpu, index - 32) = val.vval;
939
+
kvmppc_set_vsx_vr(vcpu, index - 32, &val.vval);
940
940
} else {
941
-
VCPU_VSX_FPR(vcpu, index, offset) = gpr;
941
+
kvmppc_set_vsx_fpr(vcpu, index, offset, gpr);
942
942
}
943
943
}
944
944
···
949
949
int index = vcpu->arch.io_gpr & KVM_MMIO_REG_MASK;
950
950
951
951
if (index >= 32) {
952
-
val.vval = VCPU_VSX_VR(vcpu, index - 32);
952
+
kvmppc_get_vsx_vr(vcpu, index - 32, &val.vval);
953
953
val.vsxval[0] = gpr;
954
954
val.vsxval[1] = gpr;
955
-
VCPU_VSX_VR(vcpu, index - 32) = val.vval;
955
+
kvmppc_set_vsx_vr(vcpu, index - 32, &val.vval);
956
956
} else {
957
-
VCPU_VSX_FPR(vcpu, index, 0) = gpr;
958
-
VCPU_VSX_FPR(vcpu, index, 1) = gpr;
957
+
kvmppc_set_vsx_fpr(vcpu, index, 0, gpr);
958
+
kvmppc_set_vsx_fpr(vcpu, index, 1, gpr);
959
959
}
960
960
}
961
961
···
970
970
val.vsx32val[1] = gpr;
971
971
val.vsx32val[2] = gpr;
972
972
val.vsx32val[3] = gpr;
973
-
VCPU_VSX_VR(vcpu, index - 32) = val.vval;
973
+
kvmppc_set_vsx_vr(vcpu, index - 32, &val.vval);
974
974
} else {
975
975
val.vsx32val[0] = gpr;
976
976
val.vsx32val[1] = gpr;
977
-
VCPU_VSX_FPR(vcpu, index, 0) = val.vsxval[0];
978
-
VCPU_VSX_FPR(vcpu, index, 1) = val.vsxval[0];
977
+
kvmppc_set_vsx_fpr(vcpu, index, 0, val.vsxval[0]);
978
+
kvmppc_set_vsx_fpr(vcpu, index, 1, val.vsxval[0]);
979
979
}
980
980
}
981
981
···
991
991
return;
992
992
993
993
if (index >= 32) {
994
-
val.vval = VCPU_VSX_VR(vcpu, index - 32);
994
+
kvmppc_get_vsx_vr(vcpu, index - 32, &val.vval);
995
995
val.vsx32val[offset] = gpr32;
996
-
VCPU_VSX_VR(vcpu, index - 32) = val.vval;
996
+
kvmppc_set_vsx_vr(vcpu, index - 32, &val.vval);
997
997
} else {
998
998
dword_offset = offset / 2;
999
999
word_offset = offset % 2;
1000
-
val.vsxval[0] = VCPU_VSX_FPR(vcpu, index, dword_offset);
1000
+
val.vsxval[0] = kvmppc_get_vsx_fpr(vcpu, index, dword_offset);
1001
1001
val.vsx32val[word_offset] = gpr32;
1002
-
VCPU_VSX_FPR(vcpu, index, dword_offset) = val.vsxval[0];
1002
+
kvmppc_set_vsx_fpr(vcpu, index, dword_offset, val.vsxval[0]);
1003
1003
}
1004
1004
}
1005
1005
#endif /* CONFIG_VSX */
···
1058
1058
if (offset == -1)
1059
1059
return;
1060
1060
1061
-
val.vval = VCPU_VSX_VR(vcpu, index);
1061
+
kvmppc_get_vsx_vr(vcpu, index, &val.vval);
1062
1062
val.vsxval[offset] = gpr;
1063
-
VCPU_VSX_VR(vcpu, index) = val.vval;
1063
+
kvmppc_set_vsx_vr(vcpu, index, &val.vval);
1064
1064
}
1065
1065
1066
1066
static inline void kvmppc_set_vmx_word(struct kvm_vcpu *vcpu,
···
1074
1074
if (offset == -1)
1075
1075
return;
1076
1076
1077
-
val.vval = VCPU_VSX_VR(vcpu, index);
1077
+
kvmppc_get_vsx_vr(vcpu, index, &val.vval);
1078
1078
val.vsx32val[offset] = gpr32;
1079
-
VCPU_VSX_VR(vcpu, index) = val.vval;
1079
+
kvmppc_set_vsx_vr(vcpu, index, &val.vval);
1080
1080
}
1081
1081
1082
1082
static inline void kvmppc_set_vmx_hword(struct kvm_vcpu *vcpu,
···
1090
1090
if (offset == -1)
1091
1091
return;
1092
1092
1093
-
val.vval = VCPU_VSX_VR(vcpu, index);
1093
+
kvmppc_get_vsx_vr(vcpu, index, &val.vval);
1094
1094
val.vsx16val[offset] = gpr16;
1095
-
VCPU_VSX_VR(vcpu, index) = val.vval;
1095
+
kvmppc_set_vsx_vr(vcpu, index, &val.vval);
1096
1096
}
1097
1097
1098
1098
static inline void kvmppc_set_vmx_byte(struct kvm_vcpu *vcpu,
···
1106
1106
if (offset == -1)
1107
1107
return;
1108
1108
1109
-
val.vval = VCPU_VSX_VR(vcpu, index);
1109
+
kvmppc_get_vsx_vr(vcpu, index, &val.vval);
1110
1110
val.vsx8val[offset] = gpr8;
1111
-
VCPU_VSX_VR(vcpu, index) = val.vval;
1111
+
kvmppc_set_vsx_vr(vcpu, index, &val.vval);
1112
1112
}
1113
1113
#endif /* CONFIG_ALTIVEC */
1114
1114
···
1194
1194
if (vcpu->kvm->arch.kvm_ops->giveup_ext)
1195
1195
vcpu->kvm->arch.kvm_ops->giveup_ext(vcpu, MSR_FP);
1196
1196
1197
-
VCPU_FPR(vcpu, vcpu->arch.io_gpr & KVM_MMIO_REG_MASK) = gpr;
1197
+
kvmppc_set_fpr(vcpu, vcpu->arch.io_gpr & KVM_MMIO_REG_MASK, gpr);
1198
1198
break;
1199
1199
#ifdef CONFIG_PPC_BOOK3S
1200
1200
case KVM_MMIO_REG_QPR:
1201
1201
vcpu->arch.qpr[vcpu->arch.io_gpr & KVM_MMIO_REG_MASK] = gpr;
1202
1202
break;
1203
1203
case KVM_MMIO_REG_FQPR:
1204
-
VCPU_FPR(vcpu, vcpu->arch.io_gpr & KVM_MMIO_REG_MASK) = gpr;
1204
+
kvmppc_set_fpr(vcpu, vcpu->arch.io_gpr & KVM_MMIO_REG_MASK, gpr);
1205
1205
vcpu->arch.qpr[vcpu->arch.io_gpr & KVM_MMIO_REG_MASK] = gpr;
1206
1206
break;
1207
1207
#endif
···
1419
1419
}
1420
1420
1421
1421
if (rs < 32) {
1422
-
*val = VCPU_VSX_FPR(vcpu, rs, vsx_offset);
1422
+
*val = kvmppc_get_vsx_fpr(vcpu, rs, vsx_offset);
1423
1423
} else {
1424
-
reg.vval = VCPU_VSX_VR(vcpu, rs - 32);
1424
+
kvmppc_get_vsx_vr(vcpu, rs - 32, ®.vval);
1425
1425
*val = reg.vsxval[vsx_offset];
1426
1426
}
1427
1427
break;
···
1438
1438
if (rs < 32) {
1439
1439
dword_offset = vsx_offset / 2;
1440
1440
word_offset = vsx_offset % 2;
1441
-
reg.vsxval[0] = VCPU_VSX_FPR(vcpu, rs, dword_offset);
1441
+
reg.vsxval[0] = kvmppc_get_vsx_fpr(vcpu, rs, dword_offset);
1442
1442
*val = reg.vsx32val[word_offset];
1443
1443
} else {
1444
-
reg.vval = VCPU_VSX_VR(vcpu, rs - 32);
1444
+
kvmppc_get_vsx_vr(vcpu, rs - 32, ®.vval);
1445
1445
*val = reg.vsx32val[vsx_offset];
1446
1446
}
1447
1447
break;
···
1556
1556
if (vmx_offset == -1)
1557
1557
return -1;
1558
1558
1559
-
reg.vval = VCPU_VSX_VR(vcpu, index);
1559
+
kvmppc_get_vsx_vr(vcpu, index, ®.vval);
1560
1560
*val = reg.vsxval[vmx_offset];
1561
1561
1562
1562
return result;
···
1574
1574
if (vmx_offset == -1)
1575
1575
return -1;
1576
1576
1577
-
reg.vval = VCPU_VSX_VR(vcpu, index);
1577
+
kvmppc_get_vsx_vr(vcpu, index, ®.vval);
1578
1578
*val = reg.vsx32val[vmx_offset];
1579
1579
1580
1580
return result;
···
1592
1592
if (vmx_offset == -1)
1593
1593
return -1;
1594
1594
1595
-
reg.vval = VCPU_VSX_VR(vcpu, index);
1595
+
kvmppc_get_vsx_vr(vcpu, index, ®.vval);
1596
1596
*val = reg.vsx16val[vmx_offset];
1597
1597
1598
1598
return result;
···
1610
1610
if (vmx_offset == -1)
1611
1611
return -1;
1612
1612
1613
-
reg.vval = VCPU_VSX_VR(vcpu, index);
1613
+
kvmppc_get_vsx_vr(vcpu, index, ®.vval);
1614
1614
*val = reg.vsx8val[vmx_offset];
1615
1615
1616
1616
return result;
···
1719
1719
r = -ENXIO;
1720
1720
break;
1721
1721
}
1722
-
val.vval = vcpu->arch.vr.vr[reg->id - KVM_REG_PPC_VR0];
1722
+
kvmppc_get_vsx_vr(vcpu, reg->id - KVM_REG_PPC_VR0, &val.vval);
1723
1723
break;
1724
1724
case KVM_REG_PPC_VSCR:
1725
1725
if (!cpu_has_feature(CPU_FTR_ALTIVEC)) {
1726
1726
r = -ENXIO;
1727
1727
break;
1728
1728
}
1729
-
val = get_reg_val(reg->id, vcpu->arch.vr.vscr.u[3]);
1729
+
val = get_reg_val(reg->id, kvmppc_get_vscr(vcpu));
1730
1730
break;
1731
1731
case KVM_REG_PPC_VRSAVE:
1732
-
val = get_reg_val(reg->id, vcpu->arch.vrsave);
1732
+
val = get_reg_val(reg->id, kvmppc_get_vrsave(vcpu));
1733
1733
break;
1734
1734
#endif /* CONFIG_ALTIVEC */
1735
1735
default:
···
1770
1770
r = -ENXIO;
1771
1771
break;
1772
1772
}
1773
-
vcpu->arch.vr.vr[reg->id - KVM_REG_PPC_VR0] = val.vval;
1773
+
kvmppc_set_vsx_vr(vcpu, reg->id - KVM_REG_PPC_VR0, &val.vval);
1774
1774
break;
1775
1775
case KVM_REG_PPC_VSCR:
1776
1776
if (!cpu_has_feature(CPU_FTR_ALTIVEC)) {
1777
1777
r = -ENXIO;
1778
1778
break;
1779
1779
}
1780
-
vcpu->arch.vr.vscr.u[3] = set_reg_val(reg->id, val);
1780
+
kvmppc_set_vscr(vcpu, set_reg_val(reg->id, val));
1781
1781
break;
1782
1782
case KVM_REG_PPC_VRSAVE:
1783
1783
if (!cpu_has_feature(CPU_FTR_ALTIVEC)) {
1784
1784
r = -ENXIO;
1785
1785
break;
1786
1786
}
1787
-
vcpu->arch.vrsave = set_reg_val(reg->id, val);
1787
+
kvmppc_set_vrsave(vcpu, set_reg_val(reg->id, val));
1788
1788
break;
1789
1789
#endif /* CONFIG_ALTIVEC */
1790
1790
default:
+328
arch/powerpc/kvm/test-guest-state-buffer.c
+328
arch/powerpc/kvm/test-guest-state-buffer.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-or-later
2
+
3
+
#include <linux/init.h>
4
+
#include <linux/log2.h>
5
+
#include <kunit/test.h>
6
+
7
+
#include <asm/guest-state-buffer.h>
8
+
9
+
static void test_creating_buffer(struct kunit *test)
10
+
{
11
+
struct kvmppc_gs_buff *gsb;
12
+
size_t size = 0x100;
13
+
14
+
gsb = kvmppc_gsb_new(size, 0, 0, GFP_KERNEL);
15
+
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsb);
16
+
17
+
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsb->hdr);
18
+
19
+
KUNIT_EXPECT_EQ(test, gsb->capacity, roundup_pow_of_two(size));
20
+
KUNIT_EXPECT_EQ(test, gsb->len, sizeof(__be32));
21
+
22
+
kvmppc_gsb_free(gsb);
23
+
}
24
+
25
+
static void test_adding_element(struct kunit *test)
26
+
{
27
+
const struct kvmppc_gs_elem *head, *curr;
28
+
union {
29
+
__vector128 v;
30
+
u64 dw[2];
31
+
} u;
32
+
int rem;
33
+
struct kvmppc_gs_buff *gsb;
34
+
size_t size = 0x1000;
35
+
int i, rc;
36
+
u64 data;
37
+
38
+
gsb = kvmppc_gsb_new(size, 0, 0, GFP_KERNEL);
39
+
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsb);
40
+
41
+
/* Single elements, direct use of __kvmppc_gse_put() */
42
+
data = 0xdeadbeef;
43
+
rc = __kvmppc_gse_put(gsb, KVMPPC_GSID_GPR(0), 8, &data);
44
+
KUNIT_EXPECT_GE(test, rc, 0);
45
+
46
+
head = kvmppc_gsb_data(gsb);
47
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_iden(head), KVMPPC_GSID_GPR(0));
48
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_len(head), 8);
49
+
data = 0;
50
+
memcpy(&data, kvmppc_gse_data(head), 8);
51
+
KUNIT_EXPECT_EQ(test, data, 0xdeadbeef);
52
+
53
+
/* Multiple elements, simple wrapper */
54
+
rc = kvmppc_gse_put_u64(gsb, KVMPPC_GSID_GPR(1), 0xcafef00d);
55
+
KUNIT_EXPECT_GE(test, rc, 0);
56
+
57
+
u.dw[0] = 0x1;
58
+
u.dw[1] = 0x2;
59
+
rc = kvmppc_gse_put_vector128(gsb, KVMPPC_GSID_VSRS(0), &u.v);
60
+
KUNIT_EXPECT_GE(test, rc, 0);
61
+
u.dw[0] = 0x0;
62
+
u.dw[1] = 0x0;
63
+
64
+
kvmppc_gsb_for_each_elem(i, curr, gsb, rem) {
65
+
switch (i) {
66
+
case 0:
67
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_iden(curr),
68
+
KVMPPC_GSID_GPR(0));
69
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_len(curr), 8);
70
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_get_be64(curr),
71
+
0xdeadbeef);
72
+
break;
73
+
case 1:
74
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_iden(curr),
75
+
KVMPPC_GSID_GPR(1));
76
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_len(curr), 8);
77
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_get_u64(curr),
78
+
0xcafef00d);
79
+
break;
80
+
case 2:
81
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_iden(curr),
82
+
KVMPPC_GSID_VSRS(0));
83
+
KUNIT_EXPECT_EQ(test, kvmppc_gse_len(curr), 16);
84
+
kvmppc_gse_get_vector128(curr, &u.v);
85
+
KUNIT_EXPECT_EQ(test, u.dw[0], 0x1);
86
+
KUNIT_EXPECT_EQ(test, u.dw[1], 0x2);
87
+
break;
88
+
}
89
+
}
90
+
KUNIT_EXPECT_EQ(test, i, 3);
91
+
92
+
kvmppc_gsb_reset(gsb);
93
+
KUNIT_EXPECT_EQ(test, kvmppc_gsb_nelems(gsb), 0);
94
+
KUNIT_EXPECT_EQ(test, kvmppc_gsb_len(gsb),
95
+
sizeof(struct kvmppc_gs_header));
96
+
97
+
kvmppc_gsb_free(gsb);
98
+
}
99
+
100
+
static void test_gs_parsing(struct kunit *test)
101
+
{
102
+
struct kvmppc_gs_elem *gse;
103
+
struct kvmppc_gs_parser gsp = { 0 };
104
+
struct kvmppc_gs_buff *gsb;
105
+
size_t size = 0x1000;
106
+
u64 tmp1, tmp2;
107
+
108
+
gsb = kvmppc_gsb_new(size, 0, 0, GFP_KERNEL);
109
+
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsb);
110
+
111
+
tmp1 = 0xdeadbeefull;
112
+
kvmppc_gse_put_u64(gsb, KVMPPC_GSID_GPR(0), tmp1);
113
+
114
+
KUNIT_EXPECT_GE(test, kvmppc_gse_parse(&gsp, gsb), 0);
115
+
116
+
gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_GPR(0));
117
+
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gse);
118
+
119
+
tmp2 = kvmppc_gse_get_u64(gse);
120
+
KUNIT_EXPECT_EQ(test, tmp2, 0xdeadbeefull);
121
+
122
+
kvmppc_gsb_free(gsb);
123
+
}
124
+
125
+
static void test_gs_bitmap(struct kunit *test)
126
+
{
127
+
struct kvmppc_gs_bitmap gsbm = { 0 };
128
+
struct kvmppc_gs_bitmap gsbm1 = { 0 };
129
+
struct kvmppc_gs_bitmap gsbm2 = { 0 };
130
+
u16 iden;
131
+
int i, j;
132
+
133
+
i = 0;
134
+
for (u16 iden = KVMPPC_GSID_HOST_STATE_SIZE;
135
+
iden <= KVMPPC_GSID_PROCESS_TABLE; iden++) {
136
+
kvmppc_gsbm_set(&gsbm, iden);
137
+
kvmppc_gsbm_set(&gsbm1, iden);
138
+
KUNIT_EXPECT_TRUE(test, kvmppc_gsbm_test(&gsbm, iden));
139
+
kvmppc_gsbm_clear(&gsbm, iden);
140
+
KUNIT_EXPECT_FALSE(test, kvmppc_gsbm_test(&gsbm, iden));
141
+
i++;
142
+
}
143
+
144
+
for (u16 iden = KVMPPC_GSID_RUN_INPUT; iden <= KVMPPC_GSID_VPA;
145
+
iden++) {
146
+
kvmppc_gsbm_set(&gsbm, iden);
147
+
kvmppc_gsbm_set(&gsbm1, iden);
148
+
KUNIT_EXPECT_TRUE(test, kvmppc_gsbm_test(&gsbm, iden));
149
+
kvmppc_gsbm_clear(&gsbm, iden);
150
+
KUNIT_EXPECT_FALSE(test, kvmppc_gsbm_test(&gsbm, iden));
151
+
i++;
152
+
}
153
+
154
+
for (u16 iden = KVMPPC_GSID_GPR(0); iden <= KVMPPC_GSID_CTRL; iden++) {
155
+
kvmppc_gsbm_set(&gsbm, iden);
156
+
kvmppc_gsbm_set(&gsbm1, iden);
157
+
KUNIT_EXPECT_TRUE(test, kvmppc_gsbm_test(&gsbm, iden));
158
+
kvmppc_gsbm_clear(&gsbm, iden);
159
+
KUNIT_EXPECT_FALSE(test, kvmppc_gsbm_test(&gsbm, iden));
160
+
i++;
161
+
}
162
+
163
+
for (u16 iden = KVMPPC_GSID_CR; iden <= KVMPPC_GSID_PSPB; iden++) {
164
+
kvmppc_gsbm_set(&gsbm, iden);
165
+
kvmppc_gsbm_set(&gsbm1, iden);
166
+
KUNIT_EXPECT_TRUE(test, kvmppc_gsbm_test(&gsbm, iden));
167
+
kvmppc_gsbm_clear(&gsbm, iden);
168
+
KUNIT_EXPECT_FALSE(test, kvmppc_gsbm_test(&gsbm, iden));
169
+
i++;
170
+
}
171
+
172
+
for (u16 iden = KVMPPC_GSID_VSRS(0); iden <= KVMPPC_GSID_VSRS(63);
173
+
iden++) {
174
+
kvmppc_gsbm_set(&gsbm, iden);
175
+
kvmppc_gsbm_set(&gsbm1, iden);
176
+
KUNIT_EXPECT_TRUE(test, kvmppc_gsbm_test(&gsbm, iden));
177
+
kvmppc_gsbm_clear(&gsbm, iden);
178
+
KUNIT_EXPECT_FALSE(test, kvmppc_gsbm_test(&gsbm, iden));
179
+
i++;
180
+
}
181
+
182
+
for (u16 iden = KVMPPC_GSID_HDAR; iden <= KVMPPC_GSID_ASDR; iden++) {
183
+
kvmppc_gsbm_set(&gsbm, iden);
184
+
kvmppc_gsbm_set(&gsbm1, iden);
185
+
KUNIT_EXPECT_TRUE(test, kvmppc_gsbm_test(&gsbm, iden));
186
+
kvmppc_gsbm_clear(&gsbm, iden);
187
+
KUNIT_EXPECT_FALSE(test, kvmppc_gsbm_test(&gsbm, iden));
188
+
i++;
189
+
}
190
+
191
+
j = 0;
192
+
kvmppc_gsbm_for_each(&gsbm1, iden)
193
+
{
194
+
kvmppc_gsbm_set(&gsbm2, iden);
195
+
j++;
196
+
}
197
+
KUNIT_EXPECT_EQ(test, i, j);
198
+
KUNIT_EXPECT_MEMEQ(test, &gsbm1, &gsbm2, sizeof(gsbm1));
199
+
}
200
+
201
+
struct kvmppc_gs_msg_test1_data {
202
+
u64 a;
203
+
u32 b;
204
+
struct kvmppc_gs_part_table c;
205
+
struct kvmppc_gs_proc_table d;
206
+
struct kvmppc_gs_buff_info e;
207
+
};
208
+
209
+
static size_t test1_get_size(struct kvmppc_gs_msg *gsm)
210
+
{
211
+
size_t size = 0;
212
+
u16 ids[] = {
213
+
KVMPPC_GSID_PARTITION_TABLE,
214
+
KVMPPC_GSID_PROCESS_TABLE,
215
+
KVMPPC_GSID_RUN_INPUT,
216
+
KVMPPC_GSID_GPR(0),
217
+
KVMPPC_GSID_CR,
218
+
};
219
+
220
+
for (int i = 0; i < ARRAY_SIZE(ids); i++)
221
+
size += kvmppc_gse_total_size(kvmppc_gsid_size(ids[i]));
222
+
return size;
223
+
}
224
+
225
+
static int test1_fill_info(struct kvmppc_gs_buff *gsb,
226
+
struct kvmppc_gs_msg *gsm)
227
+
{
228
+
struct kvmppc_gs_msg_test1_data *data = gsm->data;
229
+
230
+
if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_GPR(0)))
231
+
kvmppc_gse_put_u64(gsb, KVMPPC_GSID_GPR(0), data->a);
232
+
233
+
if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_CR))
234
+
kvmppc_gse_put_u32(gsb, KVMPPC_GSID_CR, data->b);
235
+
236
+
if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_PARTITION_TABLE))
237
+
kvmppc_gse_put_part_table(gsb, KVMPPC_GSID_PARTITION_TABLE,
238
+
data->c);
239
+
240
+
if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_PROCESS_TABLE))
241
+
kvmppc_gse_put_proc_table(gsb, KVMPPC_GSID_PARTITION_TABLE,
242
+
data->d);
243
+
244
+
if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_RUN_INPUT))
245
+
kvmppc_gse_put_buff_info(gsb, KVMPPC_GSID_RUN_INPUT, data->e);
246
+
247
+
return 0;
248
+
}
249
+
250
+
static int test1_refresh_info(struct kvmppc_gs_msg *gsm,
251
+
struct kvmppc_gs_buff *gsb)
252
+
{
253
+
struct kvmppc_gs_parser gsp = { 0 };
254
+
struct kvmppc_gs_msg_test1_data *data = gsm->data;
255
+
struct kvmppc_gs_elem *gse;
256
+
int rc;
257
+
258
+
rc = kvmppc_gse_parse(&gsp, gsb);
259
+
if (rc < 0)
260
+
return rc;
261
+
262
+
gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_GPR(0));
263
+
if (gse)
264
+
data->a = kvmppc_gse_get_u64(gse);
265
+
266
+
gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_CR);
267
+
if (gse)
268
+
data->b = kvmppc_gse_get_u32(gse);
269
+
270
+
return 0;
271
+
}
272
+
273
+
static struct kvmppc_gs_msg_ops gs_msg_test1_ops = {
274
+
.get_size = test1_get_size,
275
+
.fill_info = test1_fill_info,
276
+
.refresh_info = test1_refresh_info,
277
+
};
278
+
279
+
static void test_gs_msg(struct kunit *test)
280
+
{
281
+
struct kvmppc_gs_msg_test1_data test1_data = {
282
+
.a = 0xdeadbeef,
283
+
.b = 0x1,
284
+
};
285
+
struct kvmppc_gs_msg *gsm;
286
+
struct kvmppc_gs_buff *gsb;
287
+
288
+
gsm = kvmppc_gsm_new(&gs_msg_test1_ops, &test1_data, GSM_SEND,
289
+
GFP_KERNEL);
290
+
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsm);
291
+
292
+
gsb = kvmppc_gsb_new(kvmppc_gsm_size(gsm), 0, 0, GFP_KERNEL);
293
+
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsb);
294
+
295
+
kvmppc_gsm_include(gsm, KVMPPC_GSID_PARTITION_TABLE);
296
+
kvmppc_gsm_include(gsm, KVMPPC_GSID_PROCESS_TABLE);
297
+
kvmppc_gsm_include(gsm, KVMPPC_GSID_RUN_INPUT);
298
+
kvmppc_gsm_include(gsm, KVMPPC_GSID_GPR(0));
299
+
kvmppc_gsm_include(gsm, KVMPPC_GSID_CR);
300
+
301
+
kvmppc_gsm_fill_info(gsm, gsb);
302
+
303
+
memset(&test1_data, 0, sizeof(test1_data));
304
+
305
+
kvmppc_gsm_refresh_info(gsm, gsb);
306
+
KUNIT_EXPECT_EQ(test, test1_data.a, 0xdeadbeef);
307
+
KUNIT_EXPECT_EQ(test, test1_data.b, 0x1);
308
+
309
+
kvmppc_gsm_free(gsm);
310
+
}
311
+
312
+
static struct kunit_case guest_state_buffer_testcases[] = {
313
+
KUNIT_CASE(test_creating_buffer),
314
+
KUNIT_CASE(test_adding_element),
315
+
KUNIT_CASE(test_gs_bitmap),
316
+
KUNIT_CASE(test_gs_parsing),
317
+
KUNIT_CASE(test_gs_msg),
318
+
{}
319
+
};
320
+
321
+
static struct kunit_suite guest_state_buffer_test_suite = {
322
+
.name = "guest_state_buffer_test",
323
+
.test_cases = guest_state_buffer_testcases,
324
+
};
325
+
326
+
kunit_test_suites(&guest_state_buffer_test_suite);
327
+
328
+
MODULE_LICENSE("GPL");
+139
-7
arch/powerpc/lib/code-patching.c
+139
-7
arch/powerpc/lib/code-patching.c
···
38
38
return 0;
39
39
40
40
failed:
41
+
mb(); /* sync */
41
42
return -EPERM;
42
43
}
43
44
···
205
204
{
206
205
int ret;
207
206
208
-
if (!IS_ENABLED(CONFIG_STRICT_KERNEL_RWX))
209
-
return;
210
-
211
207
if (mm_patch_enabled())
212
208
ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
213
209
"powerpc/text_poke_mm:online",
···
307
309
308
310
err = __patch_instruction(addr, instr, patch_addr);
309
311
310
-
/* hwsync performed by __patch_instruction (sync) if successful */
311
-
if (err)
312
-
mb(); /* sync */
313
-
314
312
/* context synchronisation performed by __patch_instruction (isync or exception) */
315
313
stop_using_temp_mm(patching_mm, orig_mm);
316
314
···
371
377
return err;
372
378
}
373
379
NOKPROBE_SYMBOL(patch_instruction);
380
+
381
+
static int __patch_instructions(u32 *patch_addr, u32 *code, size_t len, bool repeat_instr)
382
+
{
383
+
unsigned long start = (unsigned long)patch_addr;
384
+
385
+
/* Repeat instruction */
386
+
if (repeat_instr) {
387
+
ppc_inst_t instr = ppc_inst_read(code);
388
+
389
+
if (ppc_inst_prefixed(instr)) {
390
+
u64 val = ppc_inst_as_ulong(instr);
391
+
392
+
memset64((u64 *)patch_addr, val, len / 8);
393
+
} else {
394
+
u32 val = ppc_inst_val(instr);
395
+
396
+
memset32(patch_addr, val, len / 4);
397
+
}
398
+
} else {
399
+
memcpy(patch_addr, code, len);
400
+
}
401
+
402
+
smp_wmb(); /* smp write barrier */
403
+
flush_icache_range(start, start + len);
404
+
return 0;
405
+
}
406
+
407
+
/*
408
+
* A page is mapped and instructions that fit the page are patched.
409
+
* Assumes 'len' to be (PAGE_SIZE - offset_in_page(addr)) or below.
410
+
*/
411
+
static int __do_patch_instructions_mm(u32 *addr, u32 *code, size_t len, bool repeat_instr)
412
+
{
413
+
struct mm_struct *patching_mm, *orig_mm;
414
+
unsigned long pfn = get_patch_pfn(addr);
415
+
unsigned long text_poke_addr;
416
+
spinlock_t *ptl;
417
+
u32 *patch_addr;
418
+
pte_t *pte;
419
+
int err;
420
+
421
+
patching_mm = __this_cpu_read(cpu_patching_context.mm);
422
+
text_poke_addr = __this_cpu_read(cpu_patching_context.addr);
423
+
patch_addr = (u32 *)(text_poke_addr + offset_in_page(addr));
424
+
425
+
pte = get_locked_pte(patching_mm, text_poke_addr, &ptl);
426
+
if (!pte)
427
+
return -ENOMEM;
428
+
429
+
__set_pte_at(patching_mm, text_poke_addr, pte, pfn_pte(pfn, PAGE_KERNEL), 0);
430
+
431
+
/* order PTE update before use, also serves as the hwsync */
432
+
asm volatile("ptesync" ::: "memory");
433
+
434
+
/* order context switch after arbitrary prior code */
435
+
isync();
436
+
437
+
orig_mm = start_using_temp_mm(patching_mm);
438
+
439
+
err = __patch_instructions(patch_addr, code, len, repeat_instr);
440
+
441
+
/* context synchronisation performed by __patch_instructions */
442
+
stop_using_temp_mm(patching_mm, orig_mm);
443
+
444
+
pte_clear(patching_mm, text_poke_addr, pte);
445
+
/*
446
+
* ptesync to order PTE update before TLB invalidation done
447
+
* by radix__local_flush_tlb_page_psize (in _tlbiel_va)
448
+
*/
449
+
local_flush_tlb_page_psize(patching_mm, text_poke_addr, mmu_virtual_psize);
450
+
451
+
pte_unmap_unlock(pte, ptl);
452
+
453
+
return err;
454
+
}
455
+
456
+
/*
457
+
* A page is mapped and instructions that fit the page are patched.
458
+
* Assumes 'len' to be (PAGE_SIZE - offset_in_page(addr)) or below.
459
+
*/
460
+
static int __do_patch_instructions(u32 *addr, u32 *code, size_t len, bool repeat_instr)
461
+
{
462
+
unsigned long pfn = get_patch_pfn(addr);
463
+
unsigned long text_poke_addr;
464
+
u32 *patch_addr;
465
+
pte_t *pte;
466
+
int err;
467
+
468
+
text_poke_addr = (unsigned long)__this_cpu_read(cpu_patching_context.addr) & PAGE_MASK;
469
+
patch_addr = (u32 *)(text_poke_addr + offset_in_page(addr));
470
+
471
+
pte = __this_cpu_read(cpu_patching_context.pte);
472
+
__set_pte_at(&init_mm, text_poke_addr, pte, pfn_pte(pfn, PAGE_KERNEL), 0);
473
+
/* See ptesync comment in radix__set_pte_at() */
474
+
if (radix_enabled())
475
+
asm volatile("ptesync" ::: "memory");
476
+
477
+
err = __patch_instructions(patch_addr, code, len, repeat_instr);
478
+
479
+
pte_clear(&init_mm, text_poke_addr, pte);
480
+
flush_tlb_kernel_range(text_poke_addr, text_poke_addr + PAGE_SIZE);
481
+
482
+
return err;
483
+
}
484
+
485
+
/*
486
+
* Patch 'addr' with 'len' bytes of instructions from 'code'.
487
+
*
488
+
* If repeat_instr is true, the same instruction is filled for
489
+
* 'len' bytes.
490
+
*/
491
+
int patch_instructions(u32 *addr, u32 *code, size_t len, bool repeat_instr)
492
+
{
493
+
while (len > 0) {
494
+
unsigned long flags;
495
+
size_t plen;
496
+
int err;
497
+
498
+
plen = min_t(size_t, PAGE_SIZE - offset_in_page(addr), len);
499
+
500
+
local_irq_save(flags);
501
+
if (mm_patch_enabled())
502
+
err = __do_patch_instructions_mm(addr, code, plen, repeat_instr);
503
+
else
504
+
err = __do_patch_instructions(addr, code, plen, repeat_instr);
505
+
local_irq_restore(flags);
506
+
if (err)
507
+
return err;
508
+
509
+
len -= plen;
510
+
addr = (u32 *)((unsigned long)addr + plen);
511
+
if (!repeat_instr)
512
+
code = (u32 *)((unsigned long)code + plen);
513
+
}
514
+
515
+
return 0;
516
+
}
517
+
NOKPROBE_SYMBOL(patch_instructions);
374
518
375
519
int patch_branch(u32 *addr, unsigned long target, int flags)
376
520
{
+54
-72
arch/powerpc/lib/qspinlock.c
+54
-72
arch/powerpc/lib/qspinlock.c
···
16
16
struct qnode *next;
17
17
struct qspinlock *lock;
18
18
int cpu;
19
-
int yield_cpu;
19
+
u8 sleepy; /* 1 if the previous vCPU was preempted or
20
+
* if the previous node was sleepy */
20
21
u8 locked; /* 1 if lock acquired */
21
22
};
22
23
···
44
43
static u64 pv_sleepy_lock_interval_ns __read_mostly = 0;
45
44
static int pv_sleepy_lock_factor __read_mostly = 256;
46
45
static bool pv_yield_prev __read_mostly = true;
47
-
static bool pv_yield_propagate_owner __read_mostly = true;
46
+
static bool pv_yield_sleepy_owner __read_mostly = true;
48
47
static bool pv_prod_head __read_mostly = false;
49
48
50
49
static DEFINE_PER_CPU_ALIGNED(struct qnodes, qnodes);
···
248
247
this_cpu_write(sleepy_lock_seen_clock, sched_clock());
249
248
}
250
249
251
-
static __always_inline void seen_sleepy_node(struct qspinlock *lock, u32 val)
250
+
static __always_inline void seen_sleepy_node(void)
252
251
{
253
252
if (pv_sleepy_lock) {
254
253
if (pv_sleepy_lock_interval_ns)
255
254
this_cpu_write(sleepy_lock_seen_clock, sched_clock());
256
-
if (val & _Q_LOCKED_VAL) {
257
-
if (!(val & _Q_SLEEPY_VAL))
258
-
try_set_sleepy(lock, val);
259
-
}
255
+
/* Don't set sleepy because we likely have a stale val */
260
256
}
261
257
}
262
258
263
-
static struct qnode *get_tail_qnode(struct qspinlock *lock, u32 val)
259
+
static struct qnode *get_tail_qnode(struct qspinlock *lock, int prev_cpu)
264
260
{
265
-
int cpu = decode_tail_cpu(val);
266
-
struct qnodes *qnodesp = per_cpu_ptr(&qnodes, cpu);
261
+
struct qnodes *qnodesp = per_cpu_ptr(&qnodes, prev_cpu);
267
262
int idx;
268
263
269
264
/*
···
350
353
return __yield_to_locked_owner(lock, val, paravirt, mustq);
351
354
}
352
355
353
-
static __always_inline void propagate_yield_cpu(struct qnode *node, u32 val, int *set_yield_cpu, bool paravirt)
356
+
static __always_inline void propagate_sleepy(struct qnode *node, u32 val, bool paravirt)
354
357
{
355
358
struct qnode *next;
356
359
int owner;
357
360
358
361
if (!paravirt)
359
362
return;
360
-
if (!pv_yield_propagate_owner)
361
-
return;
362
-
363
-
owner = get_owner_cpu(val);
364
-
if (*set_yield_cpu == owner)
363
+
if (!pv_yield_sleepy_owner)
365
364
return;
366
365
367
366
next = READ_ONCE(node->next);
368
367
if (!next)
369
368
return;
370
369
371
-
if (vcpu_is_preempted(owner)) {
372
-
next->yield_cpu = owner;
373
-
*set_yield_cpu = owner;
374
-
} else if (*set_yield_cpu != -1) {
375
-
next->yield_cpu = owner;
376
-
*set_yield_cpu = owner;
377
-
}
370
+
if (next->sleepy)
371
+
return;
372
+
373
+
owner = get_owner_cpu(val);
374
+
if (vcpu_is_preempted(owner))
375
+
next->sleepy = 1;
378
376
}
379
377
380
378
/* Called inside spin_begin() */
381
-
static __always_inline bool yield_to_prev(struct qspinlock *lock, struct qnode *node, u32 val, bool paravirt)
379
+
static __always_inline bool yield_to_prev(struct qspinlock *lock, struct qnode *node, int prev_cpu, bool paravirt)
382
380
{
383
-
int prev_cpu = decode_tail_cpu(val);
384
381
u32 yield_count;
385
-
int yield_cpu;
386
382
bool preempted = false;
387
383
388
384
if (!paravirt)
389
385
goto relax;
390
386
391
-
if (!pv_yield_propagate_owner)
387
+
if (!pv_yield_sleepy_owner)
392
388
goto yield_prev;
393
389
394
-
yield_cpu = READ_ONCE(node->yield_cpu);
395
-
if (yield_cpu == -1) {
396
-
/* Propagate back the -1 CPU */
397
-
if (node->next && node->next->yield_cpu != -1)
398
-
node->next->yield_cpu = yield_cpu;
399
-
goto yield_prev;
390
+
/*
391
+
* If the previous waiter was preempted it might not be able to
392
+
* propagate sleepy to us, so check the lock in that case too.
393
+
*/
394
+
if (node->sleepy || vcpu_is_preempted(prev_cpu)) {
395
+
u32 val = READ_ONCE(lock->val);
396
+
397
+
if (val & _Q_LOCKED_VAL) {
398
+
if (node->next && !node->next->sleepy) {
399
+
/*
400
+
* Propagate sleepy to next waiter. Only if
401
+
* owner is preempted, which allows the queue
402
+
* to become "non-sleepy" if vCPU preemption
403
+
* ceases to occur, even if the lock remains
404
+
* highly contended.
405
+
*/
406
+
if (vcpu_is_preempted(get_owner_cpu(val)))
407
+
node->next->sleepy = 1;
408
+
}
409
+
410
+
preempted = yield_to_locked_owner(lock, val, paravirt);
411
+
if (preempted)
412
+
return preempted;
413
+
}
414
+
node->sleepy = false;
400
415
}
401
-
402
-
yield_count = yield_count_of(yield_cpu);
403
-
if ((yield_count & 1) == 0)
404
-
goto yield_prev; /* owner vcpu is running */
405
-
406
-
if (get_owner_cpu(READ_ONCE(lock->val)) != yield_cpu)
407
-
goto yield_prev; /* re-sample lock owner */
408
-
409
-
spin_end();
410
-
411
-
preempted = true;
412
-
seen_sleepy_node(lock, val);
413
-
414
-
smp_rmb();
415
-
416
-
if (yield_cpu == node->yield_cpu) {
417
-
if (node->next && node->next->yield_cpu != yield_cpu)
418
-
node->next->yield_cpu = yield_cpu;
419
-
yield_to_preempted(yield_cpu, yield_count);
420
-
spin_begin();
421
-
return preempted;
422
-
}
423
-
spin_begin();
424
416
425
417
yield_prev:
426
418
if (!pv_yield_prev)
···
422
436
spin_end();
423
437
424
438
preempted = true;
425
-
seen_sleepy_node(lock, val);
439
+
seen_sleepy_node();
426
440
427
441
smp_rmb(); /* See __yield_to_locked_owner comment */
428
442
···
532
546
bool sleepy = false;
533
547
bool mustq = false;
534
548
int idx;
535
-
int set_yield_cpu = -1;
536
549
int iters = 0;
537
550
538
551
BUILD_BUG_ON(CONFIG_NR_CPUS >= (1U << _Q_TAIL_CPU_BITS));
···
555
570
node->next = NULL;
556
571
node->lock = lock;
557
572
node->cpu = smp_processor_id();
558
-
node->yield_cpu = -1;
573
+
node->sleepy = 0;
559
574
node->locked = 0;
560
575
561
576
tail = encode_tail_cpu(node->cpu);
···
572
587
* head of the waitqueue.
573
588
*/
574
589
if (old & _Q_TAIL_CPU_MASK) {
575
-
struct qnode *prev = get_tail_qnode(lock, old);
590
+
int prev_cpu = decode_tail_cpu(old);
591
+
struct qnode *prev = get_tail_qnode(lock, prev_cpu);
576
592
577
593
/* Link @node into the waitqueue. */
578
594
WRITE_ONCE(prev->next, node);
···
583
597
while (!READ_ONCE(node->locked)) {
584
598
spec_barrier();
585
599
586
-
if (yield_to_prev(lock, node, old, paravirt))
600
+
if (yield_to_prev(lock, node, prev_cpu, paravirt))
587
601
seen_preempted = true;
588
602
}
589
603
spec_barrier();
590
604
spin_end();
591
-
592
-
/* Clear out stale propagated yield_cpu */
593
-
if (paravirt && pv_yield_propagate_owner && node->yield_cpu != -1)
594
-
node->yield_cpu = -1;
595
605
596
606
smp_rmb(); /* acquire barrier for the mcs lock */
597
607
···
630
648
}
631
649
}
632
650
633
-
propagate_yield_cpu(node, val, &set_yield_cpu, paravirt);
651
+
propagate_sleepy(node, val, paravirt);
634
652
preempted = yield_head_to_locked_owner(lock, val, paravirt);
635
653
if (!maybe_stealers)
636
654
continue;
···
934
952
935
953
DEFINE_SIMPLE_ATTRIBUTE(fops_pv_yield_prev, pv_yield_prev_get, pv_yield_prev_set, "%llu\n");
936
954
937
-
static int pv_yield_propagate_owner_set(void *data, u64 val)
955
+
static int pv_yield_sleepy_owner_set(void *data, u64 val)
938
956
{
939
-
pv_yield_propagate_owner = !!val;
957
+
pv_yield_sleepy_owner = !!val;
940
958
941
959
return 0;
942
960
}
943
961
944
-
static int pv_yield_propagate_owner_get(void *data, u64 *val)
962
+
static int pv_yield_sleepy_owner_get(void *data, u64 *val)
945
963
{
946
-
*val = pv_yield_propagate_owner;
964
+
*val = pv_yield_sleepy_owner;
947
965
948
966
return 0;
949
967
}
950
968
951
-
DEFINE_SIMPLE_ATTRIBUTE(fops_pv_yield_propagate_owner, pv_yield_propagate_owner_get, pv_yield_propagate_owner_set, "%llu\n");
969
+
DEFINE_SIMPLE_ATTRIBUTE(fops_pv_yield_sleepy_owner, pv_yield_sleepy_owner_get, pv_yield_sleepy_owner_set, "%llu\n");
952
970
953
971
static int pv_prod_head_set(void *data, u64 val)
954
972
{
···
980
998
debugfs_create_file("qspl_pv_sleepy_lock_interval_ns", 0600, arch_debugfs_dir, NULL, &fops_pv_sleepy_lock_interval_ns);
981
999
debugfs_create_file("qspl_pv_sleepy_lock_factor", 0600, arch_debugfs_dir, NULL, &fops_pv_sleepy_lock_factor);
982
1000
debugfs_create_file("qspl_pv_yield_prev", 0600, arch_debugfs_dir, NULL, &fops_pv_yield_prev);
983
-
debugfs_create_file("qspl_pv_yield_propagate_owner", 0600, arch_debugfs_dir, NULL, &fops_pv_yield_propagate_owner);
1001
+
debugfs_create_file("qspl_pv_yield_sleepy_owner", 0600, arch_debugfs_dir, NULL, &fops_pv_yield_sleepy_owner);
984
1002
debugfs_create_file("qspl_pv_prod_head", 0600, arch_debugfs_dir, NULL, &fops_pv_prod_head);
985
1003
}
986
1004
+20
-12
arch/powerpc/mm/book3s32/hash_low.S
+20
-12
arch/powerpc/mm/book3s32/hash_low.S
···
36
36
37
37
/*
38
38
* Load a PTE into the hash table, if possible.
39
-
* The address is in r4, and r3 contains an access flag:
40
-
* _PAGE_RW (0x400) if a write.
39
+
* The address is in r4, and r3 contains required access flags:
40
+
* - For ISI: _PAGE_PRESENT | _PAGE_EXEC
41
+
* - For DSI: _PAGE_PRESENT | _PAGE_READ | _PAGE_WRITE if a write.
41
42
* r9 contains the SRR1 value, from which we use the MSR_PR bit.
42
43
* SPRG_THREAD contains the physical address of the current task's thread.
43
44
*
···
68
67
lis r0, TASK_SIZE@h /* check if kernel address */
69
68
cmplw 0,r4,r0
70
69
mfspr r8,SPRN_SPRG_THREAD /* current task's THREAD (phys) */
71
-
ori r3,r3,_PAGE_USER|_PAGE_PRESENT /* test low addresses as user */
72
70
lwz r5,PGDIR(r8) /* virt page-table root */
73
71
blt+ 112f /* assume user more likely */
74
72
lis r5,swapper_pg_dir@ha /* if kernel address, use */
73
+
andi. r0,r9,MSR_PR /* Check usermode */
75
74
addi r5,r5,swapper_pg_dir@l /* kernel page table */
76
-
rlwimi r3,r9,32-12,29,29 /* MSR_PR -> _PAGE_USER */
75
+
#ifdef CONFIG_SMP
76
+
bne- .Lhash_page_out /* return if usermode */
77
+
#else
78
+
bnelr-
79
+
#endif
77
80
112: tophys(r5, r5)
78
81
#ifndef CONFIG_PTE_64BIT
79
82
rlwimi r5,r4,12,20,29 /* insert top 10 bits of address */
···
118
113
lwarx r6,0,r8 /* get linux-style pte, flag word */
119
114
#ifdef CONFIG_PPC_KUAP
120
115
mfsrin r5,r4
121
-
rlwinm r0,r9,28,_PAGE_RW /* MSR[PR] => _PAGE_RW */
122
-
rlwinm r5,r5,12,_PAGE_RW /* Ks => _PAGE_RW */
116
+
rlwinm r0,r9,28,_PAGE_WRITE /* MSR[PR] => _PAGE_WRITE */
117
+
rlwinm r5,r5,12,_PAGE_WRITE /* Ks => _PAGE_WRITE */
123
118
andc r5,r5,r0 /* Ks & ~MSR[PR] */
124
-
andc r5,r6,r5 /* Clear _PAGE_RW when Ks = 1 && MSR[PR] = 0 */
119
+
andc r5,r6,r5 /* Clear _PAGE_WRITE when Ks = 1 && MSR[PR] = 0 */
125
120
andc. r5,r3,r5 /* check access & ~permission */
126
121
#else
127
122
andc. r5,r3,r6 /* check access & ~permission */
128
123
#endif
129
-
rlwinm r0,r3,32-3,24,24 /* _PAGE_RW access -> _PAGE_DIRTY */
124
+
rlwinm r0,r3,32-3,24,24 /* _PAGE_WRITE access -> _PAGE_DIRTY */
130
125
ori r0,r0,_PAGE_ACCESSED|_PAGE_HASHPTE
131
126
#ifdef CONFIG_SMP
132
127
bne- .Lhash_page_out /* return if access not permitted */
···
312
307
__REF
313
308
_GLOBAL(create_hpte)
314
309
/* Convert linux-style PTE (r5) to low word of PPC-style PTE (r8) */
315
-
rlwinm r8,r5,32-9,30,30 /* _PAGE_RW -> PP msb */
310
+
lis r0, TASK_SIZE@h
311
+
rlwinm r5,r5,0,~3 /* Clear PP bits */
312
+
cmplw r4,r0
313
+
rlwinm r8,r5,32-9,30,30 /* _PAGE_WRITE -> PP msb */
316
314
rlwinm r0,r5,32-6,30,30 /* _PAGE_DIRTY -> PP msb */
317
315
and r8,r8,r0 /* writable if _RW & _DIRTY */
318
-
rlwimi r5,r5,32-1,30,30 /* _PAGE_USER -> PP msb */
319
-
rlwimi r5,r5,32-2,31,31 /* _PAGE_USER -> PP lsb */
320
-
ori r8,r8,0xe04 /* clear out reserved bits */
316
+
bge- 1f /* Kernelspace ? Skip */
317
+
ori r5,r5,3 /* Userspace ? PP = 3 */
318
+
1: ori r8,r8,0xe04 /* clear out reserved bits */
321
319
andc r8,r5,r8 /* PP = user? (rw&dirty? 1: 3): 0 */
322
320
BEGIN_FTR_SECTION
323
321
rlwinm r8,r8,0,~_PAGE_COHERENT /* clear M (coherence not required) */
+3
-3
arch/powerpc/mm/book3s32/mmu.c
+3
-3
arch/powerpc/mm/book3s32/mmu.c
···
127
127
wimgxpp = (flags & _PAGE_COHERENT) | (_PAGE_EXEC ? BPP_RX : BPP_XX);
128
128
bat[0].batu = virt | (bl << 2) | 2; /* Vs=1, Vp=0 */
129
129
bat[0].batl = BAT_PHYS_ADDR(phys) | wimgxpp;
130
-
if (flags & _PAGE_USER)
130
+
if (!is_kernel_addr(virt))
131
131
bat[0].batu |= 1; /* Vp = 1 */
132
132
}
133
133
···
277
277
/* Do DBAT first */
278
278
wimgxpp = flags & (_PAGE_WRITETHRU | _PAGE_NO_CACHE
279
279
| _PAGE_COHERENT | _PAGE_GUARDED);
280
-
wimgxpp |= (flags & _PAGE_RW)? BPP_RW: BPP_RX;
280
+
wimgxpp |= (flags & _PAGE_WRITE) ? BPP_RW : BPP_RX;
281
281
bat[1].batu = virt | (bl << 2) | 2; /* Vs=1, Vp=0 */
282
282
bat[1].batl = BAT_PHYS_ADDR(phys) | wimgxpp;
283
-
if (flags & _PAGE_USER)
283
+
if (!is_kernel_addr(virt))
284
284
bat[1].batu |= 1; /* Vp = 1 */
285
285
if (flags & _PAGE_GUARDED) {
286
286
/* G bit must be zero in IBATs */
+4
-6
arch/powerpc/mm/book3s64/pgtable.c
+4
-6
arch/powerpc/mm/book3s64/pgtable.c
···
635
635
unsigned long prot;
636
636
637
637
/* Radix supports execute-only, but protection_map maps X -> RX */
638
-
if (radix_enabled() && ((vm_flags & VM_ACCESS_FLAGS) == VM_EXEC)) {
639
-
prot = pgprot_val(PAGE_EXECONLY);
640
-
} else {
641
-
prot = pgprot_val(protection_map[vm_flags &
642
-
(VM_ACCESS_FLAGS | VM_SHARED)]);
643
-
}
638
+
if (!radix_enabled() && ((vm_flags & VM_ACCESS_FLAGS) == VM_EXEC))
639
+
vm_flags |= VM_READ;
640
+
641
+
prot = pgprot_val(protection_map[vm_flags & (VM_ACCESS_FLAGS | VM_SHARED)]);
644
642
645
643
if (vm_flags & VM_SAO)
646
644
prot |= _PAGE_SAO;
+1
-1
arch/powerpc/mm/drmem.c
+1
-1
arch/powerpc/mm/drmem.c
+5
-4
arch/powerpc/mm/fault.c
+5
-4
arch/powerpc/mm/fault.c
···
266
266
}
267
267
268
268
/*
269
-
* VM_READ, VM_WRITE and VM_EXEC all imply read permissions, as
270
-
* defined in protection_map[]. Read faults can only be caused by
271
-
* a PROT_NONE mapping, or with a PROT_EXEC-only mapping on Radix.
269
+
* VM_READ, VM_WRITE and VM_EXEC may imply read permissions, as
270
+
* defined in protection_map[]. In that case Read faults can only be
271
+
* caused by a PROT_NONE mapping. However a non exec access on a
272
+
* VM_EXEC only mapping is invalid anyway, so report it as such.
272
273
*/
273
274
if (unlikely(!vma_is_accessible(vma)))
274
275
return true;
275
276
276
-
if (unlikely(radix_enabled() && ((vma->vm_flags & VM_ACCESS_FLAGS) == VM_EXEC)))
277
+
if ((vma->vm_flags & VM_ACCESS_FLAGS) == VM_EXEC)
277
278
return true;
278
279
279
280
/*
+1
arch/powerpc/mm/init_32.c
+1
arch/powerpc/mm/init_32.c
+1
-5
arch/powerpc/mm/ioremap.c
+1
-5
arch/powerpc/mm/ioremap.c
···
50
50
if (pte_write(pte))
51
51
pte = pte_mkdirty(pte);
52
52
53
-
/* we don't want to let _PAGE_USER and _PAGE_EXEC leak out */
54
-
pte = pte_exprotect(pte);
55
-
pte = pte_mkprivileged(pte);
56
-
57
53
if (iowa_is_active())
58
54
return iowa_ioremap(addr, size, pte_pgprot(pte), caller);
59
55
return __ioremap_caller(addr, size, pte_pgprot(pte), caller);
···
62
66
unsigned long i;
63
67
64
68
for (i = 0; i < size; i += PAGE_SIZE) {
65
-
int err = map_kernel_page(ea + i, pa + i, prot);
69
+
int err = map_kernel_page(ea + i, pa + i, pgprot_nx(prot));
66
70
67
71
if (WARN_ON_ONCE(err)) /* Should clean up */
68
72
return err;
+1
arch/powerpc/mm/mem.c
+1
arch/powerpc/mm/mem.c
+12
-7
arch/powerpc/mm/nohash/40x.c
+12
-7
arch/powerpc/mm/nohash/40x.c
···
48
48
*/
49
49
void __init MMU_init_hw(void)
50
50
{
51
+
int i;
52
+
unsigned long zpr;
53
+
51
54
/*
52
55
* The Zone Protection Register (ZPR) defines how protection will
53
-
* be applied to every page which is a member of a given zone. At
54
-
* present, we utilize only two of the 4xx's zones.
56
+
* be applied to every page which is a member of a given zone.
55
57
* The zone index bits (of ZSEL) in the PTE are used for software
56
-
* indicators, except the LSB. For user access, zone 1 is used,
57
-
* for kernel access, zone 0 is used. We set all but zone 1
58
-
* to zero, allowing only kernel access as indicated in the PTE.
59
-
* For zone 1, we set a 01 binary (a value of 10 will not work)
58
+
* indicators. We use the 4 upper bits of virtual address to select
59
+
* the zone. We set all zones above TASK_SIZE to zero, allowing
60
+
* only kernel access as indicated in the PTE. For zones below
61
+
* TASK_SIZE, we set a 01 binary (a value of 10 will not work)
60
62
* to allow user access as indicated in the PTE. This also allows
61
63
* kernel access as indicated in the PTE.
62
64
*/
63
65
64
-
mtspr(SPRN_ZPR, 0x10000000);
66
+
for (i = 0, zpr = 0; i < TASK_SIZE >> 28; i++)
67
+
zpr |= 1 << (30 - i * 2);
68
+
69
+
mtspr(SPRN_ZPR, zpr);
65
70
66
71
flush_instruction_cache();
67
72
+2
arch/powerpc/mm/nohash/8xx.c
+2
arch/powerpc/mm/nohash/8xx.c
+1
-1
arch/powerpc/mm/nohash/book3e_pgtable.c
+1
-1
arch/powerpc/mm/nohash/book3e_pgtable.c
···
71
71
* map_kernel_page adds an entry to the ioremap page table
72
72
* and adds an entry to the HPT, possibly bolting it
73
73
*/
74
-
int __ref map_kernel_page(unsigned long ea, unsigned long pa, pgprot_t prot)
74
+
int __ref map_kernel_page(unsigned long ea, phys_addr_t pa, pgprot_t prot)
75
75
{
76
76
pgd_t *pgdp;
77
77
p4d_t *p4dp;
+3
-3
arch/powerpc/mm/nohash/e500.c
+3
-3
arch/powerpc/mm/nohash/e500.c
···
117
117
TLBCAM[index].MAS2 |= (flags & _PAGE_ENDIAN) ? MAS2_E : 0;
118
118
119
119
TLBCAM[index].MAS3 = (phys & MAS3_RPN) | MAS3_SR;
120
-
TLBCAM[index].MAS3 |= (flags & _PAGE_RW) ? MAS3_SW : 0;
120
+
TLBCAM[index].MAS3 |= (flags & _PAGE_WRITE) ? MAS3_SW : 0;
121
121
if (mmu_has_feature(MMU_FTR_BIG_PHYS))
122
122
TLBCAM[index].MAS7 = (u64)phys >> 32;
123
123
124
124
/* Below is unlikely -- only for large user pages or similar */
125
-
if (pte_user(__pte(flags))) {
125
+
if (!is_kernel_addr(virt)) {
126
126
TLBCAM[index].MAS3 |= MAS3_UR;
127
127
TLBCAM[index].MAS3 |= (flags & _PAGE_EXEC) ? MAS3_UX : 0;
128
-
TLBCAM[index].MAS3 |= (flags & _PAGE_RW) ? MAS3_UW : 0;
128
+
TLBCAM[index].MAS3 |= (flags & _PAGE_WRITE) ? MAS3_UW : 0;
129
129
} else {
130
130
TLBCAM[index].MAS3 |= (flags & _PAGE_EXEC) ? MAS3_SX : 0;
131
131
}
+1
-2
arch/powerpc/mm/nohash/e500_hugetlbpage.c
+1
-2
arch/powerpc/mm/nohash/e500_hugetlbpage.c
···
178
178
*
179
179
* This must always be called with the pte lock held.
180
180
*/
181
-
void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma,
182
-
unsigned long address, pte_t *ptep, unsigned int nr)
181
+
void __update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
183
182
{
184
183
if (is_vm_hugetlb_page(vma))
185
184
book3e_hugetlb_preload(vma, address, *ptep);
+13
-13
arch/powerpc/mm/pgtable.c
+13
-13
arch/powerpc/mm/pgtable.c
···
46
46
* and we avoid _PAGE_SPECIAL and cache inhibited pte. We also only do that
47
47
* on userspace PTEs
48
48
*/
49
-
static inline int pte_looks_normal(pte_t pte)
49
+
static inline int pte_looks_normal(pte_t pte, unsigned long addr)
50
50
{
51
51
52
52
if (pte_present(pte) && !pte_special(pte)) {
53
53
if (pte_ci(pte))
54
54
return 0;
55
-
if (pte_user(pte))
55
+
if (!is_kernel_addr(addr))
56
56
return 1;
57
57
}
58
58
return 0;
···
79
79
* support falls into the same category.
80
80
*/
81
81
82
-
static pte_t set_pte_filter_hash(pte_t pte)
82
+
static pte_t set_pte_filter_hash(pte_t pte, unsigned long addr)
83
83
{
84
84
pte = __pte(pte_val(pte) & ~_PAGE_HPTEFLAGS);
85
-
if (pte_looks_normal(pte) && !(cpu_has_feature(CPU_FTR_COHERENT_ICACHE) ||
86
-
cpu_has_feature(CPU_FTR_NOEXECUTE))) {
85
+
if (pte_looks_normal(pte, addr) && !(cpu_has_feature(CPU_FTR_COHERENT_ICACHE) ||
86
+
cpu_has_feature(CPU_FTR_NOEXECUTE))) {
87
87
struct folio *folio = maybe_pte_to_folio(pte);
88
88
if (!folio)
89
89
return pte;
···
97
97
98
98
#else /* CONFIG_PPC_BOOK3S */
99
99
100
-
static pte_t set_pte_filter_hash(pte_t pte) { return pte; }
100
+
static pte_t set_pte_filter_hash(pte_t pte, unsigned long addr) { return pte; }
101
101
102
102
#endif /* CONFIG_PPC_BOOK3S */
103
103
···
107
107
*
108
108
* This is also called once for the folio. So only work with folio->flags here.
109
109
*/
110
-
static inline pte_t set_pte_filter(pte_t pte)
110
+
static inline pte_t set_pte_filter(pte_t pte, unsigned long addr)
111
111
{
112
112
struct folio *folio;
113
113
···
115
115
return pte;
116
116
117
117
if (mmu_has_feature(MMU_FTR_HPTE_TABLE))
118
-
return set_pte_filter_hash(pte);
118
+
return set_pte_filter_hash(pte, addr);
119
119
120
120
/* No exec permission in the first place, move on */
121
-
if (!pte_exec(pte) || !pte_looks_normal(pte))
121
+
if (!pte_exec(pte) || !pte_looks_normal(pte, addr))
122
122
return pte;
123
123
124
124
/* If you set _PAGE_EXEC on weird pages you're on your own */
···
198
198
* is called. Filter the pte value and use the filtered value
199
199
* to setup all the ptes in the range.
200
200
*/
201
-
pte = set_pte_filter(pte);
201
+
pte = set_pte_filter(pte, addr);
202
202
203
203
/*
204
204
* We don't need to call arch_enter/leave_lazy_mmu_mode()
···
314
314
*/
315
315
VM_WARN_ON(pte_hw_valid(*ptep) && !pte_protnone(*ptep));
316
316
317
-
pte = set_pte_filter(pte);
317
+
pte = set_pte_filter(pte, addr);
318
318
319
319
val = pte_val(pte);
320
320
···
505
505
[VM_READ] = PAGE_READONLY,
506
506
[VM_WRITE] = PAGE_COPY,
507
507
[VM_WRITE | VM_READ] = PAGE_COPY,
508
-
[VM_EXEC] = PAGE_READONLY_X,
508
+
[VM_EXEC] = PAGE_EXECONLY_X,
509
509
[VM_EXEC | VM_READ] = PAGE_READONLY_X,
510
510
[VM_EXEC | VM_WRITE] = PAGE_COPY_X,
511
511
[VM_EXEC | VM_WRITE | VM_READ] = PAGE_COPY_X,
···
513
513
[VM_SHARED | VM_READ] = PAGE_READONLY,
514
514
[VM_SHARED | VM_WRITE] = PAGE_SHARED,
515
515
[VM_SHARED | VM_WRITE | VM_READ] = PAGE_SHARED,
516
-
[VM_SHARED | VM_EXEC] = PAGE_READONLY_X,
516
+
[VM_SHARED | VM_EXEC] = PAGE_EXECONLY_X,
517
517
[VM_SHARED | VM_EXEC | VM_READ] = PAGE_READONLY_X,
518
518
[VM_SHARED | VM_EXEC | VM_WRITE] = PAGE_SHARED_X,
519
519
[VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_SHARED_X
-5
arch/powerpc/mm/ptdump/8xx.c
-5
arch/powerpc/mm/ptdump/8xx.c
+5
-13
arch/powerpc/net/bpf_jit.h
+5
-13
arch/powerpc/net/bpf_jit.h
···
36
36
EMIT(PPC_RAW_BRANCH(offset)); \
37
37
} while (0)
38
38
39
-
/* bl (unconditional 'branch' with link) */
40
-
#define PPC_BL(dest) EMIT(PPC_RAW_BL((dest) - (unsigned long)(image + ctx->idx)))
41
-
42
39
/* "cond" here covers BO:BI fields. */
43
40
#define PPC_BCC_SHORT(cond, dest) \
44
41
do { \
···
144
147
#define BPF_FIXUP_LEN 2 /* Two instructions => 8 bytes */
145
148
#endif
146
149
147
-
static inline void bpf_flush_icache(void *start, void *end)
148
-
{
149
-
smp_wmb(); /* smp write barrier */
150
-
flush_icache_range((unsigned long)start, (unsigned long)end);
151
-
}
152
-
153
150
static inline bool bpf_is_seen_register(struct codegen_context *ctx, int i)
154
151
{
155
152
return ctx->seen & (1 << (31 - i));
···
160
169
}
161
170
162
171
void bpf_jit_init_reg_mapping(struct codegen_context *ctx);
163
-
int bpf_jit_emit_func_call_rel(u32 *image, struct codegen_context *ctx, u64 func);
164
-
int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, struct codegen_context *ctx,
172
+
int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context *ctx, u64 func);
173
+
int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct codegen_context *ctx,
165
174
u32 *addrs, int pass, bool extra_pass);
166
175
void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx);
167
176
void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx);
168
177
void bpf_jit_realloc_regs(struct codegen_context *ctx);
169
178
int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, long exit_addr);
170
179
171
-
int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, int pass, struct codegen_context *ctx,
172
-
int insn_idx, int jmp_off, int dst_reg);
180
+
int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass,
181
+
struct codegen_context *ctx, int insn_idx,
182
+
int jmp_off, int dst_reg);
173
183
174
184
#endif
175
185
+117
-32
arch/powerpc/net/bpf_jit_comp.c
+117
-32
arch/powerpc/net/bpf_jit_comp.c
···
13
13
#include <linux/netdevice.h>
14
14
#include <linux/filter.h>
15
15
#include <linux/if_vlan.h>
16
-
#include <asm/kprobes.h>
16
+
#include <linux/kernel.h>
17
+
#include <linux/memory.h>
17
18
#include <linux/bpf.h>
19
+
20
+
#include <asm/kprobes.h>
21
+
#include <asm/code-patching.h>
18
22
19
23
#include "bpf_jit.h"
20
24
···
43
39
return 0;
44
40
}
45
41
46
-
struct powerpc64_jit_data {
47
-
struct bpf_binary_header *header;
42
+
struct powerpc_jit_data {
43
+
/* address of rw header */
44
+
struct bpf_binary_header *hdr;
45
+
/* address of ro final header */
46
+
struct bpf_binary_header *fhdr;
48
47
u32 *addrs;
49
-
u8 *image;
48
+
u8 *fimage;
50
49
u32 proglen;
51
50
struct codegen_context ctx;
52
51
};
···
66
59
u8 *image = NULL;
67
60
u32 *code_base;
68
61
u32 *addrs;
69
-
struct powerpc64_jit_data *jit_data;
62
+
struct powerpc_jit_data *jit_data;
70
63
struct codegen_context cgctx;
71
64
int pass;
72
65
int flen;
73
-
struct bpf_binary_header *bpf_hdr;
66
+
struct bpf_binary_header *fhdr = NULL;
67
+
struct bpf_binary_header *hdr = NULL;
74
68
struct bpf_prog *org_fp = fp;
75
69
struct bpf_prog *tmp_fp;
76
70
bool bpf_blinded = false;
77
71
bool extra_pass = false;
72
+
u8 *fimage = NULL;
73
+
u32 *fcode_base;
78
74
u32 extable_len;
79
75
u32 fixup_len;
80
76
···
107
97
addrs = jit_data->addrs;
108
98
if (addrs) {
109
99
cgctx = jit_data->ctx;
110
-
image = jit_data->image;
111
-
bpf_hdr = jit_data->header;
100
+
/*
101
+
* JIT compiled to a writable location (image/code_base) first.
102
+
* It is then moved to the readonly final location (fimage/fcode_base)
103
+
* using instruction patching.
104
+
*/
105
+
fimage = jit_data->fimage;
106
+
fhdr = jit_data->fhdr;
112
107
proglen = jit_data->proglen;
108
+
hdr = jit_data->hdr;
109
+
image = (void *)hdr + ((void *)fimage - (void *)fhdr);
113
110
extra_pass = true;
114
111
/* During extra pass, ensure index is reset before repopulating extable entries */
115
112
cgctx.exentry_idx = 0;
···
136
119
cgctx.stack_size = round_up(fp->aux->stack_depth, 16);
137
120
138
121
/* Scouting faux-generate pass 0 */
139
-
if (bpf_jit_build_body(fp, 0, &cgctx, addrs, 0, false)) {
122
+
if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
140
123
/* We hit something illegal or unsupported. */
141
124
fp = org_fp;
142
125
goto out_addrs;
···
151
134
*/
152
135
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
153
136
cgctx.idx = 0;
154
-
if (bpf_jit_build_body(fp, 0, &cgctx, addrs, 0, false)) {
137
+
if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
155
138
fp = org_fp;
156
139
goto out_addrs;
157
140
}
···
163
146
* update ctgtx.idx as it pretends to output instructions, then we can
164
147
* calculate total size from idx.
165
148
*/
166
-
bpf_jit_build_prologue(0, &cgctx);
149
+
bpf_jit_build_prologue(NULL, &cgctx);
167
150
addrs[fp->len] = cgctx.idx * 4;
168
-
bpf_jit_build_epilogue(0, &cgctx);
151
+
bpf_jit_build_epilogue(NULL, &cgctx);
169
152
170
153
fixup_len = fp->aux->num_exentries * BPF_FIXUP_LEN * 4;
171
154
extable_len = fp->aux->num_exentries * sizeof(struct exception_table_entry);
···
173
156
proglen = cgctx.idx * 4;
174
157
alloclen = proglen + FUNCTION_DESCR_SIZE + fixup_len + extable_len;
175
158
176
-
bpf_hdr = bpf_jit_binary_alloc(alloclen, &image, 4, bpf_jit_fill_ill_insns);
177
-
if (!bpf_hdr) {
159
+
fhdr = bpf_jit_binary_pack_alloc(alloclen, &fimage, 4, &hdr, &image,
160
+
bpf_jit_fill_ill_insns);
161
+
if (!fhdr) {
178
162
fp = org_fp;
179
163
goto out_addrs;
180
164
}
181
165
182
166
if (extable_len)
183
-
fp->aux->extable = (void *)image + FUNCTION_DESCR_SIZE + proglen + fixup_len;
167
+
fp->aux->extable = (void *)fimage + FUNCTION_DESCR_SIZE + proglen + fixup_len;
184
168
185
169
skip_init_ctx:
186
170
code_base = (u32 *)(image + FUNCTION_DESCR_SIZE);
171
+
fcode_base = (u32 *)(fimage + FUNCTION_DESCR_SIZE);
187
172
188
173
/* Code generation passes 1-2 */
189
174
for (pass = 1; pass < 3; pass++) {
···
193
174
cgctx.idx = 0;
194
175
cgctx.alt_exit_addr = 0;
195
176
bpf_jit_build_prologue(code_base, &cgctx);
196
-
if (bpf_jit_build_body(fp, code_base, &cgctx, addrs, pass, extra_pass)) {
197
-
bpf_jit_binary_free(bpf_hdr);
177
+
if (bpf_jit_build_body(fp, code_base, fcode_base, &cgctx, addrs, pass,
178
+
extra_pass)) {
179
+
bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
180
+
bpf_jit_binary_pack_free(fhdr, hdr);
198
181
fp = org_fp;
199
182
goto out_addrs;
200
183
}
···
216
195
217
196
#ifdef CONFIG_PPC64_ELF_ABI_V1
218
197
/* Function descriptor nastiness: Address + TOC */
219
-
((u64 *)image)[0] = (u64)code_base;
198
+
((u64 *)image)[0] = (u64)fcode_base;
220
199
((u64 *)image)[1] = local_paca->kernel_toc;
221
200
#endif
222
201
223
-
fp->bpf_func = (void *)image;
202
+
fp->bpf_func = (void *)fimage;
224
203
fp->jited = 1;
225
204
fp->jited_len = proglen + FUNCTION_DESCR_SIZE;
226
205
227
-
bpf_flush_icache(bpf_hdr, (u8 *)bpf_hdr + bpf_hdr->size);
228
206
if (!fp->is_func || extra_pass) {
229
-
bpf_jit_binary_lock_ro(bpf_hdr);
207
+
if (bpf_jit_binary_pack_finalize(fp, fhdr, hdr)) {
208
+
fp = org_fp;
209
+
goto out_addrs;
210
+
}
230
211
bpf_prog_fill_jited_linfo(fp, addrs);
231
212
out_addrs:
232
213
kfree(addrs);
···
238
215
jit_data->addrs = addrs;
239
216
jit_data->ctx = cgctx;
240
217
jit_data->proglen = proglen;
241
-
jit_data->image = image;
242
-
jit_data->header = bpf_hdr;
218
+
jit_data->fimage = fimage;
219
+
jit_data->fhdr = fhdr;
220
+
jit_data->hdr = hdr;
243
221
}
244
222
245
223
out:
···
254
230
* The caller should check for (BPF_MODE(code) == BPF_PROBE_MEM) before calling
255
231
* this function, as this only applies to BPF_PROBE_MEM, for now.
256
232
*/
257
-
int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, int pass, struct codegen_context *ctx,
258
-
int insn_idx, int jmp_off, int dst_reg)
233
+
int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass,
234
+
struct codegen_context *ctx, int insn_idx, int jmp_off,
235
+
int dst_reg)
259
236
{
260
237
off_t offset;
261
238
unsigned long pc;
262
-
struct exception_table_entry *ex;
239
+
struct exception_table_entry *ex, *ex_entry;
263
240
u32 *fixup;
264
241
265
242
/* Populate extable entries only in the last pass */
···
271
246
WARN_ON_ONCE(ctx->exentry_idx >= fp->aux->num_exentries))
272
247
return -EINVAL;
273
248
249
+
/*
250
+
* Program is first written to image before copying to the
251
+
* final location (fimage). Accordingly, update in the image first.
252
+
* As all offsets used are relative, copying as is to the
253
+
* final location should be alright.
254
+
*/
274
255
pc = (unsigned long)&image[insn_idx];
256
+
ex = (void *)fp->aux->extable - (void *)fimage + (void *)image;
275
257
276
-
fixup = (void *)fp->aux->extable -
258
+
fixup = (void *)ex -
277
259
(fp->aux->num_exentries * BPF_FIXUP_LEN * 4) +
278
260
(ctx->exentry_idx * BPF_FIXUP_LEN * 4);
279
261
···
291
259
fixup[BPF_FIXUP_LEN - 1] =
292
260
PPC_RAW_BRANCH((long)(pc + jmp_off) - (long)&fixup[BPF_FIXUP_LEN - 1]);
293
261
294
-
ex = &fp->aux->extable[ctx->exentry_idx];
262
+
ex_entry = &ex[ctx->exentry_idx];
295
263
296
-
offset = pc - (long)&ex->insn;
264
+
offset = pc - (long)&ex_entry->insn;
297
265
if (WARN_ON_ONCE(offset >= 0 || offset < INT_MIN))
298
266
return -ERANGE;
299
-
ex->insn = offset;
267
+
ex_entry->insn = offset;
300
268
301
-
offset = (long)fixup - (long)&ex->fixup;
269
+
offset = (long)fixup - (long)&ex_entry->fixup;
302
270
if (WARN_ON_ONCE(offset >= 0 || offset < INT_MIN))
303
271
return -ERANGE;
304
-
ex->fixup = offset;
272
+
ex_entry->fixup = offset;
305
273
306
274
ctx->exentry_idx++;
307
275
return 0;
276
+
}
277
+
278
+
void *bpf_arch_text_copy(void *dst, void *src, size_t len)
279
+
{
280
+
int err;
281
+
282
+
if (WARN_ON_ONCE(core_kernel_text((unsigned long)dst)))
283
+
return ERR_PTR(-EINVAL);
284
+
285
+
mutex_lock(&text_mutex);
286
+
err = patch_instructions(dst, src, len, false);
287
+
mutex_unlock(&text_mutex);
288
+
289
+
return err ? ERR_PTR(err) : dst;
290
+
}
291
+
292
+
int bpf_arch_text_invalidate(void *dst, size_t len)
293
+
{
294
+
u32 insn = BREAKPOINT_INSTRUCTION;
295
+
int ret;
296
+
297
+
if (WARN_ON_ONCE(core_kernel_text((unsigned long)dst)))
298
+
return -EINVAL;
299
+
300
+
mutex_lock(&text_mutex);
301
+
ret = patch_instructions(dst, &insn, len, true);
302
+
mutex_unlock(&text_mutex);
303
+
304
+
return ret;
305
+
}
306
+
307
+
void bpf_jit_free(struct bpf_prog *fp)
308
+
{
309
+
if (fp->jited) {
310
+
struct powerpc_jit_data *jit_data = fp->aux->jit_data;
311
+
struct bpf_binary_header *hdr;
312
+
313
+
/*
314
+
* If we fail the final pass of JIT (from jit_subprogs),
315
+
* the program may not be finalized yet. Call finalize here
316
+
* before freeing it.
317
+
*/
318
+
if (jit_data) {
319
+
bpf_jit_binary_pack_finalize(fp, jit_data->fhdr, jit_data->hdr);
320
+
kvfree(jit_data->addrs);
321
+
kfree(jit_data);
322
+
}
323
+
hdr = bpf_jit_binary_pack_hdr(fp);
324
+
bpf_jit_binary_pack_free(hdr, NULL);
325
+
WARN_ON_ONCE(!bpf_prog_kallsyms_verify_off(fp));
326
+
}
327
+
328
+
bpf_prog_unlock_free(fp);
308
329
}
+8
-7
arch/powerpc/net/bpf_jit_comp32.c
+8
-7
arch/powerpc/net/bpf_jit_comp32.c
···
200
200
EMIT(PPC_RAW_BLR());
201
201
}
202
202
203
-
int bpf_jit_emit_func_call_rel(u32 *image, struct codegen_context *ctx, u64 func)
203
+
/* Relative offset needs to be calculated based on final image location */
204
+
int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context *ctx, u64 func)
204
205
{
205
-
s32 rel = (s32)func - (s32)(image + ctx->idx);
206
+
s32 rel = (s32)func - (s32)(fimage + ctx->idx);
206
207
207
208
if (image && rel < 0x2000000 && rel >= -0x2000000) {
208
-
PPC_BL(func);
209
+
EMIT(PPC_RAW_BL(rel));
209
210
} else {
210
211
/* Load function address into r0 */
211
212
EMIT(PPC_RAW_LIS(_R0, IMM_H(func)));
···
279
278
}
280
279
281
280
/* Assemble the body code between the prologue & epilogue */
282
-
int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, struct codegen_context *ctx,
281
+
int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct codegen_context *ctx,
283
282
u32 *addrs, int pass, bool extra_pass)
284
283
{
285
284
const struct bpf_insn *insn = fp->insnsi;
···
941
940
* !fp->aux->verifier_zext. Emit NOP otherwise.
942
941
*
943
942
* Note that "li reg_h,0" is emitted for BPF_B/H/W case,
944
-
* if necessary. So, jump there insted of emitting an
943
+
* if necessary. So, jump there instead of emitting an
945
944
* additional "li reg_h,0" instruction.
946
945
*/
947
946
if (size == BPF_DW && !fp->aux->verifier_zext)
···
998
997
jmp_off += 4;
999
998
}
1000
999
1001
-
ret = bpf_add_extable_entry(fp, image, pass, ctx, insn_idx,
1000
+
ret = bpf_add_extable_entry(fp, image, fimage, pass, ctx, insn_idx,
1002
1001
jmp_off, dst_reg);
1003
1002
if (ret)
1004
1003
return ret;
···
1054
1053
EMIT(PPC_RAW_STW(bpf_to_ppc(BPF_REG_5), _R1, 12));
1055
1054
}
1056
1055
1057
-
ret = bpf_jit_emit_func_call_rel(image, ctx, func_addr);
1056
+
ret = bpf_jit_emit_func_call_rel(image, fimage, ctx, func_addr);
1058
1057
if (ret)
1059
1058
return ret;
1060
1059
+5
-5
arch/powerpc/net/bpf_jit_comp64.c
+5
-5
arch/powerpc/net/bpf_jit_comp64.c
···
240
240
return 0;
241
241
}
242
242
243
-
int bpf_jit_emit_func_call_rel(u32 *image, struct codegen_context *ctx, u64 func)
243
+
int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context *ctx, u64 func)
244
244
{
245
245
unsigned int i, ctx_idx = ctx->idx;
246
246
···
361
361
);
362
362
363
363
/* Assemble the body code between the prologue & epilogue */
364
-
int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, struct codegen_context *ctx,
364
+
int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct codegen_context *ctx,
365
365
u32 *addrs, int pass, bool extra_pass)
366
366
{
367
367
enum stf_barrier_type stf_barrier = stf_barrier_type_get();
···
940
940
addrs[++i] = ctx->idx * 4;
941
941
942
942
if (BPF_MODE(code) == BPF_PROBE_MEM) {
943
-
ret = bpf_add_extable_entry(fp, image, pass, ctx, ctx->idx - 1,
944
-
4, dst_reg);
943
+
ret = bpf_add_extable_entry(fp, image, fimage, pass, ctx,
944
+
ctx->idx - 1, 4, dst_reg);
945
945
if (ret)
946
946
return ret;
947
947
}
···
995
995
if (func_addr_fixed)
996
996
ret = bpf_jit_emit_func_call_hlp(image, ctx, func_addr);
997
997
else
998
-
ret = bpf_jit_emit_func_call_rel(image, ctx, func_addr);
998
+
ret = bpf_jit_emit_func_call_rel(image, fimage, ctx, func_addr);
999
999
1000
1000
if (ret)
1001
1001
return ret;
+2
-3
arch/powerpc/perf/core-book3s.c
+2
-3
arch/powerpc/perf/core-book3s.c
···
1371
1371
/*
1372
1372
* Disable instruction sampling if it was enabled
1373
1373
*/
1374
-
if (cpuhw->mmcr.mmcra & MMCRA_SAMPLE_ENABLE)
1375
-
val &= ~MMCRA_SAMPLE_ENABLE;
1374
+
val &= ~MMCRA_SAMPLE_ENABLE;
1376
1375
1377
1376
/* Disable BHRB via mmcra (BHRBRD) for p10 */
1378
1377
if (ppmu->flags & PPMU_ARCH_31)
···
1382
1383
* instruction sampling or BHRB.
1383
1384
*/
1384
1385
if (val != mmcra) {
1385
-
mtspr(SPRN_MMCRA, mmcra);
1386
+
mtspr(SPRN_MMCRA, val);
1386
1387
mb();
1387
1388
isync();
1388
1389
}
+1
-1
arch/powerpc/perf/hv-24x7.c
+1
-1
arch/powerpc/perf/hv-24x7.c
···
1338
1338
for (i = count = 0, element_data = res->elements + data_offset;
1339
1339
i < num_elements;
1340
1340
i++, element_data += data_size + data_offset)
1341
-
count += be64_to_cpu(*((u64 *) element_data));
1341
+
count += be64_to_cpu(*((__be64 *)element_data));
1342
1342
1343
1343
*countp = count;
1344
1344
+7
-6
arch/powerpc/perf/imc-pmu.c
+7
-6
arch/powerpc/perf/imc-pmu.c
···
51
51
* core and trace-imc
52
52
*/
53
53
static struct imc_pmu_ref imc_global_refc = {
54
-
.lock = __SPIN_LOCK_INITIALIZER(imc_global_refc.lock),
54
+
.lock = __SPIN_LOCK_UNLOCKED(imc_global_refc.lock),
55
55
.id = 0,
56
56
.refc = 0,
57
57
};
···
544
544
break;
545
545
}
546
546
pcni++;
547
-
} while (pcni->vbase != 0);
547
+
} while (pcni->vbase);
548
548
549
549
if (!flag)
550
550
return -ENODEV;
···
1025
1025
return false;
1026
1026
}
1027
1027
1028
-
static u64 * get_event_base_addr(struct perf_event *event)
1028
+
static __be64 *get_event_base_addr(struct perf_event *event)
1029
1029
{
1030
1030
u64 addr;
1031
1031
1032
1032
if (is_thread_imc_pmu(event)) {
1033
1033
addr = (u64)per_cpu(thread_imc_mem, smp_processor_id());
1034
-
return (u64 *)(addr + (event->attr.config & IMC_EVENT_OFFSET_MASK));
1034
+
return (__be64 *)(addr + (event->attr.config & IMC_EVENT_OFFSET_MASK));
1035
1035
}
1036
1036
1037
-
return (u64 *)event->hw.event_base;
1037
+
return (__be64 *)event->hw.event_base;
1038
1038
}
1039
1039
1040
1040
static void thread_imc_pmu_start_txn(struct pmu *pmu,
···
1058
1058
1059
1059
static u64 imc_read_counter(struct perf_event *event)
1060
1060
{
1061
-
u64 *addr, data;
1061
+
__be64 *addr;
1062
+
u64 data;
1062
1063
1063
1064
/*
1064
1065
* In-Memory Collection (IMC) counters are free flowing counters.
+28
-16
arch/powerpc/perf/power6-pmu.c
+28
-16
arch/powerpc/perf/power6-pmu.c
···
335
335
{ 0x3000fe, 0x400056 }, /* PM_DATA_FROM_L3MISS */
336
336
};
337
337
338
-
/*
339
-
* This could be made more efficient with a binary search on
340
-
* a presorted list, if necessary
341
-
*/
342
338
static int find_alternatives_list(u64 event)
343
339
{
344
-
int i, j;
345
-
unsigned int alt;
340
+
const unsigned int presorted_event_table[] = {
341
+
0x0130e8, 0x080080, 0x080088, 0x10000a, 0x10000b, 0x10000d, 0x10000e,
342
+
0x100010, 0x10001a, 0x100026, 0x100054, 0x100056, 0x1000f0, 0x1000f8,
343
+
0x1000fc, 0x200008, 0x20000e, 0x200010, 0x200012, 0x200054, 0x2000f0,
344
+
0x2000f2, 0x2000f4, 0x2000f5, 0x2000f6, 0x2000f8, 0x2000fc, 0x2000fe,
345
+
0x2d0030, 0x30000a, 0x30000c, 0x300010, 0x300012, 0x30001a, 0x300056,
346
+
0x3000f0, 0x3000f2, 0x3000f6, 0x3000f8, 0x3000fc, 0x3000fe, 0x400006,
347
+
0x400007, 0x40000a, 0x40000e, 0x400010, 0x400018, 0x400056, 0x4000f0,
348
+
0x4000f8, 0x600005
349
+
};
350
+
const unsigned int event_index_table[] = {
351
+
0, 1, 2, 3, 4, 1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 12, 14,
352
+
7, 15, 2, 9, 16, 3, 4, 0, 17, 10, 18, 19, 20, 1, 17, 15, 19,
353
+
18, 2, 16, 21, 8, 0, 22, 13, 14, 11, 21, 5, 20, 22, 1, 6, 3
354
+
};
355
+
int hi = ARRAY_SIZE(presorted_event_table) - 1;
356
+
int lo = 0;
346
357
347
-
for (i = 0; i < ARRAY_SIZE(event_alternatives); ++i) {
348
-
if (event < event_alternatives[i][0])
349
-
return -1;
350
-
for (j = 0; j < MAX_ALT; ++j) {
351
-
alt = event_alternatives[i][j];
352
-
if (!alt || event < alt)
353
-
break;
354
-
if (event == alt)
355
-
return i;
356
-
}
358
+
while (lo <= hi) {
359
+
int mid = lo + (hi - lo) / 2;
360
+
unsigned int alt = presorted_event_table[mid];
361
+
362
+
if (alt < event)
363
+
lo = mid + 1;
364
+
else if (alt > event)
365
+
hi = mid - 1;
366
+
else
367
+
return event_index_table[mid];
357
368
}
369
+
358
370
return -1;
359
371
}
360
372
+1
-1
arch/powerpc/platforms/4xx/soc.c
+1
-1
arch/powerpc/platforms/4xx/soc.c
···
112
112
}
113
113
114
114
/* Install error handler */
115
-
if (request_irq(irq, l2c_error_handler, 0, "L2C", 0) < 0) {
115
+
if (request_irq(irq, l2c_error_handler, 0, "L2C", NULL) < 0) {
116
116
printk(KERN_ERR "Cannot install L2C error handler"
117
117
", cache is not enabled\n");
118
118
of_node_put(np);
+2
arch/powerpc/platforms/83xx/misc.c
+2
arch/powerpc/platforms/83xx/misc.c
+1
arch/powerpc/platforms/8xx/cpm1.c
+1
arch/powerpc/platforms/8xx/cpm1.c
+16
-18
arch/powerpc/platforms/book3s/vas-api.c
+16
-18
arch/powerpc/platforms/book3s/vas-api.c
···
4
4
* Copyright (C) 2019 Haren Myneni, IBM Corp
5
5
*/
6
6
7
+
#define pr_fmt(fmt) "vas-api: " fmt
8
+
7
9
#include <linux/kernel.h>
8
10
#include <linux/device.h>
9
11
#include <linux/cdev.h>
···
80
78
task_ref->mm = get_task_mm(current);
81
79
if (!task_ref->mm) {
82
80
put_pid(task_ref->pid);
83
-
pr_err("VAS: pid(%d): mm_struct is not found\n",
81
+
pr_err("pid(%d): mm_struct is not found\n",
84
82
current->pid);
85
83
return -EPERM;
86
84
}
···
237
235
rc = kill_pid_info(SIGSEGV, &info, pid);
238
236
rcu_read_unlock();
239
237
240
-
pr_devel("%s(): pid %d kill_proc_info() rc %d\n", __func__,
241
-
pid_vnr(pid), rc);
238
+
pr_devel("pid %d kill_proc_info() rc %d\n", pid_vnr(pid), rc);
242
239
}
243
240
244
241
void vas_dump_crb(struct coprocessor_request_block *crb)
···
295
294
296
295
rc = copy_from_user(&uattr, uptr, sizeof(uattr));
297
296
if (rc) {
298
-
pr_err("%s(): copy_from_user() returns %d\n", __func__, rc);
297
+
pr_err("copy_from_user() returns %d\n", rc);
299
298
return -EFAULT;
300
299
}
301
300
···
312
311
txwin = cp_inst->coproc->vops->open_win(uattr.vas_id, uattr.flags,
313
312
cp_inst->coproc->cop_type);
314
313
if (IS_ERR(txwin)) {
315
-
pr_err("%s() VAS window open failed, %ld\n", __func__,
314
+
pr_err_ratelimited("VAS window open failed rc=%ld\n",
316
315
PTR_ERR(txwin));
317
316
return PTR_ERR(txwin);
318
317
}
···
406
405
* window is not opened. Shouldn't expect this error.
407
406
*/
408
407
if (!cp_inst || !cp_inst->txwin) {
409
-
pr_err("%s(): Unexpected fault on paste address with TX window closed\n",
410
-
__func__);
408
+
pr_err("Unexpected fault on paste address with TX window closed\n");
411
409
return VM_FAULT_SIGBUS;
412
410
}
413
411
···
421
421
* issue NX request.
422
422
*/
423
423
if (txwin->task_ref.vma != vmf->vma) {
424
-
pr_err("%s(): No previous mapping with paste address\n",
425
-
__func__);
424
+
pr_err("No previous mapping with paste address\n");
426
425
return VM_FAULT_SIGBUS;
427
426
}
428
427
···
480
481
txwin = cp_inst->txwin;
481
482
482
483
if ((vma->vm_end - vma->vm_start) > PAGE_SIZE) {
483
-
pr_debug("%s(): size 0x%zx, PAGE_SIZE 0x%zx\n", __func__,
484
+
pr_debug("size 0x%zx, PAGE_SIZE 0x%zx\n",
484
485
(vma->vm_end - vma->vm_start), PAGE_SIZE);
485
486
return -EINVAL;
486
487
}
487
488
488
489
/* Ensure instance has an open send window */
489
490
if (!txwin) {
490
-
pr_err("%s(): No send window open?\n", __func__);
491
+
pr_err("No send window open?\n");
491
492
return -EINVAL;
492
493
}
493
494
494
495
if (!cp_inst->coproc->vops || !cp_inst->coproc->vops->paste_addr) {
495
-
pr_err("%s(): VAS API is not registered\n", __func__);
496
+
pr_err("VAS API is not registered\n");
496
497
return -EACCES;
497
498
}
498
499
···
509
510
*/
510
511
mutex_lock(&txwin->task_ref.mmap_mutex);
511
512
if (txwin->status != VAS_WIN_ACTIVE) {
512
-
pr_err("%s(): Window is not active\n", __func__);
513
+
pr_err("Window is not active\n");
513
514
rc = -EACCES;
514
515
goto out;
515
516
}
516
517
517
518
paste_addr = cp_inst->coproc->vops->paste_addr(txwin);
518
519
if (!paste_addr) {
519
-
pr_err("%s(): Window paste address failed\n", __func__);
520
+
pr_err("Window paste address failed\n");
520
521
rc = -EINVAL;
521
522
goto out;
522
523
}
···
532
533
rc = remap_pfn_range(vma, vma->vm_start, pfn + vma->vm_pgoff,
533
534
vma->vm_end - vma->vm_start, prot);
534
535
535
-
pr_devel("%s(): paste addr %llx at %lx, rc %d\n", __func__,
536
-
paste_addr, vma->vm_start, rc);
536
+
pr_devel("paste addr %llx at %lx, rc %d\n", paste_addr,
537
+
vma->vm_start, rc);
537
538
538
539
txwin->task_ref.vma = vma;
539
540
vma->vm_ops = &vas_vm_ops;
···
608
609
goto err;
609
610
}
610
611
611
-
pr_devel("%s: Added dev [%d,%d]\n", __func__, MAJOR(devno),
612
-
MINOR(devno));
612
+
pr_devel("Added dev [%d,%d]\n", MAJOR(devno), MINOR(devno));
613
613
614
614
return 0;
615
615
+1
arch/powerpc/platforms/powermac/Kconfig
+1
arch/powerpc/platforms/powermac/Kconfig
+2
-1
arch/powerpc/platforms/powermac/feature.c
+2
-1
arch/powerpc/platforms/powermac/feature.c
+3
-1
arch/powerpc/platforms/powermac/low_i2c.c
+3
-1
arch/powerpc/platforms/powermac/low_i2c.c
···
925
925
926
926
sz = sizeof(struct pmac_i2c_bus) + sizeof(struct smu_i2c_cmd);
927
927
bus = kzalloc(sz, GFP_KERNEL);
928
-
if (bus == NULL)
928
+
if (bus == NULL) {
929
+
of_node_put(busnode);
929
930
return;
931
+
}
930
932
931
933
bus->controller = controller;
932
934
bus->busnode = of_node_get(busnode);
+3
-1
arch/powerpc/platforms/powermac/smp.c
+3
-1
arch/powerpc/platforms/powermac/smp.c
+1
-1
arch/powerpc/platforms/powernv/opal-fadump.h
+1
-1
arch/powerpc/platforms/powernv/opal-fadump.h
···
135
135
for (i = 0; i < regs_cnt; i++, bufp += reg_entry_size) {
136
136
reg_entry = (struct hdat_fadump_reg_entry *)bufp;
137
137
val = (cpu_endian ? be64_to_cpu(reg_entry->reg_val) :
138
-
(u64)(reg_entry->reg_val));
138
+
(u64 __force)(reg_entry->reg_val));
139
139
opal_fadump_set_regval_regnum(regs,
140
140
be32_to_cpu(reg_entry->reg_type),
141
141
be32_to_cpu(reg_entry->reg_num),
+2
-1
arch/powerpc/platforms/pseries/hotplug-memory.c
+2
-1
arch/powerpc/platforms/pseries/hotplug-memory.c
+4
-4
arch/powerpc/platforms/pseries/iommu.c
+4
-4
arch/powerpc/platforms/pseries/iommu.c
···
914
914
return 0;
915
915
}
916
916
917
-
static bool find_existing_ddw(struct device_node *pdn, u64 *dma_addr, int *window_shift)
917
+
static bool find_existing_ddw(struct device_node *pdn, u64 *dma_addr, int *window_shift,
918
+
bool *direct_mapping)
918
919
{
919
920
struct dma_win *window;
920
921
const struct dynamic_dma_window_prop *dma64;
···
928
927
dma64 = window->prop;
929
928
*dma_addr = be64_to_cpu(dma64->dma_base);
930
929
*window_shift = be32_to_cpu(dma64->window_shift);
930
+
*direct_mapping = window->direct;
931
931
found = true;
932
932
break;
933
933
}
···
1272
1270
1273
1271
mutex_lock(&dma_win_init_mutex);
1274
1272
1275
-
if (find_existing_ddw(pdn, &dev->dev.archdata.dma_offset, &len)) {
1276
-
direct_mapping = (len >= max_ram_len);
1273
+
if (find_existing_ddw(pdn, &dev->dev.archdata.dma_offset, &len, &direct_mapping))
1277
1274
goto out_unlock;
1278
-
}
1279
1275
1280
1276
/*
1281
1277
* If we already went through this for a previous function of
+7
-5
arch/powerpc/platforms/pseries/lpar.c
+7
-5
arch/powerpc/platforms/pseries/lpar.c
···
192
192
continue;
193
193
kmem_cache_free(dtl_cache, pp->dispatch_log);
194
194
pp->dtl_ridx = 0;
195
-
pp->dispatch_log = 0;
196
-
pp->dispatch_log_end = 0;
197
-
pp->dtl_curr = 0;
195
+
pp->dispatch_log = NULL;
196
+
pp->dispatch_log_end = NULL;
197
+
pp->dtl_curr = NULL;
198
198
199
199
if (time_limit && time_after(jiffies, *time_limit)) {
200
200
cond_resched();
···
223
223
{
224
224
kfree(vcpu_associativity);
225
225
kfree(pcpu_associativity);
226
-
vcpu_associativity = pcpu_associativity = 0;
226
+
vcpu_associativity = pcpu_associativity = NULL;
227
227
}
228
228
229
229
static __be32 *__get_cpu_associativity(int cpu, __be32 *cpu_assoc, int flag)
···
526
526
527
527
if (cmd) {
528
528
rc = init_cpu_associativity();
529
-
if (rc)
529
+
if (rc) {
530
+
destroy_cpu_associativity();
530
531
goto out;
532
+
}
531
533
532
534
for_each_possible_cpu(cpu) {
533
535
disp = per_cpu_ptr(&vcpu_disp_data, cpu);
+2
-2
arch/powerpc/platforms/pseries/plpks.c
+2
-2
arch/powerpc/platforms/pseries/plpks.c
···
150
150
ospasswordlength = maxpwsize;
151
151
ospassword = kzalloc(maxpwsize, GFP_KERNEL);
152
152
if (!ospassword) {
153
-
kfree(password);
153
+
kfree_sensitive(password);
154
154
return -ENOMEM;
155
155
}
156
156
memcpy(ospassword, password, ospasswordlength);
···
163
163
}
164
164
}
165
165
out:
166
-
kfree(password);
166
+
kfree_sensitive(password);
167
167
168
168
return pseries_status_to_err(rc);
169
169
}
+2
-2
arch/powerpc/platforms/pseries/vas.c
+2
-2
arch/powerpc/platforms/pseries/vas.c
···
341
341
342
342
if (atomic_inc_return(&cop_feat_caps->nr_used_credits) >
343
343
atomic_read(&cop_feat_caps->nr_total_credits)) {
344
-
pr_err("Credits are not available to allocate window\n");
344
+
pr_err_ratelimited("Credits are not available to allocate window\n");
345
345
rc = -EINVAL;
346
346
goto out;
347
347
}
···
424
424
425
425
put_vas_user_win_ref(&txwin->vas_win.task_ref);
426
426
rc = -EBUSY;
427
-
pr_err("No credit is available to allocate window\n");
427
+
pr_err_ratelimited("No credit is available to allocate window\n");
428
428
429
429
out_free:
430
430
/*
+4
-6
arch/powerpc/sysdev/fsl_msi.c
+4
-6
arch/powerpc/sysdev/fsl_msi.c
···
11
11
#include <linux/msi.h>
12
12
#include <linux/pci.h>
13
13
#include <linux/slab.h>
14
+
#include <linux/of.h>
14
15
#include <linux/of_address.h>
15
16
#include <linux/of_irq.h>
16
-
#include <linux/of_platform.h>
17
+
#include <linux/platform_device.h>
18
+
#include <linux/property.h>
17
19
#include <linux/interrupt.h>
18
20
#include <linux/irqdomain.h>
19
21
#include <linux/seq_file.h>
···
394
392
static const struct of_device_id fsl_of_msi_ids[];
395
393
static int fsl_of_msi_probe(struct platform_device *dev)
396
394
{
397
-
const struct of_device_id *match;
398
395
struct fsl_msi *msi;
399
396
struct resource res, msiir;
400
397
int err, i, j, irq_index, count;
···
403
402
u32 offset;
404
403
struct pci_controller *phb;
405
404
406
-
match = of_match_device(fsl_of_msi_ids, &dev->dev);
407
-
if (!match)
408
-
return -EINVAL;
409
-
features = match->data;
405
+
features = device_get_match_data(&dev->dev);
410
406
411
407
printk(KERN_DEBUG "Setting up Freescale MSI support\n");
412
408
+1
-1
arch/powerpc/sysdev/mpic.c
+1
-1
arch/powerpc/sysdev/mpic.c
···
355
355
mpic_write(mpic->gregs, MPIC_INFO(GREG_IPI_VECTOR_PRI_0), MPIC_VECPRI_MASK);
356
356
r = mpic_read(mpic->gregs, MPIC_INFO(GREG_IPI_VECTOR_PRI_0));
357
357
358
-
if (r == le32_to_cpu(MPIC_VECPRI_MASK)) {
358
+
if (r == swab32(MPIC_VECPRI_MASK)) {
359
359
printk(KERN_INFO "mpic: Detected reversed IPI registers\n");
360
360
mpic->flags |= MPIC_BROKEN_IPI;
361
361
}
+1
-1
arch/powerpc/sysdev/xive/native.c
+1
-1
arch/powerpc/sysdev/xive/native.c
+5
-6
arch/powerpc/tools/gcc-check-mprofile-kernel.sh
+5
-6
arch/powerpc/tools/gcc-check-mprofile-kernel.sh
···
7
7
# To debug, uncomment the following line
8
8
# set -x
9
9
10
-
# -mprofile-kernel is only supported on 64-bit, so this should not be invoked
11
-
# for 32-bit. We pass in -m64 explicitly, and -mbig-endian and -mlittle-endian
12
-
# are passed in from Kconfig, which takes care of toolchains defaulting to
13
-
# other targets.
10
+
# -mprofile-kernel is only supported on 64-bit with ELFv2, so this should not
11
+
# be invoked for other targets. Therefore we can pass in -m64 and -mabi
12
+
# explicitly, to take care of toolchains defaulting to other targets.
14
13
15
14
# Test whether the compile option -mprofile-kernel exists and generates
16
15
# profiling code (ie. a call to _mcount()).
17
16
echo "int func() { return 0; }" | \
18
-
$* -m64 -S -x c -O2 -p -mprofile-kernel - -o - \
17
+
$* -m64 -mabi=elfv2 -S -x c -O2 -p -mprofile-kernel - -o - \
19
18
2> /dev/null | grep -q "_mcount"
20
19
21
20
# Test whether the notrace attribute correctly suppresses calls to _mcount().
22
21
23
22
echo -e "#include <linux/compiler.h>\nnotrace int func() { return 0; }" | \
24
-
$* -m64 -S -x c -O2 -p -mprofile-kernel - -o - \
23
+
$* -m64 -mabi=elfv2 -S -x c -O2 -p -mprofile-kernel - -o - \
25
24
2> /dev/null | grep -q "_mcount" && \
26
25
exit 1
27
26
+1
drivers/macintosh/macio-adb.c
+1
drivers/macintosh/macio-adb.c
+2
-2
drivers/soc/fsl/qe/qe_common.c
+2
-2
drivers/soc/fsl/qe/qe_common.c
···
141
141
*
142
142
* This function returns a non-negative offset into the muram area, or
143
143
* a negative errno on failure.
144
-
* Use cpm_dpram_addr() to get the virtual address of the area.
144
+
* Use cpm_muram_addr() to get the virtual address of the area.
145
145
* Use cpm_muram_free() to free the allocation.
146
146
*/
147
147
s32 cpm_muram_alloc(unsigned long size, unsigned long align)
···
193
193
* @size: number of bytes to allocate
194
194
* This function returns @offset if the area was available, a negative
195
195
* errno otherwise.
196
-
* Use cpm_dpram_addr() to get the virtual address of the area.
196
+
* Use cpm_muram_addr() to get the virtual address of the area.
197
197
* Use cpm_muram_free() to free the allocation.
198
198
*/
199
199
s32 cpm_muram_alloc_fixed(unsigned long offset, unsigned long size)
+5
-3
include/linux/crash_dump.h
+5
-3
include/linux/crash_dump.h
···
50
50
#define vmcore_elf64_check_arch(x) (elf_check_arch(x) || vmcore_elf_check_arch_cross(x))
51
51
#endif
52
52
53
+
#ifndef is_kdump_kernel
53
54
/*
54
55
* is_kdump_kernel() checks whether this kernel is booting after a panic of
55
56
* previous kernel or not. This is determined by checking if previous kernel
···
65
64
{
66
65
return elfcorehdr_addr != ELFCORE_ADDR_MAX;
67
66
}
67
+
#endif
68
68
69
69
/* is_vmcore_usable() checks if the kernel is booting after a panic and
70
70
* the vmcore region is usable.
···
77
75
78
76
static inline int is_vmcore_usable(void)
79
77
{
80
-
return is_kdump_kernel() && elfcorehdr_addr != ELFCORE_ADDR_ERR ? 1 : 0;
78
+
return elfcorehdr_addr != ELFCORE_ADDR_ERR &&
79
+
elfcorehdr_addr != ELFCORE_ADDR_MAX ? 1 : 0;
81
80
}
82
81
83
82
/* vmcore_unusable() marks the vmcore as unusable,
···
87
84
88
85
static inline void vmcore_unusable(void)
89
86
{
90
-
if (is_kdump_kernel())
91
-
elfcorehdr_addr = ELFCORE_ADDR_ERR;
87
+
elfcorehdr_addr = ELFCORE_ADDR_ERR;
92
88
}
93
89
94
90
/**
+1
-1
lib/fonts/Kconfig
+1
-1
lib/fonts/Kconfig
···
98
98
99
99
config FONT_SUN8x16
100
100
bool "Sparc console 8x16 font"
101
-
depends on FRAMEBUFFER_CONSOLE && (!SPARC && FONTS || SPARC)
101
+
depends on (FRAMEBUFFER_CONSOLE && (FONTS || SPARC)) || BOOTX_TEXT
102
102
help
103
103
This is the high resolution console font for Sun machines. Say Y.
104
104