Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Merge branch kvm-arm64/vgic-v5-ppi into kvmarm-master/next
* kvm-arm64/vgic-v5-ppi: (40 commits)
: .
: Add initial GICv5 support for KVM guests, only adding PPI support
: for the time being. Patches courtesy of Sascha Bischoff.
:
: From the cover letter:
:
: "This is v7 of the patch series to add the virtual GICv5 [1] device
: (vgic_v5). Only PPIs are supported by this initial series, and the
: vgic_v5 implementation is restricted to the CPU interface,
: only. Further patch series are to follow in due course, and will add
: support for SPIs, LPIs, the GICv5 IRS, and the GICv5 ITS."
: .
KVM: arm64: selftests: Add no-vgic-v5 selftest
KVM: arm64: selftests: Introduce a minimal GICv5 PPI selftest
KVM: arm64: gic-v5: Communicate userspace-driveable PPIs via a UAPI
Documentation: KVM: Introduce documentation for VGICv5
KVM: arm64: gic-v5: Probe for GICv5 device
KVM: arm64: gic-v5: Set ICH_VCTLR_EL2.En on boot
KVM: arm64: gic-v5: Introduce kvm_arm_vgic_v5_ops and register them
KVM: arm64: gic-v5: Hide FEAT_GCIE from NV GICv5 guests
KVM: arm64: gic: Hide GICv5 for protected guests
KVM: arm64: gic-v5: Mandate architected PPI for PMU emulation on GICv5
KVM: arm64: gic-v5: Enlighten arch timer for GICv5
irqchip/gic-v5: Introduce minimal irq_set_type() for PPIs
KVM: arm64: gic-v5: Initialise ID and priority bits when resetting vcpu
KVM: arm64: gic-v5: Create and initialise vgic_v5
KVM: arm64: gic-v5: Support GICv5 interrupts with KVM_IRQ_LINE
KVM: arm64: gic-v5: Implement direct injection of PPIs
KVM: arm64: Introduce set_direct_injection irq_op
KVM: arm64: gic-v5: Trap and mask guest ICC_PPI_ENABLERx_EL1 writes
KVM: arm64: gic-v5: Check for pending PPIs
KVM: arm64: gic-v5: Clear TWI if single task running
...
Signed-off-by: Marc Zyngier <maz@kernel.org>
+3205
-383
+4
-2
Documentation/virt/kvm/api.rst
+4
-2
Documentation/virt/kvm/api.rst
···
907
907
- KVM_ARM_IRQ_TYPE_CPU:
908
908
out-of-kernel GIC: irq_id 0 is IRQ, irq_id 1 is FIQ
909
909
- KVM_ARM_IRQ_TYPE_SPI:
910
-
in-kernel GIC: SPI, irq_id between 32 and 1019 (incl.)
910
+
in-kernel GICv2/GICv3: SPI, irq_id between 32 and 1019 (incl.)
911
911
(the vcpu_index field is ignored)
912
+
in-kernel GICv5: SPI, irq_id between 0 and 65535 (incl.)
912
913
- KVM_ARM_IRQ_TYPE_PPI:
913
-
in-kernel GIC: PPI, irq_id between 16 and 31 (incl.)
914
+
in-kernel GICv2/GICv3: PPI, irq_id between 16 and 31 (incl.)
915
+
in-kernel GICv5: PPI, irq_id between 0 and 127 (incl.)
914
916
915
917
(The irq_id field thus corresponds nicely to the IRQ ID in the ARM GIC specs)
916
918
+50
Documentation/virt/kvm/devices/arm-vgic-v5.rst
+50
Documentation/virt/kvm/devices/arm-vgic-v5.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0
2
+
3
+
====================================================
4
+
ARM Virtual Generic Interrupt Controller v5 (VGICv5)
5
+
====================================================
6
+
7
+
8
+
Device types supported:
9
+
- KVM_DEV_TYPE_ARM_VGIC_V5 ARM Generic Interrupt Controller v5.0
10
+
11
+
Only one VGIC instance may be instantiated through this API. The created VGIC
12
+
will act as the VM interrupt controller, requiring emulated user-space devices
13
+
to inject interrupts to the VGIC instead of directly to CPUs.
14
+
15
+
Creating a guest GICv5 device requires a host GICv5 host. The current VGICv5
16
+
device only supports PPI interrupts. These can either be injected from emulated
17
+
in-kernel devices (such as the Arch Timer, or PMU), or via the KVM_IRQ_LINE
18
+
ioctl.
19
+
20
+
Groups:
21
+
KVM_DEV_ARM_VGIC_GRP_CTRL
22
+
Attributes:
23
+
24
+
KVM_DEV_ARM_VGIC_CTRL_INIT
25
+
request the initialization of the VGIC, no additional parameter in
26
+
kvm_device_attr.addr. Must be called after all VCPUs have been created.
27
+
28
+
KVM_DEV_ARM_VGIC_USERPSPACE_PPIs
29
+
request the mask of userspace-drivable PPIs. Only a subset of the PPIs can
30
+
be directly driven from userspace with GICv5, and the returned mask
31
+
informs userspace of which it is allowed to drive via KVM_IRQ_LINE.
32
+
33
+
Userspace must allocate and point to __u64[2] of data in
34
+
kvm_device_attr.addr. When this call returns, the provided memory will be
35
+
populated with the userspace PPI mask. The lower __u64 contains the mask
36
+
for the lower 64 PPIS, with the remaining 64 being in the second __u64.
37
+
38
+
This is a read-only attribute, and cannot be set. Attempts to set it are
39
+
rejected.
40
+
41
+
Errors:
42
+
43
+
======= ========================================================
44
+
-ENXIO VGIC not properly configured as required prior to calling
45
+
this attribute
46
+
-ENODEV no online VCPU
47
+
-ENOMEM memory shortage when allocating vgic internal data
48
+
-EFAULT Invalid guest ram access
49
+
-EBUSY One or more VCPUS are running
50
+
======= ========================================================
+1
Documentation/virt/kvm/devices/index.rst
+1
Documentation/virt/kvm/devices/index.rst
+3
-2
Documentation/virt/kvm/devices/vcpu.rst
+3
-2
Documentation/virt/kvm/devices/vcpu.rst
···
37
37
A value describing the PMUv3 (Performance Monitor Unit v3) overflow interrupt
38
38
number for this vcpu. This interrupt could be a PPI or SPI, but the interrupt
39
39
type must be same for each vcpu. As a PPI, the interrupt number is the same for
40
-
all vcpus, while as an SPI it must be a separate number per vcpu.
40
+
all vcpus, while as an SPI it must be a separate number per vcpu. For
41
+
GICv5-based guests, the architected PPI (23) must be used.
41
42
42
43
1.2 ATTRIBUTE: KVM_ARM_VCPU_PMU_V3_INIT
43
44
---------------------------------------
···
51
50
-EEXIST Interrupt number already used
52
51
-ENODEV PMUv3 not supported or GIC not initialized
53
52
-ENXIO PMUv3 not supported, missing VCPU feature or interrupt
54
-
number not set
53
+
number not set (non-GICv5 guests, only)
55
54
-EBUSY PMUv3 already initialized
56
55
======= ======================================================
57
56
+2
arch/arm64/include/asm/el2_setup.h
+2
arch/arm64/include/asm/el2_setup.h
+2
arch/arm64/include/asm/kvm_asm.h
+2
arch/arm64/include/asm/kvm_asm.h
···
81
81
__KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff,
82
82
__KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs,
83
83
__KVM_HOST_SMCCC_FUNC___vgic_v3_restore_vmcr_aprs,
84
+
__KVM_HOST_SMCCC_FUNC___vgic_v5_save_apr,
85
+
__KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr,
84
86
__KVM_HOST_SMCCC_FUNC___pkvm_reserve_vm,
85
87
__KVM_HOST_SMCCC_FUNC___pkvm_unreserve_vm,
86
88
__KVM_HOST_SMCCC_FUNC___pkvm_init_vm,
+34
arch/arm64/include/asm/kvm_host.h
+34
arch/arm64/include/asm/kvm_host.h
···
287
287
HDFGRTR2_GROUP,
288
288
HDFGWTR2_GROUP = HDFGRTR2_GROUP,
289
289
HFGITR2_GROUP,
290
+
ICH_HFGRTR_GROUP,
291
+
ICH_HFGWTR_GROUP = ICH_HFGRTR_GROUP,
292
+
ICH_HFGITR_GROUP,
290
293
291
294
/* Must be last */
292
295
__NR_FGT_GROUP_IDS__
···
623
620
VNCR(ICH_HCR_EL2),
624
621
VNCR(ICH_VMCR_EL2),
625
622
623
+
VNCR(ICH_HFGRTR_EL2),
624
+
VNCR(ICH_HFGWTR_EL2),
625
+
VNCR(ICH_HFGITR_EL2),
626
+
626
627
NR_SYS_REGS /* Nothing after this line! */
627
628
};
628
629
···
682
675
extern struct fgt_masks hfgitr2_masks;
683
676
extern struct fgt_masks hdfgrtr2_masks;
684
677
extern struct fgt_masks hdfgwtr2_masks;
678
+
extern struct fgt_masks ich_hfgrtr_masks;
679
+
extern struct fgt_masks ich_hfgwtr_masks;
680
+
extern struct fgt_masks ich_hfgitr_masks;
685
681
686
682
extern struct fgt_masks kvm_nvhe_sym(hfgrtr_masks);
687
683
extern struct fgt_masks kvm_nvhe_sym(hfgwtr_masks);
···
697
687
extern struct fgt_masks kvm_nvhe_sym(hfgitr2_masks);
698
688
extern struct fgt_masks kvm_nvhe_sym(hdfgrtr2_masks);
699
689
extern struct fgt_masks kvm_nvhe_sym(hdfgwtr2_masks);
690
+
extern struct fgt_masks kvm_nvhe_sym(ich_hfgrtr_masks);
691
+
extern struct fgt_masks kvm_nvhe_sym(ich_hfgwtr_masks);
692
+
extern struct fgt_masks kvm_nvhe_sym(ich_hfgitr_masks);
700
693
701
694
struct kvm_cpu_context {
702
695
struct user_pt_regs regs; /* sp = sp_el0 */
···
800
787
801
788
/* Last vgic_irq part of the AP list recorded in an LR */
802
789
struct vgic_irq *last_lr_irq;
790
+
791
+
/* PPI state tracking for GICv5-based guests */
792
+
struct {
793
+
/*
794
+
* For tracking the PPI pending state, we need both the entry
795
+
* state and exit state to correctly detect edges as it is
796
+
* possible that an interrupt has been injected in software in
797
+
* the interim.
798
+
*/
799
+
DECLARE_BITMAP(pendr_entry, VGIC_V5_NR_PRIVATE_IRQS);
800
+
DECLARE_BITMAP(pendr_exit, VGIC_V5_NR_PRIVATE_IRQS);
801
+
802
+
/* The saved state of the regs when leaving the guest */
803
+
DECLARE_BITMAP(activer_exit, VGIC_V5_NR_PRIVATE_IRQS);
804
+
} vgic_v5_ppi_state;
803
805
};
804
806
805
807
struct kvm_host_psci_config {
···
1690
1662
case HDFGRTR2_EL2:
1691
1663
case HDFGWTR2_EL2:
1692
1664
return HDFGRTR2_GROUP;
1665
+
case ICH_HFGRTR_EL2:
1666
+
case ICH_HFGWTR_EL2:
1667
+
return ICH_HFGRTR_GROUP;
1668
+
case ICH_HFGITR_EL2:
1669
+
return ICH_HFGITR_GROUP;
1693
1670
default:
1694
1671
BUILD_BUG_ON(1);
1695
1672
}
···
1709
1676
case HDFGWTR_EL2: \
1710
1677
case HFGWTR2_EL2: \
1711
1678
case HDFGWTR2_EL2: \
1679
+
case ICH_HFGWTR_EL2: \
1712
1680
p = &(vcpu)->arch.fgt[id].w; \
1713
1681
break; \
1714
1682
default: \
+10
arch/arm64/include/asm/kvm_hyp.h
+10
arch/arm64/include/asm/kvm_hyp.h
···
87
87
void __vgic_v3_restore_vmcr_aprs(struct vgic_v3_cpu_if *cpu_if);
88
88
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
89
89
90
+
/* GICv5 */
91
+
void __vgic_v5_save_apr(struct vgic_v5_cpu_if *cpu_if);
92
+
void __vgic_v5_restore_vmcr_apr(struct vgic_v5_cpu_if *cpu_if);
93
+
/* No hypercalls for the following */
94
+
void __vgic_v5_save_ppi_state(struct vgic_v5_cpu_if *cpu_if);
95
+
void __vgic_v5_restore_ppi_state(struct vgic_v5_cpu_if *cpu_if);
96
+
void __vgic_v5_save_state(struct vgic_v5_cpu_if *cpu_if);
97
+
void __vgic_v5_restore_state(struct vgic_v5_cpu_if *cpu_if);
98
+
90
99
#ifdef __KVM_NVHE_HYPERVISOR__
91
100
void __timer_enable_traps(struct kvm_vcpu *vcpu);
92
101
void __timer_disable_traps(struct kvm_vcpu *vcpu);
···
144
135
145
136
extern u64 kvm_nvhe_sym(id_aa64pfr0_el1_sys_val);
146
137
extern u64 kvm_nvhe_sym(id_aa64pfr1_el1_sys_val);
138
+
extern u64 kvm_nvhe_sym(id_aa64pfr2_el1_sys_val);
147
139
extern u64 kvm_nvhe_sym(id_aa64isar0_el1_sys_val);
148
140
extern u64 kvm_nvhe_sym(id_aa64isar1_el1_sys_val);
149
141
extern u64 kvm_nvhe_sym(id_aa64isar2_el1_sys_val);
+7
arch/arm64/include/asm/sysreg.h
+7
arch/arm64/include/asm/sysreg.h
···
1052
1052
#define GICV5_OP_GIC_CDPRI sys_insn(1, 0, 12, 1, 2)
1053
1053
#define GICV5_OP_GIC_CDRCFG sys_insn(1, 0, 12, 1, 5)
1054
1054
#define GICV5_OP_GICR_CDIA sys_insn(1, 0, 12, 3, 0)
1055
+
#define GICV5_OP_GICR_CDNMIA sys_insn(1, 0, 12, 3, 1)
1055
1056
1056
1057
/* Definitions for GIC CDAFF */
1057
1058
#define GICV5_GIC_CDAFF_IAFFID_MASK GENMASK_ULL(47, 32)
···
1098
1097
#define GICV5_GICR_CDIA_VALID(r) FIELD_GET(GICV5_GIC_CDIA_VALID_MASK, r)
1099
1098
#define GICV5_GIC_CDIA_TYPE_MASK GENMASK_ULL(31, 29)
1100
1099
#define GICV5_GIC_CDIA_ID_MASK GENMASK_ULL(23, 0)
1100
+
1101
+
/* Definitions for GICR CDNMIA */
1102
+
#define GICV5_GICR_CDNMIA_VALID_MASK BIT_ULL(32)
1103
+
#define GICV5_GICR_CDNMIA_VALID(r) FIELD_GET(GICV5_GICR_CDNMIA_VALID_MASK, r)
1104
+
#define GICV5_GICR_CDNMIA_TYPE_MASK GENMASK_ULL(31, 29)
1105
+
#define GICV5_GICR_CDNMIA_ID_MASK GENMASK_ULL(23, 0)
1101
1106
1102
1107
#define gicr_insn(insn) read_sysreg_s(GICV5_OP_GICR_##insn)
1103
1108
#define gic_insn(v, insn) write_sysreg_s(v, GICV5_OP_GIC_##insn)
+3
arch/arm64/include/asm/vncr_mapping.h
+3
arch/arm64/include/asm/vncr_mapping.h
+1
arch/arm64/include/uapi/asm/kvm.h
+1
arch/arm64/include/uapi/asm/kvm.h
+80
-28
arch/arm64/kvm/arch_timer.c
+80
-28
arch/arm64/kvm/arch_timer.c
···
56
56
.get_input_level = kvm_arch_timer_get_input_level,
57
57
};
58
58
59
+
static struct irq_ops arch_timer_irq_ops_vgic_v5 = {
60
+
.get_input_level = kvm_arch_timer_get_input_level,
61
+
.queue_irq_unlock = vgic_v5_ppi_queue_irq_unlock,
62
+
.set_direct_injection = vgic_v5_set_ppi_dvi,
63
+
};
64
+
59
65
static int nr_timers(struct kvm_vcpu *vcpu)
60
66
{
61
67
if (!vcpu_has_nv(vcpu))
···
182
176
map->emul_vtimer = NULL;
183
177
map->emul_ptimer = vcpu_ptimer(vcpu);
184
178
}
179
+
180
+
map->direct_vtimer->direct = true;
181
+
if (map->direct_ptimer)
182
+
map->direct_ptimer->direct = true;
185
183
186
184
trace_kvm_get_timer_map(vcpu->vcpu_id, map);
187
185
}
···
406
396
407
397
int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
408
398
{
409
-
return vcpu_has_wfit_active(vcpu) && wfit_delay_ns(vcpu) == 0;
399
+
struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
400
+
struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
401
+
402
+
return kvm_timer_should_fire(vtimer) || kvm_timer_should_fire(ptimer) ||
403
+
(vcpu_has_wfit_active(vcpu) && wfit_delay_ns(vcpu) == 0);
410
404
}
411
405
412
406
/*
···
459
445
timer_ctx->irq.level);
460
446
461
447
if (userspace_irqchip(vcpu->kvm))
448
+
return;
449
+
450
+
/* Skip injecting on GICv5 for directly injected (DVI'd) timers */
451
+
if (vgic_is_v5(vcpu->kvm) && timer_ctx->direct)
462
452
return;
463
453
464
454
kvm_vgic_inject_irq(vcpu->kvm, vcpu,
···
692
674
phys_active = kvm_vgic_map_is_active(vcpu, timer_irq(ctx));
693
675
694
676
phys_active |= ctx->irq.level;
677
+
phys_active |= vgic_is_v5(vcpu->kvm);
695
678
696
679
set_timer_irq_phys_active(ctx, phys_active);
697
680
}
···
759
740
760
741
ret = kvm_vgic_map_phys_irq(vcpu,
761
742
map->direct_vtimer->host_timer_irq,
762
-
timer_irq(map->direct_vtimer),
763
-
&arch_timer_irq_ops);
743
+
timer_irq(map->direct_vtimer));
764
744
WARN_ON_ONCE(ret);
765
745
ret = kvm_vgic_map_phys_irq(vcpu,
766
746
map->direct_ptimer->host_timer_irq,
767
-
timer_irq(map->direct_ptimer),
768
-
&arch_timer_irq_ops);
747
+
timer_irq(map->direct_ptimer));
769
748
WARN_ON_ONCE(ret);
770
749
}
771
750
}
···
881
864
get_timer_map(vcpu, &map);
882
865
883
866
if (static_branch_likely(&has_gic_active_state)) {
884
-
if (vcpu_has_nv(vcpu))
867
+
/* We don't do NV on GICv5, yet */
868
+
if (vcpu_has_nv(vcpu) && !vgic_is_v5(vcpu->kvm))
885
869
kvm_timer_vcpu_load_nested_switch(vcpu, &map);
886
870
887
871
kvm_timer_vcpu_load_gic(map.direct_vtimer);
···
952
934
953
935
if (kvm_vcpu_is_blocking(vcpu))
954
936
kvm_timer_blocking(vcpu);
937
+
938
+
if (vgic_is_v5(vcpu->kvm)) {
939
+
set_timer_irq_phys_active(map.direct_vtimer, false);
940
+
if (map.direct_ptimer)
941
+
set_timer_irq_phys_active(map.direct_ptimer, false);
942
+
}
955
943
}
956
944
957
945
void kvm_timer_sync_nested(struct kvm_vcpu *vcpu)
···
1121
1097
HRTIMER_MODE_ABS_HARD);
1122
1098
}
1123
1099
1100
+
/*
1101
+
* This is always called during kvm_arch_init_vm, but will also be
1102
+
* called from kvm_vgic_create if we have a vGICv5.
1103
+
*/
1124
1104
void kvm_timer_init_vm(struct kvm *kvm)
1125
1105
{
1106
+
/*
1107
+
* Set up the default PPIs - note that we adjust them based on
1108
+
* the model of the GIC as GICv5 uses a different way to
1109
+
* describing interrupts.
1110
+
*/
1126
1111
for (int i = 0; i < NR_KVM_TIMERS; i++)
1127
-
kvm->arch.timer_data.ppi[i] = default_ppi[i];
1112
+
kvm->arch.timer_data.ppi[i] = get_vgic_ppi(kvm, default_ppi[i]);
1128
1113
}
1129
1114
1130
1115
void kvm_timer_cpu_up(void)
···
1302
1269
1303
1270
static void timer_irq_eoi(struct irq_data *d)
1304
1271
{
1305
-
if (!irqd_is_forwarded_to_vcpu(d))
1272
+
/*
1273
+
* On a GICv5 host, we still need to call EOI on the parent for
1274
+
* PPIs. The host driver already handles irqs which are forwarded to
1275
+
* vcpus, and skips the GIC CDDI while still doing the GIC CDEOI. This
1276
+
* is required to emulate the EOIMode=1 on GICv5 hardware. Failure to
1277
+
* call EOI unsurprisingly results in *BAD* lock-ups.
1278
+
*/
1279
+
if (!irqd_is_forwarded_to_vcpu(d) ||
1280
+
kvm_vgic_global_state.type == VGIC_V5)
1306
1281
irq_chip_eoi_parent(d);
1307
1282
}
1308
1283
···
1374
1333
host_vtimer_irq = info->virtual_irq;
1375
1334
kvm_irq_fixup_flags(host_vtimer_irq, &host_vtimer_irq_flags);
1376
1335
1377
-
if (kvm_vgic_global_state.no_hw_deactivation) {
1336
+
if (kvm_vgic_global_state.no_hw_deactivation ||
1337
+
kvm_vgic_global_state.type == VGIC_V5) {
1378
1338
struct fwnode_handle *fwnode;
1379
1339
struct irq_data *data;
1380
1340
···
1393
1351
return -ENOMEM;
1394
1352
}
1395
1353
1396
-
arch_timer_irq_ops.flags |= VGIC_IRQ_SW_RESAMPLE;
1354
+
if (kvm_vgic_global_state.no_hw_deactivation)
1355
+
arch_timer_irq_ops.flags |= VGIC_IRQ_SW_RESAMPLE;
1397
1356
WARN_ON(irq_domain_push_irq(domain, host_vtimer_irq,
1398
1357
(void *)TIMER_VTIMER));
1399
1358
}
···
1545
1502
break;
1546
1503
1547
1504
/*
1548
-
* We know by construction that we only have PPIs, so
1549
-
* all values are less than 32.
1505
+
* We know by construction that we only have PPIs, so all values
1506
+
* are less than 32 for non-GICv5 VGICs. On GICv5, they are
1507
+
* architecturally defined to be under 32 too. However, we mask
1508
+
* off most of the bits as we might be presented with a GICv5
1509
+
* style PPI where the type is encoded in the top-bits.
1550
1510
*/
1551
-
ppis |= BIT(irq);
1511
+
ppis |= BIT(irq & 0x1f);
1552
1512
}
1553
1513
1554
1514
valid = hweight32(ppis) == nr_timers(vcpu);
···
1589
1543
{
1590
1544
struct arch_timer_cpu *timer = vcpu_timer(vcpu);
1591
1545
struct timer_map map;
1546
+
struct irq_ops *ops;
1592
1547
int ret;
1593
1548
1594
1549
if (timer->enabled)
···
1610
1563
1611
1564
get_timer_map(vcpu, &map);
1612
1565
1566
+
ops = vgic_is_v5(vcpu->kvm) ? &arch_timer_irq_ops_vgic_v5 :
1567
+
&arch_timer_irq_ops;
1568
+
1569
+
for (int i = 0; i < nr_timers(vcpu); i++)
1570
+
kvm_vgic_set_irq_ops(vcpu, timer_irq(vcpu_get_timer(vcpu, i)), ops);
1571
+
1613
1572
ret = kvm_vgic_map_phys_irq(vcpu,
1614
1573
map.direct_vtimer->host_timer_irq,
1615
-
timer_irq(map.direct_vtimer),
1616
-
&arch_timer_irq_ops);
1574
+
timer_irq(map.direct_vtimer));
1617
1575
if (ret)
1618
1576
return ret;
1619
1577
1620
-
if (map.direct_ptimer) {
1578
+
if (map.direct_ptimer)
1621
1579
ret = kvm_vgic_map_phys_irq(vcpu,
1622
1580
map.direct_ptimer->host_timer_irq,
1623
-
timer_irq(map.direct_ptimer),
1624
-
&arch_timer_irq_ops);
1625
-
}
1626
-
1581
+
timer_irq(map.direct_ptimer));
1627
1582
if (ret)
1628
1583
return ret;
1629
1584
···
1652
1603
if (get_user(irq, uaddr))
1653
1604
return -EFAULT;
1654
1605
1655
-
if (!(irq_is_ppi(irq)))
1606
+
if (!(irq_is_ppi(vcpu->kvm, irq)))
1656
1607
return -EINVAL;
1657
1608
1658
-
mutex_lock(&vcpu->kvm->arch.config_lock);
1609
+
guard(mutex)(&vcpu->kvm->arch.config_lock);
1659
1610
1660
1611
if (test_bit(KVM_ARCH_FLAG_TIMER_PPIS_IMMUTABLE,
1661
1612
&vcpu->kvm->arch.flags)) {
1662
-
ret = -EBUSY;
1663
-
goto out;
1613
+
return -EBUSY;
1664
1614
}
1665
1615
1666
1616
switch (attr->attr) {
···
1676
1628
idx = TIMER_HPTIMER;
1677
1629
break;
1678
1630
default:
1679
-
ret = -ENXIO;
1680
-
goto out;
1631
+
return -ENXIO;
1681
1632
}
1633
+
1634
+
/*
1635
+
* The PPIs for the Arch Timers are architecturally defined for
1636
+
* GICv5. Reject anything that changes them from the specified value.
1637
+
*/
1638
+
if (vgic_is_v5(vcpu->kvm) && vcpu->kvm->arch.timer_data.ppi[idx] != irq)
1639
+
return -EINVAL;
1682
1640
1683
1641
/*
1684
1642
* We cannot validate the IRQ unicity before we run, so take it at
···
1693
1639
*/
1694
1640
vcpu->kvm->arch.timer_data.ppi[idx] = irq;
1695
1641
1696
-
out:
1697
-
mutex_unlock(&vcpu->kvm->arch.config_lock);
1698
1642
return ret;
1699
1643
}
1700
1644
+40
-4
arch/arm64/kvm/arm.c
+40
-4
arch/arm64/kvm/arm.c
···
47
47
#include <kvm/arm_hypercalls.h>
48
48
#include <kvm/arm_pmu.h>
49
49
#include <kvm/arm_psci.h>
50
+
#include <kvm/arm_vgic.h>
51
+
52
+
#include <linux/irqchip/arm-gic-v5.h>
50
53
51
54
#include "sys_regs.h"
52
55
···
618
615
if (unlikely(kvm_wfi_trap_policy != KVM_WFX_NOTRAP_SINGLE_TASK))
619
616
return kvm_wfi_trap_policy == KVM_WFX_NOTRAP;
620
617
618
+
if (vgic_is_v5(vcpu->kvm))
619
+
return single_task_running();
620
+
621
621
return single_task_running() &&
622
622
vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3 &&
623
623
(atomic_read(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe.vlpi_count) ||
···
943
937
if (ret)
944
938
return ret;
945
939
}
940
+
941
+
ret = vgic_v5_finalize_ppi_state(kvm);
942
+
if (ret)
943
+
return ret;
946
944
947
945
if (is_protected_kvm_enabled()) {
948
946
ret = pkvm_create_hyp_vm(kvm);
···
1453
1443
int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level,
1454
1444
bool line_status)
1455
1445
{
1456
-
u32 irq = irq_level->irq;
1457
1446
unsigned int irq_type, vcpu_id, irq_num;
1458
1447
struct kvm_vcpu *vcpu = NULL;
1459
1448
bool level = irq_level->level;
1449
+
u32 irq = irq_level->irq;
1450
+
unsigned long *mask;
1460
1451
1461
1452
irq_type = (irq >> KVM_ARM_IRQ_TYPE_SHIFT) & KVM_ARM_IRQ_TYPE_MASK;
1462
1453
vcpu_id = (irq >> KVM_ARM_IRQ_VCPU_SHIFT) & KVM_ARM_IRQ_VCPU_MASK;
···
1487
1476
if (!vcpu)
1488
1477
return -EINVAL;
1489
1478
1490
-
if (irq_num < VGIC_NR_SGIS || irq_num >= VGIC_NR_PRIVATE_IRQS)
1479
+
if (vgic_is_v5(kvm)) {
1480
+
if (irq_num >= VGIC_V5_NR_PRIVATE_IRQS)
1481
+
return -EINVAL;
1482
+
1483
+
/*
1484
+
* Only allow PPIs that are explicitly exposed to
1485
+
* usespace to be driven via KVM_IRQ_LINE
1486
+
*/
1487
+
mask = kvm->arch.vgic.gicv5_vm.userspace_ppis;
1488
+
if (!test_bit(irq_num, mask))
1489
+
return -EINVAL;
1490
+
1491
+
/* Build a GICv5-style IntID here */
1492
+
irq_num = vgic_v5_make_ppi(irq_num);
1493
+
} else if (irq_num < VGIC_NR_SGIS ||
1494
+
irq_num >= VGIC_NR_PRIVATE_IRQS) {
1491
1495
return -EINVAL;
1496
+
}
1492
1497
1493
1498
return kvm_vgic_inject_irq(kvm, vcpu, irq_num, level, NULL);
1494
1499
case KVM_ARM_IRQ_TYPE_SPI:
1495
1500
if (!irqchip_in_kernel(kvm))
1496
1501
return -ENXIO;
1497
1502
1498
-
if (irq_num < VGIC_NR_PRIVATE_IRQS)
1499
-
return -EINVAL;
1503
+
if (vgic_is_v5(kvm)) {
1504
+
/* Build a GICv5-style IntID here */
1505
+
irq_num = vgic_v5_make_spi(irq_num);
1506
+
} else {
1507
+
if (irq_num < VGIC_NR_PRIVATE_IRQS)
1508
+
return -EINVAL;
1509
+
}
1500
1510
1501
1511
return kvm_vgic_inject_irq(kvm, NULL, irq_num, level, NULL);
1502
1512
}
···
2547
2515
{
2548
2516
kvm_nvhe_sym(id_aa64pfr0_el1_sys_val) = get_hyp_id_aa64pfr0_el1();
2549
2517
kvm_nvhe_sym(id_aa64pfr1_el1_sys_val) = read_sanitised_ftr_reg(SYS_ID_AA64PFR1_EL1);
2518
+
kvm_nvhe_sym(id_aa64pfr2_el1_sys_val) = read_sanitised_ftr_reg(SYS_ID_AA64PFR2_EL1);
2550
2519
kvm_nvhe_sym(id_aa64isar0_el1_sys_val) = read_sanitised_ftr_reg(SYS_ID_AA64ISAR0_EL1);
2551
2520
kvm_nvhe_sym(id_aa64isar1_el1_sys_val) = read_sanitised_ftr_reg(SYS_ID_AA64ISAR1_EL1);
2552
2521
kvm_nvhe_sym(id_aa64isar2_el1_sys_val) = read_sanitised_ftr_reg(SYS_ID_AA64ISAR2_EL1);
···
2570
2537
kvm_nvhe_sym(hfgitr2_masks) = hfgitr2_masks;
2571
2538
kvm_nvhe_sym(hdfgrtr2_masks)= hdfgrtr2_masks;
2572
2539
kvm_nvhe_sym(hdfgwtr2_masks)= hdfgwtr2_masks;
2540
+
kvm_nvhe_sym(ich_hfgrtr_masks) = ich_hfgrtr_masks;
2541
+
kvm_nvhe_sym(ich_hfgwtr_masks) = ich_hfgwtr_masks;
2542
+
kvm_nvhe_sym(ich_hfgitr_masks) = ich_hfgitr_masks;
2573
2543
2574
2544
/*
2575
2545
* Flush entire BSS since part of its data containing init symbols is read
+116
-7
arch/arm64/kvm/config.c
+116
-7
arch/arm64/kvm/config.c
···
225
225
#define FEAT_MTPMU ID_AA64DFR0_EL1, MTPMU, IMP
226
226
#define FEAT_HCX ID_AA64MMFR1_EL1, HCX, IMP
227
227
#define FEAT_S2PIE ID_AA64MMFR3_EL1, S2PIE, IMP
228
+
#define FEAT_GCIE ID_AA64PFR2_EL1, GCIE, IMP
228
229
229
230
static bool not_feat_aa64el3(struct kvm *kvm)
230
231
{
···
1278
1277
static const DECLARE_FEAT_MAP(vtcr_el2_desc, VTCR_EL2,
1279
1278
vtcr_el2_feat_map, FEAT_AA64EL2);
1280
1279
1280
+
static const struct reg_bits_to_feat_map ich_hfgrtr_feat_map[] = {
1281
+
NEEDS_FEAT(ICH_HFGRTR_EL2_ICC_APR_EL1 |
1282
+
ICH_HFGRTR_EL2_ICC_IDRn_EL1 |
1283
+
ICH_HFGRTR_EL2_ICC_CR0_EL1 |
1284
+
ICH_HFGRTR_EL2_ICC_HPPIR_EL1 |
1285
+
ICH_HFGRTR_EL2_ICC_PCR_EL1 |
1286
+
ICH_HFGRTR_EL2_ICC_ICSR_EL1 |
1287
+
ICH_HFGRTR_EL2_ICC_IAFFIDR_EL1 |
1288
+
ICH_HFGRTR_EL2_ICC_PPI_HMRn_EL1 |
1289
+
ICH_HFGRTR_EL2_ICC_PPI_ENABLERn_EL1 |
1290
+
ICH_HFGRTR_EL2_ICC_PPI_PENDRn_EL1 |
1291
+
ICH_HFGRTR_EL2_ICC_PPI_PRIORITYRn_EL1 |
1292
+
ICH_HFGRTR_EL2_ICC_PPI_ACTIVERn_EL1,
1293
+
FEAT_GCIE),
1294
+
};
1295
+
1296
+
static const DECLARE_FEAT_MAP_FGT(ich_hfgrtr_desc, ich_hfgrtr_masks,
1297
+
ich_hfgrtr_feat_map, FEAT_GCIE);
1298
+
1299
+
static const struct reg_bits_to_feat_map ich_hfgwtr_feat_map[] = {
1300
+
NEEDS_FEAT(ICH_HFGWTR_EL2_ICC_APR_EL1 |
1301
+
ICH_HFGWTR_EL2_ICC_CR0_EL1 |
1302
+
ICH_HFGWTR_EL2_ICC_PCR_EL1 |
1303
+
ICH_HFGWTR_EL2_ICC_ICSR_EL1 |
1304
+
ICH_HFGWTR_EL2_ICC_PPI_ENABLERn_EL1 |
1305
+
ICH_HFGWTR_EL2_ICC_PPI_PENDRn_EL1 |
1306
+
ICH_HFGWTR_EL2_ICC_PPI_PRIORITYRn_EL1 |
1307
+
ICH_HFGWTR_EL2_ICC_PPI_ACTIVERn_EL1,
1308
+
FEAT_GCIE),
1309
+
};
1310
+
1311
+
static const DECLARE_FEAT_MAP_FGT(ich_hfgwtr_desc, ich_hfgwtr_masks,
1312
+
ich_hfgwtr_feat_map, FEAT_GCIE);
1313
+
1314
+
static const struct reg_bits_to_feat_map ich_hfgitr_feat_map[] = {
1315
+
NEEDS_FEAT(ICH_HFGITR_EL2_GICCDEN |
1316
+
ICH_HFGITR_EL2_GICCDDIS |
1317
+
ICH_HFGITR_EL2_GICCDPRI |
1318
+
ICH_HFGITR_EL2_GICCDAFF |
1319
+
ICH_HFGITR_EL2_GICCDPEND |
1320
+
ICH_HFGITR_EL2_GICCDRCFG |
1321
+
ICH_HFGITR_EL2_GICCDHM |
1322
+
ICH_HFGITR_EL2_GICCDEOI |
1323
+
ICH_HFGITR_EL2_GICCDDI |
1324
+
ICH_HFGITR_EL2_GICRCDIA |
1325
+
ICH_HFGITR_EL2_GICRCDNMIA,
1326
+
FEAT_GCIE),
1327
+
};
1328
+
1329
+
static const DECLARE_FEAT_MAP_FGT(ich_hfgitr_desc, ich_hfgitr_masks,
1330
+
ich_hfgitr_feat_map, FEAT_GCIE);
1331
+
1281
1332
static void __init check_feat_map(const struct reg_bits_to_feat_map *map,
1282
1333
int map_size, u64 resx, const char *str)
1283
1334
{
···
1381
1328
check_reg_desc(&sctlr_el2_desc);
1382
1329
check_reg_desc(&mdcr_el2_desc);
1383
1330
check_reg_desc(&vtcr_el2_desc);
1331
+
check_reg_desc(&ich_hfgrtr_desc);
1332
+
check_reg_desc(&ich_hfgwtr_desc);
1333
+
check_reg_desc(&ich_hfgitr_desc);
1384
1334
}
1385
1335
1386
1336
static bool idreg_feat_match(struct kvm *kvm, const struct reg_bits_to_feat_map *map)
···
1516
1460
val |= compute_fgu_bits(kvm, &hdfgrtr2_desc);
1517
1461
val |= compute_fgu_bits(kvm, &hdfgwtr2_desc);
1518
1462
break;
1463
+
case ICH_HFGRTR_GROUP:
1464
+
val |= compute_fgu_bits(kvm, &ich_hfgrtr_desc);
1465
+
val |= compute_fgu_bits(kvm, &ich_hfgwtr_desc);
1466
+
break;
1467
+
case ICH_HFGITR_GROUP:
1468
+
val |= compute_fgu_bits(kvm, &ich_hfgitr_desc);
1469
+
break;
1519
1470
default:
1520
1471
BUG();
1521
1472
}
···
1594
1531
case VTCR_EL2:
1595
1532
resx = compute_reg_resx_bits(kvm, &vtcr_el2_desc, 0, 0);
1596
1533
break;
1534
+
case ICH_HFGRTR_EL2:
1535
+
resx = compute_reg_resx_bits(kvm, &ich_hfgrtr_desc, 0, 0);
1536
+
break;
1537
+
case ICH_HFGWTR_EL2:
1538
+
resx = compute_reg_resx_bits(kvm, &ich_hfgwtr_desc, 0, 0);
1539
+
break;
1540
+
case ICH_HFGITR_EL2:
1541
+
resx = compute_reg_resx_bits(kvm, &ich_hfgitr_desc, 0, 0);
1542
+
break;
1597
1543
default:
1598
1544
WARN_ON_ONCE(1);
1599
1545
resx = (typeof(resx)){};
···
1637
1565
return &hdfgrtr2_masks;
1638
1566
case HDFGWTR2_EL2:
1639
1567
return &hdfgwtr2_masks;
1568
+
case ICH_HFGRTR_EL2:
1569
+
return &ich_hfgrtr_masks;
1570
+
case ICH_HFGWTR_EL2:
1571
+
return &ich_hfgwtr_masks;
1572
+
case ICH_HFGITR_EL2:
1573
+
return &ich_hfgitr_masks;
1640
1574
default:
1641
1575
BUILD_BUG_ON(1);
1642
1576
}
···
1684
1606
*vcpu_fgt(vcpu, HDFGWTR_EL2) |= HDFGWTR_EL2_MDSCR_EL1;
1685
1607
}
1686
1608
1609
+
static void __compute_ich_hfgrtr(struct kvm_vcpu *vcpu)
1610
+
{
1611
+
__compute_fgt(vcpu, ICH_HFGRTR_EL2);
1612
+
1613
+
/*
1614
+
* ICC_IAFFIDR_EL1 *always* needs to be trapped when running a guest.
1615
+
*
1616
+
* We also trap accesses to ICC_IDR0_EL1 to allow us to completely hide
1617
+
* FEAT_GCIE_LEGACY from the guest, and to (potentially) present fewer
1618
+
* ID bits than the host supports.
1619
+
*/
1620
+
*vcpu_fgt(vcpu, ICH_HFGRTR_EL2) &= ~(ICH_HFGRTR_EL2_ICC_IAFFIDR_EL1 |
1621
+
ICH_HFGRTR_EL2_ICC_IDRn_EL1);
1622
+
}
1623
+
1624
+
static void __compute_ich_hfgwtr(struct kvm_vcpu *vcpu)
1625
+
{
1626
+
__compute_fgt(vcpu, ICH_HFGWTR_EL2);
1627
+
1628
+
/*
1629
+
* We present a different subset of PPIs the guest from what
1630
+
* exist in real hardware. We only trap writes, not reads.
1631
+
*/
1632
+
*vcpu_fgt(vcpu, ICH_HFGWTR_EL2) &= ~(ICH_HFGWTR_EL2_ICC_PPI_ENABLERn_EL1);
1633
+
}
1634
+
1687
1635
void kvm_vcpu_load_fgt(struct kvm_vcpu *vcpu)
1688
1636
{
1689
1637
if (!cpus_have_final_cap(ARM64_HAS_FGT))
···
1722
1618
__compute_hdfgwtr(vcpu);
1723
1619
__compute_fgt(vcpu, HAFGRTR_EL2);
1724
1620
1725
-
if (!cpus_have_final_cap(ARM64_HAS_FGT2))
1726
-
return;
1621
+
if (cpus_have_final_cap(ARM64_HAS_FGT2)) {
1622
+
__compute_fgt(vcpu, HFGRTR2_EL2);
1623
+
__compute_fgt(vcpu, HFGWTR2_EL2);
1624
+
__compute_fgt(vcpu, HFGITR2_EL2);
1625
+
__compute_fgt(vcpu, HDFGRTR2_EL2);
1626
+
__compute_fgt(vcpu, HDFGWTR2_EL2);
1627
+
}
1727
1628
1728
-
__compute_fgt(vcpu, HFGRTR2_EL2);
1729
-
__compute_fgt(vcpu, HFGWTR2_EL2);
1730
-
__compute_fgt(vcpu, HFGITR2_EL2);
1731
-
__compute_fgt(vcpu, HDFGRTR2_EL2);
1732
-
__compute_fgt(vcpu, HDFGWTR2_EL2);
1629
+
if (cpus_have_final_cap(ARM64_HAS_GICV5_CPUIF)) {
1630
+
__compute_ich_hfgrtr(vcpu);
1631
+
__compute_ich_hfgwtr(vcpu);
1632
+
__compute_fgt(vcpu, ICH_HFGITR_EL2);
1633
+
}
1733
1634
}
+68
arch/arm64/kvm/emulate-nested.c
+68
arch/arm64/kvm/emulate-nested.c
···
2053
2053
SR_FGT(SYS_AMEVCNTR0_EL0(2), HAFGRTR, AMEVCNTR02_EL0, 1),
2054
2054
SR_FGT(SYS_AMEVCNTR0_EL0(1), HAFGRTR, AMEVCNTR01_EL0, 1),
2055
2055
SR_FGT(SYS_AMEVCNTR0_EL0(0), HAFGRTR, AMEVCNTR00_EL0, 1),
2056
+
2057
+
/*
2058
+
* ICH_HFGRTR_EL2 & ICH_HFGWTR_EL2
2059
+
*/
2060
+
SR_FGT(SYS_ICC_APR_EL1, ICH_HFGRTR, ICC_APR_EL1, 0),
2061
+
SR_FGT(SYS_ICC_IDR0_EL1, ICH_HFGRTR, ICC_IDRn_EL1, 0),
2062
+
SR_FGT(SYS_ICC_CR0_EL1, ICH_HFGRTR, ICC_CR0_EL1, 0),
2063
+
SR_FGT(SYS_ICC_HPPIR_EL1, ICH_HFGRTR, ICC_HPPIR_EL1, 0),
2064
+
SR_FGT(SYS_ICC_PCR_EL1, ICH_HFGRTR, ICC_PCR_EL1, 0),
2065
+
SR_FGT(SYS_ICC_ICSR_EL1, ICH_HFGRTR, ICC_ICSR_EL1, 0),
2066
+
SR_FGT(SYS_ICC_IAFFIDR_EL1, ICH_HFGRTR, ICC_IAFFIDR_EL1, 0),
2067
+
SR_FGT(SYS_ICC_PPI_HMR0_EL1, ICH_HFGRTR, ICC_PPI_HMRn_EL1, 0),
2068
+
SR_FGT(SYS_ICC_PPI_HMR1_EL1, ICH_HFGRTR, ICC_PPI_HMRn_EL1, 0),
2069
+
SR_FGT(SYS_ICC_PPI_ENABLER0_EL1, ICH_HFGRTR, ICC_PPI_ENABLERn_EL1, 0),
2070
+
SR_FGT(SYS_ICC_PPI_ENABLER1_EL1, ICH_HFGRTR, ICC_PPI_ENABLERn_EL1, 0),
2071
+
SR_FGT(SYS_ICC_PPI_CPENDR0_EL1, ICH_HFGRTR, ICC_PPI_PENDRn_EL1, 0),
2072
+
SR_FGT(SYS_ICC_PPI_CPENDR1_EL1, ICH_HFGRTR, ICC_PPI_PENDRn_EL1, 0),
2073
+
SR_FGT(SYS_ICC_PPI_SPENDR0_EL1, ICH_HFGRTR, ICC_PPI_PENDRn_EL1, 0),
2074
+
SR_FGT(SYS_ICC_PPI_SPENDR1_EL1, ICH_HFGRTR, ICC_PPI_PENDRn_EL1, 0),
2075
+
SR_FGT(SYS_ICC_PPI_PRIORITYR0_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2076
+
SR_FGT(SYS_ICC_PPI_PRIORITYR1_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2077
+
SR_FGT(SYS_ICC_PPI_PRIORITYR2_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2078
+
SR_FGT(SYS_ICC_PPI_PRIORITYR3_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2079
+
SR_FGT(SYS_ICC_PPI_PRIORITYR4_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2080
+
SR_FGT(SYS_ICC_PPI_PRIORITYR5_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2081
+
SR_FGT(SYS_ICC_PPI_PRIORITYR6_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2082
+
SR_FGT(SYS_ICC_PPI_PRIORITYR7_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2083
+
SR_FGT(SYS_ICC_PPI_PRIORITYR8_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2084
+
SR_FGT(SYS_ICC_PPI_PRIORITYR9_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2085
+
SR_FGT(SYS_ICC_PPI_PRIORITYR10_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2086
+
SR_FGT(SYS_ICC_PPI_PRIORITYR11_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2087
+
SR_FGT(SYS_ICC_PPI_PRIORITYR12_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2088
+
SR_FGT(SYS_ICC_PPI_PRIORITYR13_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2089
+
SR_FGT(SYS_ICC_PPI_PRIORITYR14_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2090
+
SR_FGT(SYS_ICC_PPI_PRIORITYR15_EL1, ICH_HFGRTR, ICC_PPI_PRIORITYRn_EL1, 0),
2091
+
SR_FGT(SYS_ICC_PPI_CACTIVER0_EL1, ICH_HFGRTR, ICC_PPI_ACTIVERn_EL1, 0),
2092
+
SR_FGT(SYS_ICC_PPI_CACTIVER1_EL1, ICH_HFGRTR, ICC_PPI_ACTIVERn_EL1, 0),
2093
+
SR_FGT(SYS_ICC_PPI_SACTIVER0_EL1, ICH_HFGRTR, ICC_PPI_ACTIVERn_EL1, 0),
2094
+
SR_FGT(SYS_ICC_PPI_SACTIVER1_EL1, ICH_HFGRTR, ICC_PPI_ACTIVERn_EL1, 0),
2095
+
2096
+
/*
2097
+
* ICH_HFGITR_EL2
2098
+
*/
2099
+
SR_FGT(GICV5_OP_GIC_CDEN, ICH_HFGITR, GICCDEN, 0),
2100
+
SR_FGT(GICV5_OP_GIC_CDDIS, ICH_HFGITR, GICCDDIS, 0),
2101
+
SR_FGT(GICV5_OP_GIC_CDPRI, ICH_HFGITR, GICCDPRI, 0),
2102
+
SR_FGT(GICV5_OP_GIC_CDAFF, ICH_HFGITR, GICCDAFF, 0),
2103
+
SR_FGT(GICV5_OP_GIC_CDPEND, ICH_HFGITR, GICCDPEND, 0),
2104
+
SR_FGT(GICV5_OP_GIC_CDRCFG, ICH_HFGITR, GICCDRCFG, 0),
2105
+
SR_FGT(GICV5_OP_GIC_CDHM, ICH_HFGITR, GICCDHM, 0),
2106
+
SR_FGT(GICV5_OP_GIC_CDEOI, ICH_HFGITR, GICCDEOI, 0),
2107
+
SR_FGT(GICV5_OP_GIC_CDDI, ICH_HFGITR, GICCDDI, 0),
2108
+
SR_FGT(GICV5_OP_GICR_CDIA, ICH_HFGITR, GICRCDIA, 0),
2109
+
SR_FGT(GICV5_OP_GICR_CDNMIA, ICH_HFGITR, GICRCDNMIA, 0),
2056
2110
};
2057
2111
2058
2112
/*
···
2181
2127
FGT_MASKS(hfgitr2_masks, HFGITR2_EL2);
2182
2128
FGT_MASKS(hdfgrtr2_masks, HDFGRTR2_EL2);
2183
2129
FGT_MASKS(hdfgwtr2_masks, HDFGWTR2_EL2);
2130
+
FGT_MASKS(ich_hfgrtr_masks, ICH_HFGRTR_EL2);
2131
+
FGT_MASKS(ich_hfgwtr_masks, ICH_HFGWTR_EL2);
2132
+
FGT_MASKS(ich_hfgitr_masks, ICH_HFGITR_EL2);
2184
2133
2185
2134
static __init bool aggregate_fgt(union trap_config tc)
2186
2135
{
···
2217
2160
break;
2218
2161
case HFGITR2_GROUP:
2219
2162
rmasks = &hfgitr2_masks;
2163
+
wmasks = NULL;
2164
+
break;
2165
+
case ICH_HFGRTR_GROUP:
2166
+
rmasks = &ich_hfgrtr_masks;
2167
+
wmasks = &ich_hfgwtr_masks;
2168
+
break;
2169
+
case ICH_HFGITR_GROUP:
2170
+
rmasks = &ich_hfgitr_masks;
2220
2171
wmasks = NULL;
2221
2172
break;
2222
2173
}
···
2297
2232
&hfgitr2_masks,
2298
2233
&hdfgrtr2_masks,
2299
2234
&hdfgwtr2_masks,
2235
+
&ich_hfgrtr_masks,
2236
+
&ich_hfgwtr_masks,
2237
+
&ich_hfgitr_masks,
2300
2238
};
2301
2239
int err = 0;
2302
2240
+27
arch/arm64/kvm/hyp/include/hyp/switch.h
+27
arch/arm64/kvm/hyp/include/hyp/switch.h
···
233
233
__activate_fgt(hctxt, vcpu, HDFGWTR2_EL2);
234
234
}
235
235
236
+
static inline void __activate_traps_ich_hfgxtr(struct kvm_vcpu *vcpu)
237
+
{
238
+
struct kvm_cpu_context *hctxt = host_data_ptr(host_ctxt);
239
+
240
+
if (!cpus_have_final_cap(ARM64_HAS_GICV5_CPUIF))
241
+
return;
242
+
243
+
__activate_fgt(hctxt, vcpu, ICH_HFGRTR_EL2);
244
+
__activate_fgt(hctxt, vcpu, ICH_HFGWTR_EL2);
245
+
__activate_fgt(hctxt, vcpu, ICH_HFGITR_EL2);
246
+
}
247
+
236
248
#define __deactivate_fgt(htcxt, vcpu, reg) \
237
249
do { \
238
250
write_sysreg_s(ctxt_sys_reg(hctxt, reg), \
···
275
263
__deactivate_fgt(hctxt, vcpu, HFGITR2_EL2);
276
264
__deactivate_fgt(hctxt, vcpu, HDFGRTR2_EL2);
277
265
__deactivate_fgt(hctxt, vcpu, HDFGWTR2_EL2);
266
+
}
267
+
268
+
static inline void __deactivate_traps_ich_hfgxtr(struct kvm_vcpu *vcpu)
269
+
{
270
+
struct kvm_cpu_context *hctxt = host_data_ptr(host_ctxt);
271
+
272
+
if (!cpus_have_final_cap(ARM64_HAS_GICV5_CPUIF))
273
+
return;
274
+
275
+
__deactivate_fgt(hctxt, vcpu, ICH_HFGRTR_EL2);
276
+
__deactivate_fgt(hctxt, vcpu, ICH_HFGWTR_EL2);
277
+
__deactivate_fgt(hctxt, vcpu, ICH_HFGITR_EL2);
278
+
278
279
}
279
280
280
281
static inline void __activate_traps_mpam(struct kvm_vcpu *vcpu)
···
353
328
}
354
329
355
330
__activate_traps_hfgxtr(vcpu);
331
+
__activate_traps_ich_hfgxtr(vcpu);
356
332
__activate_traps_mpam(vcpu);
357
333
}
358
334
···
371
345
write_sysreg_s(ctxt_sys_reg(hctxt, HCRX_EL2), SYS_HCRX_EL2);
372
346
373
347
__deactivate_traps_hfgxtr(vcpu);
348
+
__deactivate_traps_ich_hfgxtr(vcpu);
374
349
__deactivate_traps_mpam();
375
350
}
376
351
+1
-1
arch/arm64/kvm/hyp/nvhe/Makefile
+1
-1
arch/arm64/kvm/hyp/nvhe/Makefile
···
26
26
hyp-main.o hyp-smp.o psci-relay.o early_alloc.o page_alloc.o \
27
27
cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o
28
28
hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \
29
-
../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o
29
+
../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o ../vgic-v5-sr.o
30
30
hyp-obj-y += ../../../kernel/smccc-call.o
31
31
hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o
32
32
hyp-obj-$(CONFIG_NVHE_EL2_TRACING) += clock.o trace.o events.o
+16
arch/arm64/kvm/hyp/nvhe/hyp-main.c
+16
arch/arm64/kvm/hyp/nvhe/hyp-main.c
···
650
650
trace_selftest(id);
651
651
}
652
652
653
+
static void handle___vgic_v5_save_apr(struct kvm_cpu_context *host_ctxt)
654
+
{
655
+
DECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt, 1);
656
+
657
+
__vgic_v5_save_apr(kern_hyp_va(cpu_if));
658
+
}
659
+
660
+
static void handle___vgic_v5_restore_vmcr_apr(struct kvm_cpu_context *host_ctxt)
661
+
{
662
+
DECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt, 1);
663
+
664
+
__vgic_v5_restore_vmcr_apr(kern_hyp_va(cpu_if));
665
+
}
666
+
653
667
typedef void (*hcall_t)(struct kvm_cpu_context *);
654
668
655
669
#define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x
···
697
683
HANDLE_FUNC(__kvm_timer_set_cntvoff),
698
684
HANDLE_FUNC(__vgic_v3_save_aprs),
699
685
HANDLE_FUNC(__vgic_v3_restore_vmcr_aprs),
686
+
HANDLE_FUNC(__vgic_v5_save_apr),
687
+
HANDLE_FUNC(__vgic_v5_restore_vmcr_apr),
700
688
HANDLE_FUNC(__pkvm_reserve_vm),
701
689
HANDLE_FUNC(__pkvm_unreserve_vm),
702
690
HANDLE_FUNC(__pkvm_init_vm),
+15
arch/arm64/kvm/hyp/nvhe/switch.c
+15
arch/arm64/kvm/hyp/nvhe/switch.c
···
44
44
struct fgt_masks hfgitr2_masks;
45
45
struct fgt_masks hdfgrtr2_masks;
46
46
struct fgt_masks hdfgwtr2_masks;
47
+
struct fgt_masks ich_hfgrtr_masks;
48
+
struct fgt_masks ich_hfgwtr_masks;
49
+
struct fgt_masks ich_hfgitr_masks;
47
50
48
51
extern void kvm_nvhe_prepare_backtrace(unsigned long fp, unsigned long pc);
49
52
···
113
110
/* Save VGICv3 state on non-VHE systems */
114
111
static void __hyp_vgic_save_state(struct kvm_vcpu *vcpu)
115
112
{
113
+
if (vgic_is_v5(kern_hyp_va(vcpu->kvm))) {
114
+
__vgic_v5_save_state(&vcpu->arch.vgic_cpu.vgic_v5);
115
+
__vgic_v5_save_ppi_state(&vcpu->arch.vgic_cpu.vgic_v5);
116
+
return;
117
+
}
118
+
116
119
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
117
120
__vgic_v3_save_state(&vcpu->arch.vgic_cpu.vgic_v3);
118
121
__vgic_v3_deactivate_traps(&vcpu->arch.vgic_cpu.vgic_v3);
···
128
119
/* Restore VGICv3 state on non-VHE systems */
129
120
static void __hyp_vgic_restore_state(struct kvm_vcpu *vcpu)
130
121
{
122
+
if (vgic_is_v5(kern_hyp_va(vcpu->kvm))) {
123
+
__vgic_v5_restore_state(&vcpu->arch.vgic_cpu.vgic_v5);
124
+
__vgic_v5_restore_ppi_state(&vcpu->arch.vgic_cpu.vgic_v5);
125
+
return;
126
+
}
127
+
131
128
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif)) {
132
129
__vgic_v3_activate_traps(&vcpu->arch.vgic_cpu.vgic_v3);
133
130
__vgic_v3_restore_state(&vcpu->arch.vgic_cpu.vgic_v3);
+8
arch/arm64/kvm/hyp/nvhe/sys_regs.c
+8
arch/arm64/kvm/hyp/nvhe/sys_regs.c
···
20
20
*/
21
21
u64 id_aa64pfr0_el1_sys_val;
22
22
u64 id_aa64pfr1_el1_sys_val;
23
+
u64 id_aa64pfr2_el1_sys_val;
23
24
u64 id_aa64isar0_el1_sys_val;
24
25
u64 id_aa64isar1_el1_sys_val;
25
26
u64 id_aa64isar2_el1_sys_val;
···
106
105
MAX_FEAT(ID_AA64PFR1_EL1, BT, IMP),
107
106
MAX_FEAT(ID_AA64PFR1_EL1, SSBS, SSBS2),
108
107
MAX_FEAT_ENUM(ID_AA64PFR1_EL1, MTE_frac, NI),
108
+
FEAT_END
109
+
};
110
+
111
+
static const struct pvm_ftr_bits pvmid_aa64pfr2[] = {
112
+
MAX_FEAT(ID_AA64PFR2_EL1, GCIE, NI),
109
113
FEAT_END
110
114
};
111
115
···
227
221
return get_restricted_features(vcpu, id_aa64pfr0_el1_sys_val, pvmid_aa64pfr0);
228
222
case SYS_ID_AA64PFR1_EL1:
229
223
return get_restricted_features(vcpu, id_aa64pfr1_el1_sys_val, pvmid_aa64pfr1);
224
+
case SYS_ID_AA64PFR2_EL1:
225
+
return get_restricted_features(vcpu, id_aa64pfr2_el1_sys_val, pvmid_aa64pfr2);
230
226
case SYS_ID_AA64ISAR0_EL1:
231
227
return id_aa64isar0_el1_sys_val;
232
228
case SYS_ID_AA64ISAR1_EL1:
+166
arch/arm64/kvm/hyp/vgic-v5-sr.c
+166
arch/arm64/kvm/hyp/vgic-v5-sr.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-only
2
+
/*
3
+
* Copyright (C) 2025, 2026 - Arm Ltd
4
+
*/
5
+
6
+
#include <linux/irqchip/arm-gic-v5.h>
7
+
8
+
#include <asm/kvm_hyp.h>
9
+
10
+
void __vgic_v5_save_apr(struct vgic_v5_cpu_if *cpu_if)
11
+
{
12
+
cpu_if->vgic_apr = read_sysreg_s(SYS_ICH_APR_EL2);
13
+
}
14
+
15
+
static void __vgic_v5_compat_mode_disable(void)
16
+
{
17
+
sysreg_clear_set_s(SYS_ICH_VCTLR_EL2, ICH_VCTLR_EL2_V3, 0);
18
+
isb();
19
+
}
20
+
21
+
void __vgic_v5_restore_vmcr_apr(struct vgic_v5_cpu_if *cpu_if)
22
+
{
23
+
__vgic_v5_compat_mode_disable();
24
+
25
+
write_sysreg_s(cpu_if->vgic_vmcr, SYS_ICH_VMCR_EL2);
26
+
write_sysreg_s(cpu_if->vgic_apr, SYS_ICH_APR_EL2);
27
+
}
28
+
29
+
void __vgic_v5_save_ppi_state(struct vgic_v5_cpu_if *cpu_if)
30
+
{
31
+
/*
32
+
* The following code assumes that the bitmap storage that we have for
33
+
* PPIs is either 64 (architected PPIs, only) or 128 bits (architected &
34
+
* impdef PPIs).
35
+
*/
36
+
BUILD_BUG_ON(VGIC_V5_NR_PRIVATE_IRQS % 64);
37
+
38
+
bitmap_write(host_data_ptr(vgic_v5_ppi_state)->activer_exit,
39
+
read_sysreg_s(SYS_ICH_PPI_ACTIVER0_EL2), 0, 64);
40
+
bitmap_write(host_data_ptr(vgic_v5_ppi_state)->pendr_exit,
41
+
read_sysreg_s(SYS_ICH_PPI_PENDR0_EL2), 0, 64);
42
+
43
+
cpu_if->vgic_ppi_priorityr[0] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR0_EL2);
44
+
cpu_if->vgic_ppi_priorityr[1] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR1_EL2);
45
+
cpu_if->vgic_ppi_priorityr[2] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR2_EL2);
46
+
cpu_if->vgic_ppi_priorityr[3] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR3_EL2);
47
+
cpu_if->vgic_ppi_priorityr[4] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR4_EL2);
48
+
cpu_if->vgic_ppi_priorityr[5] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR5_EL2);
49
+
cpu_if->vgic_ppi_priorityr[6] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR6_EL2);
50
+
cpu_if->vgic_ppi_priorityr[7] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR7_EL2);
51
+
52
+
if (VGIC_V5_NR_PRIVATE_IRQS == 128) {
53
+
bitmap_write(host_data_ptr(vgic_v5_ppi_state)->activer_exit,
54
+
read_sysreg_s(SYS_ICH_PPI_ACTIVER1_EL2), 64, 64);
55
+
bitmap_write(host_data_ptr(vgic_v5_ppi_state)->pendr_exit,
56
+
read_sysreg_s(SYS_ICH_PPI_PENDR1_EL2), 64, 64);
57
+
58
+
cpu_if->vgic_ppi_priorityr[8] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR8_EL2);
59
+
cpu_if->vgic_ppi_priorityr[9] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR9_EL2);
60
+
cpu_if->vgic_ppi_priorityr[10] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR10_EL2);
61
+
cpu_if->vgic_ppi_priorityr[11] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR11_EL2);
62
+
cpu_if->vgic_ppi_priorityr[12] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR12_EL2);
63
+
cpu_if->vgic_ppi_priorityr[13] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR13_EL2);
64
+
cpu_if->vgic_ppi_priorityr[14] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR14_EL2);
65
+
cpu_if->vgic_ppi_priorityr[15] = read_sysreg_s(SYS_ICH_PPI_PRIORITYR15_EL2);
66
+
}
67
+
68
+
/* Now that we are done, disable DVI */
69
+
write_sysreg_s(0, SYS_ICH_PPI_DVIR0_EL2);
70
+
write_sysreg_s(0, SYS_ICH_PPI_DVIR1_EL2);
71
+
}
72
+
73
+
void __vgic_v5_restore_ppi_state(struct vgic_v5_cpu_if *cpu_if)
74
+
{
75
+
DECLARE_BITMAP(pendr, VGIC_V5_NR_PRIVATE_IRQS);
76
+
77
+
/* We assume 64 or 128 PPIs - see above comment */
78
+
BUILD_BUG_ON(VGIC_V5_NR_PRIVATE_IRQS % 64);
79
+
80
+
/* Enable DVI so that the guest's interrupt config takes over */
81
+
write_sysreg_s(bitmap_read(cpu_if->vgic_ppi_dvir, 0, 64),
82
+
SYS_ICH_PPI_DVIR0_EL2);
83
+
84
+
write_sysreg_s(bitmap_read(cpu_if->vgic_ppi_activer, 0, 64),
85
+
SYS_ICH_PPI_ACTIVER0_EL2);
86
+
write_sysreg_s(bitmap_read(cpu_if->vgic_ppi_enabler, 0, 64),
87
+
SYS_ICH_PPI_ENABLER0_EL2);
88
+
89
+
/* Update the pending state of the NON-DVI'd PPIs, only */
90
+
bitmap_andnot(pendr, host_data_ptr(vgic_v5_ppi_state)->pendr_entry,
91
+
cpu_if->vgic_ppi_dvir, VGIC_V5_NR_PRIVATE_IRQS);
92
+
write_sysreg_s(bitmap_read(pendr, 0, 64), SYS_ICH_PPI_PENDR0_EL2);
93
+
94
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[0],
95
+
SYS_ICH_PPI_PRIORITYR0_EL2);
96
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[1],
97
+
SYS_ICH_PPI_PRIORITYR1_EL2);
98
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[2],
99
+
SYS_ICH_PPI_PRIORITYR2_EL2);
100
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[3],
101
+
SYS_ICH_PPI_PRIORITYR3_EL2);
102
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[4],
103
+
SYS_ICH_PPI_PRIORITYR4_EL2);
104
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[5],
105
+
SYS_ICH_PPI_PRIORITYR5_EL2);
106
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[6],
107
+
SYS_ICH_PPI_PRIORITYR6_EL2);
108
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[7],
109
+
SYS_ICH_PPI_PRIORITYR7_EL2);
110
+
111
+
if (VGIC_V5_NR_PRIVATE_IRQS == 128) {
112
+
/* Enable DVI so that the guest's interrupt config takes over */
113
+
write_sysreg_s(bitmap_read(cpu_if->vgic_ppi_dvir, 64, 64),
114
+
SYS_ICH_PPI_DVIR1_EL2);
115
+
116
+
write_sysreg_s(bitmap_read(cpu_if->vgic_ppi_activer, 64, 64),
117
+
SYS_ICH_PPI_ACTIVER1_EL2);
118
+
write_sysreg_s(bitmap_read(cpu_if->vgic_ppi_enabler, 64, 64),
119
+
SYS_ICH_PPI_ENABLER1_EL2);
120
+
write_sysreg_s(bitmap_read(pendr, 64, 64),
121
+
SYS_ICH_PPI_PENDR1_EL2);
122
+
123
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[8],
124
+
SYS_ICH_PPI_PRIORITYR8_EL2);
125
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[9],
126
+
SYS_ICH_PPI_PRIORITYR9_EL2);
127
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[10],
128
+
SYS_ICH_PPI_PRIORITYR10_EL2);
129
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[11],
130
+
SYS_ICH_PPI_PRIORITYR11_EL2);
131
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[12],
132
+
SYS_ICH_PPI_PRIORITYR12_EL2);
133
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[13],
134
+
SYS_ICH_PPI_PRIORITYR13_EL2);
135
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[14],
136
+
SYS_ICH_PPI_PRIORITYR14_EL2);
137
+
write_sysreg_s(cpu_if->vgic_ppi_priorityr[15],
138
+
SYS_ICH_PPI_PRIORITYR15_EL2);
139
+
} else {
140
+
write_sysreg_s(0, SYS_ICH_PPI_DVIR1_EL2);
141
+
142
+
write_sysreg_s(0, SYS_ICH_PPI_ACTIVER1_EL2);
143
+
write_sysreg_s(0, SYS_ICH_PPI_ENABLER1_EL2);
144
+
write_sysreg_s(0, SYS_ICH_PPI_PENDR1_EL2);
145
+
146
+
write_sysreg_s(0, SYS_ICH_PPI_PRIORITYR8_EL2);
147
+
write_sysreg_s(0, SYS_ICH_PPI_PRIORITYR9_EL2);
148
+
write_sysreg_s(0, SYS_ICH_PPI_PRIORITYR10_EL2);
149
+
write_sysreg_s(0, SYS_ICH_PPI_PRIORITYR11_EL2);
150
+
write_sysreg_s(0, SYS_ICH_PPI_PRIORITYR12_EL2);
151
+
write_sysreg_s(0, SYS_ICH_PPI_PRIORITYR13_EL2);
152
+
write_sysreg_s(0, SYS_ICH_PPI_PRIORITYR14_EL2);
153
+
write_sysreg_s(0, SYS_ICH_PPI_PRIORITYR15_EL2);
154
+
}
155
+
}
156
+
157
+
void __vgic_v5_save_state(struct vgic_v5_cpu_if *cpu_if)
158
+
{
159
+
cpu_if->vgic_vmcr = read_sysreg_s(SYS_ICH_VMCR_EL2);
160
+
cpu_if->vgic_icsr = read_sysreg_s(SYS_ICC_ICSR_EL1);
161
+
}
162
+
163
+
void __vgic_v5_restore_state(struct vgic_v5_cpu_if *cpu_if)
164
+
{
165
+
write_sysreg_s(cpu_if->vgic_icsr, SYS_ICC_ICSR_EL1);
166
+
}
+1
-1
arch/arm64/kvm/hyp/vhe/Makefile
+1
-1
arch/arm64/kvm/hyp/vhe/Makefile
+5
arch/arm64/kvm/nested.c
+5
arch/arm64/kvm/nested.c
···
1558
1558
ID_AA64PFR1_EL1_MTE);
1559
1559
break;
1560
1560
1561
+
case SYS_ID_AA64PFR2_EL1:
1562
+
/* GICv5 is not yet supported for NV */
1563
+
val &= ~ID_AA64PFR2_EL1_GCIE;
1564
+
break;
1565
+
1561
1566
case SYS_ID_AA64MMFR0_EL1:
1562
1567
/* Hide ExS, Secure Memory */
1563
1568
val &= ~(ID_AA64MMFR0_EL1_EXS |
+15
-5
arch/arm64/kvm/pmu-emul.c
+15
-5
arch/arm64/kvm/pmu-emul.c
···
939
939
* number against the dimensions of the vgic and make sure
940
940
* it's valid.
941
941
*/
942
-
if (!irq_is_ppi(irq) && !vgic_valid_spi(vcpu->kvm, irq))
942
+
if (!irq_is_ppi(vcpu->kvm, irq) &&
943
+
!vgic_valid_spi(vcpu->kvm, irq))
943
944
return -EINVAL;
944
945
} else if (kvm_arm_pmu_irq_initialized(vcpu)) {
945
946
return -EINVAL;
···
962
961
if (!vgic_initialized(vcpu->kvm))
963
962
return -ENODEV;
964
963
965
-
if (!kvm_arm_pmu_irq_initialized(vcpu))
966
-
return -ENXIO;
964
+
if (!kvm_arm_pmu_irq_initialized(vcpu)) {
965
+
if (!vgic_is_v5(vcpu->kvm))
966
+
return -ENXIO;
967
+
968
+
/* Use the architected irq number for GICv5. */
969
+
vcpu->arch.pmu.irq_num = KVM_ARMV8_PMU_GICV5_IRQ;
970
+
}
967
971
968
972
ret = kvm_vgic_set_owner(vcpu, vcpu->arch.pmu.irq_num,
969
973
&vcpu->arch.pmu);
···
993
987
unsigned long i;
994
988
struct kvm_vcpu *vcpu;
995
989
990
+
/* On GICv5, the PMUIRQ is architecturally mandated to be PPI 23 */
991
+
if (vgic_is_v5(kvm) && irq != KVM_ARMV8_PMU_GICV5_IRQ)
992
+
return false;
993
+
996
994
kvm_for_each_vcpu(i, vcpu, kvm) {
997
995
if (!kvm_arm_pmu_irq_initialized(vcpu))
998
996
continue;
999
997
1000
-
if (irq_is_ppi(irq)) {
998
+
if (irq_is_ppi(vcpu->kvm, irq)) {
1001
999
if (vcpu->arch.pmu.irq_num != irq)
1002
1000
return false;
1003
1001
} else {
···
1152
1142
return -EFAULT;
1153
1143
1154
1144
/* The PMU overflow interrupt can be a PPI or a valid SPI. */
1155
-
if (!(irq_is_ppi(irq) || irq_is_spi(irq)))
1145
+
if (!(irq_is_ppi(vcpu->kvm, irq) || irq_is_spi(vcpu->kvm, irq)))
1156
1146
return -EINVAL;
1157
1147
1158
1148
if (!pmu_irq_is_valid(kvm, irq))
+158
-18
arch/arm64/kvm/sys_regs.c
+158
-18
arch/arm64/kvm/sys_regs.c
···
681
681
return true;
682
682
}
683
683
684
+
static bool access_gicv5_idr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
685
+
const struct sys_reg_desc *r)
686
+
{
687
+
if (p->is_write)
688
+
return undef_access(vcpu, p, r);
689
+
690
+
/*
691
+
* Expose KVM's priority- and ID-bits to the guest, but not GCIE_LEGACY.
692
+
*
693
+
* Note: for GICv5 the mimic the way that the num_pri_bits and
694
+
* num_id_bits fields are used with GICv3:
695
+
* - num_pri_bits stores the actual number of priority bits, whereas the
696
+
* register field stores num_pri_bits - 1.
697
+
* - num_id_bits stores the raw field value, which is 0b0000 for 16 bits
698
+
* and 0b0001 for 24 bits.
699
+
*/
700
+
p->regval = FIELD_PREP(ICC_IDR0_EL1_PRI_BITS, vcpu->arch.vgic_cpu.num_pri_bits - 1) |
701
+
FIELD_PREP(ICC_IDR0_EL1_ID_BITS, vcpu->arch.vgic_cpu.num_id_bits);
702
+
703
+
return true;
704
+
}
705
+
706
+
static bool access_gicv5_iaffid(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
707
+
const struct sys_reg_desc *r)
708
+
{
709
+
if (p->is_write)
710
+
return undef_access(vcpu, p, r);
711
+
712
+
/*
713
+
* For GICv5 VMs, the IAFFID value is the same as the VPE ID. The VPE ID
714
+
* is the same as the VCPU's ID.
715
+
*/
716
+
p->regval = FIELD_PREP(ICC_IAFFIDR_EL1_IAFFID, vcpu->vcpu_id);
717
+
718
+
return true;
719
+
}
720
+
721
+
static bool access_gicv5_ppi_enabler(struct kvm_vcpu *vcpu,
722
+
struct sys_reg_params *p,
723
+
const struct sys_reg_desc *r)
724
+
{
725
+
unsigned long *mask = vcpu->kvm->arch.vgic.gicv5_vm.vgic_ppi_mask;
726
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
727
+
int i;
728
+
729
+
/* We never expect to get here with a read! */
730
+
if (WARN_ON_ONCE(!p->is_write))
731
+
return undef_access(vcpu, p, r);
732
+
733
+
/*
734
+
* If we're only handling architected PPIs and the guest writes to the
735
+
* enable for the non-architected PPIs, we just return as there's
736
+
* nothing to do at all. We don't even allocate the storage for them in
737
+
* this case.
738
+
*/
739
+
if (VGIC_V5_NR_PRIVATE_IRQS == 64 && p->Op2 % 2)
740
+
return true;
741
+
742
+
/*
743
+
* Merge the raw guest write into out bitmap at an offset of either 0 or
744
+
* 64, then and it with our PPI mask.
745
+
*/
746
+
bitmap_write(cpu_if->vgic_ppi_enabler, p->regval, 64 * (p->Op2 % 2), 64);
747
+
bitmap_and(cpu_if->vgic_ppi_enabler, cpu_if->vgic_ppi_enabler, mask,
748
+
VGIC_V5_NR_PRIVATE_IRQS);
749
+
750
+
/*
751
+
* Sync the change in enable states to the vgic_irqs. We consider all
752
+
* PPIs as we don't expose many to the guest.
753
+
*/
754
+
for_each_set_bit(i, mask, VGIC_V5_NR_PRIVATE_IRQS) {
755
+
u32 intid = vgic_v5_make_ppi(i);
756
+
struct vgic_irq *irq;
757
+
758
+
irq = vgic_get_vcpu_irq(vcpu, intid);
759
+
760
+
scoped_guard(raw_spinlock_irqsave, &irq->irq_lock)
761
+
irq->enabled = test_bit(i, cpu_if->vgic_ppi_enabler);
762
+
763
+
vgic_put_irq(vcpu->kvm, irq);
764
+
}
765
+
766
+
return true;
767
+
}
768
+
684
769
static bool trap_raz_wi(struct kvm_vcpu *vcpu,
685
770
struct sys_reg_params *p,
686
771
const struct sys_reg_desc *r)
···
1843
1758
1844
1759
static u64 sanitise_id_aa64pfr0_el1(const struct kvm_vcpu *vcpu, u64 val);
1845
1760
static u64 sanitise_id_aa64pfr1_el1(const struct kvm_vcpu *vcpu, u64 val);
1761
+
static u64 sanitise_id_aa64pfr2_el1(const struct kvm_vcpu *vcpu, u64 val);
1846
1762
static u64 sanitise_id_aa64dfr0_el1(const struct kvm_vcpu *vcpu, u64 val);
1847
1763
1848
1764
/* Read a sanitised cpufeature ID register by sys_reg_desc */
···
1869
1783
val = sanitise_id_aa64pfr1_el1(vcpu, val);
1870
1784
break;
1871
1785
case SYS_ID_AA64PFR2_EL1:
1872
-
val &= ID_AA64PFR2_EL1_FPMR |
1873
-
(kvm_has_mte(vcpu->kvm) ?
1874
-
ID_AA64PFR2_EL1_MTEFAR | ID_AA64PFR2_EL1_MTESTOREONLY :
1875
-
0);
1786
+
val = sanitise_id_aa64pfr2_el1(vcpu, val);
1876
1787
break;
1877
1788
case SYS_ID_AA64ISAR1_EL1:
1878
1789
if (!vcpu_has_ptrauth(vcpu))
···
2068
1985
val |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, CSV3, IMP);
2069
1986
}
2070
1987
2071
-
if (vgic_is_v3(vcpu->kvm)) {
1988
+
if (vgic_host_has_gicv3()) {
2072
1989
val &= ~ID_AA64PFR0_EL1_GIC_MASK;
2073
1990
val |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, GIC, IMP);
2074
1991
}
···
2106
2023
val &= ~ID_AA64PFR1_EL1_MTEX;
2107
2024
val &= ~ID_AA64PFR1_EL1_PFAR;
2108
2025
val &= ~ID_AA64PFR1_EL1_MPAM_frac;
2026
+
2027
+
return val;
2028
+
}
2029
+
2030
+
static u64 sanitise_id_aa64pfr2_el1(const struct kvm_vcpu *vcpu, u64 val)
2031
+
{
2032
+
val &= ID_AA64PFR2_EL1_FPMR |
2033
+
ID_AA64PFR2_EL1_MTEFAR |
2034
+
ID_AA64PFR2_EL1_MTESTOREONLY;
2035
+
2036
+
if (!kvm_has_mte(vcpu->kvm)) {
2037
+
val &= ~ID_AA64PFR2_EL1_MTEFAR;
2038
+
val &= ~ID_AA64PFR2_EL1_MTESTOREONLY;
2039
+
}
2040
+
2041
+
if (vgic_host_has_gicv5())
2042
+
val |= SYS_FIELD_PREP_ENUM(ID_AA64PFR2_EL1, GCIE, IMP);
2109
2043
2110
2044
return val;
2111
2045
}
···
2277
2177
(vcpu_has_nv(vcpu) && !FIELD_GET(ID_AA64PFR0_EL1_EL2, user_val)))
2278
2178
return -EINVAL;
2279
2179
2280
-
/*
2281
-
* If we are running on a GICv5 host and support FEAT_GCIE_LEGACY, then
2282
-
* we support GICv3. Fail attempts to do anything but set that to IMP.
2283
-
*/
2284
-
if (vgic_is_v3_compat(vcpu->kvm) &&
2285
-
FIELD_GET(ID_AA64PFR0_EL1_GIC_MASK, user_val) != ID_AA64PFR0_EL1_GIC_IMP)
2286
-
return -EINVAL;
2287
-
2288
2180
return set_id_reg(vcpu, rd, user_val);
2289
2181
}
2290
2182
···
2313
2221
user_val |= hw_val & ID_AA64PFR1_EL1_MTE_frac_MASK;
2314
2222
}
2315
2223
2224
+
return set_id_reg(vcpu, rd, user_val);
2225
+
}
2226
+
2227
+
static int set_id_aa64pfr2_el1(struct kvm_vcpu *vcpu,
2228
+
const struct sys_reg_desc *rd, u64 user_val)
2229
+
{
2316
2230
return set_id_reg(vcpu, rd, user_val);
2317
2231
}
2318
2232
···
3303
3205
ID_AA64PFR1_EL1_RES0 |
3304
3206
ID_AA64PFR1_EL1_MPAM_frac |
3305
3207
ID_AA64PFR1_EL1_MTE)),
3306
-
ID_WRITABLE(ID_AA64PFR2_EL1,
3307
-
ID_AA64PFR2_EL1_FPMR |
3308
-
ID_AA64PFR2_EL1_MTEFAR |
3309
-
ID_AA64PFR2_EL1_MTESTOREONLY),
3208
+
ID_FILTERED(ID_AA64PFR2_EL1, id_aa64pfr2_el1,
3209
+
~(ID_AA64PFR2_EL1_FPMR |
3210
+
ID_AA64PFR2_EL1_MTEFAR |
3211
+
ID_AA64PFR2_EL1_MTESTOREONLY |
3212
+
ID_AA64PFR2_EL1_GCIE)),
3310
3213
ID_UNALLOCATED(4,3),
3311
3214
ID_WRITABLE(ID_AA64ZFR0_EL1, ~ID_AA64ZFR0_EL1_RES0),
3312
3215
ID_HIDDEN(ID_AA64SMFR0_EL1),
···
3490
3391
{ SYS_DESC(SYS_ICC_AP1R1_EL1), undef_access },
3491
3392
{ SYS_DESC(SYS_ICC_AP1R2_EL1), undef_access },
3492
3393
{ SYS_DESC(SYS_ICC_AP1R3_EL1), undef_access },
3394
+
{ SYS_DESC(SYS_ICC_IDR0_EL1), access_gicv5_idr0 },
3395
+
{ SYS_DESC(SYS_ICC_IAFFIDR_EL1), access_gicv5_iaffid },
3396
+
{ SYS_DESC(SYS_ICC_PPI_ENABLER0_EL1), access_gicv5_ppi_enabler },
3397
+
{ SYS_DESC(SYS_ICC_PPI_ENABLER1_EL1), access_gicv5_ppi_enabler },
3493
3398
{ SYS_DESC(SYS_ICC_DIR_EL1), access_gic_dir },
3494
3399
{ SYS_DESC(SYS_ICC_RPR_EL1), undef_access },
3495
3400
{ SYS_DESC(SYS_ICC_SGI1R_EL1), access_gic_sgi },
···
5750
5647
compute_fgu(kvm, HFGRTR2_GROUP);
5751
5648
compute_fgu(kvm, HFGITR2_GROUP);
5752
5649
compute_fgu(kvm, HDFGRTR2_GROUP);
5650
+
compute_fgu(kvm, ICH_HFGRTR_GROUP);
5651
+
compute_fgu(kvm, ICH_HFGITR_GROUP);
5753
5652
5754
5653
set_bit(KVM_ARCH_FLAG_FGU_INITIALIZED, &kvm->arch.flags);
5755
5654
out:
···
5772
5667
5773
5668
guard(mutex)(&kvm->arch.config_lock);
5774
5669
5670
+
if (kvm_vm_has_ran_once(kvm))
5671
+
return 0;
5672
+
5775
5673
/*
5776
5674
* This hacks into the ID registers, so only perform it when the
5777
5675
* first vcpu runs, or the kvm_set_vm_id_reg() helper will scream.
5778
5676
*/
5779
-
if (!irqchip_in_kernel(kvm) && !kvm_vm_has_ran_once(kvm)) {
5677
+
if (!irqchip_in_kernel(kvm)) {
5780
5678
u64 val;
5781
5679
5782
5680
val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1) & ~ID_AA64PFR0_EL1_GIC;
5783
5681
kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1, val);
5682
+
val = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR2_EL1) & ~ID_AA64PFR2_EL1_GCIE;
5683
+
kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR2_EL1, val);
5784
5684
val = kvm_read_vm_id_reg(kvm, SYS_ID_PFR1_EL1) & ~ID_PFR1_EL1_GIC;
5785
5685
kvm_set_vm_id_reg(kvm, SYS_ID_PFR1_EL1, val);
5686
+
} else {
5687
+
/*
5688
+
* Certain userspace software - QEMU - samples the system
5689
+
* register state without creating an irqchip, then blindly
5690
+
* restores the state prior to running the final guest. This
5691
+
* means that it restores the virtualization & emulation
5692
+
* capabilities of the host system, rather than something that
5693
+
* reflects the final guest state. Moreover, it checks that the
5694
+
* state was "correctly" restored (i.e., verbatim), bailing if
5695
+
* it isn't, so masking off invalid state isn't an option.
5696
+
*
5697
+
* On GICv5 hardware that supports FEAT_GCIE_LEGACY we can run
5698
+
* both GICv3- and GICv5-based guests. Therefore, we initially
5699
+
* present both ID_AA64PFR0.GIC and ID_AA64PFR2.GCIE as IMP to
5700
+
* reflect that userspace can create EITHER a vGICv3 or a
5701
+
* vGICv5. This is an architecturally invalid combination, of
5702
+
* course. Once an in-kernel GIC is created, the sysreg state is
5703
+
* updated to reflect the actual, valid configuration.
5704
+
*
5705
+
* Setting both the GIC and GCIE features to IMP unsurprisingly
5706
+
* results in guests falling over, and hence we need to fix up
5707
+
* this mess in KVM. Before running for the first time we yet
5708
+
* again ensure that the GIC and GCIE fields accurately reflect
5709
+
* the actual hardware the guest should see.
5710
+
*
5711
+
* This hack allows legacy QEMU-based GICv3 guests to run
5712
+
* unmodified on compatible GICv5 hosts, and avoids the inverse
5713
+
* problem for GICv5-based guests in the future.
5714
+
*/
5715
+
kvm_vgic_finalize_idregs(kvm);
5786
5716
}
5787
5717
5788
5718
if (vcpu_has_nv(vcpu)) {
+151
-65
arch/arm64/kvm/vgic/vgic-init.c
+151
-65
arch/arm64/kvm/vgic/vgic-init.c
···
66
66
* or through the generic KVM_CREATE_DEVICE API ioctl.
67
67
* irqchip_in_kernel() tells you if this function succeeded or not.
68
68
* @kvm: kvm struct pointer
69
-
* @type: KVM_DEV_TYPE_ARM_VGIC_V[23]
69
+
* @type: KVM_DEV_TYPE_ARM_VGIC_V[235]
70
70
*/
71
71
int kvm_vgic_create(struct kvm *kvm, u32 type)
72
72
{
73
73
struct kvm_vcpu *vcpu;
74
-
u64 aa64pfr0, pfr1;
75
74
unsigned long i;
76
75
int ret;
77
76
···
131
132
132
133
if (type == KVM_DEV_TYPE_ARM_VGIC_V2)
133
134
kvm->max_vcpus = VGIC_V2_MAX_CPUS;
134
-
else
135
+
else if (type == KVM_DEV_TYPE_ARM_VGIC_V3)
135
136
kvm->max_vcpus = VGIC_V3_MAX_CPUS;
137
+
else if (type == KVM_DEV_TYPE_ARM_VGIC_V5)
138
+
kvm->max_vcpus = min(VGIC_V5_MAX_CPUS,
139
+
kvm_vgic_global_state.max_gic_vcpus);
136
140
137
141
if (atomic_read(&kvm->online_vcpus) > kvm->max_vcpus) {
138
142
ret = -E2BIG;
···
147
145
kvm->arch.vgic.implementation_rev = KVM_VGIC_IMP_REV_LATEST;
148
146
kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
149
147
150
-
aa64pfr0 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1) & ~ID_AA64PFR0_EL1_GIC;
151
-
pfr1 = kvm_read_vm_id_reg(kvm, SYS_ID_PFR1_EL1) & ~ID_PFR1_EL1_GIC;
152
-
153
-
if (type == KVM_DEV_TYPE_ARM_VGIC_V2) {
154
-
kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
155
-
} else {
156
-
INIT_LIST_HEAD(&kvm->arch.vgic.rd_regions);
157
-
aa64pfr0 |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, GIC, IMP);
158
-
pfr1 |= SYS_FIELD_PREP_ENUM(ID_PFR1_EL1, GIC, GICv3);
159
-
}
160
-
161
-
kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1, aa64pfr0);
162
-
kvm_set_vm_id_reg(kvm, SYS_ID_PFR1_EL1, pfr1);
148
+
/*
149
+
* We've now created the GIC. Update the system register state
150
+
* to accurately reflect what we've created.
151
+
*/
152
+
kvm_vgic_finalize_idregs(kvm);
163
153
164
154
kvm_for_each_vcpu(i, vcpu, kvm) {
165
155
ret = vgic_allocate_private_irqs_locked(vcpu, type);
···
172
178
173
179
if (type == KVM_DEV_TYPE_ARM_VGIC_V3)
174
180
kvm->arch.vgic.nassgicap = system_supports_direct_sgis();
181
+
182
+
/*
183
+
* We now know that we have a GICv5. The Arch Timer PPI interrupts may
184
+
* have been initialised at this stage, but will have done so assuming
185
+
* that we have an older GIC, meaning that the IntIDs won't be
186
+
* correct. We init them again, and this time they will be correct.
187
+
*/
188
+
if (type == KVM_DEV_TYPE_ARM_VGIC_V5)
189
+
kvm_timer_init_vm(kvm);
175
190
176
191
out_unlock:
177
192
mutex_unlock(&kvm->arch.config_lock);
···
262
259
return ret;
263
260
}
264
261
262
+
static void vgic_allocate_private_irq(struct kvm_vcpu *vcpu, int i, u32 type)
263
+
{
264
+
struct vgic_irq *irq = &vcpu->arch.vgic_cpu.private_irqs[i];
265
+
266
+
INIT_LIST_HEAD(&irq->ap_list);
267
+
raw_spin_lock_init(&irq->irq_lock);
268
+
irq->vcpu = NULL;
269
+
irq->target_vcpu = vcpu;
270
+
refcount_set(&irq->refcount, 0);
271
+
272
+
irq->intid = i;
273
+
if (vgic_irq_is_sgi(i)) {
274
+
/* SGIs */
275
+
irq->enabled = 1;
276
+
irq->config = VGIC_CONFIG_EDGE;
277
+
} else {
278
+
/* PPIs */
279
+
irq->config = VGIC_CONFIG_LEVEL;
280
+
}
281
+
282
+
switch (type) {
283
+
case KVM_DEV_TYPE_ARM_VGIC_V3:
284
+
irq->group = 1;
285
+
irq->mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
286
+
break;
287
+
case KVM_DEV_TYPE_ARM_VGIC_V2:
288
+
irq->group = 0;
289
+
irq->targets = BIT(vcpu->vcpu_id);
290
+
break;
291
+
}
292
+
}
293
+
294
+
static void vgic_v5_allocate_private_irq(struct kvm_vcpu *vcpu, int i, u32 type)
295
+
{
296
+
struct vgic_irq *irq = &vcpu->arch.vgic_cpu.private_irqs[i];
297
+
u32 intid = vgic_v5_make_ppi(i);
298
+
299
+
INIT_LIST_HEAD(&irq->ap_list);
300
+
raw_spin_lock_init(&irq->irq_lock);
301
+
irq->vcpu = NULL;
302
+
irq->target_vcpu = vcpu;
303
+
refcount_set(&irq->refcount, 0);
304
+
305
+
irq->intid = intid;
306
+
307
+
/* The only Edge architected PPI is the SW_PPI */
308
+
if (i == GICV5_ARCH_PPI_SW_PPI)
309
+
irq->config = VGIC_CONFIG_EDGE;
310
+
else
311
+
irq->config = VGIC_CONFIG_LEVEL;
312
+
313
+
/* Register the GICv5-specific PPI ops */
314
+
vgic_v5_set_ppi_ops(vcpu, intid);
315
+
}
316
+
265
317
static int vgic_allocate_private_irqs_locked(struct kvm_vcpu *vcpu, u32 type)
266
318
{
267
319
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
320
+
u32 num_private_irqs;
268
321
int i;
269
322
270
323
lockdep_assert_held(&vcpu->kvm->arch.config_lock);
···
328
269
if (vgic_cpu->private_irqs)
329
270
return 0;
330
271
272
+
if (vgic_is_v5(vcpu->kvm))
273
+
num_private_irqs = VGIC_V5_NR_PRIVATE_IRQS;
274
+
else
275
+
num_private_irqs = VGIC_NR_PRIVATE_IRQS;
276
+
331
277
vgic_cpu->private_irqs = kzalloc_objs(struct vgic_irq,
332
-
VGIC_NR_PRIVATE_IRQS,
278
+
num_private_irqs,
333
279
GFP_KERNEL_ACCOUNT);
334
280
335
281
if (!vgic_cpu->private_irqs)
···
344
280
* Enable and configure all SGIs to be edge-triggered and
345
281
* configure all PPIs as level-triggered.
346
282
*/
347
-
for (i = 0; i < VGIC_NR_PRIVATE_IRQS; i++) {
348
-
struct vgic_irq *irq = &vgic_cpu->private_irqs[i];
349
-
350
-
INIT_LIST_HEAD(&irq->ap_list);
351
-
raw_spin_lock_init(&irq->irq_lock);
352
-
irq->intid = i;
353
-
irq->vcpu = NULL;
354
-
irq->target_vcpu = vcpu;
355
-
refcount_set(&irq->refcount, 0);
356
-
if (vgic_irq_is_sgi(i)) {
357
-
/* SGIs */
358
-
irq->enabled = 1;
359
-
irq->config = VGIC_CONFIG_EDGE;
360
-
} else {
361
-
/* PPIs */
362
-
irq->config = VGIC_CONFIG_LEVEL;
363
-
}
364
-
365
-
switch (type) {
366
-
case KVM_DEV_TYPE_ARM_VGIC_V3:
367
-
irq->group = 1;
368
-
irq->mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
369
-
break;
370
-
case KVM_DEV_TYPE_ARM_VGIC_V2:
371
-
irq->group = 0;
372
-
irq->targets = BIT(vcpu->vcpu_id);
373
-
break;
374
-
}
283
+
for (i = 0; i < num_private_irqs; i++) {
284
+
if (vgic_is_v5(vcpu->kvm))
285
+
vgic_v5_allocate_private_irq(vcpu, i, type);
286
+
else
287
+
vgic_allocate_private_irq(vcpu, i, type);
375
288
}
376
289
377
290
return 0;
···
407
366
408
367
static void kvm_vgic_vcpu_reset(struct kvm_vcpu *vcpu)
409
368
{
410
-
if (kvm_vgic_global_state.type == VGIC_V2)
369
+
const struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
370
+
371
+
if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V5)
372
+
vgic_v5_reset(vcpu);
373
+
else if (kvm_vgic_global_state.type == VGIC_V2)
411
374
vgic_v2_reset(vcpu);
412
375
else
413
376
vgic_v3_reset(vcpu);
···
442
397
if (kvm->created_vcpus != atomic_read(&kvm->online_vcpus))
443
398
return -EBUSY;
444
399
445
-
/* freeze the number of spis */
446
-
if (!dist->nr_spis)
447
-
dist->nr_spis = VGIC_NR_IRQS_LEGACY - VGIC_NR_PRIVATE_IRQS;
400
+
if (!vgic_is_v5(kvm)) {
401
+
/* freeze the number of spis */
402
+
if (!dist->nr_spis)
403
+
dist->nr_spis = VGIC_NR_IRQS_LEGACY - VGIC_NR_PRIVATE_IRQS;
448
404
449
-
ret = kvm_vgic_dist_init(kvm, dist->nr_spis);
450
-
if (ret)
451
-
goto out;
452
-
453
-
/*
454
-
* Ensure vPEs are allocated if direct IRQ injection (e.g. vSGIs,
455
-
* vLPIs) is supported.
456
-
*/
457
-
if (vgic_supports_direct_irqs(kvm)) {
458
-
ret = vgic_v4_init(kvm);
405
+
ret = kvm_vgic_dist_init(kvm, dist->nr_spis);
459
406
if (ret)
460
-
goto out;
407
+
return ret;
408
+
409
+
/*
410
+
* Ensure vPEs are allocated if direct IRQ injection (e.g. vSGIs,
411
+
* vLPIs) is supported.
412
+
*/
413
+
if (vgic_supports_direct_irqs(kvm)) {
414
+
ret = vgic_v4_init(kvm);
415
+
if (ret)
416
+
return ret;
417
+
}
418
+
} else {
419
+
ret = vgic_v5_init(kvm);
420
+
if (ret)
421
+
return ret;
461
422
}
462
423
463
424
kvm_for_each_vcpu(idx, vcpu, kvm)
···
471
420
472
421
ret = kvm_vgic_setup_default_irq_routing(kvm);
473
422
if (ret)
474
-
goto out;
423
+
return ret;
475
424
476
425
vgic_debug_init(kvm);
477
426
dist->initialized = true;
478
-
out:
479
-
return ret;
427
+
428
+
return 0;
480
429
}
481
430
482
431
static void kvm_vgic_dist_destroy(struct kvm *kvm)
···
620
569
int kvm_vgic_map_resources(struct kvm *kvm)
621
570
{
622
571
struct vgic_dist *dist = &kvm->arch.vgic;
572
+
bool needs_dist = true;
623
573
enum vgic_type type;
624
574
gpa_t dist_base;
625
575
int ret = 0;
···
639
587
if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) {
640
588
ret = vgic_v2_map_resources(kvm);
641
589
type = VGIC_V2;
642
-
} else {
590
+
} else if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
643
591
ret = vgic_v3_map_resources(kvm);
644
592
type = VGIC_V3;
593
+
} else {
594
+
ret = vgic_v5_map_resources(kvm);
595
+
type = VGIC_V5;
596
+
needs_dist = false;
645
597
}
646
598
647
-
if (ret)
599
+
if (ret || !needs_dist)
648
600
goto out;
649
601
650
602
dist_base = dist->vgic_dist_base;
···
671
615
mutex_unlock(&kvm->slots_lock);
672
616
673
617
return ret;
618
+
}
619
+
620
+
void kvm_vgic_finalize_idregs(struct kvm *kvm)
621
+
{
622
+
u32 type = kvm->arch.vgic.vgic_model;
623
+
u64 aa64pfr0, aa64pfr2, pfr1;
624
+
625
+
aa64pfr0 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1) & ~ID_AA64PFR0_EL1_GIC;
626
+
aa64pfr2 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR2_EL1) & ~ID_AA64PFR2_EL1_GCIE;
627
+
pfr1 = kvm_read_vm_id_reg(kvm, SYS_ID_PFR1_EL1) & ~ID_PFR1_EL1_GIC;
628
+
629
+
switch (type) {
630
+
case KVM_DEV_TYPE_ARM_VGIC_V2:
631
+
kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
632
+
break;
633
+
case KVM_DEV_TYPE_ARM_VGIC_V3:
634
+
INIT_LIST_HEAD(&kvm->arch.vgic.rd_regions);
635
+
aa64pfr0 |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, GIC, IMP);
636
+
pfr1 |= SYS_FIELD_PREP_ENUM(ID_PFR1_EL1, GIC, GICv3);
637
+
break;
638
+
case KVM_DEV_TYPE_ARM_VGIC_V5:
639
+
aa64pfr2 |= SYS_FIELD_PREP_ENUM(ID_AA64PFR2_EL1, GCIE, IMP);
640
+
break;
641
+
default:
642
+
WARN_ONCE(1, "Unknown VGIC type!!!\n");
643
+
}
644
+
645
+
kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1, aa64pfr0);
646
+
kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR2_EL1, aa64pfr2);
647
+
kvm_set_vm_id_reg(kvm, SYS_ID_PFR1_EL1, pfr1);
674
648
}
675
649
676
650
/* GENERIC PROBE */
+106
-1
arch/arm64/kvm/vgic/vgic-kvm-device.c
+106
-1
arch/arm64/kvm/vgic/vgic-kvm-device.c
···
336
336
break;
337
337
ret = kvm_vgic_register_its_device();
338
338
break;
339
+
case KVM_DEV_TYPE_ARM_VGIC_V5:
340
+
ret = kvm_register_device_ops(&kvm_arm_vgic_v5_ops,
341
+
KVM_DEV_TYPE_ARM_VGIC_V5);
342
+
break;
339
343
}
340
344
341
345
return ret;
···
643
639
if (vgic_initialized(dev->kvm))
644
640
return -EBUSY;
645
641
646
-
if (!irq_is_ppi(val))
642
+
if (!irq_is_ppi(dev->kvm, val))
647
643
return -EINVAL;
648
644
649
645
dev->kvm->arch.vgic.mi_intid = val;
···
718
714
.set_attr = vgic_v3_set_attr,
719
715
.get_attr = vgic_v3_get_attr,
720
716
.has_attr = vgic_v3_has_attr,
717
+
};
718
+
719
+
static int vgic_v5_get_userspace_ppis(struct kvm_device *dev,
720
+
struct kvm_device_attr *attr)
721
+
{
722
+
struct vgic_v5_vm *gicv5_vm = &dev->kvm->arch.vgic.gicv5_vm;
723
+
u64 __user *uaddr = (u64 __user *)(long)attr->addr;
724
+
int ret;
725
+
726
+
guard(mutex)(&dev->kvm->arch.config_lock);
727
+
728
+
/*
729
+
* We either support 64 or 128 PPIs. In the former case, we need to
730
+
* return 0s for the second 64 bits as we have no storage backing those.
731
+
*/
732
+
ret = put_user(bitmap_read(gicv5_vm->userspace_ppis, 0, 64), uaddr);
733
+
if (ret)
734
+
return ret;
735
+
uaddr++;
736
+
737
+
if (VGIC_V5_NR_PRIVATE_IRQS == 128)
738
+
ret = put_user(bitmap_read(gicv5_vm->userspace_ppis, 64, 128), uaddr);
739
+
else
740
+
ret = put_user(0, uaddr);
741
+
742
+
return ret;
743
+
}
744
+
745
+
static int vgic_v5_set_attr(struct kvm_device *dev,
746
+
struct kvm_device_attr *attr)
747
+
{
748
+
switch (attr->group) {
749
+
case KVM_DEV_ARM_VGIC_GRP_ADDR:
750
+
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
751
+
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
752
+
return -ENXIO;
753
+
case KVM_DEV_ARM_VGIC_GRP_CTRL:
754
+
switch (attr->attr) {
755
+
case KVM_DEV_ARM_VGIC_CTRL_INIT:
756
+
return vgic_set_common_attr(dev, attr);
757
+
case KVM_DEV_ARM_VGIC_USERSPACE_PPIS:
758
+
default:
759
+
return -ENXIO;
760
+
}
761
+
default:
762
+
return -ENXIO;
763
+
}
764
+
765
+
}
766
+
767
+
static int vgic_v5_get_attr(struct kvm_device *dev,
768
+
struct kvm_device_attr *attr)
769
+
{
770
+
switch (attr->group) {
771
+
case KVM_DEV_ARM_VGIC_GRP_ADDR:
772
+
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
773
+
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
774
+
return -ENXIO;
775
+
case KVM_DEV_ARM_VGIC_GRP_CTRL:
776
+
switch (attr->attr) {
777
+
case KVM_DEV_ARM_VGIC_CTRL_INIT:
778
+
return vgic_get_common_attr(dev, attr);
779
+
case KVM_DEV_ARM_VGIC_USERSPACE_PPIS:
780
+
return vgic_v5_get_userspace_ppis(dev, attr);
781
+
default:
782
+
return -ENXIO;
783
+
}
784
+
default:
785
+
return -ENXIO;
786
+
}
787
+
}
788
+
789
+
static int vgic_v5_has_attr(struct kvm_device *dev,
790
+
struct kvm_device_attr *attr)
791
+
{
792
+
switch (attr->group) {
793
+
case KVM_DEV_ARM_VGIC_GRP_ADDR:
794
+
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
795
+
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
796
+
return -ENXIO;
797
+
case KVM_DEV_ARM_VGIC_GRP_CTRL:
798
+
switch (attr->attr) {
799
+
case KVM_DEV_ARM_VGIC_CTRL_INIT:
800
+
return 0;
801
+
case KVM_DEV_ARM_VGIC_USERSPACE_PPIS:
802
+
return 0;
803
+
default:
804
+
return -ENXIO;
805
+
}
806
+
default:
807
+
return -ENXIO;
808
+
}
809
+
}
810
+
811
+
struct kvm_device_ops kvm_arm_vgic_v5_ops = {
812
+
.name = "kvm-arm-vgic-v5",
813
+
.create = vgic_create,
814
+
.destroy = vgic_destroy,
815
+
.set_attr = vgic_v5_set_attr,
816
+
.get_attr = vgic_v5_get_attr,
817
+
.has_attr = vgic_v5_has_attr,
721
818
};
+34
-6
arch/arm64/kvm/vgic/vgic-mmio.c
+34
-6
arch/arm64/kvm/vgic/vgic-mmio.c
···
842
842
843
843
void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
844
844
{
845
-
if (kvm_vgic_global_state.type == VGIC_V2)
846
-
vgic_v2_set_vmcr(vcpu, vmcr);
847
-
else
845
+
const struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
846
+
847
+
switch (dist->vgic_model) {
848
+
case KVM_DEV_TYPE_ARM_VGIC_V5:
849
+
vgic_v5_set_vmcr(vcpu, vmcr);
850
+
break;
851
+
case KVM_DEV_TYPE_ARM_VGIC_V3:
848
852
vgic_v3_set_vmcr(vcpu, vmcr);
853
+
break;
854
+
case KVM_DEV_TYPE_ARM_VGIC_V2:
855
+
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
856
+
vgic_v3_set_vmcr(vcpu, vmcr);
857
+
else
858
+
vgic_v2_set_vmcr(vcpu, vmcr);
859
+
break;
860
+
default:
861
+
BUG();
862
+
}
849
863
}
850
864
851
865
void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
852
866
{
853
-
if (kvm_vgic_global_state.type == VGIC_V2)
854
-
vgic_v2_get_vmcr(vcpu, vmcr);
855
-
else
867
+
const struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
868
+
869
+
switch (dist->vgic_model) {
870
+
case KVM_DEV_TYPE_ARM_VGIC_V5:
871
+
vgic_v5_get_vmcr(vcpu, vmcr);
872
+
break;
873
+
case KVM_DEV_TYPE_ARM_VGIC_V3:
856
874
vgic_v3_get_vmcr(vcpu, vmcr);
875
+
break;
876
+
case KVM_DEV_TYPE_ARM_VGIC_V2:
877
+
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
878
+
vgic_v3_get_vmcr(vcpu, vmcr);
879
+
else
880
+
vgic_v2_get_vmcr(vcpu, vmcr);
881
+
break;
882
+
default:
883
+
BUG();
884
+
}
857
885
}
858
886
859
887
/*
+1
-1
arch/arm64/kvm/vgic/vgic-v3.c
+1
-1
arch/arm64/kvm/vgic/vgic-v3.c
+494
-9
arch/arm64/kvm/vgic/vgic-v5.c
+494
-9
arch/arm64/kvm/vgic/vgic-v5.c
···
1
1
// SPDX-License-Identifier: GPL-2.0-only
2
+
/*
3
+
* Copyright (C) 2025, 2026 Arm Ltd.
4
+
*/
2
5
3
6
#include <kvm/arm_vgic.h>
7
+
8
+
#include <linux/bitops.h>
4
9
#include <linux/irqchip/arm-vgic-info.h>
5
10
6
11
#include "vgic.h"
7
12
13
+
static struct vgic_v5_ppi_caps ppi_caps;
14
+
15
+
/*
16
+
* Not all PPIs are guaranteed to be implemented for GICv5. Deterermine which
17
+
* ones are, and generate a mask.
18
+
*/
19
+
static void vgic_v5_get_implemented_ppis(void)
20
+
{
21
+
if (!cpus_have_final_cap(ARM64_HAS_GICV5_CPUIF))
22
+
return;
23
+
24
+
/*
25
+
* If we have KVM, we have EL2, which means that we have support for the
26
+
* EL1 and EL2 Physical & Virtual timers.
27
+
*/
28
+
__assign_bit(GICV5_ARCH_PPI_CNTHP, ppi_caps.impl_ppi_mask, 1);
29
+
__assign_bit(GICV5_ARCH_PPI_CNTV, ppi_caps.impl_ppi_mask, 1);
30
+
__assign_bit(GICV5_ARCH_PPI_CNTHV, ppi_caps.impl_ppi_mask, 1);
31
+
__assign_bit(GICV5_ARCH_PPI_CNTP, ppi_caps.impl_ppi_mask, 1);
32
+
33
+
/* The SW_PPI should be available */
34
+
__assign_bit(GICV5_ARCH_PPI_SW_PPI, ppi_caps.impl_ppi_mask, 1);
35
+
36
+
/* The PMUIRQ is available if we have the PMU */
37
+
__assign_bit(GICV5_ARCH_PPI_PMUIRQ, ppi_caps.impl_ppi_mask, system_supports_pmuv3());
38
+
}
39
+
8
40
/*
9
41
* Probe for a vGICv5 compatible interrupt controller, returning 0 on success.
10
-
* Currently only supports GICv3-based VMs on a GICv5 host, and hence only
11
-
* registers a VGIC_V3 device.
12
42
*/
13
43
int vgic_v5_probe(const struct gic_kvm_info *info)
14
44
{
45
+
bool v5_registered = false;
15
46
u64 ich_vtr_el2;
16
47
int ret;
17
48
18
-
if (!cpus_have_final_cap(ARM64_HAS_GICV5_LEGACY))
19
-
return -ENODEV;
20
-
21
49
kvm_vgic_global_state.type = VGIC_V5;
22
-
kvm_vgic_global_state.has_gcie_v3_compat = true;
23
-
24
-
/* We only support v3 compat mode - use vGICv3 limits */
25
-
kvm_vgic_global_state.max_gic_vcpus = VGIC_V3_MAX_CPUS;
26
50
27
51
kvm_vgic_global_state.vcpu_base = 0;
28
52
kvm_vgic_global_state.vctrl_base = NULL;
···
54
30
kvm_vgic_global_state.has_gicv4 = false;
55
31
kvm_vgic_global_state.has_gicv4_1 = false;
56
32
33
+
/*
34
+
* GICv5 is currently not supported in Protected mode. Skip the
35
+
* registration of GICv5 completely to make sure no guests can create a
36
+
* GICv5-based guest.
37
+
*/
38
+
if (is_protected_kvm_enabled()) {
39
+
kvm_info("GICv5-based guests are not supported with pKVM\n");
40
+
goto skip_v5;
41
+
}
42
+
43
+
kvm_vgic_global_state.max_gic_vcpus = VGIC_V5_MAX_CPUS;
44
+
45
+
vgic_v5_get_implemented_ppis();
46
+
47
+
ret = kvm_register_vgic_device(KVM_DEV_TYPE_ARM_VGIC_V5);
48
+
if (ret) {
49
+
kvm_err("Cannot register GICv5 KVM device.\n");
50
+
goto skip_v5;
51
+
}
52
+
53
+
v5_registered = true;
54
+
kvm_info("GCIE system register CPU interface\n");
55
+
56
+
skip_v5:
57
+
/* If we don't support the GICv3 compat mode we're done. */
58
+
if (!cpus_have_final_cap(ARM64_HAS_GICV5_LEGACY)) {
59
+
if (!v5_registered)
60
+
return -ENODEV;
61
+
return 0;
62
+
}
63
+
64
+
kvm_vgic_global_state.has_gcie_v3_compat = true;
57
65
ich_vtr_el2 = kvm_call_hyp_ret(__vgic_v3_get_gic_config);
58
66
kvm_vgic_global_state.ich_vtr_el2 = (u32)ich_vtr_el2;
59
67
···
101
45
return ret;
102
46
}
103
47
48
+
/* We potentially limit the max VCPUs further than we need to here */
49
+
kvm_vgic_global_state.max_gic_vcpus = min(VGIC_V3_MAX_CPUS,
50
+
VGIC_V5_MAX_CPUS);
51
+
104
52
static_branch_enable(&kvm_vgic_global_state.gicv3_cpuif);
105
53
kvm_info("GCIE legacy system register CPU interface\n");
106
54
107
55
vgic_v3_enable_cpuif_traps();
108
56
109
57
return 0;
58
+
}
59
+
60
+
void vgic_v5_reset(struct kvm_vcpu *vcpu)
61
+
{
62
+
/*
63
+
* We always present 16-bits of ID space to the guest, irrespective of
64
+
* the host allowing more.
65
+
*/
66
+
vcpu->arch.vgic_cpu.num_id_bits = ICC_IDR0_EL1_ID_BITS_16BITS;
67
+
68
+
/*
69
+
* The GICv5 architeture only supports 5-bits of priority in the
70
+
* CPUIF (but potentially fewer in the IRS).
71
+
*/
72
+
vcpu->arch.vgic_cpu.num_pri_bits = 5;
73
+
}
74
+
75
+
int vgic_v5_init(struct kvm *kvm)
76
+
{
77
+
struct kvm_vcpu *vcpu;
78
+
unsigned long idx;
79
+
80
+
if (vgic_initialized(kvm))
81
+
return 0;
82
+
83
+
kvm_for_each_vcpu(idx, vcpu, kvm) {
84
+
if (vcpu_has_nv(vcpu)) {
85
+
kvm_err("Nested GICv5 VMs are currently unsupported\n");
86
+
return -EINVAL;
87
+
}
88
+
}
89
+
90
+
/* We only allow userspace to drive the SW_PPI, if it is implemented. */
91
+
bitmap_zero(kvm->arch.vgic.gicv5_vm.userspace_ppis,
92
+
VGIC_V5_NR_PRIVATE_IRQS);
93
+
__assign_bit(GICV5_ARCH_PPI_SW_PPI,
94
+
kvm->arch.vgic.gicv5_vm.userspace_ppis,
95
+
VGIC_V5_NR_PRIVATE_IRQS);
96
+
bitmap_and(kvm->arch.vgic.gicv5_vm.userspace_ppis,
97
+
kvm->arch.vgic.gicv5_vm.userspace_ppis,
98
+
ppi_caps.impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
99
+
100
+
return 0;
101
+
}
102
+
103
+
int vgic_v5_map_resources(struct kvm *kvm)
104
+
{
105
+
if (!vgic_initialized(kvm))
106
+
return -EBUSY;
107
+
108
+
return 0;
109
+
}
110
+
111
+
int vgic_v5_finalize_ppi_state(struct kvm *kvm)
112
+
{
113
+
struct kvm_vcpu *vcpu0;
114
+
int i;
115
+
116
+
if (!vgic_is_v5(kvm))
117
+
return 0;
118
+
119
+
/* The PPI state for all VCPUs should be the same. Pick the first. */
120
+
vcpu0 = kvm_get_vcpu(kvm, 0);
121
+
122
+
bitmap_zero(kvm->arch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
123
+
bitmap_zero(kvm->arch.vgic.gicv5_vm.vgic_ppi_hmr, VGIC_V5_NR_PRIVATE_IRQS);
124
+
125
+
for_each_set_bit(i, ppi_caps.impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS) {
126
+
const u32 intid = vgic_v5_make_ppi(i);
127
+
struct vgic_irq *irq;
128
+
129
+
irq = vgic_get_vcpu_irq(vcpu0, intid);
130
+
131
+
/* Expose PPIs with an owner or the SW_PPI, only */
132
+
scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
133
+
if (irq->owner || i == GICV5_ARCH_PPI_SW_PPI) {
134
+
__assign_bit(i, kvm->arch.vgic.gicv5_vm.vgic_ppi_mask, 1);
135
+
__assign_bit(i, kvm->arch.vgic.gicv5_vm.vgic_ppi_hmr,
136
+
irq->config == VGIC_CONFIG_LEVEL);
137
+
}
138
+
}
139
+
140
+
vgic_put_irq(vcpu0->kvm, irq);
141
+
}
142
+
143
+
return 0;
144
+
}
145
+
146
+
static u32 vgic_v5_get_effective_priority_mask(struct kvm_vcpu *vcpu)
147
+
{
148
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
149
+
u32 highest_ap, priority_mask;
150
+
151
+
/*
152
+
* If the guest's CPU has not opted to receive interrupts, then the
153
+
* effective running priority is the highest priority. Just return 0
154
+
* (the highest priority).
155
+
*/
156
+
if (!FIELD_GET(FEAT_GCIE_ICH_VMCR_EL2_EN, cpu_if->vgic_vmcr))
157
+
return 0;
158
+
159
+
/*
160
+
* Counting the number of trailing zeros gives the current active
161
+
* priority. Explicitly use the 32-bit version here as we have 32
162
+
* priorities. 32 then means that there are no active priorities.
163
+
*/
164
+
highest_ap = cpu_if->vgic_apr ? __builtin_ctz(cpu_if->vgic_apr) : 32;
165
+
166
+
/*
167
+
* An interrupt is of sufficient priority if it is equal to or
168
+
* greater than the priority mask. Add 1 to the priority mask
169
+
* (i.e., lower priority) to match the APR logic before taking
170
+
* the min. This gives us the lowest priority that is masked.
171
+
*/
172
+
priority_mask = FIELD_GET(FEAT_GCIE_ICH_VMCR_EL2_VPMR, cpu_if->vgic_vmcr);
173
+
174
+
return min(highest_ap, priority_mask + 1);
175
+
}
176
+
177
+
/*
178
+
* For GICv5, the PPIs are mostly directly managed by the hardware. We (the
179
+
* hypervisor) handle the pending, active, enable state save/restore, but don't
180
+
* need the PPIs to be queued on a per-VCPU AP list. Therefore, sanity check the
181
+
* state, unlock, and return.
182
+
*/
183
+
bool vgic_v5_ppi_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,
184
+
unsigned long flags)
185
+
__releases(&irq->irq_lock)
186
+
{
187
+
struct kvm_vcpu *vcpu;
188
+
189
+
lockdep_assert_held(&irq->irq_lock);
190
+
191
+
if (WARN_ON_ONCE(!__irq_is_ppi(KVM_DEV_TYPE_ARM_VGIC_V5, irq->intid)))
192
+
goto out_unlock_fail;
193
+
194
+
vcpu = irq->target_vcpu;
195
+
if (WARN_ON_ONCE(!vcpu))
196
+
goto out_unlock_fail;
197
+
198
+
raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
199
+
200
+
/* Directly kick the target VCPU to make sure it sees the IRQ */
201
+
kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
202
+
kvm_vcpu_kick(vcpu);
203
+
204
+
return true;
205
+
206
+
out_unlock_fail:
207
+
raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
208
+
209
+
return false;
210
+
}
211
+
212
+
/*
213
+
* Sets/clears the corresponding bit in the ICH_PPI_DVIR register.
214
+
*/
215
+
void vgic_v5_set_ppi_dvi(struct kvm_vcpu *vcpu, struct vgic_irq *irq, bool dvi)
216
+
{
217
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
218
+
u32 ppi;
219
+
220
+
lockdep_assert_held(&irq->irq_lock);
221
+
222
+
ppi = vgic_v5_get_hwirq_id(irq->intid);
223
+
__assign_bit(ppi, cpu_if->vgic_ppi_dvir, dvi);
224
+
}
225
+
226
+
static struct irq_ops vgic_v5_ppi_irq_ops = {
227
+
.queue_irq_unlock = vgic_v5_ppi_queue_irq_unlock,
228
+
.set_direct_injection = vgic_v5_set_ppi_dvi,
229
+
};
230
+
231
+
void vgic_v5_set_ppi_ops(struct kvm_vcpu *vcpu, u32 vintid)
232
+
{
233
+
kvm_vgic_set_irq_ops(vcpu, vintid, &vgic_v5_ppi_irq_ops);
234
+
}
235
+
236
+
/*
237
+
* Sync back the PPI priorities to the vgic_irq shadow state for any interrupts
238
+
* exposed to the guest (skipping all others).
239
+
*/
240
+
static void vgic_v5_sync_ppi_priorities(struct kvm_vcpu *vcpu)
241
+
{
242
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
243
+
u64 priorityr;
244
+
int i;
245
+
246
+
/*
247
+
* We have up to 16 PPI Priority regs, but only have a few interrupts
248
+
* that the guest is allowed to use. Limit our sync of PPI priorities to
249
+
* those actually exposed to the guest by first iterating over the mask
250
+
* of exposed PPIs.
251
+
*/
252
+
for_each_set_bit(i, vcpu->kvm->arch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS) {
253
+
u32 intid = vgic_v5_make_ppi(i);
254
+
struct vgic_irq *irq;
255
+
int pri_idx, pri_reg, pri_bit;
256
+
u8 priority;
257
+
258
+
/*
259
+
* Determine which priority register and the field within it to
260
+
* extract.
261
+
*/
262
+
pri_reg = i / 8;
263
+
pri_idx = i % 8;
264
+
pri_bit = pri_idx * 8;
265
+
266
+
priorityr = cpu_if->vgic_ppi_priorityr[pri_reg];
267
+
priority = field_get(GENMASK(pri_bit + 4, pri_bit), priorityr);
268
+
269
+
irq = vgic_get_vcpu_irq(vcpu, intid);
270
+
271
+
scoped_guard(raw_spinlock_irqsave, &irq->irq_lock)
272
+
irq->priority = priority;
273
+
274
+
vgic_put_irq(vcpu->kvm, irq);
275
+
}
276
+
}
277
+
278
+
bool vgic_v5_has_pending_ppi(struct kvm_vcpu *vcpu)
279
+
{
280
+
unsigned int priority_mask;
281
+
int i;
282
+
283
+
priority_mask = vgic_v5_get_effective_priority_mask(vcpu);
284
+
285
+
/*
286
+
* If the combined priority mask is 0, nothing can be signalled! In the
287
+
* case where the guest has disabled interrupt delivery for the vcpu
288
+
* (via ICV_CR0_EL1.EN->ICH_VMCR_EL2.EN), we calculate the priority mask
289
+
* as 0 too (the highest possible priority).
290
+
*/
291
+
if (!priority_mask)
292
+
return false;
293
+
294
+
for_each_set_bit(i, vcpu->kvm->arch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS) {
295
+
u32 intid = vgic_v5_make_ppi(i);
296
+
bool has_pending = false;
297
+
struct vgic_irq *irq;
298
+
299
+
irq = vgic_get_vcpu_irq(vcpu, intid);
300
+
301
+
scoped_guard(raw_spinlock_irqsave, &irq->irq_lock)
302
+
has_pending = (irq->enabled && irq_is_pending(irq) &&
303
+
irq->priority <= priority_mask);
304
+
305
+
vgic_put_irq(vcpu->kvm, irq);
306
+
307
+
if (has_pending)
308
+
return true;
309
+
}
310
+
311
+
return false;
312
+
}
313
+
314
+
/*
315
+
* Detect any PPIs state changes, and propagate the state with KVM's
316
+
* shadow structures.
317
+
*/
318
+
void vgic_v5_fold_ppi_state(struct kvm_vcpu *vcpu)
319
+
{
320
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
321
+
DECLARE_BITMAP(changed_active, VGIC_V5_NR_PRIVATE_IRQS);
322
+
DECLARE_BITMAP(changed_pending, VGIC_V5_NR_PRIVATE_IRQS);
323
+
DECLARE_BITMAP(changed_bits, VGIC_V5_NR_PRIVATE_IRQS);
324
+
unsigned long *activer, *pendr_entry, *pendr;
325
+
int i;
326
+
327
+
activer = host_data_ptr(vgic_v5_ppi_state)->activer_exit;
328
+
pendr_entry = host_data_ptr(vgic_v5_ppi_state)->pendr_entry;
329
+
pendr = host_data_ptr(vgic_v5_ppi_state)->pendr_exit;
330
+
331
+
bitmap_xor(changed_active, cpu_if->vgic_ppi_activer, activer,
332
+
VGIC_V5_NR_PRIVATE_IRQS);
333
+
bitmap_xor(changed_pending, pendr_entry, pendr,
334
+
VGIC_V5_NR_PRIVATE_IRQS);
335
+
bitmap_or(changed_bits, changed_active, changed_pending,
336
+
VGIC_V5_NR_PRIVATE_IRQS);
337
+
338
+
for_each_set_bit(i, changed_bits, VGIC_V5_NR_PRIVATE_IRQS) {
339
+
u32 intid = vgic_v5_make_ppi(i);
340
+
struct vgic_irq *irq;
341
+
342
+
irq = vgic_get_vcpu_irq(vcpu, intid);
343
+
344
+
scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
345
+
irq->active = test_bit(i, activer);
346
+
347
+
/* This is an OR to avoid losing incoming edges! */
348
+
if (irq->config == VGIC_CONFIG_EDGE)
349
+
irq->pending_latch |= test_bit(i, pendr);
350
+
}
351
+
352
+
vgic_put_irq(vcpu->kvm, irq);
353
+
}
354
+
355
+
/*
356
+
* Re-inject the exit state as entry state next time!
357
+
*
358
+
* Note that the write of the Enable state is trapped, and hence there
359
+
* is nothing to explcitly sync back here as we already have the latest
360
+
* copy by definition.
361
+
*/
362
+
bitmap_copy(cpu_if->vgic_ppi_activer, activer, VGIC_V5_NR_PRIVATE_IRQS);
363
+
}
364
+
365
+
void vgic_v5_flush_ppi_state(struct kvm_vcpu *vcpu)
366
+
{
367
+
DECLARE_BITMAP(pendr, VGIC_V5_NR_PRIVATE_IRQS);
368
+
int i;
369
+
370
+
/*
371
+
* Time to enter the guest - we first need to build the guest's
372
+
* ICC_PPI_PENDRx_EL1, however.
373
+
*/
374
+
bitmap_zero(pendr, VGIC_V5_NR_PRIVATE_IRQS);
375
+
for_each_set_bit(i, vcpu->kvm->arch.vgic.gicv5_vm.vgic_ppi_mask,
376
+
VGIC_V5_NR_PRIVATE_IRQS) {
377
+
u32 intid = vgic_v5_make_ppi(i);
378
+
struct vgic_irq *irq;
379
+
380
+
irq = vgic_get_vcpu_irq(vcpu, intid);
381
+
382
+
scoped_guard(raw_spinlock_irqsave, &irq->irq_lock)
383
+
__assign_bit(i, pendr, irq_is_pending(irq));
384
+
385
+
vgic_put_irq(vcpu->kvm, irq);
386
+
}
387
+
388
+
/*
389
+
* Copy the shadow state to the pending reg that will be written to the
390
+
* ICH_PPI_PENDRx_EL2 regs. While the guest is running we track any
391
+
* incoming changes to the pending state in the vgic_irq structures. The
392
+
* incoming changes are merged with the outgoing changes on the return
393
+
* path.
394
+
*/
395
+
bitmap_copy(host_data_ptr(vgic_v5_ppi_state)->pendr_entry, pendr,
396
+
VGIC_V5_NR_PRIVATE_IRQS);
397
+
398
+
/*
399
+
* Make sure that we can correctly detect "edges" in the PPI
400
+
* state. There's a path where we never actually enter the guest, and
401
+
* failure to do this risks losing pending state
402
+
*/
403
+
bitmap_copy(host_data_ptr(vgic_v5_ppi_state)->pendr_exit, pendr,
404
+
VGIC_V5_NR_PRIVATE_IRQS);
405
+
}
406
+
407
+
void vgic_v5_load(struct kvm_vcpu *vcpu)
408
+
{
409
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
410
+
411
+
/*
412
+
* On the WFI path, vgic_load is called a second time. The first is when
413
+
* scheduling in the vcpu thread again, and the second is when leaving
414
+
* WFI. Skip the second instance as it serves no purpose and just
415
+
* restores the same state again.
416
+
*/
417
+
if (cpu_if->gicv5_vpe.resident)
418
+
return;
419
+
420
+
kvm_call_hyp(__vgic_v5_restore_vmcr_apr, cpu_if);
421
+
422
+
cpu_if->gicv5_vpe.resident = true;
423
+
}
424
+
425
+
void vgic_v5_put(struct kvm_vcpu *vcpu)
426
+
{
427
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
428
+
429
+
/*
430
+
* Do nothing if we're not resident. This can happen in the WFI path
431
+
* where we do a vgic_put in the WFI path and again later when
432
+
* descheduling the thread. We risk losing VMCR state if we sync it
433
+
* twice, so instead return early in this case.
434
+
*/
435
+
if (!cpu_if->gicv5_vpe.resident)
436
+
return;
437
+
438
+
kvm_call_hyp(__vgic_v5_save_apr, cpu_if);
439
+
440
+
cpu_if->gicv5_vpe.resident = false;
441
+
442
+
/* The shadow priority is only updated on entering WFI */
443
+
if (vcpu_get_flag(vcpu, IN_WFI))
444
+
vgic_v5_sync_ppi_priorities(vcpu);
445
+
}
446
+
447
+
void vgic_v5_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
448
+
{
449
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
450
+
u64 vmcr = cpu_if->vgic_vmcr;
451
+
452
+
vmcrp->en = FIELD_GET(FEAT_GCIE_ICH_VMCR_EL2_EN, vmcr);
453
+
vmcrp->pmr = FIELD_GET(FEAT_GCIE_ICH_VMCR_EL2_VPMR, vmcr);
454
+
}
455
+
456
+
void vgic_v5_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
457
+
{
458
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
459
+
u64 vmcr;
460
+
461
+
vmcr = FIELD_PREP(FEAT_GCIE_ICH_VMCR_EL2_VPMR, vmcrp->pmr) |
462
+
FIELD_PREP(FEAT_GCIE_ICH_VMCR_EL2_EN, vmcrp->en);
463
+
464
+
cpu_if->vgic_vmcr = vmcr;
465
+
}
466
+
467
+
void vgic_v5_restore_state(struct kvm_vcpu *vcpu)
468
+
{
469
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
470
+
471
+
__vgic_v5_restore_state(cpu_if);
472
+
__vgic_v5_restore_ppi_state(cpu_if);
473
+
dsb(sy);
474
+
}
475
+
476
+
void vgic_v5_save_state(struct kvm_vcpu *vcpu)
477
+
{
478
+
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
479
+
480
+
__vgic_v5_save_state(cpu_if);
481
+
__vgic_v5_save_ppi_state(cpu_if);
482
+
dsb(sy);
110
483
}
+139
-34
arch/arm64/kvm/vgic/vgic.c
+139
-34
arch/arm64/kvm/vgic/vgic.c
···
86
86
*/
87
87
struct vgic_irq *vgic_get_irq(struct kvm *kvm, u32 intid)
88
88
{
89
+
/* Non-private IRQs are not yet implemented for GICv5 */
90
+
if (vgic_is_v5(kvm))
91
+
return NULL;
92
+
89
93
/* SPIs */
90
94
if (intid >= VGIC_NR_PRIVATE_IRQS &&
91
95
intid < (kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS)) {
···
98
94
}
99
95
100
96
/* LPIs */
101
-
if (intid >= VGIC_MIN_LPI)
97
+
if (irq_is_lpi(kvm, intid))
102
98
return vgic_get_lpi(kvm, intid);
103
99
104
100
return NULL;
···
108
104
{
109
105
if (WARN_ON(!vcpu))
110
106
return NULL;
107
+
108
+
if (vgic_is_v5(vcpu->kvm)) {
109
+
u32 int_num, hwirq_id;
110
+
111
+
if (!__irq_is_ppi(KVM_DEV_TYPE_ARM_VGIC_V5, intid))
112
+
return NULL;
113
+
114
+
hwirq_id = FIELD_GET(GICV5_HWIRQ_ID, intid);
115
+
int_num = array_index_nospec(hwirq_id, VGIC_V5_NR_PRIVATE_IRQS);
116
+
117
+
return &vcpu->arch.vgic_cpu.private_irqs[int_num];
118
+
}
111
119
112
120
/* SGIs and PPIs */
113
121
if (intid < VGIC_NR_PRIVATE_IRQS) {
···
139
123
140
124
static __must_check bool __vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq)
141
125
{
142
-
if (irq->intid < VGIC_MIN_LPI)
126
+
if (!irq_is_lpi(kvm, irq->intid))
143
127
return false;
144
128
145
129
return refcount_dec_and_test(&irq->refcount);
···
164
148
* Acquire/release it early on lockdep kernels to make locking issues
165
149
* in rare release paths a bit more obvious.
166
150
*/
167
-
if (IS_ENABLED(CONFIG_LOCKDEP) && irq->intid >= VGIC_MIN_LPI) {
151
+
if (IS_ENABLED(CONFIG_LOCKDEP) && irq_is_lpi(kvm, irq->intid)) {
168
152
guard(spinlock_irqsave)(&dist->lpi_xa.xa_lock);
169
153
}
170
154
···
202
186
raw_spin_lock_irqsave(&vgic_cpu->ap_list_lock, flags);
203
187
204
188
list_for_each_entry_safe(irq, tmp, &vgic_cpu->ap_list_head, ap_list) {
205
-
if (irq->intid >= VGIC_MIN_LPI) {
189
+
if (irq_is_lpi(vcpu->kvm, irq->intid)) {
206
190
raw_spin_lock(&irq->irq_lock);
207
191
list_del(&irq->ap_list);
208
192
irq->vcpu = NULL;
···
420
404
421
405
lockdep_assert_held(&irq->irq_lock);
422
406
407
+
if (irq->ops && irq->ops->queue_irq_unlock)
408
+
return irq->ops->queue_irq_unlock(kvm, irq, flags);
409
+
423
410
retry:
424
411
vcpu = vgic_target_oracle(irq);
425
412
if (irq->vcpu || !vcpu) {
···
540
521
if (ret)
541
522
return ret;
542
523
543
-
if (!vcpu && intid < VGIC_NR_PRIVATE_IRQS)
524
+
if (!vcpu && irq_is_private(kvm, intid))
544
525
return -EINVAL;
545
526
546
527
trace_vgic_update_irq_pending(vcpu ? vcpu->vcpu_idx : 0, intid, level);
547
528
548
-
if (intid < VGIC_NR_PRIVATE_IRQS)
529
+
if (irq_is_private(kvm, intid))
549
530
irq = vgic_get_vcpu_irq(vcpu, intid);
550
531
else
551
532
irq = vgic_get_irq(kvm, intid);
···
572
553
return 0;
573
554
}
574
555
556
+
void kvm_vgic_set_irq_ops(struct kvm_vcpu *vcpu, u32 vintid,
557
+
struct irq_ops *ops)
558
+
{
559
+
struct vgic_irq *irq = vgic_get_vcpu_irq(vcpu, vintid);
560
+
561
+
BUG_ON(!irq);
562
+
563
+
scoped_guard(raw_spinlock_irqsave, &irq->irq_lock)
564
+
irq->ops = ops;
565
+
566
+
vgic_put_irq(vcpu->kvm, irq);
567
+
}
568
+
569
+
void kvm_vgic_clear_irq_ops(struct kvm_vcpu *vcpu, u32 vintid)
570
+
{
571
+
kvm_vgic_set_irq_ops(vcpu, vintid, NULL);
572
+
}
573
+
575
574
/* @irq->irq_lock must be held */
576
575
static int kvm_vgic_map_irq(struct kvm_vcpu *vcpu, struct vgic_irq *irq,
577
-
unsigned int host_irq,
578
-
struct irq_ops *ops)
576
+
unsigned int host_irq)
579
577
{
580
578
struct irq_desc *desc;
581
579
struct irq_data *data;
···
612
576
irq->hw = true;
613
577
irq->host_irq = host_irq;
614
578
irq->hwintid = data->hwirq;
615
-
irq->ops = ops;
579
+
580
+
if (irq->ops && irq->ops->set_direct_injection)
581
+
irq->ops->set_direct_injection(vcpu, irq, true);
582
+
616
583
return 0;
617
584
}
618
585
619
586
/* @irq->irq_lock must be held */
620
587
static inline void kvm_vgic_unmap_irq(struct vgic_irq *irq)
621
588
{
589
+
if (irq->ops && irq->ops->set_direct_injection)
590
+
irq->ops->set_direct_injection(irq->target_vcpu, irq, false);
591
+
622
592
irq->hw = false;
623
593
irq->hwintid = 0;
624
-
irq->ops = NULL;
625
594
}
626
595
627
596
int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
628
-
u32 vintid, struct irq_ops *ops)
597
+
u32 vintid)
629
598
{
630
599
struct vgic_irq *irq = vgic_get_vcpu_irq(vcpu, vintid);
631
600
unsigned long flags;
···
639
598
BUG_ON(!irq);
640
599
641
600
raw_spin_lock_irqsave(&irq->irq_lock, flags);
642
-
ret = kvm_vgic_map_irq(vcpu, irq, host_irq, ops);
601
+
ret = kvm_vgic_map_irq(vcpu, irq, host_irq);
643
602
raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
644
603
vgic_put_irq(vcpu->kvm, irq);
645
604
···
726
685
return -EAGAIN;
727
686
728
687
/* SGIs and LPIs cannot be wired up to any device */
729
-
if (!irq_is_ppi(intid) && !vgic_valid_spi(vcpu->kvm, intid))
688
+
if (!irq_is_ppi(vcpu->kvm, intid) && !vgic_valid_spi(vcpu->kvm, intid))
730
689
return -EINVAL;
731
690
732
691
irq = vgic_get_vcpu_irq(vcpu, intid);
···
853
812
vgic_release_deleted_lpis(vcpu->kvm);
854
813
}
855
814
856
-
static inline void vgic_fold_lr_state(struct kvm_vcpu *vcpu)
815
+
static void vgic_fold_state(struct kvm_vcpu *vcpu)
857
816
{
817
+
if (vgic_is_v5(vcpu->kvm)) {
818
+
vgic_v5_fold_ppi_state(vcpu);
819
+
return;
820
+
}
821
+
858
822
if (!*host_data_ptr(last_lr_irq))
859
823
return;
860
824
···
1048
1002
1049
1003
static inline void vgic_save_state(struct kvm_vcpu *vcpu)
1050
1004
{
1051
-
if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1005
+
/* No switch statement here. See comment in vgic_restore_state() */
1006
+
if (vgic_is_v5(vcpu->kvm))
1007
+
vgic_v5_save_state(vcpu);
1008
+
else if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1052
1009
vgic_v2_save_state(vcpu);
1053
1010
else
1054
1011
__vgic_v3_save_state(&vcpu->arch.vgic_cpu.vgic_v3);
···
1060
1011
/* Sync back the hardware VGIC state into our emulation after a guest's run. */
1061
1012
void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
1062
1013
{
1063
-
/* If nesting, emulate the HW effect from L0 to L1 */
1064
-
if (vgic_state_is_nested(vcpu)) {
1065
-
vgic_v3_sync_nested(vcpu);
1066
-
return;
1067
-
}
1014
+
if (vgic_is_v3(vcpu->kvm)) {
1015
+
/* If nesting, emulate the HW effect from L0 to L1 */
1016
+
if (vgic_state_is_nested(vcpu)) {
1017
+
vgic_v3_sync_nested(vcpu);
1018
+
return;
1019
+
}
1068
1020
1069
-
if (vcpu_has_nv(vcpu))
1070
-
vgic_v3_nested_update_mi(vcpu);
1021
+
if (vcpu_has_nv(vcpu))
1022
+
vgic_v3_nested_update_mi(vcpu);
1023
+
}
1071
1024
1072
1025
if (can_access_vgic_from_kernel())
1073
1026
vgic_save_state(vcpu);
1074
1027
1075
-
vgic_fold_lr_state(vcpu);
1076
-
vgic_prune_ap_list(vcpu);
1028
+
vgic_fold_state(vcpu);
1029
+
1030
+
if (!vgic_is_v5(vcpu->kvm))
1031
+
vgic_prune_ap_list(vcpu);
1077
1032
}
1078
1033
1079
1034
/* Sync interrupts that were deactivated through a DIR trap */
···
1093
1040
1094
1041
static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
1095
1042
{
1096
-
if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1043
+
/*
1044
+
* As nice as it would be to restructure this code into a switch
1045
+
* statement as can be found elsewhere, the logic quickly gets ugly.
1046
+
*
1047
+
* __vgic_v3_restore_state() is doing a lot of heavy lifting here. It is
1048
+
* required for GICv3-on-GICv3, GICv2-on-GICv3, GICv3-on-GICv5, and the
1049
+
* no-in-kernel-irqchip case on GICv3 hardware. Hence, adding a switch
1050
+
* here results in much more complex code.
1051
+
*/
1052
+
if (vgic_is_v5(vcpu->kvm))
1053
+
vgic_v5_restore_state(vcpu);
1054
+
else if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1097
1055
vgic_v2_restore_state(vcpu);
1098
1056
else
1099
1057
__vgic_v3_restore_state(&vcpu->arch.vgic_cpu.vgic_v3);
1058
+
}
1059
+
1060
+
static void vgic_flush_state(struct kvm_vcpu *vcpu)
1061
+
{
1062
+
if (vgic_is_v5(vcpu->kvm)) {
1063
+
vgic_v5_flush_ppi_state(vcpu);
1064
+
return;
1065
+
}
1066
+
1067
+
scoped_guard(raw_spinlock, &vcpu->arch.vgic_cpu.ap_list_lock)
1068
+
vgic_flush_lr_state(vcpu);
1100
1069
}
1101
1070
1102
1071
/* Flush our emulation state into the GIC hardware before entering the guest. */
···
1157
1082
1158
1083
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
1159
1084
1160
-
scoped_guard(raw_spinlock, &vcpu->arch.vgic_cpu.ap_list_lock)
1161
-
vgic_flush_lr_state(vcpu);
1085
+
vgic_flush_state(vcpu);
1162
1086
1163
1087
if (can_access_vgic_from_kernel())
1164
1088
vgic_restore_state(vcpu);
1165
1089
1166
-
if (vgic_supports_direct_irqs(vcpu->kvm))
1090
+
if (vgic_supports_direct_irqs(vcpu->kvm) && kvm_vgic_global_state.has_gicv4)
1167
1091
vgic_v4_commit(vcpu);
1168
1092
}
1169
1093
1170
1094
void kvm_vgic_load(struct kvm_vcpu *vcpu)
1171
1095
{
1096
+
const struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
1097
+
1172
1098
if (unlikely(!irqchip_in_kernel(vcpu->kvm) || !vgic_initialized(vcpu->kvm))) {
1173
1099
if (has_vhe() && static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1174
1100
__vgic_v3_activate_traps(&vcpu->arch.vgic_cpu.vgic_v3);
1175
1101
return;
1176
1102
}
1177
1103
1178
-
if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1179
-
vgic_v2_load(vcpu);
1180
-
else
1104
+
switch (dist->vgic_model) {
1105
+
case KVM_DEV_TYPE_ARM_VGIC_V5:
1106
+
vgic_v5_load(vcpu);
1107
+
break;
1108
+
case KVM_DEV_TYPE_ARM_VGIC_V3:
1181
1109
vgic_v3_load(vcpu);
1110
+
break;
1111
+
case KVM_DEV_TYPE_ARM_VGIC_V2:
1112
+
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1113
+
vgic_v3_load(vcpu);
1114
+
else
1115
+
vgic_v2_load(vcpu);
1116
+
break;
1117
+
default:
1118
+
BUG();
1119
+
}
1182
1120
}
1183
1121
1184
1122
void kvm_vgic_put(struct kvm_vcpu *vcpu)
1185
1123
{
1124
+
const struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
1125
+
1186
1126
if (unlikely(!irqchip_in_kernel(vcpu->kvm) || !vgic_initialized(vcpu->kvm))) {
1187
1127
if (has_vhe() && static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1188
1128
__vgic_v3_deactivate_traps(&vcpu->arch.vgic_cpu.vgic_v3);
1189
1129
return;
1190
1130
}
1191
1131
1192
-
if (!static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1193
-
vgic_v2_put(vcpu);
1194
-
else
1132
+
switch (dist->vgic_model) {
1133
+
case KVM_DEV_TYPE_ARM_VGIC_V5:
1134
+
vgic_v5_put(vcpu);
1135
+
break;
1136
+
case KVM_DEV_TYPE_ARM_VGIC_V3:
1195
1137
vgic_v3_put(vcpu);
1138
+
break;
1139
+
case KVM_DEV_TYPE_ARM_VGIC_V2:
1140
+
if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
1141
+
vgic_v3_put(vcpu);
1142
+
else
1143
+
vgic_v2_put(vcpu);
1144
+
break;
1145
+
default:
1146
+
BUG();
1147
+
}
1196
1148
}
1197
1149
1198
1150
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
···
1229
1127
bool pending = false;
1230
1128
unsigned long flags;
1231
1129
struct vgic_vmcr vmcr;
1130
+
1131
+
if (vgic_is_v5(vcpu->kvm))
1132
+
return vgic_v5_has_pending_ppi(vcpu);
1232
1133
1233
1134
if (!vcpu->kvm->arch.vgic.enabled)
1234
1135
return false;
+40
-13
arch/arm64/kvm/vgic/vgic.h
+40
-13
arch/arm64/kvm/vgic/vgic.h
···
187
187
* registers regardless of the hardware backed GIC used.
188
188
*/
189
189
struct vgic_vmcr {
190
+
u32 en; /* GICv5-specific */
190
191
u32 grpen0;
191
192
u32 grpen1;
192
193
···
364
363
void vgic_debug_destroy(struct kvm *kvm);
365
364
366
365
int vgic_v5_probe(const struct gic_kvm_info *info);
366
+
void vgic_v5_reset(struct kvm_vcpu *vcpu);
367
+
int vgic_v5_init(struct kvm *kvm);
368
+
int vgic_v5_map_resources(struct kvm *kvm);
369
+
void vgic_v5_set_ppi_ops(struct kvm_vcpu *vcpu, u32 vintid);
370
+
bool vgic_v5_has_pending_ppi(struct kvm_vcpu *vcpu);
371
+
void vgic_v5_flush_ppi_state(struct kvm_vcpu *vcpu);
372
+
void vgic_v5_fold_ppi_state(struct kvm_vcpu *vcpu);
373
+
void vgic_v5_load(struct kvm_vcpu *vcpu);
374
+
void vgic_v5_put(struct kvm_vcpu *vcpu);
375
+
void vgic_v5_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
376
+
void vgic_v5_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
377
+
void vgic_v5_restore_state(struct kvm_vcpu *vcpu);
378
+
void vgic_v5_save_state(struct kvm_vcpu *vcpu);
367
379
368
380
static inline int vgic_v3_max_apr_idx(struct kvm_vcpu *vcpu)
369
381
{
···
439
425
int vgic_its_inv_lpi(struct kvm *kvm, struct vgic_irq *irq);
440
426
int vgic_its_invall(struct kvm_vcpu *vcpu);
441
427
442
-
bool system_supports_direct_sgis(void);
443
-
bool vgic_supports_direct_msis(struct kvm *kvm);
444
-
bool vgic_supports_direct_sgis(struct kvm *kvm);
445
-
446
-
static inline bool vgic_supports_direct_irqs(struct kvm *kvm)
447
-
{
448
-
return vgic_supports_direct_msis(kvm) || vgic_supports_direct_sgis(kvm);
449
-
}
450
-
451
428
int vgic_v4_init(struct kvm *kvm);
452
429
void vgic_v4_teardown(struct kvm *kvm);
453
430
void vgic_v4_configure_vsgis(struct kvm *kvm);
···
452
447
return kvm_has_feat(kvm, ID_AA64PFR0_EL1, GIC, IMP);
453
448
}
454
449
450
+
static inline bool kvm_has_gicv5(struct kvm *kvm)
451
+
{
452
+
return kvm_has_feat(kvm, ID_AA64PFR2_EL1, GCIE, IMP);
453
+
}
454
+
455
455
void vgic_v3_flush_nested(struct kvm_vcpu *vcpu);
456
456
void vgic_v3_sync_nested(struct kvm_vcpu *vcpu);
457
457
void vgic_v3_load_nested(struct kvm_vcpu *vcpu);
···
464
454
void vgic_v3_handle_nested_maint_irq(struct kvm_vcpu *vcpu);
465
455
void vgic_v3_nested_update_mi(struct kvm_vcpu *vcpu);
466
456
467
-
static inline bool vgic_is_v3_compat(struct kvm *kvm)
457
+
static inline bool vgic_host_has_gicv3(void)
468
458
{
469
-
return cpus_have_final_cap(ARM64_HAS_GICV5_CPUIF) &&
459
+
/*
460
+
* Either the host is a native GICv3, or it is GICv5 with
461
+
* FEAT_GCIE_LEGACY.
462
+
*/
463
+
return kvm_vgic_global_state.type == VGIC_V3 ||
470
464
kvm_vgic_global_state.has_gcie_v3_compat;
471
465
}
472
466
473
-
static inline bool vgic_is_v3(struct kvm *kvm)
467
+
static inline bool vgic_host_has_gicv5(void)
474
468
{
475
-
return kvm_vgic_global_state.type == VGIC_V3 || vgic_is_v3_compat(kvm);
469
+
return kvm_vgic_global_state.type == VGIC_V5;
470
+
}
471
+
472
+
bool system_supports_direct_sgis(void);
473
+
bool vgic_supports_direct_msis(struct kvm *kvm);
474
+
bool vgic_supports_direct_sgis(struct kvm *kvm);
475
+
476
+
static inline bool vgic_supports_direct_irqs(struct kvm *kvm)
477
+
{
478
+
/* GICv5 always supports direct IRQs */
479
+
if (vgic_is_v5(kvm))
480
+
return true;
481
+
482
+
return vgic_supports_direct_msis(kvm) || vgic_supports_direct_sgis(kvm);
476
483
}
477
484
478
485
int vgic_its_debug_init(struct kvm_device *dev);
+480
arch/arm64/tools/sysreg
+480
arch/arm64/tools/sysreg
···
3243
3243
EndEnum
3244
3244
EndSysreg
3245
3245
3246
+
Sysreg ICC_HPPIR_EL1 3 0 12 10 3
3247
+
Res0 63:33
3248
+
Field 32 HPPIV
3249
+
Field 31:29 TYPE
3250
+
Res0 28:24
3251
+
Field 23:0 ID
3252
+
EndSysreg
3253
+
3246
3254
Sysreg ICC_ICSR_EL1 3 0 12 10 4
3247
3255
Res0 63:48
3248
3256
Field 47:32 IAFFID
···
3263
3255
Field 2 Pending
3264
3256
Field 1 Enabled
3265
3257
Field 0 F
3258
+
EndSysreg
3259
+
3260
+
Sysreg ICC_IAFFIDR_EL1 3 0 12 10 5
3261
+
Res0 63:16
3262
+
Field 15:0 IAFFID
3266
3263
EndSysreg
3267
3264
3268
3265
SysregFields ICC_PPI_ENABLERx_EL1
···
3674
3661
Field 15 SMPS
3675
3662
Res0 14:12
3676
3663
Field 11:0 AFFINITY
3664
+
EndSysreg
3665
+
3666
+
Sysreg ICC_APR_EL1 3 1 12 0 0
3667
+
Res0 63:32
3668
+
Field 31 P31
3669
+
Field 30 P30
3670
+
Field 29 P29
3671
+
Field 28 P28
3672
+
Field 27 P27
3673
+
Field 26 P26
3674
+
Field 25 P25
3675
+
Field 24 P24
3676
+
Field 23 P23
3677
+
Field 22 P22
3678
+
Field 21 P21
3679
+
Field 20 P20
3680
+
Field 19 P19
3681
+
Field 18 P18
3682
+
Field 17 P17
3683
+
Field 16 P16
3684
+
Field 15 P15
3685
+
Field 14 P14
3686
+
Field 13 P13
3687
+
Field 12 P12
3688
+
Field 11 P11
3689
+
Field 10 P10
3690
+
Field 9 P9
3691
+
Field 8 P8
3692
+
Field 7 P7
3693
+
Field 6 P6
3694
+
Field 5 P5
3695
+
Field 4 P4
3696
+
Field 3 P3
3697
+
Field 2 P2
3698
+
Field 1 P1
3699
+
Field 0 P0
3677
3700
EndSysreg
3678
3701
3679
3702
Sysreg ICC_CR0_EL1 3 1 12 0 1
···
4736
4687
Field 15:0 PhyPARTID28
4737
4688
EndSysreg
4738
4689
4690
+
Sysreg ICH_APR_EL2 3 4 12 8 4
4691
+
Res0 63:32
4692
+
Field 31 P31
4693
+
Field 30 P30
4694
+
Field 29 P29
4695
+
Field 28 P28
4696
+
Field 27 P27
4697
+
Field 26 P26
4698
+
Field 25 P25
4699
+
Field 24 P24
4700
+
Field 23 P23
4701
+
Field 22 P22
4702
+
Field 21 P21
4703
+
Field 20 P20
4704
+
Field 19 P19
4705
+
Field 18 P18
4706
+
Field 17 P17
4707
+
Field 16 P16
4708
+
Field 15 P15
4709
+
Field 14 P14
4710
+
Field 13 P13
4711
+
Field 12 P12
4712
+
Field 11 P11
4713
+
Field 10 P10
4714
+
Field 9 P9
4715
+
Field 8 P8
4716
+
Field 7 P7
4717
+
Field 6 P6
4718
+
Field 5 P5
4719
+
Field 4 P4
4720
+
Field 3 P3
4721
+
Field 2 P2
4722
+
Field 1 P1
4723
+
Field 0 P0
4724
+
EndSysreg
4725
+
4739
4726
Sysreg ICH_HFGRTR_EL2 3 4 12 9 4
4740
4727
Res0 63:21
4741
4728
Field 20 ICC_PPI_ACTIVERn_EL1
···
4818
4733
Field 2 GICCDPRI
4819
4734
Field 1 GICCDDIS
4820
4735
Field 0 GICCDEN
4736
+
EndSysreg
4737
+
4738
+
SysregFields ICH_PPI_DVIRx_EL2
4739
+
Field 63 DVI63
4740
+
Field 62 DVI62
4741
+
Field 61 DVI61
4742
+
Field 60 DVI60
4743
+
Field 59 DVI59
4744
+
Field 58 DVI58
4745
+
Field 57 DVI57
4746
+
Field 56 DVI56
4747
+
Field 55 DVI55
4748
+
Field 54 DVI54
4749
+
Field 53 DVI53
4750
+
Field 52 DVI52
4751
+
Field 51 DVI51
4752
+
Field 50 DVI50
4753
+
Field 49 DVI49
4754
+
Field 48 DVI48
4755
+
Field 47 DVI47
4756
+
Field 46 DVI46
4757
+
Field 45 DVI45
4758
+
Field 44 DVI44
4759
+
Field 43 DVI43
4760
+
Field 42 DVI42
4761
+
Field 41 DVI41
4762
+
Field 40 DVI40
4763
+
Field 39 DVI39
4764
+
Field 38 DVI38
4765
+
Field 37 DVI37
4766
+
Field 36 DVI36
4767
+
Field 35 DVI35
4768
+
Field 34 DVI34
4769
+
Field 33 DVI33
4770
+
Field 32 DVI32
4771
+
Field 31 DVI31
4772
+
Field 30 DVI30
4773
+
Field 29 DVI29
4774
+
Field 28 DVI28
4775
+
Field 27 DVI27
4776
+
Field 26 DVI26
4777
+
Field 25 DVI25
4778
+
Field 24 DVI24
4779
+
Field 23 DVI23
4780
+
Field 22 DVI22
4781
+
Field 21 DVI21
4782
+
Field 20 DVI20
4783
+
Field 19 DVI19
4784
+
Field 18 DVI18
4785
+
Field 17 DVI17
4786
+
Field 16 DVI16
4787
+
Field 15 DVI15
4788
+
Field 14 DVI14
4789
+
Field 13 DVI13
4790
+
Field 12 DVI12
4791
+
Field 11 DVI11
4792
+
Field 10 DVI10
4793
+
Field 9 DVI9
4794
+
Field 8 DVI8
4795
+
Field 7 DVI7
4796
+
Field 6 DVI6
4797
+
Field 5 DVI5
4798
+
Field 4 DVI4
4799
+
Field 3 DVI3
4800
+
Field 2 DVI2
4801
+
Field 1 DVI1
4802
+
Field 0 DVI0
4803
+
EndSysregFields
4804
+
4805
+
Sysreg ICH_PPI_DVIR0_EL2 3 4 12 10 0
4806
+
Fields ICH_PPI_DVIx_EL2
4807
+
EndSysreg
4808
+
4809
+
Sysreg ICH_PPI_DVIR1_EL2 3 4 12 10 1
4810
+
Fields ICH_PPI_DVIx_EL2
4811
+
EndSysreg
4812
+
4813
+
SysregFields ICH_PPI_ENABLERx_EL2
4814
+
Field 63 EN63
4815
+
Field 62 EN62
4816
+
Field 61 EN61
4817
+
Field 60 EN60
4818
+
Field 59 EN59
4819
+
Field 58 EN58
4820
+
Field 57 EN57
4821
+
Field 56 EN56
4822
+
Field 55 EN55
4823
+
Field 54 EN54
4824
+
Field 53 EN53
4825
+
Field 52 EN52
4826
+
Field 51 EN51
4827
+
Field 50 EN50
4828
+
Field 49 EN49
4829
+
Field 48 EN48
4830
+
Field 47 EN47
4831
+
Field 46 EN46
4832
+
Field 45 EN45
4833
+
Field 44 EN44
4834
+
Field 43 EN43
4835
+
Field 42 EN42
4836
+
Field 41 EN41
4837
+
Field 40 EN40
4838
+
Field 39 EN39
4839
+
Field 38 EN38
4840
+
Field 37 EN37
4841
+
Field 36 EN36
4842
+
Field 35 EN35
4843
+
Field 34 EN34
4844
+
Field 33 EN33
4845
+
Field 32 EN32
4846
+
Field 31 EN31
4847
+
Field 30 EN30
4848
+
Field 29 EN29
4849
+
Field 28 EN28
4850
+
Field 27 EN27
4851
+
Field 26 EN26
4852
+
Field 25 EN25
4853
+
Field 24 EN24
4854
+
Field 23 EN23
4855
+
Field 22 EN22
4856
+
Field 21 EN21
4857
+
Field 20 EN20
4858
+
Field 19 EN19
4859
+
Field 18 EN18
4860
+
Field 17 EN17
4861
+
Field 16 EN16
4862
+
Field 15 EN15
4863
+
Field 14 EN14
4864
+
Field 13 EN13
4865
+
Field 12 EN12
4866
+
Field 11 EN11
4867
+
Field 10 EN10
4868
+
Field 9 EN9
4869
+
Field 8 EN8
4870
+
Field 7 EN7
4871
+
Field 6 EN6
4872
+
Field 5 EN5
4873
+
Field 4 EN4
4874
+
Field 3 EN3
4875
+
Field 2 EN2
4876
+
Field 1 EN1
4877
+
Field 0 EN0
4878
+
EndSysregFields
4879
+
4880
+
Sysreg ICH_PPI_ENABLER0_EL2 3 4 12 10 2
4881
+
Fields ICH_PPI_ENABLERx_EL2
4882
+
EndSysreg
4883
+
4884
+
Sysreg ICH_PPI_ENABLER1_EL2 3 4 12 10 3
4885
+
Fields ICH_PPI_ENABLERx_EL2
4886
+
EndSysreg
4887
+
4888
+
SysregFields ICH_PPI_PENDRx_EL2
4889
+
Field 63 PEND63
4890
+
Field 62 PEND62
4891
+
Field 61 PEND61
4892
+
Field 60 PEND60
4893
+
Field 59 PEND59
4894
+
Field 58 PEND58
4895
+
Field 57 PEND57
4896
+
Field 56 PEND56
4897
+
Field 55 PEND55
4898
+
Field 54 PEND54
4899
+
Field 53 PEND53
4900
+
Field 52 PEND52
4901
+
Field 51 PEND51
4902
+
Field 50 PEND50
4903
+
Field 49 PEND49
4904
+
Field 48 PEND48
4905
+
Field 47 PEND47
4906
+
Field 46 PEND46
4907
+
Field 45 PEND45
4908
+
Field 44 PEND44
4909
+
Field 43 PEND43
4910
+
Field 42 PEND42
4911
+
Field 41 PEND41
4912
+
Field 40 PEND40
4913
+
Field 39 PEND39
4914
+
Field 38 PEND38
4915
+
Field 37 PEND37
4916
+
Field 36 PEND36
4917
+
Field 35 PEND35
4918
+
Field 34 PEND34
4919
+
Field 33 PEND33
4920
+
Field 32 PEND32
4921
+
Field 31 PEND31
4922
+
Field 30 PEND30
4923
+
Field 29 PEND29
4924
+
Field 28 PEND28
4925
+
Field 27 PEND27
4926
+
Field 26 PEND26
4927
+
Field 25 PEND25
4928
+
Field 24 PEND24
4929
+
Field 23 PEND23
4930
+
Field 22 PEND22
4931
+
Field 21 PEND21
4932
+
Field 20 PEND20
4933
+
Field 19 PEND19
4934
+
Field 18 PEND18
4935
+
Field 17 PEND17
4936
+
Field 16 PEND16
4937
+
Field 15 PEND15
4938
+
Field 14 PEND14
4939
+
Field 13 PEND13
4940
+
Field 12 PEND12
4941
+
Field 11 PEND11
4942
+
Field 10 PEND10
4943
+
Field 9 PEND9
4944
+
Field 8 PEND8
4945
+
Field 7 PEND7
4946
+
Field 6 PEND6
4947
+
Field 5 PEND5
4948
+
Field 4 PEND4
4949
+
Field 3 PEND3
4950
+
Field 2 PEND2
4951
+
Field 1 PEND1
4952
+
Field 0 PEND0
4953
+
EndSysregFields
4954
+
4955
+
Sysreg ICH_PPI_PENDR0_EL2 3 4 12 10 4
4956
+
Fields ICH_PPI_PENDRx_EL2
4957
+
EndSysreg
4958
+
4959
+
Sysreg ICH_PPI_PENDR1_EL2 3 4 12 10 5
4960
+
Fields ICH_PPI_PENDRx_EL2
4961
+
EndSysreg
4962
+
4963
+
SysregFields ICH_PPI_ACTIVERx_EL2
4964
+
Field 63 ACTIVE63
4965
+
Field 62 ACTIVE62
4966
+
Field 61 ACTIVE61
4967
+
Field 60 ACTIVE60
4968
+
Field 59 ACTIVE59
4969
+
Field 58 ACTIVE58
4970
+
Field 57 ACTIVE57
4971
+
Field 56 ACTIVE56
4972
+
Field 55 ACTIVE55
4973
+
Field 54 ACTIVE54
4974
+
Field 53 ACTIVE53
4975
+
Field 52 ACTIVE52
4976
+
Field 51 ACTIVE51
4977
+
Field 50 ACTIVE50
4978
+
Field 49 ACTIVE49
4979
+
Field 48 ACTIVE48
4980
+
Field 47 ACTIVE47
4981
+
Field 46 ACTIVE46
4982
+
Field 45 ACTIVE45
4983
+
Field 44 ACTIVE44
4984
+
Field 43 ACTIVE43
4985
+
Field 42 ACTIVE42
4986
+
Field 41 ACTIVE41
4987
+
Field 40 ACTIVE40
4988
+
Field 39 ACTIVE39
4989
+
Field 38 ACTIVE38
4990
+
Field 37 ACTIVE37
4991
+
Field 36 ACTIVE36
4992
+
Field 35 ACTIVE35
4993
+
Field 34 ACTIVE34
4994
+
Field 33 ACTIVE33
4995
+
Field 32 ACTIVE32
4996
+
Field 31 ACTIVE31
4997
+
Field 30 ACTIVE30
4998
+
Field 29 ACTIVE29
4999
+
Field 28 ACTIVE28
5000
+
Field 27 ACTIVE27
5001
+
Field 26 ACTIVE26
5002
+
Field 25 ACTIVE25
5003
+
Field 24 ACTIVE24
5004
+
Field 23 ACTIVE23
5005
+
Field 22 ACTIVE22
5006
+
Field 21 ACTIVE21
5007
+
Field 20 ACTIVE20
5008
+
Field 19 ACTIVE19
5009
+
Field 18 ACTIVE18
5010
+
Field 17 ACTIVE17
5011
+
Field 16 ACTIVE16
5012
+
Field 15 ACTIVE15
5013
+
Field 14 ACTIVE14
5014
+
Field 13 ACTIVE13
5015
+
Field 12 ACTIVE12
5016
+
Field 11 ACTIVE11
5017
+
Field 10 ACTIVE10
5018
+
Field 9 ACTIVE9
5019
+
Field 8 ACTIVE8
5020
+
Field 7 ACTIVE7
5021
+
Field 6 ACTIVE6
5022
+
Field 5 ACTIVE5
5023
+
Field 4 ACTIVE4
5024
+
Field 3 ACTIVE3
5025
+
Field 2 ACTIVE2
5026
+
Field 1 ACTIVE1
5027
+
Field 0 ACTIVE0
5028
+
EndSysregFields
5029
+
5030
+
Sysreg ICH_PPI_ACTIVER0_EL2 3 4 12 10 6
5031
+
Fields ICH_PPI_ACTIVERx_EL2
5032
+
EndSysreg
5033
+
5034
+
Sysreg ICH_PPI_ACTIVER1_EL2 3 4 12 10 7
5035
+
Fields ICH_PPI_ACTIVERx_EL2
4821
5036
EndSysreg
4822
5037
4823
5038
Sysreg ICH_HCR_EL2 3 4 12 11 0
···
5174
4789
Field 0 En
5175
4790
EndSysreg
5176
4791
4792
+
Sysreg ICH_CONTEXTR_EL2 3 4 12 11 6
4793
+
Field 63 V
4794
+
Field 62 F
4795
+
Field 61 IRICHPPIDIS
4796
+
Field 60 DB
4797
+
Field 59:55 DBPM
4798
+
Res0 54:48
4799
+
Field 47:32 VPE
4800
+
Res0 31:16
4801
+
Field 15:0 VM
4802
+
EndSysreg
4803
+
5177
4804
Sysreg ICH_VMCR_EL2 3 4 12 11 7
5178
4805
Prefix FEAT_GCIE
5179
4806
Res0 63:32
···
5205
4808
Field 2 VAckCtl
5206
4809
Field 1 VENG1
5207
4810
Field 0 VENG0
4811
+
EndSysreg
4812
+
4813
+
SysregFields ICH_PPI_PRIORITYRx_EL2
4814
+
Res0 63:61
4815
+
Field 60:56 Priority7
4816
+
Res0 55:53
4817
+
Field 52:48 Priority6
4818
+
Res0 47:45
4819
+
Field 44:40 Priority5
4820
+
Res0 39:37
4821
+
Field 36:32 Priority4
4822
+
Res0 31:29
4823
+
Field 28:24 Priority3
4824
+
Res0 23:21
4825
+
Field 20:16 Priority2
4826
+
Res0 15:13
4827
+
Field 12:8 Priority1
4828
+
Res0 7:5
4829
+
Field 4:0 Priority0
4830
+
EndSysregFields
4831
+
4832
+
Sysreg ICH_PPI_PRIORITYR0_EL2 3 4 12 14 0
4833
+
Fields ICH_PPI_PRIORITYRx_EL2
4834
+
EndSysreg
4835
+
4836
+
Sysreg ICH_PPI_PRIORITYR1_EL2 3 4 12 14 1
4837
+
Fields ICH_PPI_PRIORITYRx_EL2
4838
+
EndSysreg
4839
+
4840
+
Sysreg ICH_PPI_PRIORITYR2_EL2 3 4 12 14 2
4841
+
Fields ICH_PPI_PRIORITYRx_EL2
4842
+
EndSysreg
4843
+
4844
+
Sysreg ICH_PPI_PRIORITYR3_EL2 3 4 12 14 3
4845
+
Fields ICH_PPI_PRIORITYRx_EL2
4846
+
EndSysreg
4847
+
4848
+
Sysreg ICH_PPI_PRIORITYR4_EL2 3 4 12 14 4
4849
+
Fields ICH_PPI_PRIORITYRx_EL2
4850
+
EndSysreg
4851
+
4852
+
Sysreg ICH_PPI_PRIORITYR5_EL2 3 4 12 14 5
4853
+
Fields ICH_PPI_PRIORITYRx_EL2
4854
+
EndSysreg
4855
+
4856
+
Sysreg ICH_PPI_PRIORITYR6_EL2 3 4 12 14 6
4857
+
Fields ICH_PPI_PRIORITYRx_EL2
4858
+
EndSysreg
4859
+
4860
+
Sysreg ICH_PPI_PRIORITYR7_EL2 3 4 12 14 7
4861
+
Fields ICH_PPI_PRIORITYRx_EL2
4862
+
EndSysreg
4863
+
4864
+
Sysreg ICH_PPI_PRIORITYR8_EL2 3 4 12 15 0
4865
+
Fields ICH_PPI_PRIORITYRx_EL2
4866
+
EndSysreg
4867
+
4868
+
Sysreg ICH_PPI_PRIORITYR9_EL2 3 4 12 15 1
4869
+
Fields ICH_PPI_PRIORITYRx_EL2
4870
+
EndSysreg
4871
+
4872
+
Sysreg ICH_PPI_PRIORITYR10_EL2 3 4 12 15 2
4873
+
Fields ICH_PPI_PRIORITYRx_EL2
4874
+
EndSysreg
4875
+
4876
+
Sysreg ICH_PPI_PRIORITYR11_EL2 3 4 12 15 3
4877
+
Fields ICH_PPI_PRIORITYRx_EL2
4878
+
EndSysreg
4879
+
4880
+
Sysreg ICH_PPI_PRIORITYR12_EL2 3 4 12 15 4
4881
+
Fields ICH_PPI_PRIORITYRx_EL2
4882
+
EndSysreg
4883
+
4884
+
Sysreg ICH_PPI_PRIORITYR13_EL2 3 4 12 15 5
4885
+
Fields ICH_PPI_PRIORITYRx_EL2
4886
+
EndSysreg
4887
+
4888
+
Sysreg ICH_PPI_PRIORITYR14_EL2 3 4 12 15 6
4889
+
Fields ICH_PPI_PRIORITYRx_EL2
4890
+
EndSysreg
4891
+
4892
+
Sysreg ICH_PPI_PRIORITYR15_EL2 3 4 12 15 7
4893
+
Fields ICH_PPI_PRIORITYRx_EL2
5208
4894
EndSysreg
5209
4895
5210
4896
Sysreg CONTEXTIDR_EL2 3 4 13 0 1
+18
drivers/irqchip/irq-gic-v5.c
+18
drivers/irqchip/irq-gic-v5.c
···
511
511
return !!(read_ppi_sysreg_s(hwirq, PPI_HM) & bit);
512
512
}
513
513
514
+
static int gicv5_ppi_irq_set_type(struct irq_data *d, unsigned int type)
515
+
{
516
+
/*
517
+
* GICv5's PPIs do not have a configurable trigger or handling
518
+
* mode. Check that the attempt to set a type matches what the
519
+
* hardware reports in the HMR, and error on a mismatch.
520
+
*/
521
+
522
+
if (type & IRQ_TYPE_EDGE_BOTH && gicv5_ppi_irq_is_level(d->hwirq))
523
+
return -EINVAL;
524
+
525
+
if (type & IRQ_TYPE_LEVEL_MASK && !gicv5_ppi_irq_is_level(d->hwirq))
526
+
return -EINVAL;
527
+
528
+
return 0;
529
+
}
530
+
514
531
static int gicv5_ppi_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu)
515
532
{
516
533
if (vcpu)
···
543
526
.irq_mask = gicv5_ppi_irq_mask,
544
527
.irq_unmask = gicv5_ppi_irq_unmask,
545
528
.irq_eoi = gicv5_ppi_irq_eoi,
529
+
.irq_set_type = gicv5_ppi_irq_set_type,
546
530
.irq_get_irqchip_state = gicv5_ppi_irq_get_irqchip_state,
547
531
.irq_set_irqchip_state = gicv5_ppi_irq_set_irqchip_state,
548
532
.irq_set_vcpu_affinity = gicv5_ppi_irq_set_vcpu_affinity,
+10
-1
include/kvm/arm_arch_timer.h
+10
-1
include/kvm/arm_arch_timer.h
···
10
10
#include <linux/clocksource.h>
11
11
#include <linux/hrtimer.h>
12
12
13
+
#include <linux/irqchip/arm-gic-v5.h>
14
+
13
15
enum kvm_arch_timers {
14
16
TIMER_PTIMER,
15
17
TIMER_VTIMER,
···
49
47
u64 poffset;
50
48
51
49
/* The PPI for each timer, global to the VM */
52
-
u8 ppi[NR_KVM_TIMERS];
50
+
u32 ppi[NR_KVM_TIMERS];
53
51
};
54
52
55
53
struct arch_timer_context {
···
76
74
77
75
/* Duplicated state from arch_timer.c for convenience */
78
76
u32 host_timer_irq;
77
+
78
+
/* Is this a direct timer? */
79
+
bool direct;
79
80
};
80
81
81
82
struct timer_map {
···
134
129
#define timer_context_to_vcpu(ctx) container_of((ctx), struct kvm_vcpu, arch.timer_cpu.timers[(ctx)->timer_id])
135
130
#define timer_vm_data(ctx) (&(timer_context_to_vcpu(ctx)->kvm->arch.timer_data))
136
131
#define timer_irq(ctx) (timer_vm_data(ctx)->ppi[arch_timer_ctx_index(ctx)])
132
+
133
+
#define get_vgic_ppi(k, i) (((k)->arch.vgic.vgic_model != KVM_DEV_TYPE_ARM_VGIC_V5) ? \
134
+
(i) : (FIELD_PREP(GICV5_HWIRQ_ID, i) | \
135
+
FIELD_PREP(GICV5_HWIRQ_TYPE, GICV5_HWIRQ_TYPE_PPI)))
137
136
138
137
u64 kvm_arm_timer_read_sysreg(struct kvm_vcpu *vcpu,
139
138
enum kvm_arch_timers tmr,
+4
-1
include/kvm/arm_pmu.h
+4
-1
include/kvm/arm_pmu.h
···
12
12
13
13
#define KVM_ARMV8_PMU_MAX_COUNTERS 32
14
14
15
+
/* PPI #23 - architecturally specified for GICv5 */
16
+
#define KVM_ARMV8_PMU_GICV5_IRQ 0x20000017
17
+
15
18
#if IS_ENABLED(CONFIG_HW_PERF_EVENTS) && IS_ENABLED(CONFIG_KVM)
16
19
struct kvm_pmc {
17
20
u8 idx; /* index into the pmu->pmc array */
···
41
38
};
42
39
43
40
bool kvm_supports_guest_pmuv3(void);
44
-
#define kvm_arm_pmu_irq_initialized(v) ((v)->arch.pmu.irq_num >= VGIC_NR_SGIS)
41
+
#define kvm_arm_pmu_irq_initialized(v) ((v)->arch.pmu.irq_num != 0)
45
42
u64 kvm_pmu_get_counter_value(struct kvm_vcpu *vcpu, u64 select_idx);
46
43
void kvm_pmu_set_counter_value(struct kvm_vcpu *vcpu, u64 select_idx, u64 val);
47
44
void kvm_pmu_set_counter_value_user(struct kvm_vcpu *vcpu, u64 select_idx, u64 val);
+185
-6
include/kvm/arm_vgic.h
+185
-6
include/kvm/arm_vgic.h
···
19
19
#include <linux/jump_label.h>
20
20
21
21
#include <linux/irqchip/arm-gic-v4.h>
22
+
#include <linux/irqchip/arm-gic-v5.h>
22
23
24
+
#define VGIC_V5_MAX_CPUS 512
23
25
#define VGIC_V3_MAX_CPUS 512
24
26
#define VGIC_V2_MAX_CPUS 8
25
27
#define VGIC_NR_IRQS_LEGACY 256
···
33
31
#define VGIC_MIN_LPI 8192
34
32
#define KVM_IRQCHIP_NUM_PINS (1020 - 32)
35
33
36
-
#define irq_is_ppi(irq) ((irq) >= VGIC_NR_SGIS && (irq) < VGIC_NR_PRIVATE_IRQS)
37
-
#define irq_is_spi(irq) ((irq) >= VGIC_NR_PRIVATE_IRQS && \
38
-
(irq) <= VGIC_MAX_SPI)
34
+
/*
35
+
* GICv5 supports 128 PPIs, but only the first 64 are architected. We only
36
+
* support the timers and PMU in KVM, both of which are architected. Rather than
37
+
* handling twice the state, we instead opt to only support the architected set
38
+
* in KVM for now. At a future stage, this can be bumped up to 128, if required.
39
+
*/
40
+
#define VGIC_V5_NR_PRIVATE_IRQS 64
41
+
42
+
#define is_v5_type(t, i) (FIELD_GET(GICV5_HWIRQ_TYPE, (i)) == (t))
43
+
44
+
#define __irq_is_sgi(t, i) \
45
+
({ \
46
+
bool __ret; \
47
+
\
48
+
switch (t) { \
49
+
case KVM_DEV_TYPE_ARM_VGIC_V5: \
50
+
__ret = false; \
51
+
break; \
52
+
default: \
53
+
__ret = (i) < VGIC_NR_SGIS; \
54
+
} \
55
+
\
56
+
__ret; \
57
+
})
58
+
59
+
#define __irq_is_ppi(t, i) \
60
+
({ \
61
+
bool __ret; \
62
+
\
63
+
switch (t) { \
64
+
case KVM_DEV_TYPE_ARM_VGIC_V5: \
65
+
__ret = is_v5_type(GICV5_HWIRQ_TYPE_PPI, (i)); \
66
+
break; \
67
+
default: \
68
+
__ret = (i) >= VGIC_NR_SGIS; \
69
+
__ret &= (i) < VGIC_NR_PRIVATE_IRQS; \
70
+
} \
71
+
\
72
+
__ret; \
73
+
})
74
+
75
+
#define __irq_is_spi(t, i) \
76
+
({ \
77
+
bool __ret; \
78
+
\
79
+
switch (t) { \
80
+
case KVM_DEV_TYPE_ARM_VGIC_V5: \
81
+
__ret = is_v5_type(GICV5_HWIRQ_TYPE_SPI, (i)); \
82
+
break; \
83
+
default: \
84
+
__ret = (i) <= VGIC_MAX_SPI; \
85
+
__ret &= (i) >= VGIC_NR_PRIVATE_IRQS; \
86
+
} \
87
+
\
88
+
__ret; \
89
+
})
90
+
91
+
#define __irq_is_lpi(t, i) \
92
+
({ \
93
+
bool __ret; \
94
+
\
95
+
switch (t) { \
96
+
case KVM_DEV_TYPE_ARM_VGIC_V5: \
97
+
__ret = is_v5_type(GICV5_HWIRQ_TYPE_LPI, (i)); \
98
+
break; \
99
+
default: \
100
+
__ret = (i) >= 8192; \
101
+
} \
102
+
\
103
+
__ret; \
104
+
})
105
+
106
+
#define irq_is_sgi(k, i) __irq_is_sgi((k)->arch.vgic.vgic_model, i)
107
+
#define irq_is_ppi(k, i) __irq_is_ppi((k)->arch.vgic.vgic_model, i)
108
+
#define irq_is_spi(k, i) __irq_is_spi((k)->arch.vgic.vgic_model, i)
109
+
#define irq_is_lpi(k, i) __irq_is_lpi((k)->arch.vgic.vgic_model, i)
110
+
111
+
#define irq_is_private(k, i) (irq_is_ppi(k, i) || irq_is_sgi(k, i))
112
+
113
+
#define vgic_v5_get_hwirq_id(x) FIELD_GET(GICV5_HWIRQ_ID, (x))
114
+
#define vgic_v5_set_hwirq_id(x) FIELD_PREP(GICV5_HWIRQ_ID, (x))
115
+
116
+
#define __vgic_v5_set_type(t) (FIELD_PREP(GICV5_HWIRQ_TYPE, GICV5_HWIRQ_TYPE_##t))
117
+
#define vgic_v5_make_ppi(x) (__vgic_v5_set_type(PPI) | vgic_v5_set_hwirq_id(x))
118
+
#define vgic_v5_make_spi(x) (__vgic_v5_set_type(SPI) | vgic_v5_set_hwirq_id(x))
119
+
#define vgic_v5_make_lpi(x) (__vgic_v5_set_type(LPI) | vgic_v5_set_hwirq_id(x))
120
+
121
+
#define __vgic_is_v(k, v) ((k)->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V##v)
122
+
#define vgic_is_v3(k) (__vgic_is_v(k, 3))
123
+
#define vgic_is_v5(k) (__vgic_is_v(k, 5))
39
124
40
125
enum vgic_type {
41
126
VGIC_V2, /* Good ol' GICv2 */
···
190
101
VGIC_CONFIG_LEVEL
191
102
};
192
103
104
+
struct vgic_irq;
105
+
193
106
/*
194
107
* Per-irq ops overriding some common behavious.
195
108
*
···
210
119
* peaking into the physical GIC.
211
120
*/
212
121
bool (*get_input_level)(int vintid);
122
+
123
+
/*
124
+
* Function pointer to override the queuing of an IRQ.
125
+
*/
126
+
bool (*queue_irq_unlock)(struct kvm *kvm, struct vgic_irq *irq,
127
+
unsigned long flags) __releases(&irq->irq_lock);
128
+
129
+
/*
130
+
* Callback function pointer to either enable or disable direct
131
+
* injection for a mapped interrupt.
132
+
*/
133
+
void (*set_direct_injection)(struct kvm_vcpu *vcpu,
134
+
struct vgic_irq *irq, bool direct);
213
135
};
214
136
215
137
struct vgic_irq {
···
342
238
struct list_head list;
343
239
};
344
240
241
+
struct vgic_v5_vm {
242
+
/*
243
+
* We only expose a subset of PPIs to the guest. This subset is a
244
+
* combination of the PPIs that are actually implemented and what we
245
+
* actually choose to expose.
246
+
*/
247
+
DECLARE_BITMAP(vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
248
+
249
+
/* A mask of the PPIs that are exposed for userspace to drive. */
250
+
DECLARE_BITMAP(userspace_ppis, VGIC_V5_NR_PRIVATE_IRQS);
251
+
252
+
/*
253
+
* The HMR itself is handled by the hardware, but we still need to have
254
+
* a mask that we can use when merging in pending state (only the state
255
+
* of Edge PPIs is merged back in from the guest an the HMR provides a
256
+
* convenient way to do that).
257
+
*/
258
+
DECLARE_BITMAP(vgic_ppi_hmr, VGIC_V5_NR_PRIVATE_IRQS);
259
+
};
260
+
345
261
struct vgic_dist {
346
262
bool in_kernel;
347
263
bool ready;
···
434
310
* else.
435
311
*/
436
312
struct its_vm its_vm;
313
+
314
+
/*
315
+
* GICv5 per-VM data.
316
+
*/
317
+
struct vgic_v5_vm gicv5_vm;
437
318
};
438
319
439
320
struct vgic_v2_cpu_if {
···
469
340
unsigned int used_lrs;
470
341
};
471
342
343
+
struct vgic_v5_cpu_if {
344
+
u64 vgic_apr;
345
+
u64 vgic_vmcr;
346
+
347
+
/* PPI register state */
348
+
DECLARE_BITMAP(vgic_ppi_dvir, VGIC_V5_NR_PRIVATE_IRQS);
349
+
DECLARE_BITMAP(vgic_ppi_activer, VGIC_V5_NR_PRIVATE_IRQS);
350
+
DECLARE_BITMAP(vgic_ppi_enabler, VGIC_V5_NR_PRIVATE_IRQS);
351
+
/* We have one byte (of which 5 bits are used) per PPI for priority */
352
+
u64 vgic_ppi_priorityr[VGIC_V5_NR_PRIVATE_IRQS / 8];
353
+
354
+
/*
355
+
* The ICSR is re-used across host and guest, and hence it needs to be
356
+
* saved/restored. Only one copy is required as the host should block
357
+
* preemption between executing GIC CDRCFG and acccessing the
358
+
* ICC_ICSR_EL1. A guest, of course, can never guarantee this, and hence
359
+
* it is the hyp's responsibility to keep the state constistent.
360
+
*/
361
+
u64 vgic_icsr;
362
+
363
+
struct gicv5_vpe gicv5_vpe;
364
+
};
365
+
366
+
/* What PPI capabilities does a GICv5 host have */
367
+
struct vgic_v5_ppi_caps {
368
+
DECLARE_BITMAP(impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
369
+
};
370
+
472
371
struct vgic_cpu {
473
372
/* CPU vif control registers for world switch */
474
373
union {
475
374
struct vgic_v2_cpu_if vgic_v2;
476
375
struct vgic_v3_cpu_if vgic_v3;
376
+
struct vgic_v5_cpu_if vgic_v5;
477
377
};
478
378
479
379
struct vgic_irq *private_irqs;
···
550
392
void kvm_vgic_destroy(struct kvm *kvm);
551
393
void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu);
552
394
int kvm_vgic_map_resources(struct kvm *kvm);
395
+
void kvm_vgic_finalize_idregs(struct kvm *kvm);
553
396
int kvm_vgic_hyp_init(void);
554
397
void kvm_vgic_init_cpu_hardware(void);
555
398
556
399
int kvm_vgic_inject_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,
557
400
unsigned int intid, bool level, void *owner);
401
+
void kvm_vgic_set_irq_ops(struct kvm_vcpu *vcpu, u32 vintid,
402
+
struct irq_ops *ops);
403
+
void kvm_vgic_clear_irq_ops(struct kvm_vcpu *vcpu, u32 vintid);
558
404
int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, unsigned int host_irq,
559
-
u32 vintid, struct irq_ops *ops);
405
+
u32 vintid);
560
406
int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid);
561
407
int kvm_vgic_get_map(struct kvm_vcpu *vcpu, unsigned int vintid);
562
408
bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int vintid);
···
576
414
577
415
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
578
416
#define vgic_initialized(k) ((k)->arch.vgic.initialized)
579
-
#define vgic_valid_spi(k, i) (((i) >= VGIC_NR_PRIVATE_IRQS) && \
580
-
((i) < (k)->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS))
417
+
#define vgic_valid_spi(k, i) \
418
+
({ \
419
+
bool __ret = irq_is_spi(k, i); \
420
+
\
421
+
switch ((k)->arch.vgic.vgic_model) { \
422
+
case KVM_DEV_TYPE_ARM_VGIC_V5: \
423
+
__ret &= FIELD_GET(GICV5_HWIRQ_ID, i) < (k)->arch.vgic.nr_spis; \
424
+
break; \
425
+
default: \
426
+
__ret &= (i) < ((k)->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS); \
427
+
} \
428
+
\
429
+
__ret; \
430
+
})
581
431
582
432
bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
583
433
void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
···
628
454
int vgic_v4_load(struct kvm_vcpu *vcpu);
629
455
void vgic_v4_commit(struct kvm_vcpu *vcpu);
630
456
int vgic_v4_put(struct kvm_vcpu *vcpu);
457
+
458
+
int vgic_v5_finalize_ppi_state(struct kvm *kvm);
459
+
bool vgic_v5_ppi_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,
460
+
unsigned long flags);
461
+
void vgic_v5_set_ppi_dvi(struct kvm_vcpu *vcpu, struct vgic_irq *irq, bool dvi);
631
462
632
463
bool vgic_state_is_nested(struct kvm_vcpu *vcpu);
633
464
+27
include/linux/irqchip/arm-gic-v5.h
+27
include/linux/irqchip/arm-gic-v5.h
···
25
25
#define GICV5_HWIRQ_TYPE_SPI UL(0x3)
26
26
27
27
/*
28
+
* Architected PPIs
29
+
*/
30
+
#define GICV5_ARCH_PPI_S_DB_PPI 0x0
31
+
#define GICV5_ARCH_PPI_RL_DB_PPI 0x1
32
+
#define GICV5_ARCH_PPI_NS_DB_PPI 0x2
33
+
#define GICV5_ARCH_PPI_SW_PPI 0x3
34
+
#define GICV5_ARCH_PPI_HACDBSIRQ 0xf
35
+
#define GICV5_ARCH_PPI_CNTHVS 0x13
36
+
#define GICV5_ARCH_PPI_CNTHPS 0x14
37
+
#define GICV5_ARCH_PPI_PMBIRQ 0x15
38
+
#define GICV5_ARCH_PPI_COMMIRQ 0x16
39
+
#define GICV5_ARCH_PPI_PMUIRQ 0x17
40
+
#define GICV5_ARCH_PPI_CTIIRQ 0x18
41
+
#define GICV5_ARCH_PPI_GICMNT 0x19
42
+
#define GICV5_ARCH_PPI_CNTHP 0x1a
43
+
#define GICV5_ARCH_PPI_CNTV 0x1b
44
+
#define GICV5_ARCH_PPI_CNTHV 0x1c
45
+
#define GICV5_ARCH_PPI_CNTPS 0x1d
46
+
#define GICV5_ARCH_PPI_CNTP 0x1e
47
+
#define GICV5_ARCH_PPI_TRBIRQ 0x1f
48
+
49
+
/*
28
50
* Tables attributes
29
51
*/
30
52
#define GICV5_NO_READ_ALLOC 0b0
···
386
364
int gicv5_spi_irq_set_type(struct irq_data *d, unsigned int type);
387
365
int gicv5_irs_iste_alloc(u32 lpi);
388
366
void gicv5_irs_syncr(void);
367
+
368
+
/* Embedded in kvm.arch */
369
+
struct gicv5_vpe {
370
+
bool resident;
371
+
};
389
372
390
373
struct gicv5_its_devtab_cfg {
391
374
union {
+1
include/linux/kvm_host.h
+1
include/linux/kvm_host.h
···
2366
2366
extern struct kvm_device_ops kvm_mpic_ops;
2367
2367
extern struct kvm_device_ops kvm_arm_vgic_v2_ops;
2368
2368
extern struct kvm_device_ops kvm_arm_vgic_v3_ops;
2369
+
extern struct kvm_device_ops kvm_arm_vgic_v5_ops;
2369
2370
2370
2371
#ifdef CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT
2371
2372
+2
include/uapi/linux/kvm.h
+2
include/uapi/linux/kvm.h
···
1224
1224
#define KVM_DEV_TYPE_LOONGARCH_EIOINTC KVM_DEV_TYPE_LOONGARCH_EIOINTC
1225
1225
KVM_DEV_TYPE_LOONGARCH_PCHPIC,
1226
1226
#define KVM_DEV_TYPE_LOONGARCH_PCHPIC KVM_DEV_TYPE_LOONGARCH_PCHPIC
1227
+
KVM_DEV_TYPE_ARM_VGIC_V5,
1228
+
#define KVM_DEV_TYPE_ARM_VGIC_V5 KVM_DEV_TYPE_ARM_VGIC_V5
1227
1229
1228
1230
KVM_DEV_TYPE_MAX,
1229
1231
+1
tools/arch/arm64/include/uapi/asm/kvm.h
+1
tools/arch/arm64/include/uapi/asm/kvm.h
+2
tools/include/uapi/linux/kvm.h
+2
tools/include/uapi/linux/kvm.h
···
1220
1220
#define KVM_DEV_TYPE_LOONGARCH_EIOINTC KVM_DEV_TYPE_LOONGARCH_EIOINTC
1221
1221
KVM_DEV_TYPE_LOONGARCH_PCHPIC,
1222
1222
#define KVM_DEV_TYPE_LOONGARCH_PCHPIC KVM_DEV_TYPE_LOONGARCH_PCHPIC
1223
+
KVM_DEV_TYPE_ARM_VGIC_V5,
1224
+
#define KVM_DEV_TYPE_ARM_VGIC_V5 KVM_DEV_TYPE_ARM_VGIC_V5
1223
1225
1224
1226
KVM_DEV_TYPE_MAX,
1225
1227
+2
-1
tools/testing/selftests/kvm/Makefile.kvm
+2
-1
tools/testing/selftests/kvm/Makefile.kvm
···
177
177
TEST_GEN_PROGS_arm64 += arm64/vgic_init
178
178
TEST_GEN_PROGS_arm64 += arm64/vgic_irq
179
179
TEST_GEN_PROGS_arm64 += arm64/vgic_lpi_stress
180
+
TEST_GEN_PROGS_arm64 += arm64/vgic_v5
180
181
TEST_GEN_PROGS_arm64 += arm64/vpmu_counter_access
181
-
TEST_GEN_PROGS_arm64 += arm64/no-vgic-v3
182
+
TEST_GEN_PROGS_arm64 += arm64/no-vgic
182
183
TEST_GEN_PROGS_arm64 += arm64/idreg-idst
183
184
TEST_GEN_PROGS_arm64 += arm64/kvm-uuid
184
185
TEST_GEN_PROGS_arm64 += access_tracking_perf_test
-177
tools/testing/selftests/kvm/arm64/no-vgic-v3.c
-177
tools/testing/selftests/kvm/arm64/no-vgic-v3.c
···
1
-
// SPDX-License-Identifier: GPL-2.0
2
-
3
-
// Check that, on a GICv3 system, not configuring GICv3 correctly
4
-
// results in all of the sysregs generating an UNDEF exception.
5
-
6
-
#include <test_util.h>
7
-
#include <kvm_util.h>
8
-
#include <processor.h>
9
-
10
-
static volatile bool handled;
11
-
12
-
#define __check_sr_read(r) \
13
-
({ \
14
-
uint64_t val; \
15
-
\
16
-
handled = false; \
17
-
dsb(sy); \
18
-
val = read_sysreg_s(SYS_ ## r); \
19
-
val; \
20
-
})
21
-
22
-
#define __check_sr_write(r) \
23
-
do { \
24
-
handled = false; \
25
-
dsb(sy); \
26
-
write_sysreg_s(0, SYS_ ## r); \
27
-
isb(); \
28
-
} while(0)
29
-
30
-
/* Fatal checks */
31
-
#define check_sr_read(r) \
32
-
do { \
33
-
__check_sr_read(r); \
34
-
__GUEST_ASSERT(handled, #r " no read trap"); \
35
-
} while(0)
36
-
37
-
#define check_sr_write(r) \
38
-
do { \
39
-
__check_sr_write(r); \
40
-
__GUEST_ASSERT(handled, #r " no write trap"); \
41
-
} while(0)
42
-
43
-
#define check_sr_rw(r) \
44
-
do { \
45
-
check_sr_read(r); \
46
-
check_sr_write(r); \
47
-
} while(0)
48
-
49
-
static void guest_code(void)
50
-
{
51
-
uint64_t val;
52
-
53
-
/*
54
-
* Check that we advertise that ID_AA64PFR0_EL1.GIC == 0, having
55
-
* hidden the feature at runtime without any other userspace action.
56
-
*/
57
-
__GUEST_ASSERT(FIELD_GET(ID_AA64PFR0_EL1_GIC,
58
-
read_sysreg(id_aa64pfr0_el1)) == 0,
59
-
"GICv3 wrongly advertised");
60
-
61
-
/*
62
-
* Access all GICv3 registers, and fail if we don't get an UNDEF.
63
-
* Note that we happily access all the APxRn registers without
64
-
* checking their existance, as all we want to see is a failure.
65
-
*/
66
-
check_sr_rw(ICC_PMR_EL1);
67
-
check_sr_read(ICC_IAR0_EL1);
68
-
check_sr_write(ICC_EOIR0_EL1);
69
-
check_sr_rw(ICC_HPPIR0_EL1);
70
-
check_sr_rw(ICC_BPR0_EL1);
71
-
check_sr_rw(ICC_AP0R0_EL1);
72
-
check_sr_rw(ICC_AP0R1_EL1);
73
-
check_sr_rw(ICC_AP0R2_EL1);
74
-
check_sr_rw(ICC_AP0R3_EL1);
75
-
check_sr_rw(ICC_AP1R0_EL1);
76
-
check_sr_rw(ICC_AP1R1_EL1);
77
-
check_sr_rw(ICC_AP1R2_EL1);
78
-
check_sr_rw(ICC_AP1R3_EL1);
79
-
check_sr_write(ICC_DIR_EL1);
80
-
check_sr_read(ICC_RPR_EL1);
81
-
check_sr_write(ICC_SGI1R_EL1);
82
-
check_sr_write(ICC_ASGI1R_EL1);
83
-
check_sr_write(ICC_SGI0R_EL1);
84
-
check_sr_read(ICC_IAR1_EL1);
85
-
check_sr_write(ICC_EOIR1_EL1);
86
-
check_sr_rw(ICC_HPPIR1_EL1);
87
-
check_sr_rw(ICC_BPR1_EL1);
88
-
check_sr_rw(ICC_CTLR_EL1);
89
-
check_sr_rw(ICC_IGRPEN0_EL1);
90
-
check_sr_rw(ICC_IGRPEN1_EL1);
91
-
92
-
/*
93
-
* ICC_SRE_EL1 may not be trappable, as ICC_SRE_EL2.Enable can
94
-
* be RAO/WI. Engage in non-fatal accesses, starting with a
95
-
* write of 0 to try and disable SRE, and let's see if it
96
-
* sticks.
97
-
*/
98
-
__check_sr_write(ICC_SRE_EL1);
99
-
if (!handled)
100
-
GUEST_PRINTF("ICC_SRE_EL1 write not trapping (OK)\n");
101
-
102
-
val = __check_sr_read(ICC_SRE_EL1);
103
-
if (!handled) {
104
-
__GUEST_ASSERT((val & BIT(0)),
105
-
"ICC_SRE_EL1 not trapped but ICC_SRE_EL1.SRE not set\n");
106
-
GUEST_PRINTF("ICC_SRE_EL1 read not trapping (OK)\n");
107
-
}
108
-
109
-
GUEST_DONE();
110
-
}
111
-
112
-
static void guest_undef_handler(struct ex_regs *regs)
113
-
{
114
-
/* Success, we've gracefully exploded! */
115
-
handled = true;
116
-
regs->pc += 4;
117
-
}
118
-
119
-
static void test_run_vcpu(struct kvm_vcpu *vcpu)
120
-
{
121
-
struct ucall uc;
122
-
123
-
do {
124
-
vcpu_run(vcpu);
125
-
126
-
switch (get_ucall(vcpu, &uc)) {
127
-
case UCALL_ABORT:
128
-
REPORT_GUEST_ASSERT(uc);
129
-
break;
130
-
case UCALL_PRINTF:
131
-
printf("%s", uc.buffer);
132
-
break;
133
-
case UCALL_DONE:
134
-
break;
135
-
default:
136
-
TEST_FAIL("Unknown ucall %lu", uc.cmd);
137
-
}
138
-
} while (uc.cmd != UCALL_DONE);
139
-
}
140
-
141
-
static void test_guest_no_gicv3(void)
142
-
{
143
-
struct kvm_vcpu *vcpu;
144
-
struct kvm_vm *vm;
145
-
146
-
/* Create a VM without a GICv3 */
147
-
vm = vm_create_with_one_vcpu(&vcpu, guest_code);
148
-
149
-
vm_init_descriptor_tables(vm);
150
-
vcpu_init_descriptor_tables(vcpu);
151
-
152
-
vm_install_sync_handler(vm, VECTOR_SYNC_CURRENT,
153
-
ESR_ELx_EC_UNKNOWN, guest_undef_handler);
154
-
155
-
test_run_vcpu(vcpu);
156
-
157
-
kvm_vm_free(vm);
158
-
}
159
-
160
-
int main(int argc, char *argv[])
161
-
{
162
-
struct kvm_vcpu *vcpu;
163
-
struct kvm_vm *vm;
164
-
uint64_t pfr0;
165
-
166
-
test_disable_default_vgic();
167
-
168
-
vm = vm_create_with_one_vcpu(&vcpu, NULL);
169
-
pfr0 = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_ID_AA64PFR0_EL1));
170
-
__TEST_REQUIRE(FIELD_GET(ID_AA64PFR0_EL1_GIC, pfr0),
171
-
"GICv3 not supported.");
172
-
kvm_vm_free(vm);
173
-
174
-
test_guest_no_gicv3();
175
-
176
-
return 0;
177
-
}
+297
tools/testing/selftests/kvm/arm64/no-vgic.c
+297
tools/testing/selftests/kvm/arm64/no-vgic.c
···
1
+
// SPDX-License-Identifier: GPL-2.0
2
+
3
+
// Check that, on a GICv3-capable system (GICv3 native, or GICv5 with
4
+
// FEAT_GCIE_LEGACY), not configuring GICv3 correctly results in all
5
+
// of the sysregs generating an UNDEF exception. Do the same for GICv5
6
+
// on a GICv5 host.
7
+
8
+
#include <test_util.h>
9
+
#include <kvm_util.h>
10
+
#include <processor.h>
11
+
12
+
#include <arm64/gic_v5.h>
13
+
14
+
static volatile bool handled;
15
+
16
+
#define __check_sr_read(r) \
17
+
({ \
18
+
uint64_t val; \
19
+
\
20
+
handled = false; \
21
+
dsb(sy); \
22
+
val = read_sysreg_s(SYS_ ## r); \
23
+
val; \
24
+
})
25
+
26
+
#define __check_sr_write(r) \
27
+
do { \
28
+
handled = false; \
29
+
dsb(sy); \
30
+
write_sysreg_s(0, SYS_ ## r); \
31
+
isb(); \
32
+
} while (0)
33
+
34
+
#define __check_gicv5_gicr_op(r) \
35
+
({ \
36
+
uint64_t val; \
37
+
\
38
+
handled = false; \
39
+
dsb(sy); \
40
+
val = read_sysreg_s(GICV5_OP_GICR_ ## r); \
41
+
val; \
42
+
})
43
+
44
+
#define __check_gicv5_gic_op(r) \
45
+
do { \
46
+
handled = false; \
47
+
dsb(sy); \
48
+
write_sysreg_s(0, GICV5_OP_GIC_ ## r); \
49
+
isb(); \
50
+
} while (0)
51
+
52
+
/* Fatal checks */
53
+
#define check_sr_read(r) \
54
+
do { \
55
+
__check_sr_read(r); \
56
+
__GUEST_ASSERT(handled, #r " no read trap"); \
57
+
} while (0)
58
+
59
+
#define check_sr_write(r) \
60
+
do { \
61
+
__check_sr_write(r); \
62
+
__GUEST_ASSERT(handled, #r " no write trap"); \
63
+
} while (0)
64
+
65
+
#define check_sr_rw(r) \
66
+
do { \
67
+
check_sr_read(r); \
68
+
check_sr_write(r); \
69
+
} while (0)
70
+
71
+
#define check_gicv5_gicr_op(r) \
72
+
do { \
73
+
__check_gicv5_gicr_op(r); \
74
+
__GUEST_ASSERT(handled, #r " no read trap"); \
75
+
} while (0)
76
+
77
+
#define check_gicv5_gic_op(r) \
78
+
do { \
79
+
__check_gicv5_gic_op(r); \
80
+
__GUEST_ASSERT(handled, #r " no write trap"); \
81
+
} while (0)
82
+
83
+
static void guest_code_gicv3(void)
84
+
{
85
+
uint64_t val;
86
+
87
+
/*
88
+
* Check that we advertise that ID_AA64PFR0_EL1.GIC == 0, having
89
+
* hidden the feature at runtime without any other userspace action.
90
+
*/
91
+
__GUEST_ASSERT(FIELD_GET(ID_AA64PFR0_EL1_GIC,
92
+
read_sysreg(id_aa64pfr0_el1)) == 0,
93
+
"GICv3 wrongly advertised");
94
+
95
+
/*
96
+
* Access all GICv3 registers, and fail if we don't get an UNDEF.
97
+
* Note that we happily access all the APxRn registers without
98
+
* checking their existence, as all we want to see is a failure.
99
+
*/
100
+
check_sr_rw(ICC_PMR_EL1);
101
+
check_sr_read(ICC_IAR0_EL1);
102
+
check_sr_write(ICC_EOIR0_EL1);
103
+
check_sr_rw(ICC_HPPIR0_EL1);
104
+
check_sr_rw(ICC_BPR0_EL1);
105
+
check_sr_rw(ICC_AP0R0_EL1);
106
+
check_sr_rw(ICC_AP0R1_EL1);
107
+
check_sr_rw(ICC_AP0R2_EL1);
108
+
check_sr_rw(ICC_AP0R3_EL1);
109
+
check_sr_rw(ICC_AP1R0_EL1);
110
+
check_sr_rw(ICC_AP1R1_EL1);
111
+
check_sr_rw(ICC_AP1R2_EL1);
112
+
check_sr_rw(ICC_AP1R3_EL1);
113
+
check_sr_write(ICC_DIR_EL1);
114
+
check_sr_read(ICC_RPR_EL1);
115
+
check_sr_write(ICC_SGI1R_EL1);
116
+
check_sr_write(ICC_ASGI1R_EL1);
117
+
check_sr_write(ICC_SGI0R_EL1);
118
+
check_sr_read(ICC_IAR1_EL1);
119
+
check_sr_write(ICC_EOIR1_EL1);
120
+
check_sr_rw(ICC_HPPIR1_EL1);
121
+
check_sr_rw(ICC_BPR1_EL1);
122
+
check_sr_rw(ICC_CTLR_EL1);
123
+
check_sr_rw(ICC_IGRPEN0_EL1);
124
+
check_sr_rw(ICC_IGRPEN1_EL1);
125
+
126
+
/*
127
+
* ICC_SRE_EL1 may not be trappable, as ICC_SRE_EL2.Enable can
128
+
* be RAO/WI. Engage in non-fatal accesses, starting with a
129
+
* write of 0 to try and disable SRE, and let's see if it
130
+
* sticks.
131
+
*/
132
+
__check_sr_write(ICC_SRE_EL1);
133
+
if (!handled)
134
+
GUEST_PRINTF("ICC_SRE_EL1 write not trapping (OK)\n");
135
+
136
+
val = __check_sr_read(ICC_SRE_EL1);
137
+
if (!handled) {
138
+
__GUEST_ASSERT((val & BIT(0)),
139
+
"ICC_SRE_EL1 not trapped but ICC_SRE_EL1.SRE not set\n");
140
+
GUEST_PRINTF("ICC_SRE_EL1 read not trapping (OK)\n");
141
+
}
142
+
143
+
GUEST_DONE();
144
+
}
145
+
146
+
static void guest_code_gicv5(void)
147
+
{
148
+
/*
149
+
* Check that we advertise that ID_AA64PFR2_EL1.GCIE == 0, having
150
+
* hidden the feature at runtime without any other userspace action.
151
+
*/
152
+
__GUEST_ASSERT(FIELD_GET(ID_AA64PFR2_EL1_GCIE,
153
+
read_sysreg_s(SYS_ID_AA64PFR2_EL1)) == 0,
154
+
"GICv5 wrongly advertised");
155
+
156
+
/*
157
+
* Try all GICv5 instructions, and fail if we don't get an UNDEF.
158
+
*/
159
+
check_gicv5_gic_op(CDAFF);
160
+
check_gicv5_gic_op(CDDI);
161
+
check_gicv5_gic_op(CDDIS);
162
+
check_gicv5_gic_op(CDEOI);
163
+
check_gicv5_gic_op(CDHM);
164
+
check_gicv5_gic_op(CDPEND);
165
+
check_gicv5_gic_op(CDPRI);
166
+
check_gicv5_gic_op(CDRCFG);
167
+
check_gicv5_gicr_op(CDIA);
168
+
check_gicv5_gicr_op(CDNMIA);
169
+
170
+
/* Check General System Register acccesses */
171
+
check_sr_rw(ICC_APR_EL1);
172
+
check_sr_rw(ICC_CR0_EL1);
173
+
check_sr_read(ICC_HPPIR_EL1);
174
+
check_sr_read(ICC_IAFFIDR_EL1);
175
+
check_sr_rw(ICC_ICSR_EL1);
176
+
check_sr_read(ICC_IDR0_EL1);
177
+
check_sr_rw(ICC_PCR_EL1);
178
+
179
+
/* Check PPI System Register accessess */
180
+
check_sr_rw(ICC_PPI_CACTIVER0_EL1);
181
+
check_sr_rw(ICC_PPI_CACTIVER1_EL1);
182
+
check_sr_rw(ICC_PPI_SACTIVER0_EL1);
183
+
check_sr_rw(ICC_PPI_SACTIVER1_EL1);
184
+
check_sr_rw(ICC_PPI_CPENDR0_EL1);
185
+
check_sr_rw(ICC_PPI_CPENDR1_EL1);
186
+
check_sr_rw(ICC_PPI_SPENDR0_EL1);
187
+
check_sr_rw(ICC_PPI_SPENDR1_EL1);
188
+
check_sr_rw(ICC_PPI_ENABLER0_EL1);
189
+
check_sr_rw(ICC_PPI_ENABLER1_EL1);
190
+
check_sr_read(ICC_PPI_HMR0_EL1);
191
+
check_sr_read(ICC_PPI_HMR1_EL1);
192
+
check_sr_rw(ICC_PPI_PRIORITYR0_EL1);
193
+
check_sr_rw(ICC_PPI_PRIORITYR1_EL1);
194
+
check_sr_rw(ICC_PPI_PRIORITYR2_EL1);
195
+
check_sr_rw(ICC_PPI_PRIORITYR3_EL1);
196
+
check_sr_rw(ICC_PPI_PRIORITYR4_EL1);
197
+
check_sr_rw(ICC_PPI_PRIORITYR5_EL1);
198
+
check_sr_rw(ICC_PPI_PRIORITYR6_EL1);
199
+
check_sr_rw(ICC_PPI_PRIORITYR7_EL1);
200
+
check_sr_rw(ICC_PPI_PRIORITYR8_EL1);
201
+
check_sr_rw(ICC_PPI_PRIORITYR9_EL1);
202
+
check_sr_rw(ICC_PPI_PRIORITYR10_EL1);
203
+
check_sr_rw(ICC_PPI_PRIORITYR11_EL1);
204
+
check_sr_rw(ICC_PPI_PRIORITYR12_EL1);
205
+
check_sr_rw(ICC_PPI_PRIORITYR13_EL1);
206
+
check_sr_rw(ICC_PPI_PRIORITYR14_EL1);
207
+
check_sr_rw(ICC_PPI_PRIORITYR15_EL1);
208
+
209
+
GUEST_DONE();
210
+
}
211
+
212
+
static void guest_undef_handler(struct ex_regs *regs)
213
+
{
214
+
/* Success, we've gracefully exploded! */
215
+
handled = true;
216
+
regs->pc += 4;
217
+
}
218
+
219
+
static void test_run_vcpu(struct kvm_vcpu *vcpu)
220
+
{
221
+
struct ucall uc;
222
+
223
+
do {
224
+
vcpu_run(vcpu);
225
+
226
+
switch (get_ucall(vcpu, &uc)) {
227
+
case UCALL_ABORT:
228
+
REPORT_GUEST_ASSERT(uc);
229
+
break;
230
+
case UCALL_PRINTF:
231
+
printf("%s", uc.buffer);
232
+
break;
233
+
case UCALL_DONE:
234
+
break;
235
+
default:
236
+
TEST_FAIL("Unknown ucall %lu", uc.cmd);
237
+
}
238
+
} while (uc.cmd != UCALL_DONE);
239
+
}
240
+
241
+
static void test_guest_no_vgic(void *guest_code)
242
+
{
243
+
struct kvm_vcpu *vcpu;
244
+
struct kvm_vm *vm;
245
+
246
+
/* Create a VM without a GIC */
247
+
vm = vm_create_with_one_vcpu(&vcpu, guest_code);
248
+
249
+
vm_init_descriptor_tables(vm);
250
+
vcpu_init_descriptor_tables(vcpu);
251
+
252
+
vm_install_sync_handler(vm, VECTOR_SYNC_CURRENT,
253
+
ESR_ELx_EC_UNKNOWN, guest_undef_handler);
254
+
255
+
test_run_vcpu(vcpu);
256
+
257
+
kvm_vm_free(vm);
258
+
}
259
+
260
+
int main(int argc, char *argv[])
261
+
{
262
+
struct kvm_vcpu *vcpu;
263
+
struct kvm_vm *vm;
264
+
bool has_v3, has_v5;
265
+
uint64_t pfr;
266
+
267
+
test_disable_default_vgic();
268
+
269
+
vm = vm_create_with_one_vcpu(&vcpu, NULL);
270
+
271
+
pfr = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_ID_AA64PFR0_EL1));
272
+
has_v3 = !!FIELD_GET(ID_AA64PFR0_EL1_GIC, pfr);
273
+
274
+
pfr = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_ID_AA64PFR2_EL1));
275
+
has_v5 = !!FIELD_GET(ID_AA64PFR2_EL1_GCIE, pfr);
276
+
277
+
kvm_vm_free(vm);
278
+
279
+
__TEST_REQUIRE(has_v3 || has_v5,
280
+
"Neither GICv3 nor GICv5 supported.");
281
+
282
+
if (has_v3) {
283
+
pr_info("Testing no-vgic-v3\n");
284
+
test_guest_no_vgic(guest_code_gicv3);
285
+
} else {
286
+
pr_info("No GICv3 support: skipping no-vgic-v3 test\n");
287
+
}
288
+
289
+
if (has_v5) {
290
+
pr_info("Testing no-vgic-v5\n");
291
+
test_guest_no_vgic(guest_code_gicv5);
292
+
} else {
293
+
pr_info("No GICv5 support: skipping no-vgic-v5 test\n");
294
+
}
295
+
296
+
return 0;
297
+
}
+228
tools/testing/selftests/kvm/arm64/vgic_v5.c
+228
tools/testing/selftests/kvm/arm64/vgic_v5.c
···
1
+
// SPDX-License-Identifier: GPL-2.0
2
+
3
+
#include <linux/kernel.h>
4
+
#include <sys/syscall.h>
5
+
#include <asm/kvm.h>
6
+
#include <asm/kvm_para.h>
7
+
8
+
#include <arm64/gic_v5.h>
9
+
10
+
#include "test_util.h"
11
+
#include "kvm_util.h"
12
+
#include "processor.h"
13
+
#include "vgic.h"
14
+
15
+
#define NR_VCPUS 1
16
+
17
+
struct vm_gic {
18
+
struct kvm_vm *vm;
19
+
int gic_fd;
20
+
uint32_t gic_dev_type;
21
+
};
22
+
23
+
static uint64_t max_phys_size;
24
+
25
+
#define GUEST_CMD_IRQ_CDIA 10
26
+
#define GUEST_CMD_IRQ_DIEOI 11
27
+
#define GUEST_CMD_IS_AWAKE 12
28
+
#define GUEST_CMD_IS_READY 13
29
+
30
+
static void guest_irq_handler(struct ex_regs *regs)
31
+
{
32
+
bool valid;
33
+
u32 hwirq;
34
+
u64 ia;
35
+
static int count;
36
+
37
+
/*
38
+
* We have pending interrupts. Should never actually enter WFI
39
+
* here!
40
+
*/
41
+
wfi();
42
+
GUEST_SYNC(GUEST_CMD_IS_AWAKE);
43
+
44
+
ia = gicr_insn(CDIA);
45
+
valid = GICV5_GICR_CDIA_VALID(ia);
46
+
47
+
GUEST_SYNC(GUEST_CMD_IRQ_CDIA);
48
+
49
+
if (!valid)
50
+
return;
51
+
52
+
gsb_ack();
53
+
isb();
54
+
55
+
hwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);
56
+
57
+
gic_insn(hwirq, CDDI);
58
+
gic_insn(0, CDEOI);
59
+
60
+
GUEST_SYNC(GUEST_CMD_IRQ_DIEOI);
61
+
62
+
if (++count >= 2)
63
+
GUEST_DONE();
64
+
65
+
/* Ask for the next interrupt to be injected */
66
+
GUEST_SYNC(GUEST_CMD_IS_READY);
67
+
}
68
+
69
+
static void guest_code(void)
70
+
{
71
+
local_irq_disable();
72
+
73
+
gicv5_cpu_enable_interrupts();
74
+
local_irq_enable();
75
+
76
+
/* Enable the SW_PPI (3) */
77
+
write_sysreg_s(BIT_ULL(3), SYS_ICC_PPI_ENABLER0_EL1);
78
+
79
+
/* Ask for the first interrupt to be injected */
80
+
GUEST_SYNC(GUEST_CMD_IS_READY);
81
+
82
+
/* Loop forever waiting for interrupts */
83
+
while (1);
84
+
}
85
+
86
+
87
+
/* we don't want to assert on run execution, hence that helper */
88
+
static int run_vcpu(struct kvm_vcpu *vcpu)
89
+
{
90
+
return __vcpu_run(vcpu) ? -errno : 0;
91
+
}
92
+
93
+
static void vm_gic_destroy(struct vm_gic *v)
94
+
{
95
+
close(v->gic_fd);
96
+
kvm_vm_free(v->vm);
97
+
}
98
+
99
+
static void test_vgic_v5_ppis(uint32_t gic_dev_type)
100
+
{
101
+
struct kvm_vcpu *vcpus[NR_VCPUS];
102
+
struct ucall uc;
103
+
u64 user_ppis[2];
104
+
struct vm_gic v;
105
+
int ret, i;
106
+
107
+
v.gic_dev_type = gic_dev_type;
108
+
v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
109
+
110
+
v.gic_fd = kvm_create_device(v.vm, gic_dev_type);
111
+
112
+
for (i = 0; i < NR_VCPUS; i++)
113
+
vcpus[i] = vm_vcpu_add(v.vm, i, guest_code);
114
+
115
+
vm_init_descriptor_tables(v.vm);
116
+
vm_install_exception_handler(v.vm, VECTOR_IRQ_CURRENT, guest_irq_handler);
117
+
118
+
for (i = 0; i < NR_VCPUS; i++)
119
+
vcpu_init_descriptor_tables(vcpus[i]);
120
+
121
+
kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
122
+
KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
123
+
124
+
/* Read out the PPIs that user space is allowed to drive. */
125
+
kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
126
+
KVM_DEV_ARM_VGIC_USERSPACE_PPIS, &user_ppis);
127
+
128
+
/* We should always be able to drive the SW_PPI. */
129
+
TEST_ASSERT(user_ppis[0] & BIT(GICV5_ARCH_PPI_SW_PPI),
130
+
"SW_PPI is not drivable by userspace");
131
+
132
+
while (1) {
133
+
ret = run_vcpu(vcpus[0]);
134
+
135
+
switch (get_ucall(vcpus[0], &uc)) {
136
+
case UCALL_SYNC:
137
+
/*
138
+
* The guest is ready for the next level change. Set
139
+
* high if ready, and lower if it has been consumed.
140
+
*/
141
+
if (uc.args[1] == GUEST_CMD_IS_READY ||
142
+
uc.args[1] == GUEST_CMD_IRQ_DIEOI) {
143
+
u64 irq;
144
+
bool level = uc.args[1] == GUEST_CMD_IRQ_DIEOI ? 0 : 1;
145
+
146
+
irq = FIELD_PREP(KVM_ARM_IRQ_NUM_MASK, 3);
147
+
irq |= KVM_ARM_IRQ_TYPE_PPI << KVM_ARM_IRQ_TYPE_SHIFT;
148
+
149
+
_kvm_irq_line(v.vm, irq, level);
150
+
} else if (uc.args[1] == GUEST_CMD_IS_AWAKE) {
151
+
pr_info("Guest skipping WFI due to pending IRQ\n");
152
+
} else if (uc.args[1] == GUEST_CMD_IRQ_CDIA) {
153
+
pr_info("Guest acknowledged IRQ\n");
154
+
}
155
+
156
+
continue;
157
+
case UCALL_ABORT:
158
+
REPORT_GUEST_ASSERT(uc);
159
+
break;
160
+
case UCALL_DONE:
161
+
goto done;
162
+
default:
163
+
TEST_FAIL("Unknown ucall %lu", uc.cmd);
164
+
}
165
+
}
166
+
167
+
done:
168
+
TEST_ASSERT(ret == 0, "Failed to test GICv5 PPIs");
169
+
170
+
vm_gic_destroy(&v);
171
+
}
172
+
173
+
/*
174
+
* Returns 0 if it's possible to create GIC device of a given type (V5).
175
+
*/
176
+
int test_kvm_device(uint32_t gic_dev_type)
177
+
{
178
+
struct kvm_vcpu *vcpus[NR_VCPUS];
179
+
struct vm_gic v;
180
+
int ret;
181
+
182
+
v.vm = vm_create_with_vcpus(NR_VCPUS, guest_code, vcpus);
183
+
184
+
/* try to create a non existing KVM device */
185
+
ret = __kvm_test_create_device(v.vm, 0);
186
+
TEST_ASSERT(ret && errno == ENODEV, "unsupported device");
187
+
188
+
/* trial mode */
189
+
ret = __kvm_test_create_device(v.vm, gic_dev_type);
190
+
if (ret)
191
+
return ret;
192
+
v.gic_fd = kvm_create_device(v.vm, gic_dev_type);
193
+
194
+
ret = __kvm_create_device(v.vm, gic_dev_type);
195
+
TEST_ASSERT(ret < 0 && errno == EEXIST, "create GIC device twice");
196
+
197
+
vm_gic_destroy(&v);
198
+
199
+
return 0;
200
+
}
201
+
202
+
void run_tests(uint32_t gic_dev_type)
203
+
{
204
+
pr_info("Test VGICv5 PPIs\n");
205
+
test_vgic_v5_ppis(gic_dev_type);
206
+
}
207
+
208
+
int main(int ac, char **av)
209
+
{
210
+
int ret;
211
+
int pa_bits;
212
+
213
+
test_disable_default_vgic();
214
+
215
+
pa_bits = vm_guest_mode_params[VM_MODE_DEFAULT].pa_bits;
216
+
max_phys_size = 1ULL << pa_bits;
217
+
218
+
ret = test_kvm_device(KVM_DEV_TYPE_ARM_VGIC_V5);
219
+
if (ret) {
220
+
pr_info("No GICv5 support; Not running GIC_v5 tests.\n");
221
+
exit(KSFT_SKIP);
222
+
}
223
+
224
+
pr_info("Running VGIC_V5 tests.\n");
225
+
run_tests(KVM_DEV_TYPE_ARM_VGIC_V5);
226
+
227
+
return 0;
228
+
}
+150
tools/testing/selftests/kvm/include/arm64/gic_v5.h
+150
tools/testing/selftests/kvm/include/arm64/gic_v5.h
···
1
+
/* SPDX-License-Identifier: GPL-2.0-only */
2
+
3
+
#ifndef __SELFTESTS_GIC_V5_H
4
+
#define __SELFTESTS_GIC_V5_H
5
+
6
+
#include <asm/barrier.h>
7
+
#include <asm/sysreg.h>
8
+
9
+
#include <linux/bitfield.h>
10
+
11
+
#include "processor.h"
12
+
13
+
/*
14
+
* Definitions for GICv5 instructions for the Current Domain
15
+
*/
16
+
#define GICV5_OP_GIC_CDAFF sys_insn(1, 0, 12, 1, 3)
17
+
#define GICV5_OP_GIC_CDDI sys_insn(1, 0, 12, 2, 0)
18
+
#define GICV5_OP_GIC_CDDIS sys_insn(1, 0, 12, 1, 0)
19
+
#define GICV5_OP_GIC_CDHM sys_insn(1, 0, 12, 2, 1)
20
+
#define GICV5_OP_GIC_CDEN sys_insn(1, 0, 12, 1, 1)
21
+
#define GICV5_OP_GIC_CDEOI sys_insn(1, 0, 12, 1, 7)
22
+
#define GICV5_OP_GIC_CDPEND sys_insn(1, 0, 12, 1, 4)
23
+
#define GICV5_OP_GIC_CDPRI sys_insn(1, 0, 12, 1, 2)
24
+
#define GICV5_OP_GIC_CDRCFG sys_insn(1, 0, 12, 1, 5)
25
+
#define GICV5_OP_GICR_CDIA sys_insn(1, 0, 12, 3, 0)
26
+
#define GICV5_OP_GICR_CDNMIA sys_insn(1, 0, 12, 3, 1)
27
+
28
+
/* Definitions for GIC CDAFF */
29
+
#define GICV5_GIC_CDAFF_IAFFID_MASK GENMASK_ULL(47, 32)
30
+
#define GICV5_GIC_CDAFF_TYPE_MASK GENMASK_ULL(31, 29)
31
+
#define GICV5_GIC_CDAFF_IRM_MASK BIT_ULL(28)
32
+
#define GICV5_GIC_CDAFF_ID_MASK GENMASK_ULL(23, 0)
33
+
34
+
/* Definitions for GIC CDDI */
35
+
#define GICV5_GIC_CDDI_TYPE_MASK GENMASK_ULL(31, 29)
36
+
#define GICV5_GIC_CDDI_ID_MASK GENMASK_ULL(23, 0)
37
+
38
+
/* Definitions for GIC CDDIS */
39
+
#define GICV5_GIC_CDDIS_TYPE_MASK GENMASK_ULL(31, 29)
40
+
#define GICV5_GIC_CDDIS_TYPE(r) FIELD_GET(GICV5_GIC_CDDIS_TYPE_MASK, r)
41
+
#define GICV5_GIC_CDDIS_ID_MASK GENMASK_ULL(23, 0)
42
+
#define GICV5_GIC_CDDIS_ID(r) FIELD_GET(GICV5_GIC_CDDIS_ID_MASK, r)
43
+
44
+
/* Definitions for GIC CDEN */
45
+
#define GICV5_GIC_CDEN_TYPE_MASK GENMASK_ULL(31, 29)
46
+
#define GICV5_GIC_CDEN_ID_MASK GENMASK_ULL(23, 0)
47
+
48
+
/* Definitions for GIC CDHM */
49
+
#define GICV5_GIC_CDHM_HM_MASK BIT_ULL(32)
50
+
#define GICV5_GIC_CDHM_TYPE_MASK GENMASK_ULL(31, 29)
51
+
#define GICV5_GIC_CDHM_ID_MASK GENMASK_ULL(23, 0)
52
+
53
+
/* Definitions for GIC CDPEND */
54
+
#define GICV5_GIC_CDPEND_PENDING_MASK BIT_ULL(32)
55
+
#define GICV5_GIC_CDPEND_TYPE_MASK GENMASK_ULL(31, 29)
56
+
#define GICV5_GIC_CDPEND_ID_MASK GENMASK_ULL(23, 0)
57
+
58
+
/* Definitions for GIC CDPRI */
59
+
#define GICV5_GIC_CDPRI_PRIORITY_MASK GENMASK_ULL(39, 35)
60
+
#define GICV5_GIC_CDPRI_TYPE_MASK GENMASK_ULL(31, 29)
61
+
#define GICV5_GIC_CDPRI_ID_MASK GENMASK_ULL(23, 0)
62
+
63
+
/* Definitions for GIC CDRCFG */
64
+
#define GICV5_GIC_CDRCFG_TYPE_MASK GENMASK_ULL(31, 29)
65
+
#define GICV5_GIC_CDRCFG_ID_MASK GENMASK_ULL(23, 0)
66
+
67
+
/* Definitions for GICR CDIA */
68
+
#define GICV5_GICR_CDIA_VALID_MASK BIT_ULL(32)
69
+
#define GICV5_GICR_CDIA_VALID(r) FIELD_GET(GICV5_GICR_CDIA_VALID_MASK, r)
70
+
#define GICV5_GICR_CDIA_TYPE_MASK GENMASK_ULL(31, 29)
71
+
#define GICV5_GICR_CDIA_ID_MASK GENMASK_ULL(23, 0)
72
+
#define GICV5_GICR_CDIA_INTID GENMASK_ULL(31, 0)
73
+
74
+
/* Definitions for GICR CDNMIA */
75
+
#define GICV5_GICR_CDNMIA_VALID_MASK BIT_ULL(32)
76
+
#define GICV5_GICR_CDNMIA_VALID(r) FIELD_GET(GICV5_GICR_CDNMIA_VALID_MASK, r)
77
+
#define GICV5_GICR_CDNMIA_TYPE_MASK GENMASK_ULL(31, 29)
78
+
#define GICV5_GICR_CDNMIA_ID_MASK GENMASK_ULL(23, 0)
79
+
80
+
#define gicr_insn(insn) read_sysreg_s(GICV5_OP_GICR_##insn)
81
+
#define gic_insn(v, insn) write_sysreg_s(v, GICV5_OP_GIC_##insn)
82
+
83
+
#define __GIC_BARRIER_INSN(op0, op1, CRn, CRm, op2, Rt) \
84
+
__emit_inst(0xd5000000 | \
85
+
sys_insn((op0), (op1), (CRn), (CRm), (op2)) | \
86
+
((Rt) & 0x1f))
87
+
88
+
#define GSB_SYS_BARRIER_INSN __GIC_BARRIER_INSN(1, 0, 12, 0, 0, 31)
89
+
#define GSB_ACK_BARRIER_INSN __GIC_BARRIER_INSN(1, 0, 12, 0, 1, 31)
90
+
91
+
#define gsb_ack() asm volatile(GSB_ACK_BARRIER_INSN : : : "memory")
92
+
#define gsb_sys() asm volatile(GSB_SYS_BARRIER_INSN : : : "memory")
93
+
94
+
#define REPEAT_BYTE(x) ((~0ul / 0xff) * (x))
95
+
96
+
#define GICV5_IRQ_DEFAULT_PRI 0b10000
97
+
98
+
#define GICV5_ARCH_PPI_SW_PPI 0x3
99
+
100
+
void gicv5_ppi_priority_init(void)
101
+
{
102
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR0_EL1);
103
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR1_EL1);
104
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR2_EL1);
105
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR3_EL1);
106
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR4_EL1);
107
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR5_EL1);
108
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR6_EL1);
109
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR7_EL1);
110
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR8_EL1);
111
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR9_EL1);
112
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR10_EL1);
113
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR11_EL1);
114
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR12_EL1);
115
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR13_EL1);
116
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR14_EL1);
117
+
write_sysreg_s(REPEAT_BYTE(GICV5_IRQ_DEFAULT_PRI), SYS_ICC_PPI_PRIORITYR15_EL1);
118
+
119
+
/*
120
+
* Context syncronization required to make sure system register writes
121
+
* effects are synchronised.
122
+
*/
123
+
isb();
124
+
}
125
+
126
+
void gicv5_cpu_disable_interrupts(void)
127
+
{
128
+
u64 cr0;
129
+
130
+
cr0 = FIELD_PREP(ICC_CR0_EL1_EN, 0);
131
+
write_sysreg_s(cr0, SYS_ICC_CR0_EL1);
132
+
}
133
+
134
+
void gicv5_cpu_enable_interrupts(void)
135
+
{
136
+
u64 cr0, pcr;
137
+
138
+
write_sysreg_s(0, SYS_ICC_PPI_ENABLER0_EL1);
139
+
write_sysreg_s(0, SYS_ICC_PPI_ENABLER1_EL1);
140
+
141
+
gicv5_ppi_priority_init();
142
+
143
+
pcr = FIELD_PREP(ICC_PCR_EL1_PRIORITY, GICV5_IRQ_DEFAULT_PRI);
144
+
write_sysreg_s(pcr, SYS_ICC_PCR_EL1);
145
+
146
+
cr0 = FIELD_PREP(ICC_CR0_EL1_EN, 1);
147
+
write_sysreg_s(cr0, SYS_ICC_CR0_EL1);
148
+
}
149
+
150
+
#endif