tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

KVM: Boost vCPUs that are delivering interrupts

Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during
vcpu wakeup and interrupt delivery), we want to also boost not just
lock holders but also vCPUs that are delivering interrupts. Most
smp_call_function_many calls are synchronous, so the IPI target vCPUs
are also good yield candidates. This patch introduces vcpu->ready to
boost vCPUs during wakeup and interrupt delivery time; unlike s390 we do
not reuse vcpu->preempted so that voluntarily preempted vCPUs are taken
into account by kvm_vcpu_on_spin, but vmx_vcpu_pi_put is not affected
(VT-d PI handles voluntary preemption separately, in pi_pre_block).

Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
ebizzy -M

vanilla boosting improved
1VM 21443 23520 9%
2VM 2800 8000 180%
3VM 1800 3100 72%

Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
one running ebizzy -M, the other running 'stress --cpu 2':

w/ boosting + w/o pv sched yield(vanilla)

vanilla boosting improved
1570 4000 155%

w/ boosting + w/ pv sched yield(vanilla)

vanilla boosting improved
1844 5157 179%

w/o boosting, perf top in VM:

72.33% [kernel] [k] smp_call_function_many
4.22% [kernel] [k] call_function_i
3.71% [kernel] [k] async_page_fault

w/ boosting, perf top in VM:

38.43% [kernel] [k] smp_call_function_many
6.31% [kernel] [k] async_page_fault
6.13% libc-2.23.so [.] __memcpy_avx_unaligned
4.88% [kernel] [k] call_function_interrupt

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Marc Zyngier <maz@kernel.org>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

authored by

Wanpeng Li and committed by
Paolo Bonzini d73eb57b 2417c870

+10 -5
+1 -1
arch/s390/kvm/interrupt.c
··· 1240 1240 * The vcpu gave up the cpu voluntarily, mark it as a good 1241 1241 * yield-candidate. 1242 1242 */ 1243 - vcpu->preempted = true; 1243 + vcpu->ready = true; 1244 1244 swake_up_one(&vcpu->wq); 1245 1245 vcpu->stat.halt_wakeup++; 1246 1246 }
+1
include/linux/kvm_host.h
··· 318 318 } spin_loop; 319 319 #endif 320 320 bool preempted; 321 + bool ready; 321 322 struct kvm_vcpu_arch arch; 322 323 struct dentry *debugfs_dentry; 323 324 };
+8 -4
virt/kvm/kvm_main.c
··· 314 314 kvm_vcpu_set_in_spin_loop(vcpu, false); 315 315 kvm_vcpu_set_dy_eligible(vcpu, false); 316 316 vcpu->preempted = false; 317 + vcpu->ready = false; 317 318 318 319 r = kvm_arch_vcpu_init(vcpu); 319 320 if (r < 0) ··· 2388 2387 wqp = kvm_arch_vcpu_wq(vcpu); 2389 2388 if (swq_has_sleeper(wqp)) { 2390 2389 swake_up_one(wqp); 2390 + WRITE_ONCE(vcpu->ready, true); 2391 2391 ++vcpu->stat.halt_wakeup; 2392 2392 return true; 2393 2393 } ··· 2502 2500 continue; 2503 2501 } else if (pass && i > last_boosted_vcpu) 2504 2502 break; 2505 - if (!READ_ONCE(vcpu->preempted)) 2503 + if (!READ_ONCE(vcpu->ready)) 2506 2504 continue; 2507 2505 if (vcpu == me) 2508 2506 continue; ··· 4205 4203 { 4206 4204 struct kvm_vcpu *vcpu = preempt_notifier_to_vcpu(pn); 4207 4205 4208 - if (vcpu->preempted) 4209 - vcpu->preempted = false; 4206 + vcpu->preempted = false; 4207 + WRITE_ONCE(vcpu->ready, false); 4210 4208 4211 4209 kvm_arch_sched_in(vcpu, cpu); 4212 4210 ··· 4218 4216 { 4219 4217 struct kvm_vcpu *vcpu = preempt_notifier_to_vcpu(pn); 4220 4218 4221 - if (current->state == TASK_RUNNING) 4219 + if (current->state == TASK_RUNNING) { 4222 4220 vcpu->preempted = true; 4221 + WRITE_ONCE(vcpu->ready, true); 4222 + } 4223 4223 kvm_arch_vcpu_put(vcpu); 4224 4224 } 4225 4225