Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
On 19/12/2014 06:25, Zhang, Yang Z wrote: I see your point. But from performance point, if we can schedule the vCPU to another PCPU to handle the interrupt, it would helpful. But I remember current KVM will not schedule the vCPU in run queue (even though it got preempted) to another pCPU to run(Am I right?). So it may hard to do it. Yes. If the vCPU is in the run queue, it means it exhausted its quantum. As Feng said, the scheduler can decide to migrate it to another pCPU, or it can decide to leave it runnable but not start it. KVM doesn't try to force the scheduler one way or the other. If the vCPU is I/O bound, it will not exhaust its quantum and will not be preempted. It will block, and the wakeup vector will restart it. I don't think urgent notifications are interesting. If you want to do real time work, pin the vCPU to a physical CPU, and isolate the pCPU with isolcpus. Then the vCPU will always be running. Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
-Original Message- From: Paolo Bonzini [mailto:paolo.bonz...@gmail.com] On Behalf Of Paolo Bonzini Sent: Friday, December 19, 2014 8:01 PM To: Zhang, Yang Z; Wu, Feng; Paolo Bonzini; KVM list Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 19/12/2014 06:25, Zhang, Yang Z wrote: I see your point. But from performance point, if we can schedule the vCPU to another PCPU to handle the interrupt, it would helpful. But I remember current KVM will not schedule the vCPU in run queue (even though it got preempted) to another pCPU to run(Am I right?). So it may hard to do it. Yes. If the vCPU is in the run queue, it means it exhausted its quantum. As Feng said, the scheduler can decide to migrate it to another pCPU, or it can decide to leave it runnable but not start it. KVM doesn't try to force the scheduler one way or the other. If the vCPU is I/O bound, it will not exhaust its quantum and will not be preempted. It will block, and the wakeup vector will restart it. I don't think urgent notifications are interesting. If you want to do real time work, pin the vCPU to a physical CPU, and isolate the pCPU with isolcpus. Then the vCPU will always be running. Paolo I agree, thanks Paolo! Thanks, Feng -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
On 18/12/2014 04:14, Wu, Feng wrote:
-Original Message-
From: linux-kernel-ow...@vger.kernel.org
[mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo Bonzini
Sent: Thursday, December 18, 2014 1:43 AM
To: Wu, Feng; Thomas Gleixner; Ingo Molnar; H. Peter Anvin; x...@kernel.org;
Gleb Natapov; Paolo Bonzini; dw...@infradead.org;
joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex
Williamson; Jiang Liu
Cc: io...@lists.linux-foundation.org;
linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list;
Eric Auger
Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
On 12/12/2014 16:14, Feng Wu wrote:
Currently, we don't support urgent interrupt, all interrupts
are recognized as non-urgent interrupt, so we cannot send
posted-interrupt when 'SN' is set.
Can this happen? If the vcpu is in guest mode, it cannot have been
scheduled out, and that's the only case when SN is set.
Paolo
Currently, the only place where SN is set is vCPU is preempted and waiting for
the next scheduling in the runqueue. But I am not sure whether we need to
set SN for other purpose in future. Adding SN checking here is just to follow
the Spec. non-urgent interrupts are suppressed when SN is set.
I would change that to a WARN_ON_ONCE then.
Paolo
Thanks,
Feng
Signed-off-by: Feng Wu
feng.wu-ral2jqcrhueavxtiumw...@public.gmane.org
---
arch/x86/kvm/vmx.c | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index a1c83a2..0aee151 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -4401,15 +4401,22 @@ static int vmx_vm_has_apicv(struct kvm *kvm)
static void vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int
vector)
{
struct vcpu_vmx *vmx = to_vmx(vcpu);
- int r;
+ int r, sn;
if (pi_test_and_set_pir(vector, vmx-pi_desc))
return;
+ /*
+* Currently, we don't support urgent interrupt, all interrupts
+* are recognized as non-urgent interrupt, so we cannot send
+* posted-interrupt when 'SN' is set.
+*/
+ sn = pi_test_sn(vmx-pi_desc);
+
r = pi_test_and_set_on(vmx-pi_desc);
kvm_make_request(KVM_REQ_EVENT, vcpu);
#ifdef CONFIG_SMP
- if (!r (vcpu-mode == IN_GUEST_MODE))
+ if (!r !sn (vcpu-mode == IN_GUEST_MODE))
apic-send_IPI_mask(get_cpu_mask(vcpu-cpu),
POSTED_INTR_VECTOR);
else
--
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
-Original Message- From: iommu-boun...@lists.linux-foundation.org [mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of Zhang, Yang Z Sent: Thursday, December 18, 2014 11:10 PM To: Paolo Bonzini; kvm@vger.kernel.org Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? If a vCPU is preempted, the 'SN' bit is set, the subsequent interrupts are suppressed for posting. Thanks, Feng waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang ___ iommu mailing list io...@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
Wu, Feng wrote on 2014-12-19: iommu-boun...@lists.linux-foundation.org wrote on mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of: Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? If a vCPU is preempted, the 'SN' bit is set, the subsequent interrupts are suppressed for posting. I mean what happens if we don't set SN bit. From my point, if preempter already disabled the interrupt, it is ok to leave SN bit as zero. But if preempter enabled the interrupt, doesn't this mean he allow interrupt to happen? BTW, since there already has ON bit, so this means there only have one interrupt arrived at most and it doesn't hurt performance. Do we really need to set SN bit? Thanks, Feng waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang ___ iommu mailing list io...@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu Best regards, Yang -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
-Original Message- From: Zhang, Yang Z Sent: Friday, December 19, 2014 11:33 AM To: Wu, Feng; Paolo Bonzini; kvm@vger.kernel.org Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: iommu-boun...@lists.linux-foundation.org wrote on mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of: Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? If a vCPU is preempted, the 'SN' bit is set, the subsequent interrupts are suppressed for posting. I mean what happens if we don't set SN bit. From my point, if preempter already disabled the interrupt, it is ok to leave SN bit as zero. But if preempter enabled the interrupt, doesn't this mean he allow interrupt to happen? BTW, since there already has ON bit, so this means there only have one interrupt arrived at most and it doesn't hurt performance. Do we really need to set SN bit? See this scenario: vCPU0 is running on pCPU0 -- vCPU0 is preempted by vCPU1 -- Then vCPU1 is running on pCPU0 and vCPU0 is waiting for schedule in runqueue If the we don't set SN for vCPU0, then all subsequent interrupts for vCPU0 is posted to vCPU1, this will consume hardware and software efforts and in fact it is not needed at all. If SN is set for vCPU0, VT-d hardware will not issue Notification Event for vCPU0 when an interrupt is for it, but just setting the related PIR bit. Thanks, Feng Thanks, Feng waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang ___ iommu mailing list io...@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu Best regards, Yang -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: iommu-boun...@lists.linux-foundation.org wrote on mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of: Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? If a vCPU is preempted, the 'SN' bit is set, the subsequent interrupts are suppressed for posting. I mean what happens if we don't set SN bit. From my point, if preempter already disabled the interrupt, it is ok to leave SN bit as zero. But if preempter enabled the interrupt, doesn't this mean he allow interrupt to happen? BTW, since there already has ON bit, so this means there only have one interrupt arrived at most and it doesn't hurt performance. Do we really need to set SN bit? See this scenario: vCPU0 is running on pCPU0 -- vCPU0 is preempted by vCPU1 -- Then vCPU1 is running on pCPU0 and vCPU0 is waiting for schedule -- in runqueue If the we don't set SN for vCPU0, then all subsequent interrupts for vCPU0 is posted to vCPU1, this will consume hardware and software The PI vector for vCPU1 is notification vector, but the PI vector for vCPU0 should be wakeup vector. Why vCPU1 will consume this PI event? efforts and in fact it is not needed at all. If SN is set for vCPU0, VT-d hardware will not issue Notification Event for vCPU0 when an interrupt is for it, but just setting the related PIR bit. Thanks, Feng Thanks, Feng waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang ___ iommu mailing list io...@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu Best regards, Yang Best regards, Yang -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
-Original Message- From: Zhang, Yang Z Sent: Friday, December 19, 2014 12:44 PM To: Wu, Feng; Paolo Bonzini; kvm@vger.kernel.org Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: iommu-boun...@lists.linux-foundation.org wrote on mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of: Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? If a vCPU is preempted, the 'SN' bit is set, the subsequent interrupts are suppressed for posting. I mean what happens if we don't set SN bit. From my point, if preempter already disabled the interrupt, it is ok to leave SN bit as zero. But if preempter enabled the interrupt, doesn't this mean he allow interrupt to happen? BTW, since there already has ON bit, so this means there only have one interrupt arrived at most and it doesn't hurt performance. Do we really need to set SN bit? See this scenario: vCPU0 is running on pCPU0 -- vCPU0 is preempted by vCPU1 -- Then vCPU1 is running on pCPU0 and vCPU0 is waiting for schedule -- in runqueue If the we don't set SN for vCPU0, then all subsequent interrupts for vCPU0 is posted to vCPU1, this will consume hardware and software The PI vector for vCPU1 is notification vector, but the PI vector for vCPU0 should be wakeup vector. Why vCPU1 will consume this PI event? Wakeup vector is only used for blocking case, when vCPU is preempted and waiting in the runqueue, the NV is the notification vector. Thanks, Feng efforts and in fact it is not needed at all. If SN is set for vCPU0, VT-d hardware will not issue Notification Event for vCPU0 when an interrupt is for it, but just setting the related PIR bit. Thanks, Feng Thanks, Feng waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang ___ iommu mailing list io...@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu Best regards, Yang Best regards, Yang -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: iommu-boun...@lists.linux-foundation.org wrote on mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of: Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? If a vCPU is preempted, the 'SN' bit is set, the subsequent interrupts are suppressed for posting. I mean what happens if we don't set SN bit. From my point, if preempter already disabled the interrupt, it is ok to leave SN bit as zero. But if preempter enabled the interrupt, doesn't this mean he allow interrupt to happen? BTW, since there already has ON bit, so this means there only have one interrupt arrived at most and it doesn't hurt performance. Do we really need to set SN bit? See this scenario: vCPU0 is running on pCPU0 -- vCPU0 is preempted by vCPU1 -- Then vCPU1 is running on pCPU0 and vCPU0 is waiting for -- schedule in runqueue If the we don't set SN for vCPU0, then all subsequent interrupts for vCPU0 is posted to vCPU1, this will consume hardware and software The PI vector for vCPU1 is notification vector, but the PI vector for vCPU0 should be wakeup vector. Why vCPU1 will consume this PI event? Wakeup vector is only used for blocking case, when vCPU is preempted and waiting in the runqueue, the NV is the notification vector. I see your point. But from performance point, if we can schedule the vCPU to another PCPU to handle the interrupt, it would helpful. But I remember current KVM will not schedule the vCPU in run queue (even though it got preempted) to another pCPU to run(Am I right?). So it may hard to do it. Thanks, Feng efforts and in fact it is not needed at all. If SN is set for vCPU0, VT-d hardware will not issue Notification Event for vCPU0 when an interrupt is for it, but just setting the related PIR bit. Thanks, Feng Thanks, Feng waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang ___ iommu mailing list io...@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu Best regards, Yang Best regards, Yang Best regards, Yang -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
-Original Message- From: Zhang, Yang Z Sent: Friday, December 19, 2014 1:26 PM To: Wu, Feng; Paolo Bonzini; kvm@vger.kernel.org Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: iommu-boun...@lists.linux-foundation.org wrote on mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of: Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? If a vCPU is preempted, the 'SN' bit is set, the subsequent interrupts are suppressed for posting. I mean what happens if we don't set SN bit. From my point, if preempter already disabled the interrupt, it is ok to leave SN bit as zero. But if preempter enabled the interrupt, doesn't this mean he allow interrupt to happen? BTW, since there already has ON bit, so this means there only have one interrupt arrived at most and it doesn't hurt performance. Do we really need to set SN bit? See this scenario: vCPU0 is running on pCPU0 -- vCPU0 is preempted by vCPU1 -- Then vCPU1 is running on pCPU0 and vCPU0 is waiting for -- schedule in runqueue If the we don't set SN for vCPU0, then all subsequent interrupts for vCPU0 is posted to vCPU1, this will consume hardware and software The PI vector for vCPU1 is notification vector, but the PI vector for vCPU0 should be wakeup vector. Why vCPU1 will consume this PI event? Wakeup vector is only used for blocking case, when vCPU is preempted and waiting in the runqueue, the NV is the notification vector. I see your point. But from performance point, if we can schedule the vCPU to another PCPU to handle the interrupt, it would helpful. But I remember current KVM will not schedule the vCPU in run queue (even though it got preempted) to another pCPU to run(Am I right?). So it may hard to do it. KVM is using the Linux scheduler, when the preempted vCPU (in runqueue) is scheduled again depends on the scheduling algorithm itself, I think it is a little hard for us to get involved. I think what you mentioned is a little like the urgent interrupt in VT-d PI Spec. For this kind of interrupts, if an interrupt is coming for an preempted vCPU (waiting in the run queue), we need to schedule the vCPU immediately. This is some real time things. And we don't support urgent interrupt so far. Thanks, Feng Thanks, Feng efforts and in fact it is not needed at all. If SN is set for vCPU0, VT-d hardware will not issue Notification Event for vCPU0 when an interrupt is for it, but just setting the related PIR bit. Thanks, Feng Thanks, Feng waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang ___ iommu mailing list io...@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu Best regards, Yang Best regards, Yang Best regards, Yang -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: Zhang, Yang Z wrote on 2014-12-19: Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Wu, Feng wrote on 2014-12-19: iommu-boun...@lists.linux-foundation.org wrote on mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of: Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; kvm@vger.kernel.org Subject: RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set Paolo Bonzini wrote on 2014-12-18: On 18/12/2014 04:14, Wu, Feng wrote: linux-kernel-ow...@vger.kernel.org wrote on mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo: x...@kernel.org; Gleb Natapov; Paolo Bonzini; dw...@infradead.org; joro-zlv9swrftaidnm+yrof...@public.gmane.org; Alex Williamson; joro-zLv9SwRftAIdnm+Jiang Liu Cc: io...@lists.linux-foundation.org; linux-kernel-u79uwxl29ty76z2rm5m...@public.gmane.org; KVM list; Eric Auger Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set On 12/12/2014 16:14, Feng Wu wrote: Currently, we don't support urgent interrupt, all interrupts are recognized as non-urgent interrupt, so we cannot send posted-interrupt when 'SN' is set. Can this happen? If the vcpu is in guest mode, it cannot have been scheduled out, and that's the only case when SN is set. Paolo Currently, the only place where SN is set is vCPU is preempted and If the vCPU is preempted, shouldn't the subsequent be ignored? What happens if a PI is occurs when vCPU is preempted? If a vCPU is preempted, the 'SN' bit is set, the subsequent interrupts are suppressed for posting. I mean what happens if we don't set SN bit. From my point, if preempter already disabled the interrupt, it is ok to leave SN bit as zero. But if preempter enabled the interrupt, doesn't this mean he allow interrupt to happen? BTW, since there already has ON bit, so this means there only have one interrupt arrived at most and it doesn't hurt performance. Do we really need to set SN bit? See this scenario: vCPU0 is running on pCPU0 -- vCPU0 is preempted by vCPU1 -- Then vCPU1 is running on pCPU0 and vCPU0 is waiting for -- schedule in runqueue If the we don't set SN for vCPU0, then all subsequent interrupts for vCPU0 is posted to vCPU1, this will consume hardware and software The PI vector for vCPU1 is notification vector, but the PI vector for vCPU0 should be wakeup vector. Why vCPU1 will consume this PI event? Wakeup vector is only used for blocking case, when vCPU is preempted and waiting in the runqueue, the NV is the notification vector. I see your point. But from performance point, if we can schedule the vCPU to another PCPU to handle the interrupt, it would helpful. But I remember current KVM will not schedule the vCPU in run queue (even though it got preempted) to another pCPU to run(Am I right?). So it may hard to do it. KVM is using the Linux scheduler, when the preempted vCPU (in runqueue) is scheduled again depends on the scheduling algorithm itself, I think it is a little hard for us to get involved. I think what you mentioned is a little like the urgent interrupt in VT-d PI Spec. For this kind of interrupts, if an interrupt is coming for an preempted vCPU (waiting in the run queue), we need to schedule the vCPU immediately. This is some real time things. And we don't support urgent interrupt so far. Yes. IIRC, if we use two global vectors mechanism properly, there should no need to use hardware urgent interrupt mechanism. :) Thanks, Feng Thanks, Feng efforts and in fact it is not needed at all. If SN is set for vCPU0, VT-d hardware will not issue Notification Event for vCPU0 when an interrupt is for it, but just setting the related PIR bit. Thanks, Feng Thanks, Feng waiting for the next scheduling in the runqueue. But I am not sure whether we need to set SN for other purpose in future. Adding SN checking here is just to follow the Spec. non-urgent interrupts are suppressed when SN is set. I would change that to a WARN_ON_ONCE then. Best regards, Yang ___ iommu mailing list io...@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu Best regards, Yang Best regards, Yang Best regards, Yang Best regards, Yang -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
On 12/12/2014 16:14, Feng Wu wrote:
Currently, we don't support urgent interrupt, all interrupts
are recognized as non-urgent interrupt, so we cannot send
posted-interrupt when 'SN' is set.
Can this happen? If the vcpu is in guest mode, it cannot have been
scheduled out, and that's the only case when SN is set.
Paolo
Signed-off-by: Feng Wu feng.wu-ral2jqcrhueavxtiumw...@public.gmane.org
---
arch/x86/kvm/vmx.c | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index a1c83a2..0aee151 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -4401,15 +4401,22 @@ static int vmx_vm_has_apicv(struct kvm *kvm)
static void vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector)
{
struct vcpu_vmx *vmx = to_vmx(vcpu);
- int r;
+ int r, sn;
if (pi_test_and_set_pir(vector, vmx-pi_desc))
return;
+ /*
+ * Currently, we don't support urgent interrupt, all interrupts
+ * are recognized as non-urgent interrupt, so we cannot send
+ * posted-interrupt when 'SN' is set.
+ */
+ sn = pi_test_sn(vmx-pi_desc);
+
r = pi_test_and_set_on(vmx-pi_desc);
kvm_make_request(KVM_REQ_EVENT, vcpu);
#ifdef CONFIG_SMP
- if (!r (vcpu-mode == IN_GUEST_MODE))
+ if (!r !sn (vcpu-mode == IN_GUEST_MODE))
apic-send_IPI_mask(get_cpu_mask(vcpu-cpu),
POSTED_INTR_VECTOR);
else
--
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
On 12/12/2014 16:14, Feng Wu wrote:
Currently, we don't support urgent interrupt, all interrupts
are recognized as non-urgent interrupt, so we cannot send
posted-interrupt when 'SN' is set.
Can this happen? If the vcpu is in guest mode, it cannot have been
scheduled out, and that's the only case when SN is set.
Paolo
Signed-off-by: Feng Wu feng...@intel.com
---
arch/x86/kvm/vmx.c | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index a1c83a2..0aee151 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -4401,15 +4401,22 @@ static int vmx_vm_has_apicv(struct kvm *kvm)
static void vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector)
{
struct vcpu_vmx *vmx = to_vmx(vcpu);
- int r;
+ int r, sn;
if (pi_test_and_set_pir(vector, vmx-pi_desc))
return;
+ /*
+ * Currently, we don't support urgent interrupt, all interrupts
+ * are recognized as non-urgent interrupt, so we cannot send
+ * posted-interrupt when 'SN' is set.
+ */
+ sn = pi_test_sn(vmx-pi_desc);
+
r = pi_test_and_set_on(vmx-pi_desc);
kvm_make_request(KVM_REQ_EVENT, vcpu);
#ifdef CONFIG_SMP
- if (!r (vcpu-mode == IN_GUEST_MODE))
+ if (!r !sn (vcpu-mode == IN_GUEST_MODE))
apic-send_IPI_mask(get_cpu_mask(vcpu-cpu),
POSTED_INTR_VECTOR);
else
--
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
-Original Message-
From: linux-kernel-ow...@vger.kernel.org
[mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Paolo Bonzini
Sent: Thursday, December 18, 2014 1:43 AM
To: Wu, Feng; Thomas Gleixner; Ingo Molnar; H. Peter Anvin; x...@kernel.org;
Gleb Natapov; Paolo Bonzini; dw...@infradead.org; j...@8bytes.org; Alex
Williamson; Jiang Liu
Cc: io...@lists.linux-foundation.org; linux-ker...@vger.kernel.org; KVM list;
Eric Auger
Subject: Re: [v3 25/26] KVM: Suppress posted-interrupt when 'SN' is set
On 12/12/2014 16:14, Feng Wu wrote:
Currently, we don't support urgent interrupt, all interrupts
are recognized as non-urgent interrupt, so we cannot send
posted-interrupt when 'SN' is set.
Can this happen? If the vcpu is in guest mode, it cannot have been
scheduled out, and that's the only case when SN is set.
Paolo
Currently, the only place where SN is set is vCPU is preempted and waiting for
the next scheduling in the runqueue. But I am not sure whether we need to
set SN for other purpose in future. Adding SN checking here is just to follow
the Spec. non-urgent interrupts are suppressed when SN is set.
Thanks,
Feng
Signed-off-by: Feng Wu
feng.wu-ral2jqcrhueavxtiumw...@public.gmane.org
---
arch/x86/kvm/vmx.c | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index a1c83a2..0aee151 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -4401,15 +4401,22 @@ static int vmx_vm_has_apicv(struct kvm *kvm)
static void vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int
vector)
{
struct vcpu_vmx *vmx = to_vmx(vcpu);
- int r;
+ int r, sn;
if (pi_test_and_set_pir(vector, vmx-pi_desc))
return;
+ /*
+* Currently, we don't support urgent interrupt, all interrupts
+* are recognized as non-urgent interrupt, so we cannot send
+* posted-interrupt when 'SN' is set.
+*/
+ sn = pi_test_sn(vmx-pi_desc);
+
r = pi_test_and_set_on(vmx-pi_desc);
kvm_make_request(KVM_REQ_EVENT, vcpu);
#ifdef CONFIG_SMP
- if (!r (vcpu-mode == IN_GUEST_MODE))
+ if (!r !sn (vcpu-mode == IN_GUEST_MODE))
apic-send_IPI_mask(get_cpu_mask(vcpu-cpu),
POSTED_INTR_VECTOR);
else
--
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
