Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-07 Thread Radim Krčmář
2015-10-07 11:31+0200, Paolo Bonzini:
> What's the issue with
> handle_irq?

I get #PF instead of callback after redirecting to VCPU 1.
No idea what causes it, yet -- seeing handle_irq's iplementation made me
postpone debugging :)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-07 Thread Radim Krčmář
2015-10-07 11:29+0200, Paolo Bonzini:
> On 13/08/2015 15:46, Radim Krčmář wrote:
>> +if (kvm_x86_ops->sync_pir_to_irr(vcpu))
>> +kvm_make_request(KVM_REQ_EVENT, vcpu);
>> +
> 
> The call to sync_pir_to_irr belongs more in vcpu_scan_ioapic, I think.

Will do so in v2.

> More importantly, I think that KVM_REQ_EVENT is a latent bug for
> kvm_vcpu_ioctl_get_lapic as well, so the call to kvm_make_request should
> go in vmx_sync_pir_to_irr or in a new kvm_sync_pir_to_irr wrapper.

True, thanks.  I'll make the request in kvm_apic_update_irr (unless
you'd prefer to have it in new kvm_sync_pir_to_irr).

>> +(e->fields.trig_mode == IOAPIC_EDGE_TRIG &&
>> + kvm_apic_pending_eoi(vcpu, e->fields.vector)))
> 
> Should we test again here that kvm_irq_has_notifier(ioapic->kvm,
> KVM_IRQCHIP_IOAPIC, index), to avoid unnecessarily marking other
> edge-triggered interrupts?

Other edge-triggered interrupts are skipped by a previous condition:

  if (e->fields.trig_mode == IOAPIC_LEVEL_TRIG ||
  kvm_irq_has_notifier(ioapic->kvm, KVM_IRQCHIP_IOAPIC, index) ||
  index == RTC_GSI)
 [we're here]

I think it is ok to ignore level-triggered RTC, but we do want to
include edge-triggered.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-07 Thread Paolo Bonzini


On 06/10/2015 22:33, Radim Krčmář wrote:
> 2015-08-15 02:00+0200, Paolo Bonzini:
>> On 14/08/2015 10:38, Radim Krčmář wrote:
 How do you reproduce the bug?
>>> I run rhel4 (2.6.9) kernel on 2 VCPUs and frequently alternate
>>> smp_affinity of "timer".  The bug is hit within seconds.
>>
>> Nice, I'll try to make a unit test for it on the plane. :)
> 
> (Time on planes is best spent by sleeping ;])

That's what I ended up doing. :)

> What do you think about the series?

I had just a few small comments.  I had all but forgotten it, sorry.
The unit test makes the bug clear, thanks.  What's the issue with
handle_irq?

Paolo

> I made a prototype kvm-unit-test ...
> (Reproduces with APICv or split irqchip builds.)
> ---8<---
> x86: test IO-APIC dest_id modification before
> 
> Regression test for kvm commit $TODO.
> Run with '-smp 2'.
> 
> I had problems with handle_irq so this version uses asm workaround;
> will fix that in final version.  Initialization also presumes that
> everything will work as it did on my machines :)
> ---
>  x86/ioapic.c | 47 +--
>  1 file changed, 45 insertions(+), 2 deletions(-)
> 
> diff --git a/x86/ioapic.c b/x86/ioapic.c
> index 1fe1ccc9eadd..55f8eea03496 100644
> --- a/x86/ioapic.c
> +++ b/x86/ioapic.c
> @@ -4,21 +4,27 @@
>  #include "smp.h"
>  #include "desc.h"
>  #include "isr.h"
> +#include "io.h"
>  
>  #define EDGE_TRIGGERED 0
>  #define LEVEL_TRIGGERED 1
>  
> -static void set_ioapic_redir(unsigned line, unsigned vec, unsigned trig_mode)
> +static void __set_ioapic_redir(unsigned line, u8 vec, bool trig_mode, u8 
> dest_id)
>  {
>   ioapic_redir_entry_t e = {
>   .vector = vec,
> - .delivery_mode = 0,
>   .trig_mode = trig_mode,
> + .dest_id = dest_id,
>   };
>  
>   ioapic_write_redir(line, e);
>  }
>  
> +static void set_ioapic_redir(unsigned line, unsigned vec, unsigned trig_mode)
> +{
> + __set_ioapic_redir(line, vec, trig_mode, 0);
> +}
> +
>  static void set_irq_line(unsigned line, int val)
>  {
>   asm volatile("out %0, %1" : : "a"((u8)val), "d"((u16)(0x2000 + line)));
> @@ -298,6 +304,41 @@ static void test_ioapic_level_retrigger_mask(void)
>   set_mask(0x0e, false);
>  }
>  
> +static volatile int pit_working = -1;
> +
> +static void __attribute__((used)) pit_isr_fn(void)
> +{
> + if (!pit_working++);
> + __set_ioapic_redir(0x2, 0x84, EDGE_TRIGGERED, 1);
> + eoi();
> +}
> +
> +void pit_isr(void);
> +asm (
> + "pit_isr: \n"
> + "   call pit_isr_fn \n"
> +#ifndef __x86_64__
> + "   iret"
> +#else
> + "   iretq"
> +#endif
> + );
> +
> +static void test_ioapic_eoi_bug(void)
> +{
> + if (cpu_count() < 2)
> + return;
> +
> + set_idt_entry(0x84, pit_isr, 0);
> + __set_ioapic_redir(0x2, 0x84, EDGE_TRIGGERED, 0);
> +
> + outb(0x34, 0x43);
> +
> + for (unsigned loops = 1; loops && pit_working < 1; loops--)
> + asm volatile ("nop");
> +
> + report("dest_id reconfiguration before EOI", pit_working >= 1);
> +}
>  
>  int main(void)
>  {
> @@ -325,5 +366,7 @@ int main(void)
>   test_ioapic_level_mask();
>   test_ioapic_level_retrigger_mask();
>  
> + test_ioapic_eoi_bug();
> +
>   return report_summary();
>  }
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-07 Thread Paolo Bonzini


On 13/08/2015 15:46, Radim Krčmář wrote:
> + if (kvm_x86_ops->sync_pir_to_irr(vcpu))
> + kvm_make_request(KVM_REQ_EVENT, vcpu);
> +

The call to sync_pir_to_irr belongs more in vcpu_scan_ioapic, I think.

More importantly, I think that KVM_REQ_EVENT is a latent bug for
kvm_vcpu_ioctl_get_lapic as well, so the call to kvm_make_request should
go in vmx_sync_pir_to_irr or in a new kvm_sync_pir_to_irr wrapper.

> + (e->fields.trig_mode == IOAPIC_EDGE_TRIG &&
> +  kvm_apic_pending_eoi(vcpu, e->fields.vector)))

Should we test again here that kvm_irq_has_notifier(ioapic->kvm,
KVM_IRQCHIP_IOAPIC, index), to avoid unnecessarily marking other
edge-triggered interrupts?

Paolo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-07 Thread Paolo Bonzini


On 06/10/2015 22:33, Radim Krčmář wrote:
> 2015-08-15 02:00+0200, Paolo Bonzini:
>> On 14/08/2015 10:38, Radim Krčmář wrote:
 How do you reproduce the bug?
>>> I run rhel4 (2.6.9) kernel on 2 VCPUs and frequently alternate
>>> smp_affinity of "timer".  The bug is hit within seconds.
>>
>> Nice, I'll try to make a unit test for it on the plane. :)
> 
> (Time on planes is best spent by sleeping ;])

That's what I ended up doing. :)

> What do you think about the series?

I had just a few small comments.  I had all but forgotten it, sorry.
The unit test makes the bug clear, thanks.  What's the issue with
handle_irq?

Paolo

> I made a prototype kvm-unit-test ...
> (Reproduces with APICv or split irqchip builds.)
> ---8<---
> x86: test IO-APIC dest_id modification before
> 
> Regression test for kvm commit $TODO.
> Run with '-smp 2'.
> 
> I had problems with handle_irq so this version uses asm workaround;
> will fix that in final version.  Initialization also presumes that
> everything will work as it did on my machines :)
> ---
>  x86/ioapic.c | 47 +--
>  1 file changed, 45 insertions(+), 2 deletions(-)
> 
> diff --git a/x86/ioapic.c b/x86/ioapic.c
> index 1fe1ccc9eadd..55f8eea03496 100644
> --- a/x86/ioapic.c
> +++ b/x86/ioapic.c
> @@ -4,21 +4,27 @@
>  #include "smp.h"
>  #include "desc.h"
>  #include "isr.h"
> +#include "io.h"
>  
>  #define EDGE_TRIGGERED 0
>  #define LEVEL_TRIGGERED 1
>  
> -static void set_ioapic_redir(unsigned line, unsigned vec, unsigned trig_mode)
> +static void __set_ioapic_redir(unsigned line, u8 vec, bool trig_mode, u8 
> dest_id)
>  {
>   ioapic_redir_entry_t e = {
>   .vector = vec,
> - .delivery_mode = 0,
>   .trig_mode = trig_mode,
> + .dest_id = dest_id,
>   };
>  
>   ioapic_write_redir(line, e);
>  }
>  
> +static void set_ioapic_redir(unsigned line, unsigned vec, unsigned trig_mode)
> +{
> + __set_ioapic_redir(line, vec, trig_mode, 0);
> +}
> +
>  static void set_irq_line(unsigned line, int val)
>  {
>   asm volatile("out %0, %1" : : "a"((u8)val), "d"((u16)(0x2000 + line)));
> @@ -298,6 +304,41 @@ static void test_ioapic_level_retrigger_mask(void)
>   set_mask(0x0e, false);
>  }
>  
> +static volatile int pit_working = -1;
> +
> +static void __attribute__((used)) pit_isr_fn(void)
> +{
> + if (!pit_working++);
> + __set_ioapic_redir(0x2, 0x84, EDGE_TRIGGERED, 1);
> + eoi();
> +}
> +
> +void pit_isr(void);
> +asm (
> + "pit_isr: \n"
> + "   call pit_isr_fn \n"
> +#ifndef __x86_64__
> + "   iret"
> +#else
> + "   iretq"
> +#endif
> + );
> +
> +static void test_ioapic_eoi_bug(void)
> +{
> + if (cpu_count() < 2)
> + return;
> +
> + set_idt_entry(0x84, pit_isr, 0);
> + __set_ioapic_redir(0x2, 0x84, EDGE_TRIGGERED, 0);
> +
> + outb(0x34, 0x43);
> +
> + for (unsigned loops = 1; loops && pit_working < 1; loops--)
> + asm volatile ("nop");
> +
> + report("dest_id reconfiguration before EOI", pit_working >= 1);
> +}
>  
>  int main(void)
>  {
> @@ -325,5 +366,7 @@ int main(void)
>   test_ioapic_level_mask();
>   test_ioapic_level_retrigger_mask();
>  
> + test_ioapic_eoi_bug();
> +
>   return report_summary();
>  }
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-07 Thread Paolo Bonzini


On 13/08/2015 15:46, Radim Krčmář wrote:
> + if (kvm_x86_ops->sync_pir_to_irr(vcpu))
> + kvm_make_request(KVM_REQ_EVENT, vcpu);
> +

The call to sync_pir_to_irr belongs more in vcpu_scan_ioapic, I think.

More importantly, I think that KVM_REQ_EVENT is a latent bug for
kvm_vcpu_ioctl_get_lapic as well, so the call to kvm_make_request should
go in vmx_sync_pir_to_irr or in a new kvm_sync_pir_to_irr wrapper.

> + (e->fields.trig_mode == IOAPIC_EDGE_TRIG &&
> +  kvm_apic_pending_eoi(vcpu, e->fields.vector)))

Should we test again here that kvm_irq_has_notifier(ioapic->kvm,
KVM_IRQCHIP_IOAPIC, index), to avoid unnecessarily marking other
edge-triggered interrupts?

Paolo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-07 Thread Radim Krčmář
2015-10-07 11:29+0200, Paolo Bonzini:
> On 13/08/2015 15:46, Radim Krčmář wrote:
>> +if (kvm_x86_ops->sync_pir_to_irr(vcpu))
>> +kvm_make_request(KVM_REQ_EVENT, vcpu);
>> +
> 
> The call to sync_pir_to_irr belongs more in vcpu_scan_ioapic, I think.

Will do so in v2.

> More importantly, I think that KVM_REQ_EVENT is a latent bug for
> kvm_vcpu_ioctl_get_lapic as well, so the call to kvm_make_request should
> go in vmx_sync_pir_to_irr or in a new kvm_sync_pir_to_irr wrapper.

True, thanks.  I'll make the request in kvm_apic_update_irr (unless
you'd prefer to have it in new kvm_sync_pir_to_irr).

>> +(e->fields.trig_mode == IOAPIC_EDGE_TRIG &&
>> + kvm_apic_pending_eoi(vcpu, e->fields.vector)))
> 
> Should we test again here that kvm_irq_has_notifier(ioapic->kvm,
> KVM_IRQCHIP_IOAPIC, index), to avoid unnecessarily marking other
> edge-triggered interrupts?

Other edge-triggered interrupts are skipped by a previous condition:

  if (e->fields.trig_mode == IOAPIC_LEVEL_TRIG ||
  kvm_irq_has_notifier(ioapic->kvm, KVM_IRQCHIP_IOAPIC, index) ||
  index == RTC_GSI)
 [we're here]

I think it is ok to ignore level-triggered RTC, but we do want to
include edge-triggered.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-07 Thread Radim Krčmář
2015-10-07 11:31+0200, Paolo Bonzini:
> What's the issue with
> handle_irq?

I get #PF instead of callback after redirecting to VCPU 1.
No idea what causes it, yet -- seeing handle_irq's iplementation made me
postpone debugging :)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-06 Thread Radim Krčmář
2015-08-15 02:00+0200, Paolo Bonzini:
>On 14/08/2015 10:38, Radim Krčmář wrote:
>>> How do you reproduce the bug?
>> I run rhel4 (2.6.9) kernel on 2 VCPUs and frequently alternate
>> smp_affinity of "timer".  The bug is hit within seconds.
> 
> Nice, I'll try to make a unit test for it on the plane. :)

(Time on planes is best spent by sleeping ;])

What do you think about the series?

I made a prototype kvm-unit-test ...
(Reproduces with APICv or split irqchip builds.)
---8<---
x86: test IO-APIC dest_id modification before

Regression test for kvm commit $TODO.
Run with '-smp 2'.

I had problems with handle_irq so this version uses asm workaround;
will fix that in final version.  Initialization also presumes that
everything will work as it did on my machines :)
---
 x86/ioapic.c | 47 +--
 1 file changed, 45 insertions(+), 2 deletions(-)

diff --git a/x86/ioapic.c b/x86/ioapic.c
index 1fe1ccc9eadd..55f8eea03496 100644
--- a/x86/ioapic.c
+++ b/x86/ioapic.c
@@ -4,21 +4,27 @@
 #include "smp.h"
 #include "desc.h"
 #include "isr.h"
+#include "io.h"
 
 #define EDGE_TRIGGERED 0
 #define LEVEL_TRIGGERED 1
 
-static void set_ioapic_redir(unsigned line, unsigned vec, unsigned trig_mode)
+static void __set_ioapic_redir(unsigned line, u8 vec, bool trig_mode, u8 
dest_id)
 {
ioapic_redir_entry_t e = {
.vector = vec,
-   .delivery_mode = 0,
.trig_mode = trig_mode,
+   .dest_id = dest_id,
};
 
ioapic_write_redir(line, e);
 }
 
+static void set_ioapic_redir(unsigned line, unsigned vec, unsigned trig_mode)
+{
+   __set_ioapic_redir(line, vec, trig_mode, 0);
+}
+
 static void set_irq_line(unsigned line, int val)
 {
asm volatile("out %0, %1" : : "a"((u8)val), "d"((u16)(0x2000 + line)));
@@ -298,6 +304,41 @@ static void test_ioapic_level_retrigger_mask(void)
set_mask(0x0e, false);
 }
 
+static volatile int pit_working = -1;
+
+static void __attribute__((used)) pit_isr_fn(void)
+{
+   if (!pit_working++);
+   __set_ioapic_redir(0x2, 0x84, EDGE_TRIGGERED, 1);
+   eoi();
+}
+
+void pit_isr(void);
+asm (
+ "pit_isr: \n"
+ "   call pit_isr_fn \n"
+#ifndef __x86_64__
+ "   iret"
+#else
+ "   iretq"
+#endif
+ );
+
+static void test_ioapic_eoi_bug(void)
+{
+   if (cpu_count() < 2)
+   return;
+
+   set_idt_entry(0x84, pit_isr, 0);
+   __set_ioapic_redir(0x2, 0x84, EDGE_TRIGGERED, 0);
+
+   outb(0x34, 0x43);
+
+   for (unsigned loops = 1; loops && pit_working < 1; loops--)
+   asm volatile ("nop");
+
+   report("dest_id reconfiguration before EOI", pit_working >= 1);
+}
 
 int main(void)
 {
@@ -325,5 +366,7 @@ int main(void)
test_ioapic_level_mask();
test_ioapic_level_retrigger_mask();
 
+   test_ioapic_eoi_bug();
+
return report_summary();
 }
-- 
2.5.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-10-06 Thread Radim Krčmář
2015-08-15 02:00+0200, Paolo Bonzini:
>On 14/08/2015 10:38, Radim Krčmář wrote:
>>> How do you reproduce the bug?
>> I run rhel4 (2.6.9) kernel on 2 VCPUs and frequently alternate
>> smp_affinity of "timer".  The bug is hit within seconds.
> 
> Nice, I'll try to make a unit test for it on the plane. :)

(Time on planes is best spent by sleeping ;])

What do you think about the series?

I made a prototype kvm-unit-test ...
(Reproduces with APICv or split irqchip builds.)
---8<---
x86: test IO-APIC dest_id modification before

Regression test for kvm commit $TODO.
Run with '-smp 2'.

I had problems with handle_irq so this version uses asm workaround;
will fix that in final version.  Initialization also presumes that
everything will work as it did on my machines :)
---
 x86/ioapic.c | 47 +--
 1 file changed, 45 insertions(+), 2 deletions(-)

diff --git a/x86/ioapic.c b/x86/ioapic.c
index 1fe1ccc9eadd..55f8eea03496 100644
--- a/x86/ioapic.c
+++ b/x86/ioapic.c
@@ -4,21 +4,27 @@
 #include "smp.h"
 #include "desc.h"
 #include "isr.h"
+#include "io.h"
 
 #define EDGE_TRIGGERED 0
 #define LEVEL_TRIGGERED 1
 
-static void set_ioapic_redir(unsigned line, unsigned vec, unsigned trig_mode)
+static void __set_ioapic_redir(unsigned line, u8 vec, bool trig_mode, u8 
dest_id)
 {
ioapic_redir_entry_t e = {
.vector = vec,
-   .delivery_mode = 0,
.trig_mode = trig_mode,
+   .dest_id = dest_id,
};
 
ioapic_write_redir(line, e);
 }
 
+static void set_ioapic_redir(unsigned line, unsigned vec, unsigned trig_mode)
+{
+   __set_ioapic_redir(line, vec, trig_mode, 0);
+}
+
 static void set_irq_line(unsigned line, int val)
 {
asm volatile("out %0, %1" : : "a"((u8)val), "d"((u16)(0x2000 + line)));
@@ -298,6 +304,41 @@ static void test_ioapic_level_retrigger_mask(void)
set_mask(0x0e, false);
 }
 
+static volatile int pit_working = -1;
+
+static void __attribute__((used)) pit_isr_fn(void)
+{
+   if (!pit_working++);
+   __set_ioapic_redir(0x2, 0x84, EDGE_TRIGGERED, 1);
+   eoi();
+}
+
+void pit_isr(void);
+asm (
+ "pit_isr: \n"
+ "   call pit_isr_fn \n"
+#ifndef __x86_64__
+ "   iret"
+#else
+ "   iretq"
+#endif
+ );
+
+static void test_ioapic_eoi_bug(void)
+{
+   if (cpu_count() < 2)
+   return;
+
+   set_idt_entry(0x84, pit_isr, 0);
+   __set_ioapic_redir(0x2, 0x84, EDGE_TRIGGERED, 0);
+
+   outb(0x34, 0x43);
+
+   for (unsigned loops = 1; loops && pit_working < 1; loops--)
+   asm volatile ("nop");
+
+   report("dest_id reconfiguration before EOI", pit_working >= 1);
+}
 
 int main(void)
 {
@@ -325,5 +366,7 @@ int main(void)
test_ioapic_level_mask();
test_ioapic_level_retrigger_mask();
 
+   test_ioapic_eoi_bug();
+
return report_summary();
 }
-- 
2.5.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-08-14 Thread Paolo Bonzini


On 14/08/2015 10:38, Radim Krčmář wrote:
>> How do you reproduce the bug?
> I run rhel4 (2.6.9) kernel on 2 VCPUs and frequently alternate
> smp_affinity of "timer".  The bug is hit within seconds.

Nice, I'll try to make a unit test for it on the plane. :)

Paolo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-08-14 Thread Radim Krčmář
2015-08-13 16:53+0200, Paolo Bonzini:
> On 13/08/2015 15:46, Radim Krčmář wrote:
>>  1) IOAPIC inject a vector from i8254
>>  2) guest reconfigures that vector's VCPU and therefore eoi_exit_bitmap
>> on original VCPU gets cleared
>>  3) guest's handler for the vector does EOI
>>  4) KVM's EOI handler doesn't pass that vector to IOAPIC because it is
>> not in that VCPU's eoi_exit_bitmap
>>  5) i8254 stops working
>> 
>> This creates an unwanted situation if the vector is reused by a
>> non-IOAPIC source, but I think it is so rare that we don't want to make
>> the solution more sophisticated. 
> 
> What happens if the vector is changed in step 2?
> __kvm_ioapic_update_eoi won't match the redirection table entry.

Yes, the EOI is going to be ignored.  (With APICv, VMX won't even exit.)
In the patch, I dissmissed it as "shouldn't happen in the wild" because
we've always had the vector-change bug :) (Unlike the destination-change
one, which was APICv-only before recent changes.)

A simple solution to the vector-change would have a list of one-time
fixups (vector, *ioapic) and hooks in ioapic reconfig, scan and EOI.

A complex solution would replace ioapic scanning with an array of list
of ioapics (it needs to be a list or small array because vectors can be
shared).
An ioapic would be added to list[vector] on reconfig and removed on
reconfig unless an edge fixup was needed, then it would last til next
EOI  (I guess we won't need to consider vector in IRR and ISR).
Callbacks would update the eoi_exit_bitmap on relevant changes.

I considered doing the complex one, but then it occured to me that we
want the destination-change fixed in stable as APICv machines are
starting to get used and people might migrate old guests on them.

> How do you reproduce the bug?

I run rhel4 (2.6.9) kernel on 2 VCPUs and frequently alternate
smp_affinity of "timer".  The bug is hit within seconds.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-08-14 Thread Radim Krčmář
2015-08-13 16:53+0200, Paolo Bonzini:
 On 13/08/2015 15:46, Radim Krčmář wrote:
  1) IOAPIC inject a vector from i8254
  2) guest reconfigures that vector's VCPU and therefore eoi_exit_bitmap
 on original VCPU gets cleared
  3) guest's handler for the vector does EOI
  4) KVM's EOI handler doesn't pass that vector to IOAPIC because it is
 not in that VCPU's eoi_exit_bitmap
  5) i8254 stops working
 
 This creates an unwanted situation if the vector is reused by a
 non-IOAPIC source, but I think it is so rare that we don't want to make
 the solution more sophisticated. 
 
 What happens if the vector is changed in step 2?
 __kvm_ioapic_update_eoi won't match the redirection table entry.

Yes, the EOI is going to be ignored.  (With APICv, VMX won't even exit.)
In the patch, I dissmissed it as shouldn't happen in the wild because
we've always had the vector-change bug :) (Unlike the destination-change
one, which was APICv-only before recent changes.)

A simple solution to the vector-change would have a list of one-time
fixups (vector, *ioapic) and hooks in ioapic reconfig, scan and EOI.

A complex solution would replace ioapic scanning with an array of list
of ioapics (it needs to be a list or small array because vectors can be
shared).
An ioapic would be added to list[vector] on reconfig and removed on
reconfig unless an edge fixup was needed, then it would last til next
EOI  (I guess we won't need to consider vector in IRR and ISR).
Callbacks would update the eoi_exit_bitmap on relevant changes.

I considered doing the complex one, but then it occured to me that we
want the destination-change fixed in stable as APICv machines are
starting to get used and people might migrate old guests on them.

 How do you reproduce the bug?

I run rhel4 (2.6.9) kernel on 2 VCPUs and frequently alternate
smp_affinity of timer.  The bug is hit within seconds.
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-08-14 Thread Paolo Bonzini


On 14/08/2015 10:38, Radim Krčmář wrote:
 How do you reproduce the bug?
 I run rhel4 (2.6.9) kernel on 2 VCPUs and frequently alternate
 smp_affinity of timer.  The bug is hit within seconds.

Nice, I'll try to make a unit test for it on the plane. :)

Paolo
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-08-13 Thread Paolo Bonzini


On 13/08/2015 15:46, Radim Krčmář wrote:
>  1) IOAPIC inject a vector from i8254
>  2) guest reconfigures that vector's VCPU and therefore eoi_exit_bitmap
> on original VCPU gets cleared
>  3) guest's handler for the vector does EOI
>  4) KVM's EOI handler doesn't pass that vector to IOAPIC because it is
> not in that VCPU's eoi_exit_bitmap
>  5) i8254 stops working
> 
> This creates an unwanted situation if the vector is reused by a
> non-IOAPIC source, but I think it is so rare that we don't want to make
> the solution more sophisticated. 

What happens if the vector is changed in step 2?
__kvm_ioapic_update_eoi won't match the redirection table entry.

How do you reproduce the bug?

Paolo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 2/2] KVM: x86: fix edge EOI and IOAPIC reconfig race

2015-08-13 Thread Paolo Bonzini


On 13/08/2015 15:46, Radim Krčmář wrote:
  1) IOAPIC inject a vector from i8254
  2) guest reconfigures that vector's VCPU and therefore eoi_exit_bitmap
 on original VCPU gets cleared
  3) guest's handler for the vector does EOI
  4) KVM's EOI handler doesn't pass that vector to IOAPIC because it is
 not in that VCPU's eoi_exit_bitmap
  5) i8254 stops working
 
 This creates an unwanted situation if the vector is reused by a
 non-IOAPIC source, but I think it is so rare that we don't want to make
 the solution more sophisticated. 

What happens if the vector is changed in step 2?
__kvm_ioapic_update_eoi won't match the redirection table entry.

How do you reproduce the bug?

Paolo
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/