Re: cpu hotplug
On Sun, Sep 19, 2010 at 06:03:31PM -0400, Kevin O'Connor wrote: > On Sun, Sep 19, 2010 at 06:12:14PM +0200, Gleb Natapov wrote: > > On Sun, Sep 19, 2010 at 12:04:51PM -0400, Kevin O'Connor wrote: > > > I'm confused. The "cpu_set x offline" doesn't appear to do anything > > > for me. It does not disable the cpu, nor does it inform the guest to > > > disable the cpu. > [...] > > Then cpu eject method in seabios has a bug. When "cpu_set x offline" is > > called qemu sets status bit in gpe and injects ACPI interrupt. ACPI > > should do Notify() on cpu object when it happens. > > I was wrong. The "cpu_set x offline" does send an event to the guest > OS. SeaBIOS even forwards the event along - as far as I can tell a > Notify(CPxx, 3) event is generated by SeaBIOS. > > My Windows 7 ultimate beta seems to receive the event okay (it pops up > a dialog box which says you can't unplug cpus). > It may react to Eject() method. > Unfortunately, my test linux guest OS (FC13) doesn't seem to do > anything with the unplug Notify event. I've tried with the original > FC13 and with a fully updated version - no luck. > > So, I'm guessing this has something to do with the guest OS. > Can you verify that _STA() return zero after cpu unplug? -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 05:57:25PM -0400, Kevin O'Connor wrote:
> On Sun, Sep 19, 2010 at 06:24:33PM +0200, Conrad Wood wrote:
> >
> > a script with my exact steps is below. Result is reproducible.
> >
> [...]
> > echo "Ejecting CPU #4"
> > echo "cpu_set 4 offline" | nc ${MONITORHOST} ${MONITORPORT} >/dev/null
> >
> > printInfo
> >
> > echo "Setting all available cpus to online..."
> > ssh -lroot ${VM} "find /sys/devices/system/cpu/ -type f -path
> > '*/cpu?/online' -exec bash -c \"echo 1 >{}\" \;"
>
> It's not clear if you can trust your guests or not. If you can trust
> the guest OS, it should be safe to run:
>
> echo 1 > /sys/devices/LNXSYSTM:00/LNXSYBUS:00/LNXCPU:xx/eject
>
> in the guest. A normal linux guest wont reactivate the cpu after
> that.
>
You can trust you guest more then you can trust a users of your guest.
It is easy for a user to do echo, it is much harder to modify guest to not
remove /sys entry for un-plugged cpu.
But I agree that kvm shouldn't allow to reactivate un-plugged cpu even
if it is not destroy it on un-plug.
--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] Correct handling of idt vectoring info
On Thu, Jun 17, 2010, Gleb Natapov wrote about "Re: [PATCH 22/24] Correct handling of idt vectoring info": > On Sun, Jun 13, 2010 at 03:33:50PM +0300, Nadav Har'El wrote: > > In the normal non-nested case, the idt_vectoring_info case is treated after > > the exit. However, in the nested case a decision of whether to return to L2 > This is not correct. On the normal non-nested case the idt_vectoring_info is > parsed into vmx/svm independent data structure (which is saved/restored during > VM migartion) after exit. The reinjection happens on vmentry path. You're right that perhaps I overstated the difference between the nested and normal case. In both cases, the code at the end of vmx_vcpu_run just remembers what to do, and the actual injection (namely, setting the vmcs) happens in the beginning of the next entry. I'll change the wording of the patch's description to make it more accurate. > Why can't you do that using existing exception/nmi/interrupt queues that > we have, but instead you effectively disable vmx_complete_interrupts() > by patch 18 when in nested mode and add logically same code in this > patch. I.e after exit you save info about idt event into nested_vmx > and reinject it on vm entry. This is an interesting point. The basic problem is that (as I explained in the patch's description) when L2 exits to L1 with idt vectoring info, L0 should *not* do its normal thing of injecting the event - it should basically do nothing, and just leave the IDT_VECTORING_INFO_FIELD in vmcs12 for L1 to find and act upon. So in this case we must eliminate the normal decision that KVM would make to inject the event. Perhaps it would have been possible to leave the decision as-is (i.e., not change vmx_complete_interrupts()), but instead disable the injection itself in inject_pending_event() (in x86.c, not vmx.c) or in all of vmx_queue_exception, vmx_set_nmi and vmx_set_irq. But I'm not sure this will be a cleaner patch (and I'd especially like to avoid nested-specific changes in x86.c), and I'm pretty sure that however I change this code, it's bound to break in subtle ways. The current patch took some blood, toil, tears and sweat (well, maybe everything except the blood...) of my coworkers to get right :-) Nadav. -- Nadav Har'El| Sunday, Sep 19 2010, 12 Tishri 5771 n...@math.technion.ac.il |- Phone +972-523-790466, ICQ 13349191 |Life is what happens to you while you're http://nadav.harel.org.il |busy making other plans. - John Lennon -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 06:12:14PM +0200, Gleb Natapov wrote: > On Sun, Sep 19, 2010 at 12:04:51PM -0400, Kevin O'Connor wrote: > > I'm confused. The "cpu_set x offline" doesn't appear to do anything > > for me. It does not disable the cpu, nor does it inform the guest to > > disable the cpu. [...] > Then cpu eject method in seabios has a bug. When "cpu_set x offline" is > called qemu sets status bit in gpe and injects ACPI interrupt. ACPI > should do Notify() on cpu object when it happens. I was wrong. The "cpu_set x offline" does send an event to the guest OS. SeaBIOS even forwards the event along - as far as I can tell a Notify(CPxx, 3) event is generated by SeaBIOS. My Windows 7 ultimate beta seems to receive the event okay (it pops up a dialog box which says you can't unplug cpus). Unfortunately, my test linux guest OS (FC13) doesn't seem to do anything with the unplug Notify event. I've tried with the original FC13 and with a fully updated version - no luck. So, I'm guessing this has something to do with the guest OS. -Kevin -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 06:24:33PM +0200, Conrad Wood wrote:
>
> a script with my exact steps is below. Result is reproducible.
>
[...]
> echo "Ejecting CPU #4"
> echo "cpu_set 4 offline" | nc ${MONITORHOST} ${MONITORPORT} >/dev/null
>
> printInfo
>
> echo "Setting all available cpus to online..."
> ssh -lroot ${VM} "find /sys/devices/system/cpu/ -type f -path
> '*/cpu?/online' -exec bash -c \"echo 1 >{}\" \;"
It's not clear if you can trust your guests or not. If you can trust
the guest OS, it should be safe to run:
echo 1 > /sys/devices/LNXSYSTM:00/LNXSYBUS:00/LNXCPU:xx/eject
in the guest. A normal linux guest wont reactivate the cpu after
that.
-Kevin
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm: fix irqfd assign/deassign race
>>> On 9/19/2010 at 01:02 PM, in message <20100919170231.ga12...@redhat.com>, "Michael S. Tsirkin" wrote: > I think I see the following (theoretical) race: > > During irqfd assign, we drop irqfds lock before we > schedule inject work. Therefore, deassign running > on another CPU could cause shutdown and flush to run > before inject, causing user after free in inject. > > A simple fix it to schedule inject under the lock. I swear there was some reason why the schedule_work() was done outside of the lock, but I can't for the life of me remember why anymore (it obviously was a failing on my part to _comment_ why if there was such a reason). So, short of recalling what that reason was, and the fact that Michael's theory seems rational and legit... Acked-by: Gregory Haskins > > Signed-off-by: Michael S. Tsirkin > --- > > If the issue is real, this might be a 2.6.36 and -stable > candidate. Comments? > > virt/kvm/eventfd.c |3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c > index 66cf65b..c1f1e3c 100644 > --- a/virt/kvm/eventfd.c > +++ b/virt/kvm/eventfd.c > @@ -218,7 +218,6 @@ kvm_irqfd_assign(struct kvm *kvm, int fd, int gsi) > events = file->f_op->poll(file, &irqfd->pt); > > list_add_tail(&irqfd->list, &kvm->irqfds.items); > - spin_unlock_irq(&kvm->irqfds.lock); > > /* >* Check if there was an event already pending on the eventfd > @@ -227,6 +226,8 @@ kvm_irqfd_assign(struct kvm *kvm, int fd, int gsi) > if (events & POLLIN) > schedule_work(&irqfd->inject); > > + spin_unlock_irq(&kvm->irqfds.lock); > + > /* >* do not drop the file until the irqfd is fully initialized, otherwise >* we might race against the POLLHUP -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] SVM: do not generate "external interrupt exit" if other exit is pending
On Sun, Sep 19, 2010 at 07:50:27PM +0200, Gleb Natapov wrote: > On Sun, Sep 19, 2010 at 07:29:41PM +0200, Joerg Roedel wrote: > > On Sun, Sep 19, 2010 at 06:41:27PM +0200, Gleb Natapov wrote: > > > Nested SVM checks for external interrupt after injecting nested exception. > > > In case there is external interrupt pending the code generates "external > > > interrupt exit" and overwrites previous exit info. If previously injected > > > exception already generated exit it will be lost. > > > > Right. Have you seen specific mismehavior due to this problem? I am just > > curious how you found this :-) > Yes. Trying to make async page fault work with nested svm :) It was hard > to fix with reproducible test case. I am not dreaming to fix it just by > code review. Interesting. Hope this was the only problem and you have it working now? > > And another question, can you put the reason for this "if(...) return" > > into an comment before the statement? The whole svm-emulation thing is > > complicated enough so that we should document it well in the code :-) > > > Complicated is understatement. Will add a comment. Thanks, the most complicated thing is to get all the interrupt- exception- and event-reinjection thing right. You just fixed another bug there, thanks again :) Joerg -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] SVM: do not generate "external interrupt exit" if other exit is pending
On Sun, Sep 19, 2010 at 07:29:41PM +0200, Joerg Roedel wrote: > On Sun, Sep 19, 2010 at 06:41:27PM +0200, Gleb Natapov wrote: > > Nested SVM checks for external interrupt after injecting nested exception. > > In case there is external interrupt pending the code generates "external > > interrupt exit" and overwrites previous exit info. If previously injected > > exception already generated exit it will be lost. > > Right. Have you seen specific mismehavior due to this problem? I am just > curious how you found this :-) Yes. Trying to make async page fault work with nested svm :) It was hard to fix with reproducible test case. I am not dreaming to fix it just by code review. > And another question, can you put the reason for this "if(...) return" > into an comment before the statement? The whole svm-emulation thing is > complicated enough so that we should document it well in the code :-) > Complicated is understatement. Will add a comment. > Joerg > > > > > Signed-off-by: Gleb Natapov > > diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c > > index 43f5558..1a1f86b 100644 > > --- a/arch/x86/kvm/svm.c > > +++ b/arch/x86/kvm/svm.c > > @@ -1707,6 +1707,9 @@ static inline bool nested_svm_intr(struct vcpu_svm > > *svm) > > if (!(svm->vcpu.arch.hflags & HF_HIF_MASK)) > > return false; > > > > + if (svm->nested.exit_required) > > + return false; > > + > > svm->vmcb->control.exit_code = SVM_EXIT_INTR; > > svm->vmcb->control.exit_info_1 = 0; > > svm->vmcb->control.exit_info_2 = 0; > > -- > > Gleb. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] SVM: do not generate "external interrupt exit" if other exit is pending
On Sun, Sep 19, 2010 at 06:41:27PM +0200, Gleb Natapov wrote: > Nested SVM checks for external interrupt after injecting nested exception. > In case there is external interrupt pending the code generates "external > interrupt exit" and overwrites previous exit info. If previously injected > exception already generated exit it will be lost. Right. Have you seen specific mismehavior due to this problem? I am just curious how you found this :-) And another question, can you put the reason for this "if(...) return" into an comment before the statement? The whole svm-emulation thing is complicated enough so that we should document it well in the code :-) Joerg > > Signed-off-by: Gleb Natapov > diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c > index 43f5558..1a1f86b 100644 > --- a/arch/x86/kvm/svm.c > +++ b/arch/x86/kvm/svm.c > @@ -1707,6 +1707,9 @@ static inline bool nested_svm_intr(struct vcpu_svm *svm) > if (!(svm->vcpu.arch.hflags & HF_HIF_MASK)) > return false; > > + if (svm->nested.exit_required) > + return false; > + > svm->vmcb->control.exit_code = SVM_EXIT_INTR; > svm->vmcb->control.exit_info_1 = 0; > svm->vmcb->control.exit_info_2 = 0; > -- > Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] kvm: fix irqfd assign/deassign race
I think I see the following (theoretical) race: During irqfd assign, we drop irqfds lock before we schedule inject work. Therefore, deassign running on another CPU could cause shutdown and flush to run before inject, causing user after free in inject. A simple fix it to schedule inject under the lock. Signed-off-by: Michael S. Tsirkin --- If the issue is real, this might be a 2.6.36 and -stable candidate. Comments? virt/kvm/eventfd.c |3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index 66cf65b..c1f1e3c 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -218,7 +218,6 @@ kvm_irqfd_assign(struct kvm *kvm, int fd, int gsi) events = file->f_op->poll(file, &irqfd->pt); list_add_tail(&irqfd->list, &kvm->irqfds.items); - spin_unlock_irq(&kvm->irqfds.lock); /* * Check if there was an event already pending on the eventfd @@ -227,6 +226,8 @@ kvm_irqfd_assign(struct kvm *kvm, int fd, int gsi) if (events & POLLIN) schedule_work(&irqfd->inject); + spin_unlock_irq(&kvm->irqfds.lock); + /* * do not drop the file until the irqfd is fully initialized, otherwise * we might race against the POLLHUP -- 1.7.2.2 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: raw_spinlock_t rules
On 09/19/2010 06:38 PM, Steven Rostedt wrote: On Sun, 2010-09-19 at 15:09 +0200, Avi Kivity wrote: > Some time ago, the i8259 emulation code in kvm was changed to use > raw_spinlock_t, as it was called in a preempt_disable() and > local_irq_disable() context, which doesn't work with preemptible > spinlocks used with CONFIG_PREEMPT_RT. > > In Linux 2.6.37, the spinlock will no longer be taken in these contexts, > so I'd like to change it to a normal spinlock_t. However, it is still > taken in a spin_lock_irq() context. > > Is it okay to do this change? I figured since spin_lock_irq() is part > of the spinlock infrastructure it might to the right thing. > In PREEMPT_RT all interrupts (besides the timer) is converted to a thread. The spin_lock_irq() will not disable interrupts in PREEMPT_RT, because it is not needed (irqs are threaded and wont cause a deadlock if they share the same spin_locks). Note, spin_locks used by the timer will must be raw_spin_lock(), and threaded context must disable interrupts for real before using it. Thanks. So it looks like we can convert that spinlock. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] KVM: Convert PIC lock from raw spinlock to ordinary spinlock
The PIC code used to be called from preempt_disable() context, which
wasn't very good for PREEMPT_RT. That is no longer the case, so move
back from raw_spinlock_t to spinlock_t.
Signed-off-by: Avi Kivity
---
arch/x86/kvm/i8259.c |6 +++---
arch/x86/kvm/irq.h |2 +-
arch/x86/kvm/x86.c |8
3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kvm/i8259.c b/arch/x86/kvm/i8259.c
index ab1bb8f..dd54c5b 100644
--- a/arch/x86/kvm/i8259.c
+++ b/arch/x86/kvm/i8259.c
@@ -39,7 +39,7 @@ static void pic_irq_request(struct kvm *kvm, int level);
static void pic_lock(struct kvm_pic *s)
__acquires(&s->lock)
{
- raw_spin_lock(&s->lock);
+ spin_lock(&s->lock);
}
static void pic_unlock(struct kvm_pic *s)
@@ -51,7 +51,7 @@ static void pic_unlock(struct kvm_pic *s)
s->wakeup_needed = false;
- raw_spin_unlock(&s->lock);
+ spin_unlock(&s->lock);
if (wakeup) {
kvm_for_each_vcpu(i, vcpu, s->kvm) {
@@ -569,7 +569,7 @@ struct kvm_pic *kvm_create_pic(struct kvm *kvm)
s = kzalloc(sizeof(struct kvm_pic), GFP_KERNEL);
if (!s)
return NULL;
- raw_spin_lock_init(&s->lock);
+ spin_lock_init(&s->lock);
s->kvm = kvm;
s->pics[0].elcr_mask = 0xf8;
s->pics[1].elcr_mask = 0xde;
diff --git a/arch/x86/kvm/irq.h b/arch/x86/kvm/irq.h
index 63c3145..ba910d1 100644
--- a/arch/x86/kvm/irq.h
+++ b/arch/x86/kvm/irq.h
@@ -60,7 +60,7 @@ struct kvm_kpic_state {
};
struct kvm_pic {
- raw_spinlock_t lock;
+ spinlock_t lock;
bool wakeup_needed;
unsigned pending_acks;
struct kvm *kvm;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index a51635e..fba4d93 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3010,18 +3010,18 @@ static int kvm_vm_ioctl_set_irqchip(struct kvm *kvm,
struct kvm_irqchip *chip)
r = 0;
switch (chip->chip_id) {
case KVM_IRQCHIP_PIC_MASTER:
- raw_spin_lock(&pic_irqchip(kvm)->lock);
+ spin_lock(&pic_irqchip(kvm)->lock);
memcpy(&pic_irqchip(kvm)->pics[0],
&chip->chip.pic,
sizeof(struct kvm_pic_state));
- raw_spin_unlock(&pic_irqchip(kvm)->lock);
+ spin_unlock(&pic_irqchip(kvm)->lock);
break;
case KVM_IRQCHIP_PIC_SLAVE:
- raw_spin_lock(&pic_irqchip(kvm)->lock);
+ spin_lock(&pic_irqchip(kvm)->lock);
memcpy(&pic_irqchip(kvm)->pics[1],
&chip->chip.pic,
sizeof(struct kvm_pic_state));
- raw_spin_unlock(&pic_irqchip(kvm)->lock);
+ spin_unlock(&pic_irqchip(kvm)->lock);
break;
case KVM_IRQCHIP_IOAPIC:
r = kvm_set_ioapic(kvm, &chip->chip.ioapic);
--
1.7.2.3
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] SVM: do not generate "external interrupt exit" if other exit is pending
Nested SVM checks for external interrupt after injecting nested exception. In case there is external interrupt pending the code generates "external interrupt exit" and overwrites previous exit info. If previously injected exception already generated exit it will be lost. Signed-off-by: Gleb Natapov diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 43f5558..1a1f86b 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -1707,6 +1707,9 @@ static inline bool nested_svm_intr(struct vcpu_svm *svm) if (!(svm->vcpu.arch.hflags & HF_HIF_MASK)) return false; + if (svm->nested.exit_required) + return false; + svm->vmcb->control.exit_code = SVM_EXIT_INTR; svm->vmcb->control.exit_info_1 = 0; svm->vmcb->control.exit_info_2 = 0; -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: raw_spinlock_t rules
On Sun, 2010-09-19 at 15:09 +0200, Avi Kivity wrote: > Some time ago, the i8259 emulation code in kvm was changed to use > raw_spinlock_t, as it was called in a preempt_disable() and > local_irq_disable() context, which doesn't work with preemptible > spinlocks used with CONFIG_PREEMPT_RT. > > In Linux 2.6.37, the spinlock will no longer be taken in these contexts, > so I'd like to change it to a normal spinlock_t. However, it is still > taken in a spin_lock_irq() context. > > Is it okay to do this change? I figured since spin_lock_irq() is part > of the spinlock infrastructure it might to the right thing. > In PREEMPT_RT all interrupts (besides the timer) is converted to a thread. The spin_lock_irq() will not disable interrupts in PREEMPT_RT, because it is not needed (irqs are threaded and wont cause a deadlock if they share the same spin_locks). Note, spin_locks used by the timer will must be raw_spin_lock(), and threaded context must disable interrupts for real before using it. -- Steve -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] x86, nmi: workaround sti; hlt race vs nmi; intr
On machines without monitor/mwait we use an sti; hlt sequence to atomically
enable interrupts and put the cpu to sleep. The sequence uses the "interrupt
shadow" property of the sti instruction: interrupts are enabled only after
the instruction following sti has been executed. This means an interrupt
cannot happen in the middle of the sequence, which would leave us with
the interrupt processed but the cpu halted.
The interrupt shadow, however, can be broken by an nmi; the following
sequence
sti
nmi ... iret
# interrupt shadow disabled
intr ... iret
hlt
puts the cpu to sleep, even though the interrupt may need additional processing
after the hlt (like scheduling a task).
sti is explicitly documented not to force an interrupt shadow; though many
processors do inhibit nmi immediately after sti.
Avoid the race by checking, during an nmi, if we hit the safe halt sequence.
If we did, increment the instruction pointer past the hlt instruction; this
allows an immediately following interrupt to return to a safe place.
Signed-off-by: Avi Kivity
---
arch/x86/include/asm/irqflags.h | 18 +-
arch/x86/kernel/traps.c | 14 ++
2 files changed, 31 insertions(+), 1 deletions(-)
diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h
index 9e2b952..f412167 100644
--- a/arch/x86/include/asm/irqflags.h
+++ b/arch/x86/include/asm/irqflags.h
@@ -44,9 +44,25 @@ static inline void native_irq_enable(void)
asm volatile("sti": : :"memory");
}
+extern void safe_halt_addr(void);
+
static inline void native_safe_halt(void)
{
- asm volatile("sti; hlt": : :"memory");
+ asm volatile("sti \n\t"
+/*
+ * If NMI hits us here, it negates the interrupt shadow
+ * induced by STI. So the NMI handler checks for
+ * safe_halt_addr and skips the hlt if it loses the
+ * interrupt shadow.
+ *
+ * If native_safe_halt() is ever instantiated more
+ * than once, this will fail to build, and we'll need
+ * a list of addresses in a special section.
+ */
+".globl safe_halt_addr \n\t"
+"safe_halt_addr: \n\t"
+"hlt"
+: : :"memory");
}
static inline void native_halt(void)
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 60788de..f67da48 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -438,6 +438,20 @@ do_nmi(struct pt_regs *regs, long error_code)
inc_irq_stat(__nmi_count);
+ /*
+* We hit in the middle of an sti; hlt instruction. When we return,
+* the interrupt shadow cast by sti will no longer be in effect; then,
+* if an interrupt causes a wakeup, we won't notice it since the hlt
+* will take effect and block the cpu.
+*
+* If we detect this situation, fix it by advancing the instruction
+* pointer past the hlt instruction; if the interrupt doesn't happen,
+* we'll spend a few cycles falling asleep again, but that's better
+* than a missed wakeup.
+*/
+ if (regs->cs == __KERNEL_CS && regs->ip == (ulong)safe_halt_addr)
+ ++regs->ip;
+
if (!ignore_nmis)
default_do_nmi(regs);
--
1.7.2.3
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 12:04 -0400, Kevin O'Connor wrote:
> On Sun, Sep 19, 2010 at 05:53:47PM +0200, Gleb Natapov wrote:
> > On Sun, Sep 19, 2010 at 05:44:00PM +0200, Conrad Wood wrote:
> > >
> > > > > However after step 7 the guest can turn the cpu online again by
> > > > > issuing
> > > > > echo 1 >/sys/devices/.../cpuX/online
> > > > There will be no /sys/devices/.../cpuX/online in guest after step 7.
> > >
> > > Well then at least in my version there's a bug, because it still is
> > > there and never goes away. (kvm 0.12.5)
> > > And changing it to online works as well.
> > Can you provide exact steps you are using? After doing cpu_set x offline
> > cpu x should disappear from a guest. Otherwise cpu eject didn't work.
>
> I'm confused. The "cpu_set x offline" doesn't appear to do anything
> for me. It does not disable the cpu, nor does it inform the guest to
> disable the cpu.
I second that - same here.
a script with my exact steps is below. Result is reproducible.
When run on my system it will output:
"Known CPUs" are defined as cpus with an entry
in /sys/devices/system/cpu on the guest
"Inserted CPUs" are defined as cpus listed by info cpus in qemu monitor
"online CPUs" are defined as cpus listed in /proc/cpuinfo on the guest
Inserted CPUs: 4, CPUs known to the guest: 4, ONLINE in guest: 4
Inserting CPU #4
Inserted CPUs: 5, CPUs known to the guest: 5, ONLINE in guest: 4
Ejecting CPU #4
Inserted CPUs: 5, CPUs known to the guest: 5, ONLINE in guest: 4
Setting all available cpus to online...
bash: line 0: echo: write error: Invalid argument
bash: line 0: echo: write error: Invalid argument
bash: line 0: echo: write error: Invalid argument
Inserted CPUs: 5, CPUs known to the guest: 5, ONLINE in guest: 5
^
after cpu_set 4 offline this should not come online again!!
---
#!/bin/bash
VM=192.168.2.35
MONITORHOST=localhost
MONITORPORT=9401
# count how many cpus are in /proc/cpuinfo on the guest
countOnline() {
ONLINECPUS=`ssh -lroot ${VM} "grep processor /proc/cpuinfo | wc -l"`
}
# count how many cpus are in /sys/devices/...
countCPUsKnown() {
KNOWNCPUS=`ssh -lroot ${VM} "find /sys/devices/system/cpu/ -name
'cpu?'| wc -l"`
}
# count how many cpus does kvm list
countInserted() {
INSERTEDCPUS=`echo "info cpus" | nc ${MONITORHOST} ${MONITORPORT} |
grep -av '(qemu)'|grep -va 'for more inform'|wc -l`
}
printInfo() {
countOnline
countCPUsKnown
countInserted
echo "Inserted CPUs: ${INSERTEDCPUS}, CPUs known to the guest:
${KNOWNCPUS}, ONLINE in guest: ${ONLINECPUS}"
}
echo "\"Known CPUs\" are defined as cpus with an entry
in /sys/devices/system/cpu on the guest"
echo "\"Inserted CPUs\" are defined as cpus listed by info cpus in qemu
monitor"
echo "\"online CPUs\" are defined as cpus listed in /proc/cpuinfo on the
guest"
echo
printInfo
echo "Inserting CPU #4"
echo "cpu_set 4 online" | nc ${MONITORHOST} ${MONITORPORT} >/dev/null
printInfo
echo "Ejecting CPU #4"
echo "cpu_set 4 offline" | nc ${MONITORHOST} ${MONITORPORT} >/dev/null
printInfo
echo "Setting all available cpus to online..."
ssh -lroot ${VM} "find /sys/devices/system/cpu/ -type f -path
'*/cpu?/online' -exec bash -c \"echo 1 >{}\" \;"
printInfo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 12:04:51PM -0400, Kevin O'Connor wrote: > On Sun, Sep 19, 2010 at 05:53:47PM +0200, Gleb Natapov wrote: > > On Sun, Sep 19, 2010 at 05:44:00PM +0200, Conrad Wood wrote: > > > > > > > > However after step 7 the guest can turn the cpu online again by > > > > > issuing > > > > > echo 1 >/sys/devices/.../cpuX/online > > > > There will be no /sys/devices/.../cpuX/online in guest after step 7. > > > > > > Well then at least in my version there's a bug, because it still is > > > there and never goes away. (kvm 0.12.5) > > > And changing it to online works as well. > > Can you provide exact steps you are using? After doing cpu_set x offline > > cpu x should disappear from a guest. Otherwise cpu eject didn't work. > > I'm confused. The "cpu_set x offline" doesn't appear to do anything > for me. It does not disable the cpu, nor does it inform the guest to > disable the cpu. > > The only way I'm able to remove a cpu is to run in the guest: > > echo 1 > /sys/devices/LNXSYSTM:00/LNXSYBUS:00/LNXCPU:xx/eject > > which (as far as I know) is purely a voluntary guest disable of the > cpu. It doesn't (as far as I know) even tell kvm that it is no longer > using the cpu. (The acpi cpu eject method in seabios just issues a > sleep(200) call.) > Then cpu eject method in seabios has a bug. When "cpu_set x offline" is called qemu sets status bit in gpe and injects ACPI interrupt. ACPI should do Notify() on cpu object when it happens. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 05:53:47PM +0200, Gleb Natapov wrote: > On Sun, Sep 19, 2010 at 05:44:00PM +0200, Conrad Wood wrote: > > > > > > However after step 7 the guest can turn the cpu online again by issuing > > > > echo 1 >/sys/devices/.../cpuX/online > > > There will be no /sys/devices/.../cpuX/online in guest after step 7. > > > > Well then at least in my version there's a bug, because it still is > > there and never goes away. (kvm 0.12.5) > > And changing it to online works as well. > Can you provide exact steps you are using? After doing cpu_set x offline > cpu x should disappear from a guest. Otherwise cpu eject didn't work. I'm confused. The "cpu_set x offline" doesn't appear to do anything for me. It does not disable the cpu, nor does it inform the guest to disable the cpu. The only way I'm able to remove a cpu is to run in the guest: echo 1 > /sys/devices/LNXSYSTM:00/LNXSYBUS:00/LNXCPU:xx/eject which (as far as I know) is purely a voluntary guest disable of the cpu. It doesn't (as far as I know) even tell kvm that it is no longer using the cpu. (The acpi cpu eject method in seabios just issues a sleep(200) call.) -Kevin -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 05:44:00PM +0200, Conrad Wood wrote: > > > > However after step 7 the guest can turn the cpu online again by issuing > > > echo 1 >/sys/devices/.../cpuX/online > > There will be no /sys/devices/.../cpuX/online in guest after step 7. > > Well then at least in my version there's a bug, because it still is > there and never goes away. (kvm 0.12.5) > And changing it to online works as well. Can you provide exact steps you are using? After doing cpu_set x offline cpu x should disappear from a guest. Otherwise cpu eject didn't work. > Which version are you using? > Nothing :) I am saying how it should work. Last time I played with cpu hot-plug was with qemu-kvm 0.11.x. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
> > However after step 7 the guest can turn the cpu online again by issuing > > echo 1 >/sys/devices/.../cpuX/online > There will be no /sys/devices/.../cpuX/online in guest after step 7. Well then at least in my version there's a bug, because it still is there and never goes away. (kvm 0.12.5) And changing it to online works as well. Which version are you using? Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 05:37:18PM +0200, Conrad Wood wrote: > > > > I need to know if a cpu is currently inserted or ejected. > > > > > The only way to do that currently is to track it in your management app. > > The Management App does know which one should be inserted or ejected. > I see now way of enforcing it though. > > This is what I had in mind: (simplified) > > 1. Management App increase cpus > 2. kvm (as instructed by app): insert cpu x > 3. guest os: online cpu x > 4. do some stuff in the guest... > 5. guest os: offline cpu x > 6. Management App decrease cpus > 7. kvm (as instructed by app): eject cpu x > > However after step 7 the guest can turn the cpu online again by issuing > echo 1 >/sys/devices/.../cpuX/online There will be no /sys/devices/.../cpuX/online in guest after step 7. > If the guest does so, the Management App has no way of knowing and thus > cannot provide accurate logs - that is why it is so important. > > Conrad > > -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
> > I need to know if a cpu is currently inserted or ejected. > > > The only way to do that currently is to track it in your management app. The Management App does know which one should be inserted or ejected. I see now way of enforcing it though. This is what I had in mind: (simplified) 1. Management App increase cpus 2. kvm (as instructed by app): insert cpu x 3. guest os: online cpu x 4. do some stuff in the guest... 5. guest os: offline cpu x 6. Management App decrease cpus 7. kvm (as instructed by app): eject cpu x However after step 7 the guest can turn the cpu online again by issuing echo 1 >/sys/devices/.../cpuX/online If the guest does so, the Management App has no way of knowing and thus cannot provide accurate logs - that is why it is so important. Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] Real mode interrupt injection emulation
On 09/19/2010 02:34 PM, Avi Kivity wrote: Our current real mode interrupt injection injects external interrupts as software interrupts, which is somewhat hacky. This is problematic in big real mode (can't use vmx there) and on via processors (a cpu bug prevents this from working correctly). Replace the current mechanism with emulation; we now inject the interrupt by looking up the vector in the IVT, updating the stack, etc. using the emulator. This is somewhat slower but works in all cases. This patchset survived an autotest run. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 05:20:38PM +0200, Avi Kivity wrote: > On 09/19/2010 01:59 PM, Gleb Natapov wrote: > >> > >I fixed it several times in the past[0][1][3], but it keeps re-appearing :) I > >hope current qemu-kvm head is OK. > > > >[0] 865aa3fec2b3d83c9b6edd452050a30ef6eaf3b8 > >[1] 8c7d4cb7163d1c97df4439e2e0edb5918abf106b > >[2] d6d233543d543fcf34df6190bbce5daf38d870f5 > > > > kvm-autotest and you won't have to fix it again... > Heh. Can't argue with this. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On 09/19/2010 01:59 PM, Gleb Natapov wrote: > I fixed it several times in the past[0][1][3], but it keeps re-appearing :) I hope current qemu-kvm head is OK. [0] 865aa3fec2b3d83c9b6edd452050a30ef6eaf3b8 [1] 8c7d4cb7163d1c97df4439e2e0edb5918abf106b [2] d6d233543d543fcf34df6190bbce5daf38d870f5 kvm-autotest and you won't have to fix it again... -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 05:16:24PM +0200, Conrad Wood wrote: > [..] > > > Lets establish some terminology to understand each other better: > > Lets use CPU inserted/ejected to refer to cpu being available/not > > available to a guest. > > Lest use CPU online/offline to refer to cpu been used by a guest OS. > > > > CPU hot-plug works like this: > > cpu is inserted (plugged into cpu socket on MB) > > guest OS is notifies through ACPI > > guest OS starts to use newly added CPU (makes it online) > > > > CPU hot-unplug works like this: > > guest OS is notifies through ACPI that cpu is going to be removed > > guest OS stops to use cpu (makes it offline) > > cpu is ejected (removed from a socket on MB) (kvm never does that part) > > > > So which part do you want to know about from a host? > > > > You can't really know from a host if cpu is used by guest OS (if it is > > online/offline in s guest in our terminology) > > > > inserted/ejected is management action (it can't be initiated by a > > guest), so you can keep track of it by yourself. KVM doesn't do it > > for you right now. More precisely it does, but doesn't have a way to > > quire it. > > I need to know if a cpu is currently inserted or ejected. > The only way to do that currently is to track it in your management app. > I gather a cpu which is currently "inserted", may potentially be used by > a guest os. (Wether he actually uses a cpu, puts it online or offline, > isn't my concern). > > The amount of time and number of cpus available to a guest is what I > like to keep track of. > > Conrad > -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
[..] > Lets establish some terminology to understand each other better: > Lets use CPU inserted/ejected to refer to cpu being available/not > available to a guest. > Lest use CPU online/offline to refer to cpu been used by a guest OS. > > CPU hot-plug works like this: > cpu is inserted (plugged into cpu socket on MB) > guest OS is notifies through ACPI > guest OS starts to use newly added CPU (makes it online) > > CPU hot-unplug works like this: > guest OS is notifies through ACPI that cpu is going to be removed > guest OS stops to use cpu (makes it offline) > cpu is ejected (removed from a socket on MB) (kvm never does that part) > > So which part do you want to know about from a host? > > You can't really know from a host if cpu is used by guest OS (if it is > online/offline in s guest in our terminology) > > inserted/ejected is management action (it can't be initiated by a > guest), so you can keep track of it by yourself. KVM doesn't do it > for you right now. More precisely it does, but doesn't have a way to > quire it. I need to know if a cpu is currently inserted or ejected. I gather a cpu which is currently "inserted", may potentially be used by a guest os. (Wether he actually uses a cpu, puts it online or offline, isn't my concern). The amount of time and number of cpus available to a guest is what I like to keep track of. Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] svm: Add VMLOAD/VMSAVE latency test
Just figured out that it makes a lot of sens to measure the latency of
stgi and clgi too in this test. Here is an updated one.
>From 350d33d155db8928f81cc801587787c84d6bc8c5 Mon Sep 17 00:00:00 2001
From: Joerg Roedel
Date: Sun, 19 Sep 2010 15:34:37 +0200
Subject: [PATCH] svm: Add VMLOAD/VMSAVE and STGI/CLGI latency test
This patch adds a test to measure the latency of the VMLOAD
and VMSAVE instructions as well as for STGI and CLGI.
Signed-off-by: Joerg Roedel
---
x86/svm.c | 79 +
1 files changed, 79 insertions(+), 0 deletions(-)
diff --git a/x86/svm.c b/x86/svm.c
index babd77d..ed784c4 100644
--- a/x86/svm.c
+++ b/x86/svm.c
@@ -19,10 +19,20 @@ u64 tsc_start;
u64 tsc_end;
u64 vmrun_sum, vmexit_sum;
+u64 vmsave_sum, vmload_sum;
+u64 stgi_sum, clgi_sum;
u64 latvmrun_max;
u64 latvmrun_min;
u64 latvmexit_max;
u64 latvmexit_min;
+u64 latvmload_max;
+u64 latvmload_min;
+u64 latvmsave_max;
+u64 latvmsave_min;
+u64 latstgi_max;
+u64 latstgi_min;
+u64 latclgi_max;
+u64 latclgi_min;
u64 runs;
static bool npt_supported(void)
@@ -661,6 +671,73 @@ static bool latency_check(struct test *test)
return true;
}
+static void lat_svm_insn_prepare(struct test *test)
+{
+default_prepare(test);
+runs = LATENCY_RUNS;
+latvmload_min = latvmsave_min = latstgi_min = latclgi_min = -1ULL;
+latvmload_max = latvmsave_max = latstgi_max = latclgi_max = 0;
+vmload_sum = vmsave_sum = stgi_sum = clgi_sum;
+}
+
+static bool lat_svm_insn_finished(struct test *test)
+{
+u64 vmcb_phys = virt_to_phys(test->vmcb);
+u64 cycles;
+
+for ( ; runs != 0; runs--) {
+tsc_start = rdtsc();
+asm volatile("vmload\n\t" : : "a"(vmcb_phys) : "memory");
+cycles = rdtsc() - tsc_start;
+if (cycles > latvmload_max)
+latvmload_max = cycles;
+if (cycles < latvmload_min)
+latvmload_min = cycles;
+vmload_sum += cycles;
+
+tsc_start = rdtsc();
+asm volatile("vmsave\n\t" : : "a"(vmcb_phys) : "memory");
+cycles = rdtsc() - tsc_start;
+if (cycles > latvmsave_max)
+latvmsave_max = cycles;
+if (cycles < latvmsave_min)
+latvmsave_min = cycles;
+vmsave_sum += cycles;
+
+tsc_start = rdtsc();
+asm volatile("stgi\n\t");
+cycles = rdtsc() - tsc_start;
+if (cycles > latstgi_max)
+latstgi_max = cycles;
+if (cycles < latstgi_min)
+latstgi_min = cycles;
+stgi_sum += cycles;
+
+tsc_start = rdtsc();
+asm volatile("clgi\n\t");
+cycles = rdtsc() - tsc_start;
+if (cycles > latclgi_max)
+latclgi_max = cycles;
+if (cycles < latclgi_min)
+latclgi_min = cycles;
+clgi_sum += cycles;
+}
+
+return true;
+}
+
+static bool lat_svm_insn_check(struct test *test)
+{
+printf("Latency VMLOAD: max: %d min: %d avg: %d\n", latvmload_max,
+latvmload_min, vmload_sum / LATENCY_RUNS);
+printf("Latency VMSAVE: max: %d min: %d avg: %d\n", latvmsave_max,
+latvmsave_min, vmsave_sum / LATENCY_RUNS);
+printf("Latency STGI: max: %d min: %d avg: %d\n", latstgi_max,
+latstgi_min, stgi_sum / LATENCY_RUNS);
+printf("Latency CLGI: max: %d min: %d avg: %d\n", latclgi_max,
+latclgi_min, clgi_sum / LATENCY_RUNS);
+return true;
+}
static struct test tests[] = {
{ "null", default_supported, default_prepare, null_test,
default_finished, null_check },
@@ -695,6 +772,8 @@ static struct test tests[] = {
default_finished, npt_pfwalk_check },
{ "latency_run_exit", default_supported, latency_prepare, latency_test,
latency_finished, latency_check },
+{ "latency_svm_insn", default_supported, lat_svm_insn_prepare, null_test,
+ lat_svm_insn_finished, lat_svm_insn_check },
};
int main(int ac, char **av)
--
1.7.0.4
--
AMD Operating System Research Center
Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 04:07:50PM +0200, Conrad Wood wrote: > On Sun, 2010-09-19 at 15:50 +0200, Gleb Natapov wrote: > > On Sun, Sep 19, 2010 at 09:46:21AM -0400, Kevin O'Connor wrote: > > > On Sun, Sep 19, 2010 at 03:40:45PM +0200, Gleb Natapov wrote: > > > > halted state is not the way to check for whether cpu is online or > > > > offline. cpu may be online but executing hlt instruction so > > > > its state will be halted, but cpu itself is online. Actually with kvm > > > > today you are not able to check that cpu is offline if it was ever > > > > online. Query a guest for cpu status. > > > > > > In that regard, "info cpus" does seem to work - as it only shows me > > > the cpus that have ever been online. Ejecting a cpu in the guest does > > > not remove it from "info cpus", but then, I don't think kvm does > > > remove it from being available after an eject. > > > > > Exactly. That what I meant by "Actually with kvm today you are not able > > to check that cpu is offline if it was ever online". So if cpu was never > > online it is easy to check that it is offline (it will not be reported > > by "info cpus"). But if it was inserted and then ejected it will still > > be present in "info cpus", but not used by a guest OS. > > > > -- > > Gleb. > > > > 1) Thanks for clarifying "online" vs "halted" - that makes sense and is > probably part of what confused me. I need to get the "online/offline" > status of cpus, not if they are halted or not. I understand this is > currently not possible with kvm. As I have no direct control over the > guest OS, I do however need a solution on the host. Any clues how this > could be implemented would be much appreciated. > This is important to me. Lets establish some terminology to understand each other better: Lets use CPU inserted/ejected to refer to cpu being available/not available to a guest. Lest use CPU online/offline to refer to cpu been used by a guest OS. CPU hot-plug works like this: cpu is inserted (plugged into cpu socket on MB) guest OS is notifies through ACPI guest OS starts to use newly added CPU (makes it online) CPU hot-unplug works like this: guest OS is notifies through ACPI that cpu is going to be removed guest OS stops to use cpu (makes it offline) cpu is ejected (removed from a socket on MB) (kvm never does that part) So which part do you want to know about from a host? You can't really know from a host if cpu is used by guest OS (if it is online/offline in s guest in our terminology) inserted/ejected is management action (it can't be initiated by a guest), so you can keep track of it by yourself. KVM doesn't do it for you right now. More precisely it does, but doesn't have a way to quire it. > > 2) In my case, the cpus are ALWAYS reported as halted, regardless of the > current state in the guest. This is cosmetic but. It does not affect any operation. > After the guest booted, The "cpu infos" displays precisely the amount of > cpus as initially set by the -smp parameter of kvm. (as halted). > any additional cpus added later will be displayed, also as halted - > again regardless whether or not the guest switches or had switched them > to online. It is reported as "halted" even whilst the guest is running a > cpu-hogger which would stop the cpu from halting. > I don't mind so much about this cosmetic display problem ;) > Rightly so since even when it works information it displays is already outdated at the moment you see it anyway. It is useful for debugging, thus I keep fixing it when it breaks. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH trace-cmd] plugin_kvm: disassemble instructions for kvm_emulate_insn
Override kvm_emulate_insn formatting to use a disassembler to format
the emulated instruction. If a disassembler (udis86) is not available,
fall back to showing the instruction bytes in hex.
Signed-off-by: Avi Kivity
---
Note 1: on top of 'master' with 'trace-cmd-kvm' cherry-picked on top.
Note 2: I get output of the form
... kvm_emulate_insn: 0:f800010527b5: mov $0x0, 0xfffe00b0CAN'T FIND
FIELD "guest_rip"
which leads me to believe there is a bug in trace_seq_printf when the input
to %s is "".
Makefile | 11 +-
plugin_kvm.c | 111 ++
2 files changed, 121 insertions(+), 1 deletions(-)
diff --git a/Makefile b/Makefile
index 5282f94..fe34d1c 100644
--- a/Makefile
+++ b/Makefile
@@ -74,6 +74,14 @@ ifeq ($(shell sh -c "python-config --includes > /dev/null
2>&1 && echo y"), y)
PYTHON_PY_INSTALL := event-viewer.install tracecmd.install
tracecmdgui.install
endif
+# $(call test-build, snippet, ret) -> ret if snippet compiles
+# -> empty otherwise
+test-build = $(if $(shell $(CC) -o /dev/null -c -x c - > /dev/null 2>&1 \
+ <<<'$1' && echo y), $2)
+
+# have udis86 disassembler library?
+udis86-flags := $(call test-build,\#include ,-DHAVE_UDIS86 -ludis86)
+
ifeq ("$(origin O)", "command line")
BUILD_OUTPUT := $(O)
endif
@@ -188,6 +196,7 @@ CFLAGS ?= -g -Wall
# Append required CFLAGS
override CFLAGS += $(CONFIG_FLAGS) $(INCLUDES) $(PLUGIN_DIR_SQ)
+override CFLAGS += $(udis86-flags)
ifeq ($(VERBOSE),1)
Q =
@@ -228,7 +237,7 @@ do_compile_plugin_obj = \
do_plugin_build = \
($(print_plugin_build) \
- $(CC) -shared -nostartfiles -o $@ $<)
+ $(CC) $(CFLAGS) -shared -nostartfiles -o $@ $<)
do_build_static_lib = \
($(print_static_lib_build) \
diff --git a/plugin_kvm.c b/plugin_kvm.c
index 7217e85..00cac5a 100644
--- a/plugin_kvm.c
+++ b/plugin_kvm.c
@@ -21,9 +21,68 @@
#include
#include
#include
+#include
#include "parse-events.h"
+#ifdef HAVE_UDIS86
+
+#include
+
+static ud_t ud;
+
+static void init_disassembler(void)
+{
+ ud_init(&ud);
+ ud_set_syntax(&ud, UD_SYN_ATT);
+}
+
+static const char *disassemble(unsigned char *insn, int len, uint64_t rip,
+ int cr0_pe, int eflags_vm,
+ int cs_d, int cs_l)
+{
+ int mode;
+
+ if (!cr0_pe)
+ mode = 16;
+ else if (eflags_vm)
+ mode = 16;
+ else if (cs_l)
+ mode = 64;
+ else if (cs_d)
+ mode = 32;
+ else
+ mode = 16;
+
+ ud_set_pc(&ud, rip);
+ ud_set_mode(&ud, mode);
+ ud_set_input_buffer(&ud, insn, len);
+ ud_disassemble(&ud);
+ return ud_insn_asm(&ud);
+}
+
+#else
+
+static void init_disassembler(void)
+{
+}
+
+static const char *disassemble(unsigned char *insn, int len, uint64_t rip,
+ int cr0_pe, int eflags_vm,
+ int cs_d, int cs_l)
+{
+ static char out[15*3+1];
+ int i;
+
+ for (i = 0; i < len; ++i)
+ sprintf(out + i * 3, "%02x ", insn[i]);
+ out[len*3-1] = '\0';
+ return out;
+}
+
+#endif
+
+
#define VMX_EXIT_REASONS \
_ER(EXCEPTION_NMI, 0) \
_ER(EXTERNAL_INTERRUPT, 1) \
@@ -99,6 +158,53 @@ static int kvm_exit_handler(struct trace_seq *s, struct
record *record,
return 0;
}
+#define KVM_EMUL_INSN_F_CR0_PE (1 << 0)
+#define KVM_EMUL_INSN_F_EFL_VM (1 << 1)
+#define KVM_EMUL_INSN_F_CS_D (1 << 2)
+#define KVM_EMUL_INSN_F_CS_L (1 << 3)
+
+static int kvm_emulate_insn_handler(struct trace_seq *s, struct record *record,
+ struct event_format *event, void *context)
+{
+ unsigned long long rip, csbase, len, flags, failed;
+ int llen;
+ uint8_t *insn;
+ const char *disasm;
+
+ if (pevent_get_field_val(s, event, "rip", record, &rip, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "csbase", record, &csbase, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "len", record, &len, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "flags", record, &flags, 1) < 0)
+ return -1;
+
+ if (pevent_get_field_val(s, event, "failed", record, &failed, 1) < 0)
+ return -1;
+
+ insn = pevent_get_field_raw(s, event, "insn", record, &llen, 1);
+ if (!insn)
+ return -1;
+
+ disasm = disassemble(insn, len, rip,
+flags & KVM_EMUL_INSN_F_CR0_PE,
+flags & KVM_EMUL_INSN_F_EFL_VM,
+flags & KVM_EMUL_INSN_F_CS_D,
Re: cpu hotplug
On Sun, Sep 19, 2010 at 04:07:50PM +0200, Conrad Wood wrote: > 1) Thanks for clarifying "online" vs "halted" - that makes sense and is > probably part of what confused me. I need to get the "online/offline" > status of cpus, not if they are halted or not. I understand this is > currently not possible with kvm. As I have no direct control over the > guest OS, I do however need a solution on the host. Any clues how this > could be implemented would be much appreciated. > This is important to me. I thought I read somewhere that kvm does not currently support taking a cpu offline - the guest can only voluntarily stop using a cpu. (Gleb would know better, but I think he was trying to say the same thing.) > 2) In my case, the cpus are ALWAYS reported as halted, regardless of the > current state in the guest. > After the guest booted, The "cpu infos" displays precisely the amount of > cpus as initially set by the -smp parameter of kvm. (as halted). > any additional cpus added later will be displayed, also as halted - > again regardless whether or not the guest switches or had switched them > to online. It is reported as "halted" even whilst the guest is running a > cpu-hogger which would stop the cpu from halting. > I don't mind so much about this cosmetic display problem ;) I suppose one could check the host cpu usage of the thread reported in "info cpus". -Kevin -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 15:50 +0200, Gleb Natapov wrote: > On Sun, Sep 19, 2010 at 09:46:21AM -0400, Kevin O'Connor wrote: > > On Sun, Sep 19, 2010 at 03:40:45PM +0200, Gleb Natapov wrote: > > > halted state is not the way to check for whether cpu is online or > > > offline. cpu may be online but executing hlt instruction so > > > its state will be halted, but cpu itself is online. Actually with kvm > > > today you are not able to check that cpu is offline if it was ever > > > online. Query a guest for cpu status. > > > > In that regard, "info cpus" does seem to work - as it only shows me > > the cpus that have ever been online. Ejecting a cpu in the guest does > > not remove it from "info cpus", but then, I don't think kvm does > > remove it from being available after an eject. > > > Exactly. That what I meant by "Actually with kvm today you are not able > to check that cpu is offline if it was ever online". So if cpu was never > online it is easy to check that it is offline (it will not be reported > by "info cpus"). But if it was inserted and then ejected it will still > be present in "info cpus", but not used by a guest OS. > > -- > Gleb. > 1) Thanks for clarifying "online" vs "halted" - that makes sense and is probably part of what confused me. I need to get the "online/offline" status of cpus, not if they are halted or not. I understand this is currently not possible with kvm. As I have no direct control over the guest OS, I do however need a solution on the host. Any clues how this could be implemented would be much appreciated. This is important to me. 2) In my case, the cpus are ALWAYS reported as halted, regardless of the current state in the guest. After the guest booted, The "cpu infos" displays precisely the amount of cpus as initially set by the -smp parameter of kvm. (as halted). any additional cpus added later will be displayed, also as halted - again regardless whether or not the guest switches or had switched them to online. It is reported as "halted" even whilst the guest is running a cpu-hogger which would stop the cpu from halting. I don't mind so much about this cosmetic display problem ;) Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] svm: Add VMLOAD/VMSAVE latency test
This patch adds a test to measure the latency of the VMLOAD
and VMSAVE instructions.
Signed-off-by: Joerg Roedel
---
x86/svm.c | 52
1 files changed, 52 insertions(+), 0 deletions(-)
diff --git a/x86/svm.c b/x86/svm.c
index babd77d..26bf566 100644
--- a/x86/svm.c
+++ b/x86/svm.c
@@ -19,10 +19,15 @@ u64 tsc_start;
u64 tsc_end;
u64 vmrun_sum, vmexit_sum;
+u64 vmsave_sum, vmload_sum;
u64 latvmrun_max;
u64 latvmrun_min;
u64 latvmexit_max;
u64 latvmexit_min;
+u64 latvmload_max;
+u64 latvmload_min;
+u64 latvmsave_max;
+u64 latvmsave_min;
u64 runs;
static bool npt_supported(void)
@@ -661,6 +666,51 @@ static bool latency_check(struct test *test)
return true;
}
+static void load_save_prepare(struct test *test)
+{
+default_prepare(test);
+runs = LATENCY_RUNS;
+latvmload_min = latvmsave_min = -1ULL;
+latvmload_max = latvmsave_max = 0;
+vmload_sum = vmsave_sum = 0;
+}
+
+static bool load_save_finished(struct test *test)
+{
+u64 vmcb_phys = virt_to_phys(test->vmcb);
+u64 cycles;
+
+for ( ; runs != 0; runs--) {
+tsc_start = rdtsc();
+asm volatile("vmload\n\t" : : "a"(vmcb_phys) : "memory");
+cycles = rdtsc() - tsc_start;
+if (cycles > latvmload_max)
+latvmload_max = cycles;
+if (cycles < latvmload_min)
+latvmload_min = cycles;
+vmload_sum += cycles;
+
+tsc_start = rdtsc();
+asm volatile("vmsave\n\t" : : "a"(vmcb_phys) : "memory");
+cycles = rdtsc() - tsc_start;
+if (cycles > latvmsave_max)
+latvmsave_max = cycles;
+if (cycles < latvmsave_min)
+latvmsave_min = cycles;
+vmsave_sum += cycles;
+}
+
+return true;
+}
+
+static bool load_save_check(struct test *test)
+{
+printf("Latency VMLOAD: max: %d min: %d avg: %d\n", latvmload_max,
+latvmload_min, vmload_sum / LATENCY_RUNS);
+printf("Latency VMSAVE: max: %d min: %d avg: %d\n", latvmsave_max,
+latvmsave_min, vmsave_sum / LATENCY_RUNS);
+return true;
+}
static struct test tests[] = {
{ "null", default_supported, default_prepare, null_test,
default_finished, null_check },
@@ -695,6 +745,8 @@ static struct test tests[] = {
default_finished, npt_pfwalk_check },
{ "latency_run_exit", default_supported, latency_prepare, latency_test,
latency_finished, latency_check },
+{ "latency_load_save", default_supported, load_save_prepare, null_test,
+ load_save_finished, load_save_check },
};
int main(int ac, char **av)
--
1.7.0.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] svm: Add VMRUN/VMEXIT latency test
This patch adds a test to measure the latency of VMRUN and
VMEXIT.
Signed-off-by: Joerg Roedel
---
x86/svm.c | 81 +
1 files changed, 81 insertions(+), 0 deletions(-)
diff --git a/x86/svm.c b/x86/svm.c
index dc3098f..babd77d 100644
--- a/x86/svm.c
+++ b/x86/svm.c
@@ -13,6 +13,18 @@ u64 *pde[4];
u64 *pte[2048];
u64 *scratch_page;
+#define LATENCY_RUNS 100
+
+u64 tsc_start;
+u64 tsc_end;
+
+u64 vmrun_sum, vmexit_sum;
+u64 latvmrun_max;
+u64 latvmrun_min;
+u64 latvmexit_max;
+u64 latvmexit_min;
+u64 runs;
+
static bool npt_supported(void)
{
return cpuid(0x800A).d & 1;
@@ -162,6 +174,7 @@ static bool test_run(struct test *test, struct vmcb *vmcb)
vmcb->save.rip = (ulong)test_thunk;
vmcb->save.rsp = (ulong)(guest_stack + ARRAY_SIZE(guest_stack));
do {
+tsc_start = rdtsc();
asm volatile (
"clgi \n\t"
"vmload \n\t"
@@ -176,9 +189,11 @@ static bool test_run(struct test *test, struct vmcb *vmcb)
: "rbx", "rcx", "rdx", "rsi",
"r8", "r9", "r10", "r11" , "r12", "r13", "r14", "r15",
"memory");
+ tsc_end = rdtsc();
++test->exits;
} while (!test->finished(test));
+
success = test->succeeded(test);
printf("%s: %s\n", test->name, success ? "PASS" : "FAIL");
@@ -582,6 +597,70 @@ static bool npt_pfwalk_check(struct test *test)
&& (test->vmcb->control.exit_info_2 == read_cr3());
}
+static void latency_prepare(struct test *test)
+{
+default_prepare(test);
+runs = LATENCY_RUNS;
+latvmrun_min = latvmexit_min = -1ULL;
+latvmrun_max = latvmexit_max = 0;
+vmrun_sum = vmexit_sum = 0;
+}
+
+static void latency_test(struct test *test)
+{
+u64 cycles;
+
+start:
+tsc_end = rdtsc();
+
+cycles = tsc_end - tsc_start;
+
+if (cycles > latvmrun_max)
+latvmrun_max = cycles;
+
+if (cycles < latvmrun_min)
+latvmrun_min = cycles;
+
+vmrun_sum += cycles;
+
+tsc_start = rdtsc();
+
+asm volatile ("vmmcall" : : : "memory");
+goto start;
+}
+
+static bool latency_finished(struct test *test)
+{
+u64 cycles;
+
+tsc_end = rdtsc();
+
+cycles = tsc_end - tsc_start;
+
+if (cycles > latvmexit_max)
+latvmexit_max = cycles;
+
+if (cycles < latvmexit_min)
+latvmexit_min = cycles;
+
+vmexit_sum += cycles;
+
+test->vmcb->save.rip += 3;
+
+runs -= 1;
+
+return runs == 0;
+}
+
+static bool latency_check(struct test *test)
+{
+printf("Latency VMRUN : max: %d min: %d avg: %d\n", latvmrun_max,
+latvmrun_min, vmrun_sum / LATENCY_RUNS);
+printf("Latency VMEXIT: max: %d min: %d avg: %d\n", latvmexit_max,
+latvmexit_min, vmexit_sum / LATENCY_RUNS);
+return true;
+}
+
static struct test tests[] = {
{ "null", default_supported, default_prepare, null_test,
default_finished, null_check },
@@ -614,6 +693,8 @@ static struct test tests[] = {
default_finished, npt_rw_check },
{ "npt_pfwalk", npt_supported, npt_pfwalk_prepare, null_test,
default_finished, npt_pfwalk_check },
+{ "latency_run_exit", default_supported, latency_prepare, latency_test,
+ latency_finished, latency_check },
};
int main(int ac, char **av)
--
1.7.0.4
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/2] unit-tests: Add tests to measure svm instruction latencys
Hi, here are two patches for the svm unit-test framework that measure the latency of the emulated vmrun, vmload, and vmsave instructions as well as the latency of emulated vmexits. On my Phenom II X6 1090T running in P0 it produces output like this: Latency VMRUN : max: 140956 min: 6770 avg: 6819 Latency VMEXIT: max: 141042 min: 7475 avg: 7575 latency_run_exit: PASS Latency VMLOAD: max: 23248 min: 1915 avg: 1925 Latency VMSAVE: max: 130888 min: 1917 avg: 1955 latency_load_save: PASS These two tests report the tsc-cycles between events. Each latency is measured one million times (which explains the large -max values). Joerg -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 09:46:21AM -0400, Kevin O'Connor wrote: > On Sun, Sep 19, 2010 at 03:40:45PM +0200, Gleb Natapov wrote: > > halted state is not the way to check for whether cpu is online or > > offline. cpu may be online but executing hlt instruction so > > its state will be halted, but cpu itself is online. Actually with kvm > > today you are not able to check that cpu is offline if it was ever > > online. Query a guest for cpu status. > > In that regard, "info cpus" does seem to work - as it only shows me > the cpus that have ever been online. Ejecting a cpu in the guest does > not remove it from "info cpus", but then, I don't think kvm does > remove it from being available after an eject. > Exactly. That what I meant by "Actually with kvm today you are not able to check that cpu is offline if it was ever online". So if cpu was never online it is easy to check that it is offline (it will not be reported by "info cpus"). But if it was inserted and then ejected it will still be present in "info cpus", but not used by a guest OS. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 03:40:45PM +0200, Gleb Natapov wrote: > halted state is not the way to check for whether cpu is online or > offline. cpu may be online but executing hlt instruction so > its state will be halted, but cpu itself is online. Actually with kvm > today you are not able to check that cpu is offline if it was ever > online. Query a guest for cpu status. In that regard, "info cpus" does seem to work - as it only shows me the cpus that have ever been online. Ejecting a cpu in the guest does not remove it from "info cpus", but then, I don't think kvm does remove it from being available after an eject. -Kevin -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 03:35:47PM +0200, Conrad Wood wrote: > > > > hm... that's the version I am using und it does not work here ;( > > > the hotplug itself seems to work but they're reported as halted. > > I told you this is cosmetic. They are working. > > I agree, the hotplug itself seems to be working! So why are you saying "I am using und it does not work here" above? > > > > > > > Then it seems to be a local misconfiguration. > > and if so, I would like to know what I have misconfigured. ;-) > and what I can do to persuade this version to provide me with > information about which cpus are online and offline. > halted state is not the way to check for whether cpu is online or offline. cpu may be online but executing hlt instruction so its state will be halted, but cpu itself is online. Actually with kvm today you are not able to check that cpu is offline if it was ever online. Query a guest for cpu status. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 09:38 -0400, Kevin O'Connor wrote: > On Sun, Sep 19, 2010 at 03:29:35PM +0200, Conrad Wood wrote: > > On Sun, 2010-09-19 at 09:26 -0400, Kevin O'Connor wrote: > > > On Sun, Sep 19, 2010 at 08:38:12AM +0200, Gleb Natapov wrote: > > > > Known problem in qemu. There was a patch for this, but qemu maintainers > > > > think it is not good enough. > > > Old versions of kvm do work - for example, qemu-kvm-0.12.5 works for > > > me. > > hm... that's the version I am using und it does not work here ;( > > the hotplug itself seems to work but they're reported as halted. > > Could you confirm that the cpus in the monitor as reported by "info > > cpus" are not reported as "(halted)" ? > > I sent my email before I saw that you and Gleb had continued the > thread. > > It works in the sense that if I run two cpu hogs in a guest with one > cpu I use only one host cpu used. When I turn on an additional guest > cpu, I see two host cpus being used. > > But otherwise, "info cpus" does seem inaccurate. > > -Kevin > thanks for confirming that. got a little puzzled there. Cheers, Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 03:29:35PM +0200, Conrad Wood wrote: > On Sun, 2010-09-19 at 09:26 -0400, Kevin O'Connor wrote: > > On Sun, Sep 19, 2010 at 08:38:12AM +0200, Gleb Natapov wrote: > > > Known problem in qemu. There was a patch for this, but qemu maintainers > > > think it is not good enough. > > Old versions of kvm do work - for example, qemu-kvm-0.12.5 works for > > me. > hm... that's the version I am using und it does not work here ;( > the hotplug itself seems to work but they're reported as halted. > Could you confirm that the cpus in the monitor as reported by "info > cpus" are not reported as "(halted)" ? I sent my email before I saw that you and Gleb had continued the thread. It works in the sense that if I run two cpu hogs in a guest with one cpu I use only one host cpu used. When I turn on an additional guest cpu, I see two host cpus being used. But otherwise, "info cpus" does seem inaccurate. -Kevin -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
> > hm... that's the version I am using und it does not work here ;( > > the hotplug itself seems to work but they're reported as halted. > I told you this is cosmetic. They are working. I agree, the hotplug itself seems to be working! > > > > Then it seems to be a local misconfiguration. and if so, I would like to know what I have misconfigured. ;-) and what I can do to persuade this version to provide me with information about which cpus are online and offline. Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 03:29:35PM +0200, Conrad Wood wrote: > On Sun, 2010-09-19 at 09:26 -0400, Kevin O'Connor wrote: > > On Sun, Sep 19, 2010 at 08:38:12AM +0200, Gleb Natapov wrote: > > > On Sat, Sep 18, 2010 at 08:27:54PM +0200, Conrad Wood wrote: > > > > hm... after upgrading to seabios 0.6.1 and qemu-kvm 0.13.50 (git today) > > > > I get: > > [...] > > > > any ideas ? > > > > > > > Known problem in qemu. There was a patch for this, but qemu maintainers > > > think it is not good enough. > > > > Old versions of kvm do work - for example, qemu-kvm-0.12.5 works for > > me. > > > > -Kevin > > > > hm... that's the version I am using und it does not work here ;( > the hotplug itself seems to work but they're reported as halted. I told you this is cosmetic. They are working. > Could you confirm that the cpus in the monitor as reported by "info > cpus" are not reported as "(halted)" ? > > Then it seems to be a local misconfiguration. > > (with -enable-kvm on amd) > > Thanks! > > Conrad -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 09:26 -0400, Kevin O'Connor wrote: > On Sun, Sep 19, 2010 at 08:38:12AM +0200, Gleb Natapov wrote: > > On Sat, Sep 18, 2010 at 08:27:54PM +0200, Conrad Wood wrote: > > > hm... after upgrading to seabios 0.6.1 and qemu-kvm 0.13.50 (git today) > > > I get: > [...] > > > any ideas ? > > > > > Known problem in qemu. There was a patch for this, but qemu maintainers > > think it is not good enough. > > Old versions of kvm do work - for example, qemu-kvm-0.12.5 works for > me. > > -Kevin > hm... that's the version I am using und it does not work here ;( the hotplug itself seems to work but they're reported as halted. Could you confirm that the cpus in the monitor as reported by "info cpus" are not reported as "(halted)" ? Then it seems to be a local misconfiguration. (with -enable-kvm on amd) Thanks! Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 08:38:12AM +0200, Gleb Natapov wrote: > On Sat, Sep 18, 2010 at 08:27:54PM +0200, Conrad Wood wrote: > > hm... after upgrading to seabios 0.6.1 and qemu-kvm 0.13.50 (git today) > > I get: [...] > > any ideas ? > > > Known problem in qemu. There was a patch for this, but qemu maintainers > think it is not good enough. Old versions of kvm do work - for example, qemu-kvm-0.12.5 works for me. -Kevin -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
raw_spinlock_t rules
Some time ago, the i8259 emulation code in kvm was changed to use raw_spinlock_t, as it was called in a preempt_disable() and local_irq_disable() context, which doesn't work with preemptible spinlocks used with CONFIG_PREEMPT_RT. In Linux 2.6.37, the spinlock will no longer be taken in these contexts, so I'd like to change it to a normal spinlock_t. However, it is still taken in a spin_lock_irq() context. Is it okay to do this change? I figured since spin_lock_irq() is part of the spinlock infrastructure it might to the right thing. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [v2 RFC PATCH 0/4] Implement multiqueue virtio-net
On Fri, Sep 17, 2010 at 03:33:07PM +0530, Krishna Kumar wrote:
> For 1 TCP netperf, I ran 7 iterations and summed it. Explanation
> for degradation for 1 stream case:
Could you document how exactly do you measure multistream bandwidth:
netperf flags, etc?
> 1. Without any tuning, BW falls -6.5%.
Any idea where does this come from?
Do you see more TX interrupts? RX interrupts? Exits?
Do interrupts bounce more between guest CPUs?
> 2. When vhosts on server were bound to CPU0, BW was as good
>as with original code.
> 3. When new code was started with numtxqs=1 (or mq=off, which
>is the default), there was no degradation.
>
>Next steps:
>---
> 1. MQ RX patch is also complete - plan to submit once TX is OK (as
>well as after identifying bandwidth degradations for some test
>cases).
> 2. Cache-align data structures: I didn't see any BW/SD improvement
>after making the sq's (and similarly for vhost) cache-aligned
>statically:
> struct virtnet_info {
> ...
> struct send_queue sq[16] cacheline_aligned_in_smp;
> ...
> };
> 3. Migration is not tested.
4. Identify reasons for single netperf BW regression.
5. Test perf in more scenarious:
small packets
host -> guest
guest <-> external
in last case:
find some other way to measure host CPU utilization,
try multiqueue and single queue devices
6. Use above to figure out what is a sane default for numtxqs.
>
> Review/feedback appreciated.
>
> Signed-off-by: Krishna Kumar
> ---
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/4] KVM: Add kvm_inject_realmode_interrupt() wrapper
From: Mohammed Gamal
This adds a wrapper function kvm_inject_realmode_interrupt() around the
emulator function emulate_int_real() to allow real mode interrupt injection.
[avi: initialize operand and address sizes before emulating interrupts]
[avi: initialize rip for real mode interrupt injection]
[avi: clear interrupt pending flag after emulating interrupt injection]
Signed-off-by: Mohammed Gamal
Signed-off-by: Avi Kivity
---
arch/x86/kvm/x86.c | 29 +
arch/x86/kvm/x86.h |1 +
2 files changed, 30 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index a51635e..a3fc151 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4187,6 +4187,35 @@ static void init_emulate_ctxt(struct kvm_vcpu *vcpu)
memcpy(c->regs, vcpu->arch.regs, sizeof c->regs);
}
+int kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq)
+{
+ struct decode_cache *c = &vcpu->arch.emulate_ctxt.decode;
+ int ret;
+
+ init_emulate_ctxt(vcpu);
+
+ vcpu->arch.emulate_ctxt.decode.op_bytes = 2;
+ vcpu->arch.emulate_ctxt.decode.ad_bytes = 2;
+ vcpu->arch.emulate_ctxt.decode.eip = vcpu->arch.emulate_ctxt.eip;
+ ret = emulate_int_real(&vcpu->arch.emulate_ctxt, &emulate_ops, irq);
+
+ if (ret != X86EMUL_CONTINUE)
+ return EMULATE_FAIL;
+
+ vcpu->arch.emulate_ctxt.eip = c->eip;
+ memcpy(vcpu->arch.regs, c->regs, sizeof c->regs);
+ kvm_rip_write(vcpu, vcpu->arch.emulate_ctxt.eip);
+ kvm_x86_ops->set_rflags(vcpu, vcpu->arch.emulate_ctxt.eflags);
+
+ if (irq == NMI_VECTOR)
+ vcpu->arch.nmi_pending = false;
+ else
+ vcpu->arch.interrupt.pending = false;
+
+ return EMULATE_DONE;
+}
+EXPORT_SYMBOL_GPL(kvm_inject_realmode_interrupt);
+
static int handle_emulation_failure(struct kvm_vcpu *vcpu)
{
++vcpu->stat.insn_emulation_fail;
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index bf4dc2f..2cea414 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -72,6 +72,7 @@ static inline int is_paging(struct kvm_vcpu *vcpu)
void kvm_before_handle_nmi(struct kvm_vcpu *vcpu);
void kvm_after_handle_nmi(struct kvm_vcpu *vcpu);
+int kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq);
void kvm_write_tsc(struct kvm_vcpu *vcpu, u64 data);
--
1.7.2.3
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 4/4] KVM: VMX: Respect interrupt window in big real mode
If an interrupt is pending, we need to stop emulation so we
can inject it.
Signed-off-by: Avi Kivity
---
arch/x86/kvm/vmx.c |9 +
1 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 2572153..1a5ecfd 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -3588,8 +3588,17 @@ static int handle_invalid_guest_state(struct kvm_vcpu
*vcpu)
struct vcpu_vmx *vmx = to_vmx(vcpu);
enum emulation_result err = EMULATE_DONE;
int ret = 1;
+ u32 cpu_exec_ctrl;
+ bool intr_window_requested;
+
+ cpu_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL);
+ intr_window_requested = cpu_exec_ctrl & CPU_BASED_VIRTUAL_INTR_PENDING;
while (!guest_state_valid(vcpu)) {
+ if (intr_window_requested
+ && (kvm_get_rflags(&vmx->vcpu) & X86_EFLAGS_IF))
+ return handle_interrupt_window(&vmx->vcpu);
+
err = emulate_instruction(vcpu, 0, 0, 0);
if (err == EMULATE_DO_MMIO) {
--
1.7.2.3
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/4] KVM: x86 emulator: Expose emulate_int_real()
From: Mohammed Gamal Signed-off-by: Mohammed Gamal Signed-off-by: Avi Kivity --- arch/x86/include/asm/kvm_emulate.h |3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/arch/x86/include/asm/kvm_emulate.h b/arch/x86/include/asm/kvm_emulate.h index 5187dd8..b36c6b3 100644 --- a/arch/x86/include/asm/kvm_emulate.h +++ b/arch/x86/include/asm/kvm_emulate.h @@ -260,5 +260,6 @@ int x86_emulate_insn(struct x86_emulate_ctxt *ctxt); int emulator_task_switch(struct x86_emulate_ctxt *ctxt, u16 tss_selector, int reason, bool has_error_code, u32 error_code); - +int emulate_int_real(struct x86_emulate_ctxt *ctxt, +struct x86_emulate_ops *ops, int irq); #endif /* _ASM_X86_KVM_X86_EMULATE_H */ -- 1.7.2.3 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/4] KVM: VMX: Emulated real mode interrupt injection
From: Mohammed Gamal
Replace the inject-as-software-interrupt hack we currently have with
emulated injection.
Signed-off-by: Mohammed Gamal
Signed-off-by: Avi Kivity
---
arch/x86/kvm/vmx.c | 65 ---
1 files changed, 6 insertions(+), 59 deletions(-)
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 8ef6199..2572153 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -155,11 +155,6 @@ struct vcpu_vmx {
u32 limit;
u32 ar;
} tr, es, ds, fs, gs;
- struct {
- bool pending;
- u8 vector;
- unsigned rip;
- } irq;
} rmode;
int vpid;
bool emulation_required;
@@ -1034,16 +1029,8 @@ static void vmx_queue_exception(struct kvm_vcpu *vcpu,
unsigned nr,
}
if (vmx->rmode.vm86_active) {
- vmx->rmode.irq.pending = true;
- vmx->rmode.irq.vector = nr;
- vmx->rmode.irq.rip = kvm_rip_read(vcpu);
- if (kvm_exception_is_soft(nr))
- vmx->rmode.irq.rip +=
- vmx->vcpu.arch.event_exit_inst_len;
- intr_info |= INTR_TYPE_SOFT_INTR;
- vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);
- vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, 1);
- kvm_rip_write(vcpu, vmx->rmode.irq.rip - 1);
+ if (kvm_inject_realmode_interrupt(vcpu, nr) != EMULATE_DONE)
+ kvm_make_request(KVM_REQ_TRIPLE_FAULT, vcpu);
return;
}
@@ -2822,16 +2809,8 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu)
++vcpu->stat.irq_injections;
if (vmx->rmode.vm86_active) {
- vmx->rmode.irq.pending = true;
- vmx->rmode.irq.vector = irq;
- vmx->rmode.irq.rip = kvm_rip_read(vcpu);
- if (vcpu->arch.interrupt.soft)
- vmx->rmode.irq.rip +=
- vmx->vcpu.arch.event_exit_inst_len;
- vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
-irq | INTR_TYPE_SOFT_INTR | INTR_INFO_VALID_MASK);
- vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, 1);
- kvm_rip_write(vcpu, vmx->rmode.irq.rip - 1);
+ if (kvm_inject_realmode_interrupt(vcpu, irq) != EMULATE_DONE)
+ kvm_make_request(KVM_REQ_TRIPLE_FAULT, vcpu);
return;
}
intr = irq | INTR_INFO_VALID_MASK;
@@ -2863,14 +2842,8 @@ static void vmx_inject_nmi(struct kvm_vcpu *vcpu)
++vcpu->stat.nmi_injections;
if (vmx->rmode.vm86_active) {
- vmx->rmode.irq.pending = true;
- vmx->rmode.irq.vector = NMI_VECTOR;
- vmx->rmode.irq.rip = kvm_rip_read(vcpu);
- vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
-NMI_VECTOR | INTR_TYPE_SOFT_INTR |
-INTR_INFO_VALID_MASK);
- vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, 1);
- kvm_rip_write(vcpu, vmx->rmode.irq.rip - 1);
+ if (kvm_inject_realmode_interrupt(vcpu, NMI_VECTOR) !=
EMULATE_DONE)
+ kvm_make_request(KVM_REQ_TRIPLE_FAULT, vcpu);
return;
}
vmcs_write32(VM_ENTRY_INTR_INFO_FIELD,
@@ -3832,29 +3805,6 @@ static void vmx_recover_nmi_blocking(struct vcpu_vmx
*vmx)
ktime_to_ns(ktime_sub(ktime_get(), vmx->entry_time));
}
-/*
- * Failure to inject an interrupt should give us the information
- * in IDT_VECTORING_INFO_FIELD. However, if the failure occurs
- * when fetching the interrupt redirection bitmap in the real-mode
- * tss, this doesn't happen. So we do it ourselves.
- */
-static void fixup_rmode_irq(struct vcpu_vmx *vmx, u32 *idt_vectoring_info)
-{
- vmx->rmode.irq.pending = 0;
- if (kvm_rip_read(&vmx->vcpu) + 1 != vmx->rmode.irq.rip)
- return;
- kvm_rip_write(&vmx->vcpu, vmx->rmode.irq.rip);
- if (*idt_vectoring_info & VECTORING_INFO_VALID_MASK) {
- *idt_vectoring_info &= ~VECTORING_INFO_TYPE_MASK;
- *idt_vectoring_info |= INTR_TYPE_EXT_INTR;
- return;
- }
- *idt_vectoring_info =
- VECTORING_INFO_VALID_MASK
- | INTR_TYPE_EXT_INTR
- | vmx->rmode.irq.vector;
-}
-
static void __vmx_complete_interrupts(struct vcpu_vmx *vmx,
u32 idt_vectoring_info,
int instr_len_field,
@@ -3864,9 +3814,6 @@ static void __vmx_complete_interrupts(struct vcpu_vmx
*vmx,
int type;
bool idtv_info_valid;
- if (vmx->rmode.irq.pending)
- fixup_rmode_irq(vmx, &idt_vectoring_info);
-
idtv_info_valid = idt_vectoring_info & VECTORING_INFO_VALID_M
[PATCH 0/4] Real mode interrupt injection emulation
Our current real mode interrupt injection injects external interrupts as software interrupts, which is somewhat hacky. This is problematic in big real mode (can't use vmx there) and on via processors (a cpu bug prevents this from working correctly). Replace the current mechanism with emulation; we now inject the interrupt by looking up the vector in the IVT, updating the stack, etc. using the emulator. This is somewhat slower but works in all cases. Avi Kivity (1): KVM: VMX: Respect interrupt window in big real mode Mohammed Gamal (3): KVM: x86 emulator: Expose emulate_int_real() KVM: Add kvm_inject_realmode_interrupt() wrapper KVM: VMX: Emulated real mode interrupt injection arch/x86/include/asm/kvm_emulate.h |3 +- arch/x86/kvm/vmx.c | 74 +++ arch/x86/kvm/x86.c | 29 ++ arch/x86/kvm/x86.h |1 + 4 files changed, 47 insertions(+), 60 deletions(-) -- 1.7.2.3 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 13:59 +0200, Gleb Natapov wrote: > On Sun, Sep 19, 2010 at 01:51:29PM +0200, Conrad Wood wrote: > > On Sun, 2010-09-19 at 13:39 +0200, Gleb Natapov wrote: > > > On Sun, Sep 19, 2010 at 01:32:28PM +0200, Conrad Wood wrote: > > > > On Sun, 2010-09-19 at 13:13 +0200, Gleb Natapov wrote: > > > > > On Sun, Sep 19, 2010 at 01:09:39PM +0200, Conrad Wood wrote: > > > > > > On Sun, 2010-09-19 at 12:46 +0200, Gleb Natapov wrote: > > > > > > [...] > > > > > > > > But... why does the command "info cpus" in the monitor report > > > > > > > > all cpus > > > > > > > > apart from CPU0 as (halted) ? > > > > > > > > > > > > > > > Because they are halted? Run multiple cpu hogs in the guest and > > > > > > > see if > > > > > > > cpus will still be halted as reported by monitor. Also kvm-72 is > > > > > > > so > > > > > > > ancient that I am not sure it reports cpu state correctly in > > > > > > > monitor at > > > > > > > all. > > > > > > > > > > > > > > -- > > > > > > fair enough, upgraded to 0.12.5, same results. > > > > > > > > > > > > tried a cpu-hog (small loop forking 5 times and each fork doing some > > > > > > integer arithmetics for a while) - no change in info cpus > > > > > > > > > > > > thank you, but I still don't get it ;-) > > > > > > > > > > > At what rip are they halted? Try to run without kvm and see if the > > > > > result is the same. Can you provide output of "info cpus" here? > > > > > Are you using qemu-kvm or qemu BTW? > > > > > > > > I am using qemu-kvm (debian package Version: 0.12.5+dfsg-3). > > > > Here's the "info cpus" with and without kvm acceleration. > > > > basically qemu shows the behavior I'd expect, kvm does not. Or do I > > > > misunderstand something? > > > > > > > No you are not. So your qemu-kvm version has cpu state reporting bug too. > > > It is > > > not serious bug, just annoying. pc shows that otherwise everything is > > > working as expected. > > > > I see. Is that a known bug, is someone working on that already? > > If not... could you point me to the relevant code sections/docs where I > > can fix that? > > It actually is quite relevant for me, as I have to manage quite a lot of > > VMs and need to be able to query how many cpus/cores they're using ;( > > > I fixed it several times in the past[0][1][3], but it keeps re-appearing :) I > hope current qemu-kvm head is OK. > > [0] 865aa3fec2b3d83c9b6edd452050a30ef6eaf3b8 > [1] 8c7d4cb7163d1c97df4439e2e0edb5918abf106b > [2] d6d233543d543fcf34df6190bbce5daf38d870f5 got it. I'll keep an eye on head and once cpu-hotplug works with head I'll test the state reporting ;) Thank you!!! Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 01:51:29PM +0200, Conrad Wood wrote: > On Sun, 2010-09-19 at 13:39 +0200, Gleb Natapov wrote: > > On Sun, Sep 19, 2010 at 01:32:28PM +0200, Conrad Wood wrote: > > > On Sun, 2010-09-19 at 13:13 +0200, Gleb Natapov wrote: > > > > On Sun, Sep 19, 2010 at 01:09:39PM +0200, Conrad Wood wrote: > > > > > On Sun, 2010-09-19 at 12:46 +0200, Gleb Natapov wrote: > > > > > [...] > > > > > > > But... why does the command "info cpus" in the monitor report all > > > > > > > cpus > > > > > > > apart from CPU0 as (halted) ? > > > > > > > > > > > > > Because they are halted? Run multiple cpu hogs in the guest and see > > > > > > if > > > > > > cpus will still be halted as reported by monitor. Also kvm-72 is so > > > > > > ancient that I am not sure it reports cpu state correctly in > > > > > > monitor at > > > > > > all. > > > > > > > > > > > > -- > > > > > fair enough, upgraded to 0.12.5, same results. > > > > > > > > > > tried a cpu-hog (small loop forking 5 times and each fork doing some > > > > > integer arithmetics for a while) - no change in info cpus > > > > > > > > > > thank you, but I still don't get it ;-) > > > > > > > > > At what rip are they halted? Try to run without kvm and see if the > > > > result is the same. Can you provide output of "info cpus" here? > > > > Are you using qemu-kvm or qemu BTW? > > > > > > I am using qemu-kvm (debian package Version: 0.12.5+dfsg-3). > > > Here's the "info cpus" with and without kvm acceleration. > > > basically qemu shows the behavior I'd expect, kvm does not. Or do I > > > misunderstand something? > > > > > No you are not. So your qemu-kvm version has cpu state reporting bug too. > > It is > > not serious bug, just annoying. pc shows that otherwise everything is > > working as expected. > > I see. Is that a known bug, is someone working on that already? > If not... could you point me to the relevant code sections/docs where I > can fix that? > It actually is quite relevant for me, as I have to manage quite a lot of > VMs and need to be able to query how many cpus/cores they're using ;( > I fixed it several times in the past[0][1][3], but it keeps re-appearing :) I hope current qemu-kvm head is OK. [0] 865aa3fec2b3d83c9b6edd452050a30ef6eaf3b8 [1] 8c7d4cb7163d1c97df4439e2e0edb5918abf106b [2] d6d233543d543fcf34df6190bbce5daf38d870f5 -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 13:39 +0200, Gleb Natapov wrote: > On Sun, Sep 19, 2010 at 01:32:28PM +0200, Conrad Wood wrote: > > On Sun, 2010-09-19 at 13:13 +0200, Gleb Natapov wrote: > > > On Sun, Sep 19, 2010 at 01:09:39PM +0200, Conrad Wood wrote: > > > > On Sun, 2010-09-19 at 12:46 +0200, Gleb Natapov wrote: > > > > [...] > > > > > > But... why does the command "info cpus" in the monitor report all > > > > > > cpus > > > > > > apart from CPU0 as (halted) ? > > > > > > > > > > > Because they are halted? Run multiple cpu hogs in the guest and see if > > > > > cpus will still be halted as reported by monitor. Also kvm-72 is so > > > > > ancient that I am not sure it reports cpu state correctly in monitor > > > > > at > > > > > all. > > > > > > > > > > -- > > > > fair enough, upgraded to 0.12.5, same results. > > > > > > > > tried a cpu-hog (small loop forking 5 times and each fork doing some > > > > integer arithmetics for a while) - no change in info cpus > > > > > > > > thank you, but I still don't get it ;-) > > > > > > > At what rip are they halted? Try to run without kvm and see if the > > > result is the same. Can you provide output of "info cpus" here? > > > Are you using qemu-kvm or qemu BTW? > > > > I am using qemu-kvm (debian package Version: 0.12.5+dfsg-3). > > Here's the "info cpus" with and without kvm acceleration. > > basically qemu shows the behavior I'd expect, kvm does not. Or do I > > misunderstand something? > > > No you are not. So your qemu-kvm version has cpu state reporting bug too. It > is > not serious bug, just annoying. pc shows that otherwise everything is > working as expected. I see. Is that a known bug, is someone working on that already? If not... could you point me to the relevant code sections/docs where I can fix that? It actually is quite relevant for me, as I have to manage quite a lot of VMs and need to be able to query how many cpus/cores they're using ;( -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 01:32:28PM +0200, Conrad Wood wrote: > On Sun, 2010-09-19 at 13:13 +0200, Gleb Natapov wrote: > > On Sun, Sep 19, 2010 at 01:09:39PM +0200, Conrad Wood wrote: > > > On Sun, 2010-09-19 at 12:46 +0200, Gleb Natapov wrote: > > > [...] > > > > > But... why does the command "info cpus" in the monitor report all cpus > > > > > apart from CPU0 as (halted) ? > > > > > > > > > Because they are halted? Run multiple cpu hogs in the guest and see if > > > > cpus will still be halted as reported by monitor. Also kvm-72 is so > > > > ancient that I am not sure it reports cpu state correctly in monitor at > > > > all. > > > > > > > > -- > > > fair enough, upgraded to 0.12.5, same results. > > > > > > tried a cpu-hog (small loop forking 5 times and each fork doing some > > > integer arithmetics for a while) - no change in info cpus > > > > > > thank you, but I still don't get it ;-) > > > > > At what rip are they halted? Try to run without kvm and see if the > > result is the same. Can you provide output of "info cpus" here? > > Are you using qemu-kvm or qemu BTW? > > I am using qemu-kvm (debian package Version: 0.12.5+dfsg-3). > Here's the "info cpus" with and without kvm acceleration. > basically qemu shows the behavior I'd expect, kvm does not. Or do I > misunderstand something? > No you are not. So your qemu-kvm version has cpu state reporting bug too. It is not serious bug, just annoying. pc shows that otherwise everything is working as expected. > I wonder. if this is relevant: The host only has a single cpu, with 4 > cores, I had expected those to be seen as cpus in the guest, similar how > linux lists each core as a cpu in the host. Guest and host has different cpu topology. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 13:13 +0200, Gleb Natapov wrote: > On Sun, Sep 19, 2010 at 01:09:39PM +0200, Conrad Wood wrote: > > On Sun, 2010-09-19 at 12:46 +0200, Gleb Natapov wrote: > > [...] > > > > But... why does the command "info cpus" in the monitor report all cpus > > > > apart from CPU0 as (halted) ? > > > > > > > Because they are halted? Run multiple cpu hogs in the guest and see if > > > cpus will still be halted as reported by monitor. Also kvm-72 is so > > > ancient that I am not sure it reports cpu state correctly in monitor at > > > all. > > > > > > -- > > fair enough, upgraded to 0.12.5, same results. > > > > tried a cpu-hog (small loop forking 5 times and each fork doing some > > integer arithmetics for a while) - no change in info cpus > > > > thank you, but I still don't get it ;-) > > > At what rip are they halted? Try to run without kvm and see if the > result is the same. Can you provide output of "info cpus" here? > Are you using qemu-kvm or qemu BTW? I am using qemu-kvm (debian package Version: 0.12.5+dfsg-3). Here's the "info cpus" with and without kvm acceleration. basically qemu shows the behavior I'd expect, kvm does not. Or do I misunderstand something? I wonder. if this is relevant: The host only has a single cpu, with 4 cores, I had expected those to be seen as cpus in the guest, similar how linux lists each core as a cpu in the host. kvm -no-kvm: after guest has booted: (qemu) info cpus * CPU #0: pc=0x81047730 (halted) thread_id=17121 CPU #1: pc=0x81047730 (halted) thread_id=17121 CPU #2: pc=0x81047730 (halted) thread_id=17121 CPU #3: pc=0x81047730 (halted) thread_id=17121 running cpuhog with 5 forks: (qemu) info cpus * CPU #0: pc=0x00400698 thread_id=17121 CPU #1: pc=0x00400698 thread_id=17121 CPU #2: pc=0x00400698 thread_id=17121 CPU #3: pc=0x00400698 thread_id=17121 kvm -enable-kvm after guest has booted: (qemu) info cpus * CPU #0: pc=0x81047730 thread_id=27082 CPU #1: pc=0x81047730 (halted) thread_id=27083 CPU #2: pc=0x81047730 (halted) thread_id=27084 CPU #3: pc=0x81047730 (halted) thread_id=27085 running cpuhog with 5 forks: (qemu) info cpus * CPU #0: pc=0x004006b6 thread_id=27082 CPU #1: pc=0x004006b6 (halted) thread_id=27083 CPU #2: pc=0x004006b6 (halted) thread_id=27084 CPU #3: pc=0x004006b6 (halted) thread_id=27085 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC] kvm: enable irq injection from interrupt context
On Sun, Sep 19, 2010 at 01:05:09PM +0200, Avi Kivity wrote: > On 09/19/2010 12:55 PM, Michael S. Tsirkin wrote: > >On Sun, Sep 19, 2010 at 12:56:12PM +0200, Avi Kivity wrote: > >> On 09/19/2010 12:45 PM, Michael S. Tsirkin wrote: > >> >On Fri, Sep 17, 2010 at 09:59:29AM +0200, Gleb Natapov wrote: > >> >> > writing 0 to eventfd does nothing. The way to deassert irq > >> >> That is implementation detail of current irqfd. It was designed for > >> MSI > >> >> not level triggered interrupts. > >> > > >> >Maybe we should add a check that gsi is mapped to MSI (or unmapped) then? > >> >Level which switches to 1 and back to 0 immediately will be racy anyway > >> >... > >> > > >> > >> Add a check where? > > > >I would make sure that if you bind irqfd to a non-MSI GSI, > >signalling it has no effect. > > > > Certainly. We should also update the documentation, when we have > irqfd documentation. When and if we start supporing level, we'll have to add a capability flag. By way of documentation, reserve a bit value in header right now? > -- > error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC] kvm: enable irq injection from interrupt context
On Sun, Sep 19, 2010 at 01:18:44PM +0200, Gleb Natapov wrote: > On Sun, Sep 19, 2010 at 12:55:12PM +0200, Michael S. Tsirkin wrote: > > On Sun, Sep 19, 2010 at 12:56:12PM +0200, Avi Kivity wrote: > > > On 09/19/2010 12:45 PM, Michael S. Tsirkin wrote: > > > >On Fri, Sep 17, 2010 at 09:59:29AM +0200, Gleb Natapov wrote: > > > >> > writing 0 to eventfd does nothing. The way to deassert irq > > > >> That is implementation detail of current irqfd. It was designed for > > > >> MSI > > > >> not level triggered interrupts. > > > > > > > >Maybe we should add a check that gsi is mapped to MSI (or unmapped) then? > > > >Level which switches to 1 and back to 0 immediately will be racy anyway > > > >... > > > > > > > > > > Add a check where? > > > > I would make sure that if you bind irqfd to a non-MSI GSI, > > signalling it has no effect. > > > Why would you do that? I am not against checking per se, but why a user of > the API can't check for that? > > -- > Gleb. Currently using irqfd with level will kind of work, sometimes. Better have it consistently not working so application writers get the message that they can't rely on it. -- MST -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC] kvm: enable irq injection from interrupt context
On Sun, Sep 19, 2010 at 12:55:12PM +0200, Michael S. Tsirkin wrote: > On Sun, Sep 19, 2010 at 12:56:12PM +0200, Avi Kivity wrote: > > On 09/19/2010 12:45 PM, Michael S. Tsirkin wrote: > > >On Fri, Sep 17, 2010 at 09:59:29AM +0200, Gleb Natapov wrote: > > >> > writing 0 to eventfd does nothing. The way to deassert irq > > >> That is implementation detail of current irqfd. It was designed for MSI > > >> not level triggered interrupts. > > > > > >Maybe we should add a check that gsi is mapped to MSI (or unmapped) then? > > >Level which switches to 1 and back to 0 immediately will be racy anyway > > >... > > > > > > > Add a check where? > > I would make sure that if you bind irqfd to a non-MSI GSI, > signalling it has no effect. > Why would you do that? I am not against checking per se, but why a user of the API can't check for that? -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 01:09:39PM +0200, Conrad Wood wrote: > On Sun, 2010-09-19 at 12:46 +0200, Gleb Natapov wrote: > [...] > > > But... why does the command "info cpus" in the monitor report all cpus > > > apart from CPU0 as (halted) ? > > > > > Because they are halted? Run multiple cpu hogs in the guest and see if > > cpus will still be halted as reported by monitor. Also kvm-72 is so > > ancient that I am not sure it reports cpu state correctly in monitor at > > all. > > > > -- > fair enough, upgraded to 0.12.5, same results. > > tried a cpu-hog (small loop forking 5 times and each fork doing some > integer arithmetics for a while) - no change in info cpus > > thank you, but I still don't get it ;-) > At what rip are they halted? Try to run without kvm and see if the result is the same. Can you provide output of "info cpus" here? Are you using qemu-kvm or qemu BTW? -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 12:46 +0200, Gleb Natapov wrote: [...] > > But... why does the command "info cpus" in the monitor report all cpus > > apart from CPU0 as (halted) ? > > > Because they are halted? Run multiple cpu hogs in the guest and see if > cpus will still be halted as reported by monitor. Also kvm-72 is so > ancient that I am not sure it reports cpu state correctly in monitor at > all. > > -- fair enough, upgraded to 0.12.5, same results. tried a cpu-hog (small loop forking 5 times and each fork doing some integer arithmetics for a while) - no change in info cpus thank you, but I still don't get it ;-) Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC] kvm: enable irq injection from interrupt context
On 09/19/2010 12:55 PM, Michael S. Tsirkin wrote: On Sun, Sep 19, 2010 at 12:56:12PM +0200, Avi Kivity wrote: > On 09/19/2010 12:45 PM, Michael S. Tsirkin wrote: > >On Fri, Sep 17, 2010 at 09:59:29AM +0200, Gleb Natapov wrote: > >> > writing 0 to eventfd does nothing. The way to deassert irq > >> That is implementation detail of current irqfd. It was designed for MSI > >> not level triggered interrupts. > > > >Maybe we should add a check that gsi is mapped to MSI (or unmapped) then? > >Level which switches to 1 and back to 0 immediately will be racy anyway > >... > > > > Add a check where? I would make sure that if you bind irqfd to a non-MSI GSI, signalling it has no effect. Certainly. We should also update the documentation, when we have irqfd documentation. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC] kvm: enable irq injection from interrupt context
On Sun, Sep 19, 2010 at 12:56:12PM +0200, Avi Kivity wrote: > On 09/19/2010 12:45 PM, Michael S. Tsirkin wrote: > >On Fri, Sep 17, 2010 at 09:59:29AM +0200, Gleb Natapov wrote: > >> > writing 0 to eventfd does nothing. The way to deassert irq > >> That is implementation detail of current irqfd. It was designed for MSI > >> not level triggered interrupts. > > > >Maybe we should add a check that gsi is mapped to MSI (or unmapped) then? > >Level which switches to 1 and back to 0 immediately will be racy anyway > >... > > > > Add a check where? I would make sure that if you bind irqfd to a non-MSI GSI, signalling it has no effect. > -- > error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC] kvm: enable irq injection from interrupt context
On 09/19/2010 12:45 PM, Michael S. Tsirkin wrote: On Fri, Sep 17, 2010 at 09:59:29AM +0200, Gleb Natapov wrote: > > writing 0 to eventfd does nothing. The way to deassert irq > That is implementation detail of current irqfd. It was designed for MSI > not level triggered interrupts. Maybe we should add a check that gsi is mapped to MSI (or unmapped) then? Level which switches to 1 and back to 0 immediately will be racy anyway ... Add a check where? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC] kvm: enable irq injection from interrupt context
On Fri, Sep 17, 2010 at 09:59:29AM +0200, Gleb Natapov wrote: > > writing 0 to eventfd does nothing. The way to deassert irq > That is implementation detail of current irqfd. It was designed for MSI > not level triggered interrupts. Maybe we should add a check that gsi is mapped to MSI (or unmapped) then? Level which switches to 1 and back to 0 immediately will be racy anyway ... Avi? -- MST -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, Sep 19, 2010 at 12:41:37PM +0200, Conrad Wood wrote: > On Sun, 2010-09-19 at 08:38 +0200, Gleb Natapov wrote: > > On Sat, Sep 18, 2010 at 08:27:54PM +0200, Conrad Wood wrote: > > > On Sat, 2010-09-18 at 10:32 -0400, Kevin O'Connor wrote: > > > > On Fri, Sep 17, 2010 at 11:39:36PM +0200, Conrad Wood wrote: > > > > > Hi everyone, > > > > > > > > > > I'm currently looking into hotplugging CPUs. > > > > > This exclusively with linux-guests and linux-host. > > > > > I have found some (conflicting) information about this on the net. > > > > > > [...] > > Well, I got a bit further - thanks so far for your help. > Given > 1. kvm-72 > 2. linux-guest-kernel 2.6.35.4 > 3. kernel cmdline "maxcpus=32 possible_cpus=32" > > I can set "cpu_set 5 online" in the monitor and get indeed a new cpu > appearing in /sys/devices..., which I can then set online in the guest > by doing echo 1 >/sys/devices/.../online. > > /proc/cpuinfo in the guest then reports the new cpu. > > But... why does the command "info cpus" in the monitor report all cpus > apart from CPU0 as (halted) ? > Because they are halted? Run multiple cpu hogs in the guest and see if cpus will still be halted as reported by monitor. Also kvm-72 is so ancient that I am not sure it reports cpu state correctly in monitor at all. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cpu hotplug
On Sun, 2010-09-19 at 08:38 +0200, Gleb Natapov wrote: > On Sat, Sep 18, 2010 at 08:27:54PM +0200, Conrad Wood wrote: > > On Sat, 2010-09-18 at 10:32 -0400, Kevin O'Connor wrote: > > > On Fri, Sep 17, 2010 at 11:39:36PM +0200, Conrad Wood wrote: > > > > Hi everyone, > > > > > > > > I'm currently looking into hotplugging CPUs. > > > > This exclusively with linux-guests and linux-host. > > > > I have found some (conflicting) information about this on the net. > > > > [...] Well, I got a bit further - thanks so far for your help. Given 1. kvm-72 2. linux-guest-kernel 2.6.35.4 3. kernel cmdline "maxcpus=32 possible_cpus=32" I can set "cpu_set 5 online" in the monitor and get indeed a new cpu appearing in /sys/devices..., which I can then set online in the guest by doing echo 1 >/sys/devices/.../online. /proc/cpuinfo in the guest then reports the new cpu. But... why does the command "info cpus" in the monitor report all cpus apart from CPU0 as (halted) ? echo 1 >/sys/devices.../online makes no difference to what "info cpus" reports, even though the guests /proc/cpuinfo shows expected behaviour. Sorry to bother everyone, just trying to understand this. How, if at all, are the guests' cpus mapped to physical cpus or cores on the host? Conrad -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 0/6] Nonatomic interrupt injection
On 09/19/2010 11:28 AM, Avi Kivity wrote: On 09/19/2010 11:25 AM, Avi Kivity wrote: Let's assume that this is so (I'll check). It's trivially so. If a completion causes an interrupt to be raised, the vcpu's apic code is executed in the iothread context. However, that's a bug even with the optimization removed: iothreadvcpu ioapic interrrupt apic: set isr mov $large_value, cr8 on_vcpu(inject) inject interrupt even though cr8 inhibits it Setting the isr and injecting the interrupt should be an atomic operation (easily done be running the apic code on the vcpu thread). -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/7] New Unit-Tests for KVM SVM emulation v2
On 09/14/2010 05:59 PM, Joerg Roedel wrote: Hi Avi, here is the second version of the new unit-tests for the KVM SVM emulation. The changes to the previous version are really minor: * Fixed coding-style * Fixed comment in the code that builds the nested page table * Renamed sel_cr0 test to sel_cr0_bug test to add a real sel_cr0 test later which checks if the feature itself is working All-in-all, not a lot of changes. I re-ran all tests and they still all PASS. Applied, thanks. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 0/6] Nonatomic interrupt injection
On 09/19/2010 11:25 AM, Avi Kivity wrote: Let's assume that this is so (I'll check). It's trivially so. If a completion causes an interrupt to be raised, the vcpu's apic code is executed in the iothread context. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 0/6] Nonatomic interrupt injection
On 09/17/2010 09:12 PM, Marcelo Tosatti wrote:
> This is now merged, with the change pointed out by Marcelo. Windows
> XP x64 fails installation without
>
> (vmx.c handle_cr())
> case 8: {
> u8 cr8_prev = kvm_get_cr8(vcpu);
> u8 cr8 = kvm_register_read(vcpu, reg);
> kvm_set_cr8(vcpu, cr8);
> skip_emulated_instruction(vcpu);
> if (irqchip_in_kernel(vcpu->kvm))
> return 1;
> -if (cr8_prev<= cr8)
> -return 1;
> vcpu->run->exit_reason = KVM_EXIT_SET_TPR;
> return 0;
> }
>
> Which doesn't make any sense (anyone?). The failure is present even
> without the patchset, and is fixed by the same hack, so a regression
> was not introduced.
If userspace does not have an uptodate TPR value, it can signal an
interrupt that is now blocked? Say:
- cr8 write 0
- cr8 write 5
- no exit to userspace
- userspace signals interrupt with priority
4 because it knows about tpr == 0.
To signal an interrupt, userspace needs to force an exit. The exit will
sync cr8.
However, it may be that the decision to inject the interrupt is taken
before the exit, so the interrupt is injected even though it shouldn't be.
Let's assume that this is so (I'll check). Is this a bug in the kernel
or userspace?
My feeling is that this is a kernel bug, and the optimization should be
removed.
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
