[Qemu-devel] livepatch kernel to use md-clear without reboot

2019-05-17 Thread Stefan Priebe - Profihost AG
Hello,

i'm interested if it is possible to livepatch the guest kernel without
reboot.

(host kernel and microcode is patched)

Greets,
Stefan



Re: [Qemu-devel] cpu.fail / MDS fixes

2019-05-15 Thread Stefan Priebe - Profihost AG


Am 15.05.19 um 19:54 schrieb Daniel P. Berrangé:
> On Wed, May 15, 2019 at 07:13:56PM +0200, Stefan Priebe - Profihost AG wrote:
>> Hello list,
>>
>> i've updated my host to kernel 4.19.43 and applied the following patch
>> to my qemu 2.12.1:
>> https://bugzilla.suse.com/attachment.cgi?id=798722
>>
>> But my guest running 4.19.43 still says:
>> Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state
>> unknown
>>
>> while the host says:
>> Vulnerable: Clear CPU buffers attempted, SMT Host state unknown
> 
> That suggests your host OS hasn't got the new microcode installed
> or has not loaded it.

No it does not. A not loaded Microcode looks like this:
Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable

but in my case it is:
Mitigation: Clear CPU buffers; SMT vulnerable

on the host as hyper threading is still enabled.

> You want the host to report that it is Mitigated, and for the
> host's /proc/cpuinfo to report "md-clear" exists.
> 
>> I expected the guest can use the new microcode.
> 
> You've not said what CPU model you've given to the guest.
> 
> You need either "-cpu host", or if using a named CPU model
> you need to explicitly turn on the "md-clear" feature
> (and all previous fixes)
> 
>eg  "-cpu Haswell,+spec-ctrl,+ssbd,+md-clear"
hah yes you're true i need to specifiy +md-clean

Thanks!

> Regards,
> Daniel
> 



[Qemu-devel] cpu.fail / MDS fixes

2019-05-15 Thread Stefan Priebe - Profihost AG
Hello list,

i've updated my host to kernel 4.19.43 and applied the following patch
to my qemu 2.12.1:
https://bugzilla.suse.com/attachment.cgi?id=798722

But my guest running 4.19.43 still says:
Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state
unknown

while the host says:
Vulnerable: Clear CPU buffers attempted, SMT Host state unknown

I expected the guest can use the new microcode.

Greets,
Stefan



Re: [Qemu-devel] Overcommiting cpu results in all vms offline

2018-09-17 Thread Stefan Priebe - Profihost AG
Am 17.09.2018 um 11:40 schrieb Jack Wang:
> Stefan Priebe - Profihost AG  于2018年9月17日周一 上午9:00写道:
>>
>> Hi,
>>
>> Am 17.09.2018 um 08:38 schrieb Jack Wang:
>>> Stefan Priebe - Profihost AG  于2018年9月16日周日 下午3:31写道:
>>>>
>>>> Hello,
>>>>
>>>> while overcommiting cpu I had several situations where all vms gone 
>>>> offline while two vms saturated all cores.
>>>>
>>>> I believed all vms would stay online but would just not be able to use all 
>>>> their cores?
>>>>
>>>> My original idea was to automate live migration on high host load to move 
>>>> vms to another node but that makes only sense if all vms stay online.
>>>>
>>>> Is this expected? Anything special needed to archive this?
>>>>
>>>> Greets,
>>>> Stefan
>>>>
>>> Hi, Stefan,
>>>
>>> Do you have any logs when all VMs go offline?
>>> Maybe OOMkiller play a role there?
>>
>> After reviewing i think this is memory related but OOM did not play a role.
>> All kvm processes where spinning trying to get > 100% CPU and i was not
>> able to even login to ssh. After 5-10 minutes i was able to login.
> So the VMs are not really offline, what the result if you run
> query-status via qmp?

I can't as i can't connect to the host in that stage.

>> There were about 150GB free mem.
>>
>> Relevant settings (no local storage involved):
>> vm.dirty_background_ratio:
>> 3
>> vm.dirty_ratio:
>> 10
>> vm.min_free_kbytes:
>> 10567004
>>
>> # cat /sys/kernel/mm/transparent_hugepage/defrag
>> always defer [defer+madvise] madvise never
>>
>> # cat /sys/kernel/mm/transparent_hugepage/enabled
>> [always] madvise never
>>
>> After that i had the following traces on the host node:
>> https://pastebin.com/raw/0VhyQmAv
> 
> The call trace looks ceph related deadlock or hung.

Yes but i can also show you traces where nothing from ceph is involved
the only thing they have in common is the beginning in page_fault.

>> Thanks!
>>
>> Greets,
>> Stefan



Re: [Qemu-devel] Overcommiting cpu results in all vms offline

2018-09-17 Thread Stefan Priebe - Profihost AG
May be amissing piece:
vm.overcommit_memory=0

Greets,
Stefan


Am 17.09.2018 um 09:00 schrieb Stefan Priebe - Profihost AG:
> Hi,
> 
> Am 17.09.2018 um 08:38 schrieb Jack Wang:
>> Stefan Priebe - Profihost AG  于2018年9月16日周日 下午3:31写道:
>>>
>>> Hello,
>>>
>>> while overcommiting cpu I had several situations where all vms gone offline 
>>> while two vms saturated all cores.
>>>
>>> I believed all vms would stay online but would just not be able to use all 
>>> their cores?
>>>
>>> My original idea was to automate live migration on high host load to move 
>>> vms to another node but that makes only sense if all vms stay online.
>>>
>>> Is this expected? Anything special needed to archive this?
>>>
>>> Greets,
>>> Stefan
>>>
>> Hi, Stefan,
>>
>> Do you have any logs when all VMs go offline?
>> Maybe OOMkiller play a role there?
> 
> After reviewing i think this is memory related but OOM did not play a role.
> All kvm processes where spinning trying to get > 100% CPU and i was not
> able to even login to ssh. After 5-10 minutes i was able to login.
> 
> There were about 150GB free mem.
> 
> Relevant settings (no local storage involved):
> vm.dirty_background_ratio:
> 3
> vm.dirty_ratio:
> 10
> vm.min_free_kbytes:
> 10567004
> 
> # cat /sys/kernel/mm/transparent_hugepage/defrag
> always defer [defer+madvise] madvise never
> 
> # cat /sys/kernel/mm/transparent_hugepage/enabled
> [always] madvise never
> 
> After that i had the following traces on the host node:
> https://pastebin.com/raw/0VhyQmAv
> 
> Thanks!
> 
> Greets,
> Stefan
> 



Re: [Qemu-devel] Overcommiting cpu results in all vms offline

2018-09-17 Thread Stefan Priebe - Profihost AG
Hi,

Am 17.09.2018 um 08:38 schrieb Jack Wang:
> Stefan Priebe - Profihost AG  于2018年9月16日周日 下午3:31写道:
>>
>> Hello,
>>
>> while overcommiting cpu I had several situations where all vms gone offline 
>> while two vms saturated all cores.
>>
>> I believed all vms would stay online but would just not be able to use all 
>> their cores?
>>
>> My original idea was to automate live migration on high host load to move 
>> vms to another node but that makes only sense if all vms stay online.
>>
>> Is this expected? Anything special needed to archive this?
>>
>> Greets,
>> Stefan
>>
> Hi, Stefan,
> 
> Do you have any logs when all VMs go offline?
> Maybe OOMkiller play a role there?

After reviewing i think this is memory related but OOM did not play a role.
All kvm processes where spinning trying to get > 100% CPU and i was not
able to even login to ssh. After 5-10 minutes i was able to login.

There were about 150GB free mem.

Relevant settings (no local storage involved):
vm.dirty_background_ratio:
3
vm.dirty_ratio:
10
vm.min_free_kbytes:
10567004

# cat /sys/kernel/mm/transparent_hugepage/defrag
always defer [defer+madvise] madvise never

# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

After that i had the following traces on the host node:
https://pastebin.com/raw/0VhyQmAv

Thanks!

Greets,
Stefan



[Qemu-devel] Overcommiting cpu results in all vms offline

2018-09-16 Thread Stefan Priebe - Profihost AG
Hello,

while overcommiting cpu I had several situations where all vms gone offline 
while two vms saturated all cores.

I believed all vms would stay online but would just not be able to use all 
their cores?

My original idea was to automate live migration on high host load to move vms 
to another node but that makes only sense if all vms stay online.

Is this expected? Anything special needed to archive this?

Greets,
Stefan



Re: [Qemu-devel] Qemu and Spectre_V4 + l1tf + IBRS_FW

2018-08-21 Thread Stefan Priebe - Profihost AG


Am 17.08.2018 um 11:41 schrieb Daniel P. Berrangé:
> On Fri, Aug 17, 2018 at 08:44:38AM +0200, Stefan Priebe - Profihost AG wrote:
>> Hello,
>>
>> i haven't found anything on the web regarding qemu and mentioned variants.
>>
>> While my host says:
>> l1tf:Mitigation: PTE Inversion; VMX: SMT vulnerable, L1D conditional
>> cache flushes
>> meltdown:Mitigation: PTI
>> spec_store_bypass:Mitigation: Speculative Store Bypass disabled via
>> prctl and seccomp
>> spectre_v1:Mitigation: __user pointer sanitization
>> spectre_v2:Mitigation: Full generic retpoline, IBPB, IBRS_FW
>>
>> My guests bootet with pcid and spec-ctrl only say:
>> l1tf:Mitigation: PTE Inversion
>> meltdown:Mitigation: PTI
>> spec_store_bypass:Vulnerable
>> spectre_v1:Mitigation: __user pointer sanitization
>> spectre_v2:Mitigation: Full generic retpoline, IBPB
>>
>> * What is about spec_store_bypass in Qemu?
> 
> The guest needs an 'ssbd' feature for Intel CPU models and either a
> 'virt-ssbd' or 'amd-ssbd' feature for AMD CPU models.

Ah thanks. That works fine.

>> * What is about IBRS_FW feature?
> 
> I'm not sure what IBRS_FW is referring to, but don't worry about it.
> The fact the the guest kernel says "Mitigation" instead of "Vulnerable"
> means you are protected with your current config.
> 
> For Intel CPU models Spectre v2 needs the guest to have the 'spec-ctrl'
> feature. On AMD models Spectre v2 the guest needs 'ibpb' feature.
> 
>> * What is about L1TF?
> 
> No extra CPU flags are required for QEMU guests for L1TF. The new CPU
> feature is merely an perf optimization for the host hypervisor fixes.
> 
> Note that with L1TF there are extra steps you need to consider wrt
> hyperthreading, that won't be reflected in the 'vulnerabilities'
> data published by the kernel.
> 
> You can read more about the procedure for dealing with L1TF in
> virt hosts in the "Resolve" tab of this article:
> 
>   https://access.redhat.com/security/vulnerabilities/L1TF
> 
>> Or are those just irrelevant to Qemu guests? Would be great to have some
>> informations.
> 
> We have some QEMU docs providing guidance on guest CPU model/feature config
> but they are not yet published. In the meantime this blog post of mine gives
> the same info, covering what's needed for Spectre v2, Meltdown and SSBD and
> guidance in general for CPU config:
> 
>   
> https://www.berrange.com/posts/2018/06/29/cpu-model-configuration-for-qemu-kvm-on-x86-hosts/

Thanks, that one was really helpful.

Greets,
Stefan



[Qemu-devel] Qemu and Spectre_V4 + l1tf + IBRS_FW

2018-08-16 Thread Stefan Priebe - Profihost AG
Hello,

i haven't found anything on the web regarding qemu and mentioned variants.

While my host says:
l1tf:Mitigation: PTE Inversion; VMX: SMT vulnerable, L1D conditional
cache flushes
meltdown:Mitigation: PTI
spec_store_bypass:Mitigation: Speculative Store Bypass disabled via
prctl and seccomp
spectre_v1:Mitigation: __user pointer sanitization
spectre_v2:Mitigation: Full generic retpoline, IBPB, IBRS_FW

My guests bootet with pcid and spec-ctrl only say:
l1tf:Mitigation: PTE Inversion
meltdown:Mitigation: PTI
spec_store_bypass:Vulnerable
spectre_v1:Mitigation: __user pointer sanitization
spectre_v2:Mitigation: Full generic retpoline, IBPB

* What is about spec_store_bypass in Qemu?
* What is about IBRS_FW feature?
* What is about L1TF?

Or are those just irrelevant to Qemu guests? Would be great to have some
informations.

Thanks a lot!

Greets,
Stefan



Re: [Qemu-devel] how to pass pcid to guest?

2018-01-09 Thread Stefan Priebe - Profihost AG

Am 08.01.2018 um 23:07 schrieb Eric Blake:
> On 01/08/2018 02:03 PM, Stefan Priebe - Profihost AG wrote:
>> Hello,
>>
>> for meltdown mitigation and performance it's important to have the pcid
>> flag passed down to the guest (f.e.
>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).
> 
> Indeed; you are still waiting on the qemu patch mentioned here:
> https://www.qemu.org/2018/01/04/spectre/
> 
> which is still undergoing the review process, but should be up (in the
> form of 2.11.1) "in the next few days".

OK thanks as the performance difference is significant:

no pcid:
# time for i in $(seq 1 1 50); do du -sx /; done
...
real0m26.614s
user0m17.548s
sys 0m9.056s


kvm started with +pcid:
# time for i in $(seq 1 1 50); do du -sx /; done
...
real0m14.734s
user0m7.755s
sys 0m6.973s

Greets,
Stefan




[Qemu-devel] how to pass pcid to guest?

2018-01-08 Thread Stefan Priebe - Profihost AG
Hello,

for meltdown mitigation and performance it's important to have the pcid
flag passed down to the guest (f.e.
https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).

My host shows the flag:
# grep ' pcid ' /proc/cpuinfo  | wc -l
56

But the guest does not:
# grep pcid /proc/cpuinfo
#

Guest was started with:
-cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel

Qemu is 2.9.1

Thanks!

Greets,
Stefan



[Qemu-devel] What's needed to pass pciid cpu flag to guest?

2018-01-08 Thread Stefan Priebe - Profihost AG
Hello,

for meltdown mitigation and performance it's important to have the pcid
flag passed down to the guest (f.e.
https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).

My host shows the flag:
# grep ' pcid ' /proc/cpuinfo  | wc -l
56

But the guest does not:
# grep pcid /proc/cpuinfo
#

Guest was started with:
-cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel

Qemu is 2.9.1

Thanks!

Greets,
Stefan



Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches

2018-01-05 Thread Stefan Priebe - Profihost AG
Thanks! But it‘s very difficult to get all opinions all together.

SuSE Enterprise told me to update:
- kernel
- qemu
- Intel microcode

And the released already updates for all of them.

Stefan

Excuse my typo sent from my mobile phone.

> Am 05.01.2018 um 09:33 schrieb Paolo Bonzini :
> 
>> On 04/01/2018 21:15, Stefan Priebe - Profihost AG wrote:
>> attached the relevant patch for everybody who needs it.
> 
> This is the original patch from Intel, which doesn't work unless you
> have a patched kernel (which you almost certainly don't have) and
> doesn't even warn you about that.
> 
> In other words, it's rubbish.  Please read
> https://www.qemu.org/2018/01/04/spectre/ several times, until you
> understand why there is no urgent need to update QEMU.
> 
> Days are 24 hours for QEMU developers just like for you (and believe me,
> we wished several times that they weren't during the last two months).
> We are prioritizing the fixes according to their effect in mitigating
> the vulnerability, their applicability and the availability of patches
> to the lower levels of the stack.  Right now, the most urgent part is
> the simple mitigations that can go in Linux 4.15 and stable kernels.
> 
> Paolo


Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches

2018-01-04 Thread Stefan Priebe - Profihost AG
attached the relevant patch for everybody who needs it.

Greets,
Stefan
Am 04.01.2018 um 16:53 schrieb Paolo Bonzini:
> On 04/01/2018 09:35, Alexandre DERUMIER wrote:
>>>> So you need: 
>>>> 1.) intel / amd cpu microcode update 
>>>> 2.) qemu update to pass the new MSR and CPU flags from the microcode 
>>>> update 
>>>> 3.) host kernel update 
>>>> 4.) guest kernel update 
>>
>> are you sure we need to patch guest kernel if we are able to patch qemu ?
> 
> Patching the guest kernel is only required to protect the guest kernel
> from guest usermode.
> 
>> If I understand, patching the host kernel, should avoid that a vm is reading 
>> memory of another vm.
>> (the most critical)
> 
> Correct.
> 
>> patching the guest kernel, to avoid that a process from the vm have access 
>> to memory of another process of same vm.
> 
> Correct.
> 
> The QEMU updates are pretty boring, mostly taking care of new MSR and
> CPUID flags (and adding new CPU models).
> 
> They are not needed to protect the guest from "Meltdown", only
> "Spectre"---the former only needs a guest kernel update.  Also, to have
> any effect, the guest kernels must also have "Spectre" patches which
> aren't upstream yet for either KVM or the rest of Linux.  So the QEMU
> patches are much less important than the kernel side.
> 
>>> https://access.redhat.com/solutions/3307851 
>>> "Impacts of CVE-2017-5754, CVE-2017-5753, and CVE-2017-5715 to Red Hat 
>>> Virtualization products" 
> 
> It mostly repeats the contents of the RHEL document
> https://access.redhat.com/security/vulnerabilities/speculativeexecution,
> with some information specific to RHV.
> 
> Thanks,
> 
> Paolo
> 
>> i don't have one but the content might be something like this: 
>> https://www.suse.com/de-de/support/kb/doc/?id=7022512 
>>
>> So you need: 
>> 1.) intel / amd cpu microcode update 
>> 2.) qemu update to pass the new MSR and CPU flags from the microcode update 
>> 3.) host kernel update 
>> 4.) guest kernel update 
>>
>> The microcode update and the kernel update is publicly available but i'm 
>> missing the qemu one. 
>>
>> Greets, 
>> Stefan 
>>
>>> - Mail original - 
>>> De: "aderumier"  
>>> À: "Stefan Priebe, Profihost AG"  
>>> Cc: "qemu-devel"  
>>> Envoyé: Jeudi 4 Janvier 2018 08:24:34 
>>> Objet: Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches 
>>>
>>>>> Can anybody point me to the relevant qemu patches? 
>>>
>>> I don't have find them yet. 
>>>
>>> Do you known if a vm using kvm64 cpu model is protected or not ? 
>>>
>>> - Mail original - 
>>> De: "Stefan Priebe, Profihost AG"  
>>> À: "qemu-devel"  
>>> Envoyé: Jeudi 4 Janvier 2018 07:27:01 
>>> Objet: [Qemu-devel] CVE-2017-5715: relevant qemu patches 
>>>
>>> Hello, 
>>>
>>> i've seen some vendors have updated qemu regarding meltdown / spectre. 
>>>
>>> f.e.: 
>>>
>>> CVE-2017-5715: QEMU was updated to allow passing through new MSR and 
>>> CPUID flags from the host VM to the CPU, to allow enabling/disabling 
>>> branch prediction features in the Intel CPU. (bsc#1068032) 
>>>
>>> Can anybody point me to the relevant qemu patches? 
>>>
>>> Thanks! 
>>>
>>> Greets, 
>>> Stefan 
>>>
>>
>>
> 
>From b4fdfeb4545c09a0fdf01edc938f9cce8fcaa5c6 Mon Sep 17 00:00:00 2001
From: Wei Wang 
Date: Tue, 7 Nov 2017 16:39:49 +0800
Subject: [PATCH] i386/kvm: MSR_IA32_SPEC_CTRL and MSR_IA32_PRED_CMD

CPUID(EAX=0X7,ECX=0).EDX[26]/[27] indicates the support of
MSR_IA32_SPEC_CTRL and MSR_IA32_PRED_CMD. Expose the CPUID
to the guest. Also add the support of transferring the MSRs during live
migration.

Signed-off-by: Wei Wang 
[BR: BSC#1068032 CVE-2017-5715]
Signed-off-by: Bruce Rogers 
---
 target/i386/cpu.c |  3 ++-
 target/i386/cpu.h |  4 
 target/i386/kvm.c | 15 ++-
 target/i386/machine.c | 20 
 4 files changed, 40 insertions(+), 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 55f72b679f..01761db3fc 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -2823,13 +2823,14 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
 case 7:
 /* Structured Extended Feature Flags Enumeration Leaf */
 if (count == 0) {
+host_c

Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches

2018-01-04 Thread Stefan Priebe - Profihost AG
Nobody? Is this something they did on their own?

Stefan

Am 04.01.2018 um 07:27 schrieb Stefan Priebe - Profihost AG:
> Hello,
> 
> i've seen some vendors have updated qemu regarding meltdown / spectre.
> 
> f.e.:
> 
>  CVE-2017-5715: QEMU was updated to allow passing through new MSR and
>  CPUID flags from the host VM to the CPU, to allow enabling/disabling
>  branch prediction features in the Intel CPU. (bsc#1068032)
> 
> Can anybody point me to the relevant qemu patches?
> 
> Thanks!
> 
> Greets,
> Stefan
> 



Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches

2018-01-04 Thread Stefan Priebe - Profihost AG
Am 04.01.2018 um 09:35 schrieb Alexandre DERUMIER:
>>> So you need: 
>>> 1.) intel / amd cpu microcode update 
>>> 2.) qemu update to pass the new MSR and CPU flags from the microcode update 
>>> 3.) host kernel update 
>>> 4.) guest kernel update 
> 
> are you sure we need to patch guest kernel if we are able to patch qemu ?
>> I have some pretty old guest (linux and windows)
>
> If I understand, patching the host kernel, should avoid that a vm is reading 
> memory of another vm.
> (the most critical)

Yes - this was just to complete the mitigation on all layers.

> 
> patching the guest kernel, to avoid that a process from the vm have access to 
> memory of another process of same vm.
Yes.

Stefan

> 
> 
> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" 
> À: "aderumier" 
> Cc: "qemu-devel" 
> Envoyé: Jeudi 4 Janvier 2018 09:17:41
> Objet: Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches
> 
> Am 04.01.2018 um 08:27 schrieb Alexandre DERUMIER: 
>> does somebody have a redhat account to see te content of: 
>>
>> https://access.redhat.com/solutions/3307851 
>> "Impacts of CVE-2017-5754, CVE-2017-5753, and CVE-2017-5715 to Red Hat 
>> Virtualization products" 
> 
> i don't have one but the content might be something like this: 
> https://www.suse.com/de-de/support/kb/doc/?id=7022512 
> 
> So you need: 
> 1.) intel / amd cpu microcode update 
> 2.) qemu update to pass the new MSR and CPU flags from the microcode update 
> 3.) host kernel update 
> 4.) guest kernel update 
> 
> The microcode update and the kernel update is publicly available but i'm 
> missing the qemu one. 
> 
> Greets, 
> Stefan 
> 
>> - Mail original - 
>> De: "aderumier"  
>> À: "Stefan Priebe, Profihost AG"  
>> Cc: "qemu-devel"  
>> Envoyé: Jeudi 4 Janvier 2018 08:24:34 
>> Objet: Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches 
>>
>>>> Can anybody point me to the relevant qemu patches? 
>>
>> I don't have find them yet. 
>>
>> Do you known if a vm using kvm64 cpu model is protected or not ? 
>>
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG"  
>> À: "qemu-devel"  
>> Envoyé: Jeudi 4 Janvier 2018 07:27:01 
>> Objet: [Qemu-devel] CVE-2017-5715: relevant qemu patches 
>>
>> Hello, 
>>
>> i've seen some vendors have updated qemu regarding meltdown / spectre. 
>>
>> f.e.: 
>>
>> CVE-2017-5715: QEMU was updated to allow passing through new MSR and 
>> CPUID flags from the host VM to the CPU, to allow enabling/disabling 
>> branch prediction features in the Intel CPU. (bsc#1068032) 
>>
>> Can anybody point me to the relevant qemu patches? 
>>
>> Thanks! 
>>
>> Greets, 
>> Stefan 
>>
> 



Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches

2018-01-04 Thread Stefan Priebe - Profihost AG

Am 04.01.2018 um 08:27 schrieb Alexandre DERUMIER:
> does somebody have a redhat account to see te content of: 
> 
> https://access.redhat.com/solutions/3307851
> "Impacts of CVE-2017-5754, CVE-2017-5753, and CVE-2017-5715 to Red Hat 
> Virtualization products"

i don't have one but the content might be something like this:
https://www.suse.com/de-de/support/kb/doc/?id=7022512

So you need:
1.) intel / amd cpu microcode update
2.) qemu update to pass the new MSR and CPU flags from the microcode update
3.) host kernel update
4.) guest kernel update

The microcode update and the kernel update is publicly available but i'm
missing the qemu one.

Greets,
Stefan

> - Mail original -
> De: "aderumier" 
> À: "Stefan Priebe, Profihost AG" 
> Cc: "qemu-devel" 
> Envoyé: Jeudi 4 Janvier 2018 08:24:34
> Objet: Re: [Qemu-devel] CVE-2017-5715: relevant qemu patches
> 
>>> Can anybody point me to the relevant qemu patches? 
> 
> I don't have find them yet. 
> 
> Do you known if a vm using kvm64 cpu model is protected or not ? 
> 
> - Mail original - 
> De: "Stefan Priebe, Profihost AG"  
> À: "qemu-devel"  
> Envoyé: Jeudi 4 Janvier 2018 07:27:01 
> Objet: [Qemu-devel] CVE-2017-5715: relevant qemu patches 
> 
> Hello, 
> 
> i've seen some vendors have updated qemu regarding meltdown / spectre. 
> 
> f.e.: 
> 
> CVE-2017-5715: QEMU was updated to allow passing through new MSR and 
> CPUID flags from the host VM to the CPU, to allow enabling/disabling 
> branch prediction features in the Intel CPU. (bsc#1068032) 
> 
> Can anybody point me to the relevant qemu patches? 
> 
> Thanks! 
> 
> Greets, 
> Stefan 
> 



[Qemu-devel] CVE-2017-5715: relevant qemu patches

2018-01-03 Thread Stefan Priebe - Profihost AG
Hello,

i've seen some vendors have updated qemu regarding meltdown / spectre.

f.e.:

 CVE-2017-5715: QEMU was updated to allow passing through new MSR and
 CPUID flags from the host VM to the CPU, to allow enabling/disabling
 branch prediction features in the Intel CPU. (bsc#1068032)

Can anybody point me to the relevant qemu patches?

Thanks!

Greets,
Stefan



Re: [Qemu-devel] dropped pkts with Qemu on tap interace (RX)

2018-01-03 Thread Stefan Priebe - Profihost AG
Am 03.01.2018 um 09:14 schrieb Alexandre DERUMIER:
> Hi Stefan,
> 
>>> The tap devices on the target vm shows dropped RX packages on BOTH tap 
>>> interfaces - strangely with the same amount of pkts? 
> 
> that's strange indeed. 
> if you tcpdump tap interfaces, do you see incoming traffic only on 1 
> interface, or both random ?

complete independend random traffic as it should.

> (can you provide the network configuration in the guest for both interfaces ?)

inside the guest? where the drop counter stays 0?

auto eth0
iface eth0 inet dhcp

auto eth1
iface eth1 inet static
address 192.168.0.2
netmask 255.255.255.0

that's it.

> I'm seeing that you have enable multiqueue on 1 of the interfaces, do you 
> have setup correctly the multiqueue part inside the guest.
uh oh? What is needed inside the guest?

> do you have enough vcpu to handle all the queues ?
Yes.

Stefan

> - Mail original -
> De: "Stefan Priebe, Profihost AG" 
> À: "qemu-devel" 
> Envoyé: Mardi 2 Janvier 2018 12:17:29
> Objet: [Qemu-devel] dropped pkts with Qemu on tap interace (RX)
> 
> Hello, 
> 
> currently i'm trying to fix a problem where we have "random" missing 
> packets. 
> 
> We're doing an ssh connect from machine a to machine b every 5 minutes 
> via rsync and ssh. 
> 
> Sometimes it happens that we get this cron message: 
> "Connection to 192.168.0.2 closed by remote host. 
> rsync: connection unexpectedly closed (0 bytes received so far) [sender] 
> rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2] 
> ssh: connect to host 192.168.0.2 port 22: Connection refused" 
> 
> The tap devices on the target vm shows dropped RX packages on BOTH tap 
> interfaces - strangely with the same amount of pkts? 
> 
> # ifconfig tap317i0; ifconfig tap317i1 
> tap317i0 Link encap:Ethernet HWaddr 6e:cb:65:94:bb:bf 
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 
> RX packets:2238445 errors:0 dropped:13159 overruns:0 frame:0 
> TX packets:9655853 errors:0 dropped:0 overruns:0 carrier:0 
> collisions:0 txqueuelen:1000 
> RX bytes:177991267 (169.7 MiB) TX bytes:910412749 (868.2 MiB) 
> 
> tap317i1 Link encap:Ethernet HWaddr 96:f8:b5:d0:9a:07 
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 
> RX packets:1516085 errors:0 dropped:13159 overruns:0 frame:0 
> TX packets:1446964 errors:0 dropped:0 overruns:0 carrier:0 
> collisions:0 txqueuelen:1000 
> RX bytes:1597564313 (1.4 GiB) TX bytes:3517734365 (3.2 GiB) 
> 
> Any ideas how to inspect this issue? 
> 
> Greets, 
> Stefan 
> 



Re: [Qemu-devel] dropped pkts with Qemu on tap interace (RX)

2018-01-03 Thread Stefan Priebe - Profihost AG

Am 03.01.2018 um 04:57 schrieb Wei Xu:
> On Tue, Jan 02, 2018 at 10:17:25PM +0100, Stefan Priebe - Profihost AG wrote:
>>
>> Am 02.01.2018 um 18:04 schrieb Wei Xu:
>>> On Tue, Jan 02, 2018 at 04:24:33PM +0100, Stefan Priebe - Profihost AG 
>>> wrote:
>>>> Hi,
>>>> Am 02.01.2018 um 15:20 schrieb Wei Xu:
>>>>> On Tue, Jan 02, 2018 at 12:17:29PM +0100, Stefan Priebe - Profihost AG 
>>>>> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> currently i'm trying to fix a problem where we have "random" missing
>>>>>> packets.
>>>>>>
>>>>>> We're doing an ssh connect from machine a to machine b every 5 minutes
>>>>>> via rsync and ssh.
>>>>>>
>>>>>> Sometimes it happens that we get this cron message:
>>>>>> "Connection to 192.168.0.2 closed by remote host.
>>>>>> rsync: connection unexpectedly closed (0 bytes received so far) [sender]
>>>>>> rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
>>>>>> ssh: connect to host 192.168.0.2 port 22: Connection refused"
>>>>>
>>>>> Hi Stefan,
>>>>> What kind of virtio-net backend are you using? Can you paste your qemu
>>>>> command line here?
>>>>
>>>> Sure netdev part:
>>>> -netdev
>>>> type=tap,id=net0,ifname=tap317i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on
>>>> -device
>>>> virtio-net-pci,mac=EA:37:42:5C:F3:33,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300
>>>> -netdev
>>>> type=tap,id=net1,ifname=tap317i1,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on,queues=4
>>>> -device
>>>> virtio-net-pci,mac=6A:8E:74:45:1A:0B,nedev=net1,bus=pci.0,addr=0x13,id=net1,vectors=10,mq=on,bootindex=301
>>>
>>> According to what you have mentioned, the traffic is not heavy for the 
>>> guests,
>>> the dropping shouldn't happen for regular case.
>>
>> The avg traffic is around 300kb/s.
>>
>>> What is your hardware platform?
>>
>> Dual Intel Xeon E5-2680 v4
>>
>>> and Which versions are you using for both
>>> guest/host kernel
>> Kernel v4.4.103
>>
>>> and qemu?
>> 2.9.1
>>
>>> Are there other VMs on the same host?
>> Yes.
> 
> What about the CPU load? 

Host:
80-90% Idle
LoadAvg: 6-7

VM:
97%-99% Idle

>>>>> 'Connection refused' usually means that the client gets a TCP Reset rather
>>>>> than losing packets, so this might not be a relevant issue.
>>>>
>>>> Mhm so you mean these might be two seperate ones?
>>>
>>> Yes.
>>>
>>>>
>>>>> Also you can do a tcpdump on both guests and see what happened to SSH 
>>>>> packets
>>>>> (tcpdump -i tapXXX port 22).
>>>>
>>>> Sadly not as there's too much traffic on that part as rsync is syncing
>>>> every 5 minutes through ssh.
>>>
>>> You can do a tcpdump for the entire traffic from the guest and host and 
>>> compare
>>> what kind of packets are dropped if the traffic is not overloaded.
>>
>> Are you sure? I don't get why the same amount and same kind of packets
>> should be received by both tap which are connected to different bridges
>> to different HW and physical interfaces.
> 
> Exactly, possibly this would be a host or guest kernel bug cos than qemu issue
> you are using vhost kernel as the backend and the two stats are independent,
> you might have to check out what is happening inside the traffic.

What do you mean by inside the traffic?

Stefan



Re: [Qemu-devel] dropped pkts with Qemu on tap interace (RX)

2018-01-02 Thread Stefan Priebe - Profihost AG

Am 02.01.2018 um 18:04 schrieb Wei Xu:
> On Tue, Jan 02, 2018 at 04:24:33PM +0100, Stefan Priebe - Profihost AG wrote:
>> Hi,
>> Am 02.01.2018 um 15:20 schrieb Wei Xu:
>>> On Tue, Jan 02, 2018 at 12:17:29PM +0100, Stefan Priebe - Profihost AG 
>>> wrote:
>>>> Hello,
>>>>
>>>> currently i'm trying to fix a problem where we have "random" missing
>>>> packets.
>>>>
>>>> We're doing an ssh connect from machine a to machine b every 5 minutes
>>>> via rsync and ssh.
>>>>
>>>> Sometimes it happens that we get this cron message:
>>>> "Connection to 192.168.0.2 closed by remote host.
>>>> rsync: connection unexpectedly closed (0 bytes received so far) [sender]
>>>> rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
>>>> ssh: connect to host 192.168.0.2 port 22: Connection refused"
>>>
>>> Hi Stefan,
>>> What kind of virtio-net backend are you using? Can you paste your qemu
>>> command line here?
>>
>> Sure netdev part:
>> -netdev
>> type=tap,id=net0,ifname=tap317i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on
>> -device
>> virtio-net-pci,mac=EA:37:42:5C:F3:33,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300
>> -netdev
>> type=tap,id=net1,ifname=tap317i1,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on,queues=4
>> -device
>> virtio-net-pci,mac=6A:8E:74:45:1A:0B,nedev=net1,bus=pci.0,addr=0x13,id=net1,vectors=10,mq=on,bootindex=301
> 
> According to what you have mentioned, the traffic is not heavy for the guests,
> the dropping shouldn't happen for regular case.

The avg traffic is around 300kb/s.

> What is your hardware platform?

Dual Intel Xeon E5-2680 v4

> and Which versions are you using for both
> guest/host kernel
Kernel v4.4.103

> and qemu?
2.9.1

> Are there other VMs on the same host?
Yes.


>>> 'Connection refused' usually means that the client gets a TCP Reset rather
>>> than losing packets, so this might not be a relevant issue.
>>
>> Mhm so you mean these might be two seperate ones?
> 
> Yes.
> 
>>
>>> Also you can do a tcpdump on both guests and see what happened to SSH 
>>> packets
>>> (tcpdump -i tapXXX port 22).
>>
>> Sadly not as there's too much traffic on that part as rsync is syncing
>> every 5 minutes through ssh.
> 
> You can do a tcpdump for the entire traffic from the guest and host and 
> compare
> what kind of packets are dropped if the traffic is not overloaded.

Are you sure? I don't get why the same amount and same kind of packets
should be received by both tap which are connected to different bridges
to different HW and physical interfaces.

Stefan

> Wei
> 
>>
>>>> The tap devices on the target vm shows dropped RX packages on BOTH tap
>>>> interfaces - strangely with the same amount of pkts?
>>>>
>>>> # ifconfig tap317i0; ifconfig tap317i1
>>>> tap317i0  Link encap:Ethernet  HWaddr 6e:cb:65:94:bb:bf
>>>>   UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>>>>   RX packets:2238445 errors:0 dropped:13159 overruns:0 frame:0
>>>>   TX packets:9655853 errors:0 dropped:0 overruns:0 carrier:0
>>>>   collisions:0 txqueuelen:1000
>>>>   RX bytes:177991267 (169.7 MiB)  TX bytes:910412749 (868.2 MiB)
>>>>
>>>> tap317i1  Link encap:Ethernet  HWaddr 96:f8:b5:d0:9a:07
>>>>   UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>>>>   RX packets:1516085 errors:0 dropped:13159 overruns:0 frame:0
>>>>   TX packets:1446964 errors:0 dropped:0 overruns:0 carrier:0
>>>>   collisions:0 txqueuelen:1000
>>>>   RX bytes:1597564313 (1.4 GiB)  TX bytes:3517734365 (3.2 GiB)
>>>>
>>>> Any ideas how to inspect this issue?
>>>
>>> It seems both tap interfaces lose RX pkts, dropping pkts of RX means the
>>> host(backend) cann't receive packets from the guest as fast as the guest 
>>> sends.
>>
>> Inside the guest i see no dropped packets at all. It's only on the host
>> and strangely on both taps at the same value? And both are connected to
>> absolutely different networks.
>>
>>> Are you running some symmetrical test on both guests? 
>>
>> No.
>>
>> Stefan
>>
>>
>>> Wei
>>>
>>>>
>>>> Greets,
>>>> Stefan
>>>>



Re: [Qemu-devel] dropped pkts with Qemu on tap interace (RX)

2018-01-02 Thread Stefan Priebe - Profihost AG
Hi,
Am 02.01.2018 um 15:20 schrieb Wei Xu:
> On Tue, Jan 02, 2018 at 12:17:29PM +0100, Stefan Priebe - Profihost AG wrote:
>> Hello,
>>
>> currently i'm trying to fix a problem where we have "random" missing
>> packets.
>>
>> We're doing an ssh connect from machine a to machine b every 5 minutes
>> via rsync and ssh.
>>
>> Sometimes it happens that we get this cron message:
>> "Connection to 192.168.0.2 closed by remote host.
>> rsync: connection unexpectedly closed (0 bytes received so far) [sender]
>> rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
>> ssh: connect to host 192.168.0.2 port 22: Connection refused"
> 
> Hi Stefan,
> What kind of virtio-net backend are you using? Can you paste your qemu
> command line here?

Sure netdev part:
-netdev
type=tap,id=net0,ifname=tap317i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on
-device
virtio-net-pci,mac=EA:37:42:5C:F3:33,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300
-netdev
type=tap,id=net1,ifname=tap317i1,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on,queues=4
-device
virtio-net-pci,mac=6A:8E:74:45:1A:0B,nedev=net1,bus=pci.0,addr=0x13,id=net1,vectors=10,mq=on,bootindex=301


> 'Connection refused' usually means that the client gets a TCP Reset rather
> than losing packets, so this might not be a relevant issue.

Mhm so you mean these might be two seperate ones?

> Also you can do a tcpdump on both guests and see what happened to SSH packets
> (tcpdump -i tapXXX port 22).

Sadly not as there's too much traffic on that part as rsync is syncing
every 5 minutes through ssh.

>> The tap devices on the target vm shows dropped RX packages on BOTH tap
>> interfaces - strangely with the same amount of pkts?
>>
>> # ifconfig tap317i0; ifconfig tap317i1
>> tap317i0  Link encap:Ethernet  HWaddr 6e:cb:65:94:bb:bf
>>   UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>>   RX packets:2238445 errors:0 dropped:13159 overruns:0 frame:0
>>   TX packets:9655853 errors:0 dropped:0 overruns:0 carrier:0
>>   collisions:0 txqueuelen:1000
>>   RX bytes:177991267 (169.7 MiB)  TX bytes:910412749 (868.2 MiB)
>>
>> tap317i1  Link encap:Ethernet  HWaddr 96:f8:b5:d0:9a:07
>>   UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>>   RX packets:1516085 errors:0 dropped:13159 overruns:0 frame:0
>>   TX packets:1446964 errors:0 dropped:0 overruns:0 carrier:0
>>   collisions:0 txqueuelen:1000
>>   RX bytes:1597564313 (1.4 GiB)  TX bytes:3517734365 (3.2 GiB)
>>
>> Any ideas how to inspect this issue?
> 
> It seems both tap interfaces lose RX pkts, dropping pkts of RX means the
> host(backend) cann't receive packets from the guest as fast as the guest 
> sends.

Inside the guest i see no dropped packets at all. It's only on the host
and strangely on both taps at the same value? And both are connected to
absolutely different networks.

> Are you running some symmetrical test on both guests? 

No.

Stefan


> Wei
> 
>>
>> Greets,
>> Stefan
>>



[Qemu-devel] dropped pkts with Qemu on tap interace (RX)

2018-01-02 Thread Stefan Priebe - Profihost AG
Hello,

currently i'm trying to fix a problem where we have "random" missing
packets.

We're doing an ssh connect from machine a to machine b every 5 minutes
via rsync and ssh.

Sometimes it happens that we get this cron message:
"Connection to 192.168.0.2 closed by remote host.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
ssh: connect to host 192.168.0.2 port 22: Connection refused"

The tap devices on the target vm shows dropped RX packages on BOTH tap
interfaces - strangely with the same amount of pkts?

# ifconfig tap317i0; ifconfig tap317i1
tap317i0  Link encap:Ethernet  HWaddr 6e:cb:65:94:bb:bf
  UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
  RX packets:2238445 errors:0 dropped:13159 overruns:0 frame:0
  TX packets:9655853 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:177991267 (169.7 MiB)  TX bytes:910412749 (868.2 MiB)

tap317i1  Link encap:Ethernet  HWaddr 96:f8:b5:d0:9a:07
  UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
  RX packets:1516085 errors:0 dropped:13159 overruns:0 frame:0
  TX packets:1446964 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:1597564313 (1.4 GiB)  TX bytes:3517734365 (3.2 GiB)

Any ideas how to inspect this issue?

Greets,
Stefan



Re: [Qemu-devel] kvm: virtio-net: saved image requires TUN_F_UFO support

2017-11-22 Thread Stefan Priebe - Profihost AG
Hello,

Am 22.11.2017 um 20:41 schrieb Dr. David Alan Gilbert:
> * Paolo Bonzini (pbonz...@redhat.com) wrote:
>> On 06/11/2017 12:09, Stefan Priebe - Profihost AG wrote:
>>> HI Paolo,
>>>
>>> could this patchset be related?
>>
>> Uh oh, yes it should.  Jason, any ways to fix it?  I suppose we need to
>> disable UFO in the newest machine types, but do we also have to do
>> (software) UFO in vhost-net and QEMU for migration compatibility?
> 
> Was there a solution to this?

it will be this one:
https://patchwork.ozlabs.org/patch/840094/


Stefan

> Dave
> 
>> Thanks,
>>
>> Paolo
>>
>>> Greets,
>>> Stefan
>>>
>>> Am 06.11.2017 um 10:52 schrieb Stefan Priebe - Profihost AG:
>>>> Hi Paolo,
>>>>
>>>> Am 06.11.2017 um 10:49 schrieb Paolo Bonzini:
>>>>> On 06/11/2017 10:48, Stefan Priebe - Profihost AG wrote:
>>>>>> Hi Paolo,
>>>>>>
>>>>>> Am 06.11.2017 um 10:40 schrieb Paolo Bonzini:
>>>>>>> On 06/11/2017 10:38, Stefan Priebe - Profihost AG wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> i've upgraded some servers from kernel 4.4 to 4.12 - both running Qemu
>>>>>>>> 2.9.1.
>>>>>>>>
>>>>>>>> If i migrate a VM from a host running kernel 4.4 to a host running 4.12
>>>>>>>> i get:
>>>>>>>>
>>>>>>>> kvm: virtio-net: saved image requires TUN_F_UFO support
>>>>>>>> kvm: Failed to load virtio-net-device:tmp
>>>>>>>> kvm: Failed to load virtio-net:virtio
>>>>>>>> kvm: error while loading state for instance 0x0 of device
>>>>>>>> ':00:12.0/virtio-net'
>>>>>>>> kvm: load of migration failed: Invalid argument
>>>>>>>>
>>>>>>>>
>>>>>>>> while migrating from 4.12 to 4.4 works fine.
>>>>>>>>
>>>>>>>> Can anybody help? Is this expected?
>>>>>>>
>>>>>>> Can you check why peer_has_ufo failed (in hw/net/virtio-net.c)?
>>>>>>
>>>>>> May be - how can i archieve this? Patching the code is not a problem if
>>>>>> you can give me a hint.
>>>>>>
>>>>>>> Also, did this ioctl fail when the tap device was set up on the 4.12 
>>>>>>> destination?
>>>>>>> int tap_probe_has_ufo(int fd)
>>>>>>> {
>>>>>>> unsigned offload;
>>>>>>>
>>>>>>> offload = TUN_F_CSUM | TUN_F_UFO;
>>>>>>>
>>>>>>> if (ioctl(fd, TUNSETOFFLOAD, offload) < 0)
>>>>>>> return 0;
>>>>>>>
>>>>>>> return 1;
>>>>>>> }
>>>>>>
>>>>>> Should there be any kernel output or how can i detect / check it?
>>>>>
>>>>> For both, the simplest answer is probably just using printf.
>>>>
>>>> arg i missed an important part. The kernel is an opensuse SLE15 one.
>>>>
>>>> I've seen it contains the following patchset:
>>>> https://www.spinics.net/lists/netdev/msg443821.html
>>>>
>>>> Greets,
>>>> Stefan
>>>>
>>
>>
> --
> Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
> 



[Qemu-devel] kvm: Failed to flush the L2 table cache: Input/output error

2017-11-20 Thread Stefan Priebe - Profihost AG
Hello,

while using qemu 2.9.1 and doing a backup of a disk:

I have sometimes the following output:

Formatting '/mnt/qemu-249-2017_11_19-04_00_05.qcow2', fmt=qcow2
size=236223201280 encryption=off cluster_size=65536 lazy_refcounts=off
refcount_bits=16


followed by:

kvm: Failed to flush the L2 table cache: Input/output error
kvm: Failed to flush the refcount block cache: Input/output error

If this error happens (Failed to flush) the whole backup is incomplete.
Host kernel is a 4.4 based openSuSE 42.3 kernel.

Is this a known bug?

Greets,
Stefan



Re: [Qemu-devel] kvm: virtio-net: saved image requires TUN_F_UFO support

2017-11-11 Thread Stefan Priebe - Profihost AG
Hello,

Am 10.11.2017 um 05:18 schrieb Jason Wang:
> 
> 
> On 2017年11月08日 19:22, Jason Wang wrote:
>>
>>
>> On 2017年11月08日 18:46, Paolo Bonzini wrote:
>>> On 08/11/2017 09:21, Jason Wang wrote:
>>>>
>>>> On 2017年11月08日 17:05, Stefan Priebe - Profihost AG wrote:
>>>>> Am 08.11.2017 um 08:54 schrieb Jason Wang:
>>>>>> On 2017年11月08日 15:41, Stefan Priebe - Profihost AG wrote:
>>>>>>> Hi Paolo,
>>>>>>>
>>>>>>> Am 06.11.2017 um 12:27 schrieb Paolo Bonzini:
>>>>>>>> On 06/11/2017 12:09, Stefan Priebe - Profihost AG wrote:
>>>>>>>>> HI Paolo,
>>>>>>>>>
>>>>>>>>> could this patchset be related?
>>>>>>>> Uh oh, yes it should.  Jason, any ways to fix it?  I suppose we
>>>>>>>> need to
>>>>>>>> disable UFO in the newest machine types, but do we also have to do
>>>>>>>> (software) UFO in vhost-net and QEMU for migration compatibility?
>>>>>>> Any news on this?
>>>>>>>
>>>>>>> Greets,
>>>>>> Since we probe UFO support, it will be disabled automatically on
>>>>>> device
>>>>>> startup.
>>>>>>
>>>>>> For the issue of migration, I think the only way is trying to fix
>>>>>> it in
>>>>>> kernel.
>>>>> But 4.14 is around the corner and the patchset is already included.
>>>>> Shouldn't this get fixed before 4.14 is released?
>>>> We will try to seek a solution as soon as possible. If we can't catch
>>>> 4.14, we will do it for stable for sure.
>>> Jason, can you write to netdev and Cc Linus and me?
>>>
>>> Thanks,
>>>
>>> Paolo
>>
>> Paolo, see this https://www.spinics.net/lists/netdev/msg465454.html
>>
>> Just notice this today since I'm not on the cc list.
>>
>> Thanks
>>
>>
> 
> An update:
> 
> After some discussions on netdev, we will try to bring UFO back
> partially for just TAP. Willem promise to fix this. 4.14 is too late
> consider the fix is not trivial, it will go through stable tree.

OK is it save to just revert the UFO patchset for my local branch?

Greets,
Stefan

> 
> Thanks



Re: [Qemu-devel] kvm: virtio-net: saved image requires TUN_F_UFO support

2017-11-08 Thread Stefan Priebe - Profihost AG

Am 08.11.2017 um 08:54 schrieb Jason Wang:
> 
> 
> On 2017年11月08日 15:41, Stefan Priebe - Profihost AG wrote:
>> Hi Paolo,
>>
>> Am 06.11.2017 um 12:27 schrieb Paolo Bonzini:
>>> On 06/11/2017 12:09, Stefan Priebe - Profihost AG wrote:
>>>> HI Paolo,
>>>>
>>>> could this patchset be related?
>>> Uh oh, yes it should.  Jason, any ways to fix it?  I suppose we need to
>>> disable UFO in the newest machine types, but do we also have to do
>>> (software) UFO in vhost-net and QEMU for migration compatibility?
>> Any news on this?
>>
>> Greets,
> 
> Since we probe UFO support, it will be disabled automatically on device
> startup.
> 
> For the issue of migration, I think the only way is trying to fix it in
> kernel.

But 4.14 is around the corner and the patchset is already included.
Shouldn't this get fixed before 4.14 is released?

Stefan
> 
> Thanks



Re: [Qemu-devel] kvm: virtio-net: saved image requires TUN_F_UFO support

2017-11-07 Thread Stefan Priebe - Profihost AG
Hi Paolo,

Am 06.11.2017 um 12:27 schrieb Paolo Bonzini:
> On 06/11/2017 12:09, Stefan Priebe - Profihost AG wrote:
>> HI Paolo,
>>
>> could this patchset be related?
> 
> Uh oh, yes it should.  Jason, any ways to fix it?  I suppose we need to
> disable UFO in the newest machine types, but do we also have to do
> (software) UFO in vhost-net and QEMU for migration compatibility?

Any news on this?

Greets,
Stefan


> Thanks,
> 
> Paolo
> 
>> Greets,
>> Stefan
>>
>> Am 06.11.2017 um 10:52 schrieb Stefan Priebe - Profihost AG:
>>> Hi Paolo,
>>>
>>> Am 06.11.2017 um 10:49 schrieb Paolo Bonzini:
>>>> On 06/11/2017 10:48, Stefan Priebe - Profihost AG wrote:
>>>>> Hi Paolo,
>>>>>
>>>>> Am 06.11.2017 um 10:40 schrieb Paolo Bonzini:
>>>>>> On 06/11/2017 10:38, Stefan Priebe - Profihost AG wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> i've upgraded some servers from kernel 4.4 to 4.12 - both running Qemu
>>>>>>> 2.9.1.
>>>>>>>
>>>>>>> If i migrate a VM from a host running kernel 4.4 to a host running 4.12
>>>>>>> i get:
>>>>>>>
>>>>>>> kvm: virtio-net: saved image requires TUN_F_UFO support
>>>>>>> kvm: Failed to load virtio-net-device:tmp
>>>>>>> kvm: Failed to load virtio-net:virtio
>>>>>>> kvm: error while loading state for instance 0x0 of device
>>>>>>> ':00:12.0/virtio-net'
>>>>>>> kvm: load of migration failed: Invalid argument
>>>>>>>
>>>>>>>
>>>>>>> while migrating from 4.12 to 4.4 works fine.
>>>>>>>
>>>>>>> Can anybody help? Is this expected?
>>>>>>
>>>>>> Can you check why peer_has_ufo failed (in hw/net/virtio-net.c)?
>>>>>
>>>>> May be - how can i archieve this? Patching the code is not a problem if
>>>>> you can give me a hint.
>>>>>
>>>>>> Also, did this ioctl fail when the tap device was set up on the 4.12 
>>>>>> destination?
>>>>>> int tap_probe_has_ufo(int fd)
>>>>>> {
>>>>>> unsigned offload;
>>>>>>
>>>>>> offload = TUN_F_CSUM | TUN_F_UFO;
>>>>>>
>>>>>> if (ioctl(fd, TUNSETOFFLOAD, offload) < 0)
>>>>>> return 0;
>>>>>>
>>>>>> return 1;
>>>>>> }
>>>>>
>>>>> Should there be any kernel output or how can i detect / check it?
>>>>
>>>> For both, the simplest answer is probably just using printf.
>>>
>>> arg i missed an important part. The kernel is an opensuse SLE15 one.
>>>
>>> I've seen it contains the following patchset:
>>> https://www.spinics.net/lists/netdev/msg443821.html
>>>
>>> Greets,
>>> Stefan
>>>
> 



Re: [Qemu-devel] kvm: virtio-net: saved image requires TUN_F_UFO support

2017-11-06 Thread Stefan Priebe - Profihost AG
HI Paolo,

could this patchset be related?

Greets,
Stefan

Am 06.11.2017 um 10:52 schrieb Stefan Priebe - Profihost AG:
> Hi Paolo,
> 
> Am 06.11.2017 um 10:49 schrieb Paolo Bonzini:
>> On 06/11/2017 10:48, Stefan Priebe - Profihost AG wrote:
>>> Hi Paolo,
>>>
>>> Am 06.11.2017 um 10:40 schrieb Paolo Bonzini:
>>>> On 06/11/2017 10:38, Stefan Priebe - Profihost AG wrote:
>>>>> Hello,
>>>>>
>>>>> i've upgraded some servers from kernel 4.4 to 4.12 - both running Qemu
>>>>> 2.9.1.
>>>>>
>>>>> If i migrate a VM from a host running kernel 4.4 to a host running 4.12
>>>>> i get:
>>>>>
>>>>> kvm: virtio-net: saved image requires TUN_F_UFO support
>>>>> kvm: Failed to load virtio-net-device:tmp
>>>>> kvm: Failed to load virtio-net:virtio
>>>>> kvm: error while loading state for instance 0x0 of device
>>>>> ':00:12.0/virtio-net'
>>>>> kvm: load of migration failed: Invalid argument
>>>>>
>>>>>
>>>>> while migrating from 4.12 to 4.4 works fine.
>>>>>
>>>>> Can anybody help? Is this expected?
>>>>
>>>> Can you check why peer_has_ufo failed (in hw/net/virtio-net.c)?
>>>
>>> May be - how can i archieve this? Patching the code is not a problem if
>>> you can give me a hint.
>>>
>>>> Also, did this ioctl fail when the tap device was set up on the 4.12 
>>>> destination?
>>>> int tap_probe_has_ufo(int fd)
>>>> {
>>>> unsigned offload;
>>>>
>>>> offload = TUN_F_CSUM | TUN_F_UFO;
>>>>
>>>> if (ioctl(fd, TUNSETOFFLOAD, offload) < 0)
>>>> return 0;
>>>>
>>>> return 1;
>>>> }
>>>
>>> Should there be any kernel output or how can i detect / check it?
>>
>> For both, the simplest answer is probably just using printf.
> 
> arg i missed an important part. The kernel is an opensuse SLE15 one.
> 
> I've seen it contains the following patchset:
> https://www.spinics.net/lists/netdev/msg443821.html
> 
> Greets,
> Stefan
> 



Re: [Qemu-devel] kvm: virtio-net: saved image requires TUN_F_UFO support

2017-11-06 Thread Stefan Priebe - Profihost AG
Hi Paolo,

Am 06.11.2017 um 10:49 schrieb Paolo Bonzini:
> On 06/11/2017 10:48, Stefan Priebe - Profihost AG wrote:
>> Hi Paolo,
>>
>> Am 06.11.2017 um 10:40 schrieb Paolo Bonzini:
>>> On 06/11/2017 10:38, Stefan Priebe - Profihost AG wrote:
>>>> Hello,
>>>>
>>>> i've upgraded some servers from kernel 4.4 to 4.12 - both running Qemu
>>>> 2.9.1.
>>>>
>>>> If i migrate a VM from a host running kernel 4.4 to a host running 4.12
>>>> i get:
>>>>
>>>> kvm: virtio-net: saved image requires TUN_F_UFO support
>>>> kvm: Failed to load virtio-net-device:tmp
>>>> kvm: Failed to load virtio-net:virtio
>>>> kvm: error while loading state for instance 0x0 of device
>>>> ':00:12.0/virtio-net'
>>>> kvm: load of migration failed: Invalid argument
>>>>
>>>>
>>>> while migrating from 4.12 to 4.4 works fine.
>>>>
>>>> Can anybody help? Is this expected?
>>>
>>> Can you check why peer_has_ufo failed (in hw/net/virtio-net.c)?
>>
>> May be - how can i archieve this? Patching the code is not a problem if
>> you can give me a hint.
>>
>>> Also, did this ioctl fail when the tap device was set up on the 4.12 
>>> destination?
>>> int tap_probe_has_ufo(int fd)
>>> {
>>> unsigned offload;
>>>
>>> offload = TUN_F_CSUM | TUN_F_UFO;
>>>
>>> if (ioctl(fd, TUNSETOFFLOAD, offload) < 0)
>>> return 0;
>>>
>>> return 1;
>>> }
>>
>> Should there be any kernel output or how can i detect / check it?
> 
> For both, the simplest answer is probably just using printf.

arg i missed an important part. The kernel is an opensuse SLE15 one.

I've seen it contains the following patchset:
https://www.spinics.net/lists/netdev/msg443821.html

Greets,
Stefan



Re: [Qemu-devel] kvm: virtio-net: saved image requires TUN_F_UFO support

2017-11-06 Thread Stefan Priebe - Profihost AG
Hi Paolo,

Am 06.11.2017 um 10:40 schrieb Paolo Bonzini:
> On 06/11/2017 10:38, Stefan Priebe - Profihost AG wrote:
>> Hello,
>>
>> i've upgraded some servers from kernel 4.4 to 4.12 - both running Qemu
>> 2.9.1.
>>
>> If i migrate a VM from a host running kernel 4.4 to a host running 4.12
>> i get:
>>
>> kvm: virtio-net: saved image requires TUN_F_UFO support
>> kvm: Failed to load virtio-net-device:tmp
>> kvm: Failed to load virtio-net:virtio
>> kvm: error while loading state for instance 0x0 of device
>> ':00:12.0/virtio-net'
>> kvm: load of migration failed: Invalid argument
>>
>>
>> while migrating from 4.12 to 4.4 works fine.
>>
>> Can anybody help? Is this expected?
> 
> Can you check why peer_has_ufo failed (in hw/net/virtio-net.c)?

May be - how can i archieve this? Patching the code is not a problem if
you can give me a hint.

> Also, did this ioctl fail when the tap device was set up on the 4.12 
> destination?
> int tap_probe_has_ufo(int fd)
> {
> unsigned offload;
> 
> offload = TUN_F_CSUM | TUN_F_UFO;
> 
> if (ioctl(fd, TUNSETOFFLOAD, offload) < 0)
> return 0;
> 
> return 1;
> }

Should there be any kernel output or how can i detect / check it?

Greets,
Stefan

> Thanks,
> 
> Paolo
> 



[Qemu-devel] kvm: virtio-net: saved image requires TUN_F_UFO support

2017-11-06 Thread Stefan Priebe - Profihost AG
Hello,

i've upgraded some servers from kernel 4.4 to 4.12 - both running Qemu
2.9.1.

If i migrate a VM from a host running kernel 4.4 to a host running 4.12
i get:

kvm: virtio-net: saved image requires TUN_F_UFO support
kvm: Failed to load virtio-net-device:tmp
kvm: Failed to load virtio-net:virtio
kvm: error while loading state for instance 0x0 of device
':00:12.0/virtio-net'
kvm: load of migration failed: Invalid argument


while migrating from 4.12 to 4.4 works fine.

Can anybody help? Is this expected?

Greets,
Stefan



Re: [Qemu-devel] virtio: network: lost tcp/ip packets

2017-08-30 Thread Stefan Priebe - Profihost AG
Hello Stefan,


Am 30.08.2017 um 19:17 schrieb Stefan Hajnoczi:
> On Fri, Aug 18, 2017 at 04:40:36PM +0200, Stefan Priebe - Profihost AG wrote:
>> i've a problem with two VMs running on the SAME host machine using qemu
>> 2.7.1 or 2.9.0 and vhost_net + virtio.
>>
>> Sometimes TCP packets going from machine a to machine b are simply lost.
>> I see them in VM A using tcpdump going out but they never come in on
>> machine B. Both machines have iptables and stuff turned off.
>>
>> On the host both VMs network interfaces are connected to the same bridge.
>>
>> Any ideas? May be a known bug?
>>
>> Host and Guest Kernel is an OpenSuSE 42.3 kernel based on vanilla 4.4.82.
> 
> Have you tcpdumped the tap interfaces on the host?
> 
currently i can't reproduce the issue - even i was for two days.
Currently i've no idea what has happend and will report back if it
happens again.

Thanks!

Greets,
Stefan



Re: [Qemu-devel] virtio: network: lost tcp/ip packets

2017-08-21 Thread Stefan Priebe - Profihost AG
Hello,

does nobody have an idea?

Greets,
Stefam

Am 18.08.2017 um 16:40 schrieb Stefan Priebe - Profihost AG:
> Hello,
> 
> i've a problem with two VMs running on the SAME host machine using qemu
> 2.7.1 or 2.9.0 and vhost_net + virtio.
> 
> Sometimes TCP packets going from machine a to machine b are simply lost.
> I see them in VM A using tcpdump going out but they never come in on
> machine B. Both machines have iptables and stuff turned off.
> 
> On the host both VMs network interfaces are connected to the same bridge.
> 
> Any ideas? May be a known bug?
> 
> Host and Guest Kernel is an OpenSuSE 42.3 kernel based on vanilla 4.4.82.
> 
> Thanks!
> 
> Greets,
> Stefan
> 



[Qemu-devel] virtio: network: lost tcp/ip packets

2017-08-18 Thread Stefan Priebe - Profihost AG
Hello,

i've a problem with two VMs running on the SAME host machine using qemu
2.7.1 or 2.9.0 and vhost_net + virtio.

Sometimes TCP packets going from machine a to machine b are simply lost.
I see them in VM A using tcpdump going out but they never come in on
machine B. Both machines have iptables and stuff turned off.

On the host both VMs network interfaces are connected to the same bridge.

Any ideas? May be a known bug?

Host and Guest Kernel is an OpenSuSE 42.3 kernel based on vanilla 4.4.82.

Thanks!

Greets,
Stefan



Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

2016-12-19 Thread Stefan Priebe - Profihost AG

Am 19.12.2016 um 12:03 schrieb Stefan Hajnoczi:
> On Fri, Dec 16, 2016 at 10:00:36PM +0100, Stefan Priebe - Profihost AG wrote:
>>
>> Am 15.12.2016 um 07:46 schrieb Alexandre DERUMIER:
>>> does rollbacking the kernel to previous version fix the problem ?
>>
>> The culprit is the used tuned agent from Redhat
>> (https://github.com/redhat-performance/tuned). The used profile
>> virtual-host results in these problems. Stopping tuned or using another
>> profile like throughput-performance everything is fine again.
> 
> Interesting discovery.  Have you filed a bug report about it?

As we're not using RHEL, nor CentOS i didn't expect this to make sense
;-) I just believed that something in the kernel changed.

Greets,
Stefan

>> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and
>> volatile network speeds inside my VMs.
>>
>> Currently I've no idea what causes this but it's related to the host
>> upgrades. Before i was running Qemu 2.6.2.
>>
>> I'm using virtio for the network cards.
> 
> Stefan
> 



Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

2016-12-16 Thread Stefan Priebe - Profihost AG

Am 15.12.2016 um 07:46 schrieb Alexandre DERUMIER:
> does rollbacking the kernel to previous version fix the problem ?

The culprit is the used tuned agent from Redhat
(https://github.com/redhat-performance/tuned). The used profile
virtual-host results in these problems. Stopping tuned or using another
profile like throughput-performance everything is fine again.

Geets,
Stefan

> 
> i'm not sure if "perf" could give you some hints
> - Mail original -
> De: "Stefan Priebe, Profihost AG" 
> À: "aderumier" 
> Cc: "qemu-devel" 
> Envoyé: Mercredi 14 Décembre 2016 21:36:23
> Objet: Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?
> 
> Am 14.12.2016 um 16:33 schrieb Alexandre DERUMIER: 
>> Hi Stefan, 
>>
>> do you have upgraded kernel ? 
> 
> Yes sure. But i'm out of ideas how to debug. Sometimes it gives me 
> constant 80MB/s, sometimes 125 and sometimes only 6. While on the host 
> the cards are not busy. 
> 
> Greets, 
> Stefan 
> 
>>
>> maybe it could be related to vhost-net module too. 
>>
>>
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG"  
>> À: "qemu-devel"  
>> Envoyé: Mercredi 14 Décembre 2016 16:04:08 
>> Objet: [Qemu-devel] any known virtio-net regressions in Qemu 2.7? 
>>
>> Hello, 
>>
>> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and 
>> volatile network speeds inside my VMs. 
>>
>> Currently I've no idea what causes this but it's related to the host 
>> upgrades. Before i was running Qemu 2.6.2. 
>>
>> I'm using virtio for the network cards. 
>>
>> Greets, 
>> Stefan 
>>
> 



Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

2016-12-14 Thread Stefan Priebe - Profihost AG
Am 14.12.2016 um 16:33 schrieb Alexandre DERUMIER:
> Hi Stefan,
> 
> do you have upgraded kernel ?

Yes sure. But i'm out of ideas how to debug. Sometimes it gives me
constant 80MB/s, sometimes 125 and sometimes only 6. While on the host
the cards are not busy.

Greets,
Stefan

> 
> maybe it could be related to vhost-net module too.
> 
> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" 
> À: "qemu-devel" 
> Envoyé: Mercredi 14 Décembre 2016 16:04:08
> Objet: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?
> 
> Hello, 
> 
> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and 
> volatile network speeds inside my VMs. 
> 
> Currently I've no idea what causes this but it's related to the host 
> upgrades. Before i was running Qemu 2.6.2. 
> 
> I'm using virtio for the network cards. 
> 
> Greets, 
> Stefan 
> 



[Qemu-devel] any known virtio-net regressions in Qemu 2.7?

2016-12-14 Thread Stefan Priebe - Profihost AG
Hello,

after upgrading a cluster OS, Qemu, ... i'm experiencing slow and
volatile network speeds inside my VMs.

Currently I've no idea what causes this but it's related to the host
upgrades. Before i was running Qemu 2.6.2.

I'm using virtio for the network cards.

Greets,
Stefan



Re: [Qemu-devel] Qemu 2.6 => Qemu 2.7 migration: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-balloon'

2016-11-15 Thread Stefan Priebe - Profihost AG
Am 15.11.2016 um 12:07 schrieb Ladi Prosek:
> Hi,
> 
> On Tue, Nov 15, 2016 at 11:37 AM, Stefan Priebe - Profihost AG
>  wrote:
>> Hello,
>>
>> Am 15.11.2016 um 11:30 schrieb Dr. David Alan Gilbert:
>>> * Stefan Priebe - Profihost AG (s.pri...@profihost.ag) wrote:
>>>> Hello,
>>>>
>>>> today i did a first live migration from Qemu 2.6.2 to Qemu 2.7.0. The VM
>>>> is running windows and virtio-balloon and with machine type pc-i440fx-2.5.
>>>>
>>>> The output of the target qemu process was:
>>>> kvm_apic_post_load: Yeh
>>>> kvm_apic_post_load: Yeh
>>>> kvm_apic_post_load: Yeh
>>>> kvm_apic_post_load: Yeh
>>>> kvm: VQ 2 size 0x80 < last_avail_idx 0x1 - used_idx 0x4
>>>> kvm: error while loading state for instance 0x0 of device
>>>> ':00:03.0/virtio-balloon'
>>>> kvm: load of migration failed: Operation not permitted
>>>
>>> Yes that's a known bug; only seems to affect windows guests, and I believe
>>> doesn't even need to cross versions.
>>>
>>> There's a bunch of fixes that Stefan applied to virtio code
>>> that I think fix this; I see that he cc'd qemu-stable.
>>> I think it's 4b7f91ed, but I'm not sure if there are others needed.
>>
>> thanks for pointing to that commit.
>>
>> Stefan can you tell me whether it's enough to cherry-pick 4b7f91ed into
>> 2.7.0 ?
> 
> I don't believe that 4b7f91ed will help here (no device reset on
> migration). We've seen this error with QEMU running without:
> 
>   commit 4eae2a657d1ff5ada56eb9b4966eae0eff333b0b
>   Author: Ladi Prosek 
>   Date:   Tue Mar 1 12:14:03 2016 +0100
> 
>   balloon: fix segfault and harden the stats queue
> 
> 
> Is it possible that the VM has run on such a QEMU, then was
> live-migrated to 2.6.2, and then to 2.7.0?

Hi,

yes, it was started under Qemu 2.5.0. Was then migrated to 2.6.2 and
then to 2.7.0.

Greets,
Stefan

> 
> Thanks,
> Ladi
> 
>> Greets,
>> Stefan
>>
>>>
>>> Dave
>>>>
>>>> Greets,
>>>> Stefan
>>>>
>>> --
>>> Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
>>>
>>



Re: [Qemu-devel] Qemu 2.6 => Qemu 2.7 migration: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-balloon'

2016-11-15 Thread Stefan Priebe - Profihost AG
Hello,

Am 15.11.2016 um 11:30 schrieb Dr. David Alan Gilbert:
> * Stefan Priebe - Profihost AG (s.pri...@profihost.ag) wrote:
>> Hello,
>>
>> today i did a first live migration from Qemu 2.6.2 to Qemu 2.7.0. The VM
>> is running windows and virtio-balloon and with machine type pc-i440fx-2.5.
>>
>> The output of the target qemu process was:
>> kvm_apic_post_load: Yeh
>> kvm_apic_post_load: Yeh
>> kvm_apic_post_load: Yeh
>> kvm_apic_post_load: Yeh
>> kvm: VQ 2 size 0x80 < last_avail_idx 0x1 - used_idx 0x4
>> kvm: error while loading state for instance 0x0 of device
>> ':00:03.0/virtio-balloon'
>> kvm: load of migration failed: Operation not permitted
> 
> Yes that's a known bug; only seems to affect windows guests, and I believe
> doesn't even need to cross versions.
> 
> There's a bunch of fixes that Stefan applied to virtio code
> that I think fix this; I see that he cc'd qemu-stable.
> I think it's 4b7f91ed, but I'm not sure if there are others needed.

thanks for pointing to that commit.

Stefan can you tell me whether it's enough to cherry-pick 4b7f91ed into
2.7.0 ?

Greets,
Stefan

> 
> Dave
>>
>> Greets,
>> Stefan
>>
> --
> Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
> 



[Qemu-devel] Qemu 2.6 => Qemu 2.7 migration: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-balloon'

2016-11-14 Thread Stefan Priebe - Profihost AG
Hello,

today i did a first live migration from Qemu 2.6.2 to Qemu 2.7.0. The VM
is running windows and virtio-balloon and with machine type pc-i440fx-2.5.

The output of the target qemu process was:
kvm_apic_post_load: Yeh
kvm_apic_post_load: Yeh
kvm_apic_post_load: Yeh
kvm_apic_post_load: Yeh
kvm: VQ 2 size 0x80 < last_avail_idx 0x1 - used_idx 0x4
kvm: error while loading state for instance 0x0 of device
':00:03.0/virtio-balloon'
kvm: load of migration failed: Operation not permitted

Greets,
Stefan



Re: [Qemu-devel] [PATCH] net: fix qemu_announce_self not emitting packets

2016-06-07 Thread Stefan Priebe - Profihost AG

Am 07.06.2016 um 09:37 schrieb Peter Lieven:
> commit fefe2a78 accidently dropped the code path for injecting
> raw packets. This feature is needed for sending gratuitous ARPs
> after an incoming migration has completed. The result is increased
> network downtime for vservers where the network card is not virtio-net
> with the VIRTIO_NET_F_GUEST_ANNOUNCE feature.
> 
> Fixes: fefe2a78abde932e0f340b21bded2c86def1d242
> Cc: qemu-sta...@nongnu.org
> Cc: yan...@cn.fujitsu.com
> Signed-off-by: Peter Lieven 
> ---
>  net/net.c | 6 +-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/net/net.c b/net/net.c
> index 5f3e5a9..d5834ea 100644
> --- a/net/net.c
> +++ b/net/net.c
> @@ -722,7 +722,11 @@ ssize_t qemu_deliver_packet_iov(NetClientState *sender,
>  return 0;
>  }
>  
> -if (nc->info->receive_iov) {
> +if (flags & QEMU_NET_PACKET_FLAG_RAW && iovcnt == 1 &&
> +nc->info->receive_raw) {
> +/* this is required for qemu_announce_self() */
> +ret = nc->info->receive_raw(nc, iov[0].iov_base, iov[0].iov_len);
> +} else  if (nc->info->receive_iov) {
>  ret = nc->info->receive_iov(nc, iov, iovcnt);
>  } else {
>  ret = nc_sendv_compat(nc, iov, iovcnt, flags);
> 

Thanks for the patch. Sadly it does not fix our problem. So we still
have another Problem.

Thanks!

Greets,
Stefan



Re: [Qemu-devel] [Qemu-stable] Broken live Migration in Qemu 2.5.1.1?

2016-06-07 Thread Stefan Priebe - Profihost AG

Am 07.06.2016 um 09:38 schrieb Peter Lieven:
> Am 06.06.2016 um 18:13 schrieb Stefan Priebe - Profihost AG:
>> We're most probably seeing the same while migrating a machine running
>> balanceng but haven't thought this might be a qemu bug. Instead we're
>> investigating with balanceng people.
>>
>> Waiting for your further results.
> 
> Can you try the patch
> 
> net: fix qemu_announce_self not emitting packets

Thx - will wait for a new patch based on Paolo's comments - if that's ok.

Stefan

> I just send to the mailing list.
> 
> Thanks,
> Peter
> 



Re: [Qemu-devel] [Qemu-stable] Broken live Migration in Qemu 2.5.1.1?

2016-06-06 Thread Stefan Priebe - Profihost AG
We're most probably seeing the same while migrating a machine running balanceng 
but haven't thought this might be a qemu bug. Instead we're investigating with 
balanceng people.

Waiting for your further results.

Greets,
Stefan

Excuse my typo sent from my mobile phone.

> Am 06.06.2016 um 17:51 schrieb Peter Lieven :
> 
>> Am 06.06.2016 um 15:32 schrieb Peter Lieven:
>> Hi,
>> 
>> during internal testing of Qemu 2.5.1.1 I found a vServer running Ubuntu 
>> 12.04 (kernel 3.13) and a slave SQL server to
>> stop replicating from the master. This seems to be reproducible. It is 
>> possible to continue replication when issuing a slave stop / slave start.
>> There is no error visible on the vServer.
>> 
>> Has anyone a fix in mind that could be related to such an issue?
>> 
>> Host kernel in Linux 4.4, Guest kernel 3.13. Guest driver is virtio-blk via 
>> iSCSI. Emulated vCPU is Westmere.
> 
> After a lot of testing I found out that obviously thats no block driver 
> problem, but a regression in the virtio-net or the network stack.
> 
> qemu_announce_self() is generating packets for all NICs, but it seems they 
> are no longer emitted. This worked at least in qemu-2.2.0 with
> the same guest kernel and host kernel.
> 
> I will continue debugging tomorrow why this happens.
> 
> Peter
> 


Re: [Qemu-devel] drive-backup

2016-02-26 Thread Stefan Priebe


Am 25.02.2016 um 20:53 schrieb John Snow:



On 02/25/2016 02:49 AM, Stefan Priebe - Profihost AG wrote:


Am 22.02.2016 um 23:08 schrieb John Snow:



On 02/22/2016 03:21 PM, Stefan Priebe wrote:

Hello,

is there any chance or hack to work with a bigger cluster size for the
drive backup job?

See:
http://git.qemu.org/?p=qemu.git;a=blob;f=block/backup.c;h=16105d40b193be9bb40346027bdf58e62b956a96;hb=98d2c6f2cd80afaa2dc10091f5e35a97c181e4f5


This is very slow with ceph - may be due to the 64k block size. I would
like to check whether this is faster with cephs native block size of 4mb.

Greets,
Stefan



It's hardcoded to 64K at the moment, but I am checking in a patch to
round up the cluster size to be the bigger of (64k,
$target_cluster_size) in order to make sure that incremental backups in
particular never copy a fraction of a cluster. As a side-effect, the
same round-up will happen for all modes (sync=top,none,full).

If QEMU is aware of the target cluster size of 4MB, this would
immediately jump the copy-size up to 4MB clusters for you.

See: https://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg02839.html


Thanks for your patches and thanks for your great answer. But our
problem is not the target but the source ;-) The target has a local
cache and don't care about the cluster size but the source does not.

But it works fine if we change the default cluster size to 4MB. So it
has point us to the right direction.

Stefan



Ah, sorry, I misunderstood.

It's easy to change anyway! I am in favor of adding a configurable
parameter, as long as it keeps the other constraints I mentioned in mind.


ah great and thanks!

Stefan



--js



Otherwise, after my trivial fix, you should find cluster_size to be a
mutable concept and perhaps one that you could introduce a runtime
parameter for if you could convince the necessary parties that it's
needed in the API.

You'd have to be careful in the case of incremental that all the various
cluster sizes work well together:

- Granularity of bitmap (Defaults to cluster size of source, or 64K if
unknown. can be configured to be arbitrarily larger.)
- Cluster size of source file (For qcow2, defaults to 64k)
- Cluster size of target file
- Cluster size of backup routine (Currently always 64K)

I think that LCM(source_cluster_size, target_cluster_size,
backup_cluster_size) = MAX(A, B, C) will always be a safe minimum.

Bitmap granularity is more flexible, and it is more efficient when the
bitmap granularity matches the backup cluster size, but it can cope with
mismatches. If the bitmap is smaller or larger than the backup cluster
size, it generally means that more data that is clean is going to be
transferred across the pipe.

(Hmm, I wonder if it's worth checking in code to adjust a bitmap
granularity after it has been created so people can easily experiment
with performance tweaking here.)


Let me know if any of my ramble sounds interesting :)
--John







Re: [Qemu-devel] drive-backup

2016-02-24 Thread Stefan Priebe - Profihost AG

Am 22.02.2016 um 23:08 schrieb John Snow:
> 
> 
> On 02/22/2016 03:21 PM, Stefan Priebe wrote:
>> Hello,
>>
>> is there any chance or hack to work with a bigger cluster size for the
>> drive backup job?
>>
>> See:
>> http://git.qemu.org/?p=qemu.git;a=blob;f=block/backup.c;h=16105d40b193be9bb40346027bdf58e62b956a96;hb=98d2c6f2cd80afaa2dc10091f5e35a97c181e4f5
>>
>>
>> This is very slow with ceph - may be due to the 64k block size. I would
>> like to check whether this is faster with cephs native block size of 4mb.
>>
>> Greets,
>> Stefan
>>
> 
> It's hardcoded to 64K at the moment, but I am checking in a patch to
> round up the cluster size to be the bigger of (64k,
> $target_cluster_size) in order to make sure that incremental backups in
> particular never copy a fraction of a cluster. As a side-effect, the
> same round-up will happen for all modes (sync=top,none,full).
> 
> If QEMU is aware of the target cluster size of 4MB, this would
> immediately jump the copy-size up to 4MB clusters for you.
> 
> See: https://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg02839.html

Thanks for your patches and thanks for your great answer. But our
problem is not the target but the source ;-) The target has a local
cache and don't care about the cluster size but the source does not.

But it works fine if we change the default cluster size to 4MB. So it
has point us to the right direction.

Stefan

> 
> Otherwise, after my trivial fix, you should find cluster_size to be a
> mutable concept and perhaps one that you could introduce a runtime
> parameter for if you could convince the necessary parties that it's
> needed in the API.
> 
> You'd have to be careful in the case of incremental that all the various
> cluster sizes work well together:
> 
> - Granularity of bitmap (Defaults to cluster size of source, or 64K if
> unknown. can be configured to be arbitrarily larger.)
> - Cluster size of source file (For qcow2, defaults to 64k)
> - Cluster size of target file
> - Cluster size of backup routine (Currently always 64K)
> 
> I think that LCM(source_cluster_size, target_cluster_size,
> backup_cluster_size) = MAX(A, B, C) will always be a safe minimum.
> 
> Bitmap granularity is more flexible, and it is more efficient when the
> bitmap granularity matches the backup cluster size, but it can cope with
> mismatches. If the bitmap is smaller or larger than the backup cluster
> size, it generally means that more data that is clean is going to be
> transferred across the pipe.
> 
> (Hmm, I wonder if it's worth checking in code to adjust a bitmap
> granularity after it has been created so people can easily experiment
> with performance tweaking here.)
> 
> 
> Let me know if any of my ramble sounds interesting :)
> --John
> 



[Qemu-devel] drive-backup

2016-02-22 Thread Stefan Priebe

Hello,

is there any chance or hack to work with a bigger cluster size for the 
drive backup job?


See:
http://git.qemu.org/?p=qemu.git;a=blob;f=block/backup.c;h=16105d40b193be9bb40346027bdf58e62b956a96;hb=98d2c6f2cd80afaa2dc10091f5e35a97c181e4f5

This is very slow with ceph - may be due to the 64k block size. I would 
like to check whether this is faster with cephs native block size of 4mb.


Greets,
Stefan



Re: [Qemu-devel] kernel 4.4.2: kvm_irq_delivery_to_api / rwsem_down_read_failed

2016-02-22 Thread Stefan Priebe


Am 22.02.2016 um 18:36 schrieb Paolo Bonzini:



On 20/02/2016 11:44, Stefan Priebe wrote:

Hi,

while testing Kernel 4.4.2 and starting 20 Qemu 2.4.1 virtual machines.
I got those traces and a load of 500 on those system. I was only abler
to recover by sysrq-trigger.


It seems like something happening at the VM level.  A task took the mm
semaphore and hung everyone else.  Difficult to debug without a core
(and without knowing who held the semaphore).  Sorry.


OK thank you anyway. Is there anything i can do if this happens again?

Stefan


Paolo



All traces:

INFO: task pvedaemon worke:7470 blocked for more than 120 seconds.
Not tainted 4.4.2+1-ph #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
pvedaemon worke D 88239c367ca0 0 7470 7468 0x0008
88239c367ca0 8840a6232500 8823ed83a500 88239c367c90
88239c368000 8845f5f070e8 8845f5f07100 
7ffc73b48e58 88239c367cc0 b66a4d89 88239c367cf0
Call Trace:
[] schedule+0x39/0x80
[] rwsem_down_read_failed+0xc7/0x120
[] call_rwsem_down_read_failed+0x14/0x30
[] ? down_read+0x17/0x20
[] __access_remote_vm+0x3e/0x1c0
[] ? call_rwsem_down_read_failed+0x14/0x30
[] access_remote_vm+0x1f/0x30
[] proc_pid_cmdline_read+0x16e/0x4f0
[] ? acct_account_cputime+0x1c/0x20
[] __vfs_read+0x18/0x40
[] vfs_read+0x8e/0x140
[] SyS_read+0x4f/0xa0
[] entry_SYSCALL_64_fastpath+0x12/0x71
INFO: task pvestatd:7633 blocked for more than 120 seconds.
Not tainted 4.4.2+1-ph #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
pvestatd D 88239f16fd40 0 7633 1 0x0008
88239f16fd40 8824e76a8000 8823e5fc2500 8823e5fc2500
88239f17 8845f5f070e8 8845f5f07100 8845f5f07080
0341bf10 88239f16fd60 b66a4d89 024000d00058
Call Trace:
[] schedule+0x39/0x80
[] rwsem_down_read_failed+0xc7/0x120
[] call_rwsem_down_read_failed+0x14/0x30
[] ? down_read+0x17/0x20
[] proc_pid_cmdline_read+0xac/0x4f0
[] ? acct_account_cputime+0x1c/0x20
[] ? account_user_time+0x73/0x80
[] ? vtime_account_user+0x4e/0x70
[] __vfs_read+0x18/0x40
[] vfs_read+0x8e/0x140
[] SyS_read+0x4f/0xa0
[] entry_SYSCALL_64_fastpath+0x12/0x71




[Qemu-devel] kernel 4.4.2: kvm_irq_delivery_to_api / rwsem_down_read_failed

2016-02-20 Thread Stefan Priebe

Hi,

while testing Kernel 4.4.2 and starting 20 Qemu 2.4.1 virtual machines. 
I got those traces and a load of 500 on those system. I was only abler 
to recover by sysrq-trigger.


All traces:

INFO: task pvedaemon worke:7470 blocked for more than 120 seconds.
Not tainted 4.4.2+1-ph #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
pvedaemon worke D 88239c367ca0 0 7470 7468 0x0008
88239c367ca0 8840a6232500 8823ed83a500 88239c367c90
88239c368000 8845f5f070e8 8845f5f07100 
7ffc73b48e58 88239c367cc0 b66a4d89 88239c367cf0
Call Trace:
[] schedule+0x39/0x80
[] rwsem_down_read_failed+0xc7/0x120
[] call_rwsem_down_read_failed+0x14/0x30
[] ? down_read+0x17/0x20
[] __access_remote_vm+0x3e/0x1c0
[] ? call_rwsem_down_read_failed+0x14/0x30
[] access_remote_vm+0x1f/0x30
[] proc_pid_cmdline_read+0x16e/0x4f0
[] ? acct_account_cputime+0x1c/0x20
[] __vfs_read+0x18/0x40
[] vfs_read+0x8e/0x140
[] SyS_read+0x4f/0xa0
[] entry_SYSCALL_64_fastpath+0x12/0x71
INFO: task pvestatd:7633 blocked for more than 120 seconds.
Not tainted 4.4.2+1-ph #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
pvestatd D 88239f16fd40 0 7633 1 0x0008
88239f16fd40 8824e76a8000 8823e5fc2500 8823e5fc2500
88239f17 8845f5f070e8 8845f5f07100 8845f5f07080
0341bf10 88239f16fd60 b66a4d89 024000d00058
Call Trace:
[] schedule+0x39/0x80
[] rwsem_down_read_failed+0xc7/0x120
[] call_rwsem_down_read_failed+0x14/0x30
[] ? down_read+0x17/0x20
[] proc_pid_cmdline_read+0xac/0x4f0
[] ? acct_account_cputime+0x1c/0x20
[] ? account_user_time+0x73/0x80
[] ? vtime_account_user+0x4e/0x70
[] __vfs_read+0x18/0x40
[] vfs_read+0x8e/0x140
[] SyS_read+0x4f/0xa0
[] entry_SYSCALL_64_fastpath+0x12/0x71
INFO: task kvm:11766 blocked for more than 120 seconds.
Not tainted 4.4.2+1-ph #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kvm D 88452a2d3dd0 0 11766 1 0x0008
88452a2d3dd0 880166c74a00 8845b7354a00 b617fc8e
88452a2d4000 8845f5f070e8 8845f5f07100 88452a2d3f58
8845b7354a00 88452a2d3df0 b66a4d89 7fa807abbf80
Call Trace:
[] ? __handle_mm_fault+0xd1e/0x1260
[] schedule+0x39/0x80
[] rwsem_down_read_failed+0xc7/0x120
[] call_rwsem_down_read_failed+0x14/0x30
[] ? down_read+0x17/0x20
[] __do_page_fault+0x2b7/0x380
[] ? account_user_time+0x73/0x80
[] ? vtime_account_user+0x4e/0x70
[] do_page_fault+0x37/0x90
[] page_fault+0x28/0x30
INFO: task kvm:11824 blocked for more than 120 seconds.
Not tainted 4.4.2+1-ph #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kvm D 8840a867faa0 0 11824 1 0x0008
8840a867faa0 8845866a4a00 8840a6232500 0001
8840a868 8845f5f070e8 8845f5f07100 8840a867fc0e
 8840a867fac0 b66a4d89 c0606a06
Call Trace:
[] schedule+0x39/0x80
[] ? kvm_irq_delivery_to_apic+0x56/0x220 [kvm]
[] rwsem_down_read_failed+0xc7/0x120
[] call_rwsem_down_read_failed+0x14/0x30
[] ? down_read+0x17/0x20
[] kvm_host_page_size+0x60/0xa0 [kvm]
[] mapping_level+0x5c/0x130 [kvm]
[] tdp_page_fault+0x9b/0x260 [kvm]
[] ? kernel_pio+0x2d/0x40 [kvm]
[] kvm_mmu_page_fault+0x31/0x120 [kvm]
[] handle_ept_violation+0xa4/0x170 [kvm_intel]
[] vmx_handle_exit+0x257/0x490 [kvm_intel]
[] ? __vtime_account_system+0x31/0x40
[] vcpu_enter_guest+0x6af/0xff0 [kvm]
[] ? kvm_apic_local_deliver+0x5d/0x60 [kvm]
[] kvm_arch_vcpu_ioctl_run+0xc4/0x3c0 [kvm]
[] kvm_vcpu_ioctl+0x324/0x5d0 [kvm]
[] ? acct_account_cputime+0x1c/0x20
[] ? account_user_time+0x73/0x80
[] do_vfs_ioctl+0x83/0x4e0
[] ? enter_from_user_mode+0x1f/0x50
[] ? syscall_trace_enter_phase1+0xc1/0x110
[] SyS_ioctl+0x4c/0x80
[] entry_SYSCALL_64_fastpath+0x12/0x71
INFO: task kvm:11825 blocked for more than 120 seconds.
Not tainted 4.4.2+1-ph #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kvm D 88458d6a3aa0 0 11825 1 0x00080002
88458d6a3aa0 880167302500 8840a6234a00 0001
88458d6a4000 8845f5f070e8 8845f5f07100 88458d6a3c0e
 88458d6a3ac0 b66a4d89 c0606a06
Call Trace:
[] schedule+0x39/0x80
[] ? kvm_irq_delivery_to_apic+0x56/0x220 [kvm]
[] rwsem_down_read_failed+0xc7/0x120
[] call_rwsem_down_read_failed+0x14/0x30
[] ? down_read+0x17/0x20
[] kvm_host_page_size+0x60/0xa0 [kvm]
[] mapping_level+0x5c/0x130 [kvm]
[] tdp_page_fault+0x9b/0x260 [kvm]
[] kvm_mmu_page_fault+0x31/0x120 [kvm]
[] handle_ept_violation+0xa4/0x170 [kvm_intel]
[] vmx_handle_exit+0x257/0x490 [kvm_intel]
[] ? __vtime_account_system+0x31/0x40
[] vcpu_enter_guest+0x6af/0xff0 [kvm]
[] ? kvm_apic_local_deliver+0x5d/0x60 [kvm]
[] kvm_arch_vcpu_ioctl_run+0xc4/0x3c0 [kvm]
[] kvm_vcpu_ioctl+0x324/0x5d0 [kvm]
[] ? acct_account_cputime+0x1c/0x20
[] ? account_user_time+0x73/0x80
[] do_vfs_ioctl+0x83/0x4e0
[] ? enter_from_

Re: [Qemu-devel] trace in arch/x86/kernel/apic/apic.c:1309 setup_local_APIC

2016-01-26 Thread Stefan Priebe - Profihost AG

Am 26.01.2016 um 12:39 schrieb Yang Zhang:
> On 2016/1/26 18:43, Stefan Priebe - Profihost AG wrote:
>>
>> Am 26.01.2016 um 11:13 schrieb Yang Zhang:
>>> On 2016/1/26 15:22, Stefan Priebe - Profihost AG wrote:
>>>> Hi,
>>>>
>>>> Am 26.01.2016 um 02:46 schrieb Han, Huaitong:
>>>>> What is the host kernel version and host dmesg information? And does
>>>>> the problem exist when you use latest kernel and QEMU to replace old
>>>>> binary file?
>>>>
>>>> Guest and Host is both 4.1.15. You mean the complete dmesg output from
>>>> host?
>>>>
>>>> What do you mean with replace old binary file? I haven't tested Kernel
>>>> 4.4 as we use 4.1 as it is a long term stable kernel release.
>>>
>>> Have you seen this before? I mean use the old KVM like 3.10?
>>
>> Guest or host? To test with a guest would be quite easy. Downgrading the
>> host is very difficult not sure if the hw is supported.
> 
> Host. Does the issue only exist on the Westmere CPU?

Yes. All E5 Xeons v1, v2, v3 are working fine and i've never seen this
message.

Stefan

> 
>>
>>>
>>>>
>>>> Stefan
>>>>
>>>>> On Mon, 2016-01-25 at 14:51 +0100, Stefan Priebe - Profihost AG wrote:
>>>>>> Hi,
>>>>>>
>>>>>> while running qemu 2.4 on whestmere CPUs i'm pretty often getting
>>>>>> this
>>>>>> one while booting:
>>>>>> [0.811645] Switched APIC routing to physical x2apic.
>>>>>> [1.835678] [ cut here ]
>>>>>> [1.835704] WARNING: CPU: 0 PID: 1 at
>>>>>> arch/x86/kernel/apic/apic.c:1309 setup_local_APIC+0x284/0x340()
>>>>>> [1.835714] Modules linked in:
>>>>>> [1.835724] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.1.15+72-ph
>>>>>> #1
>>>>>> [1.835731] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>>>> BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
>>>>>> [1.835743]  b69ffcea 88042d5e3d68 b669c37b
>>>>>> 0918
>>>>>> [1.835754]   88042d5e3da8 b6080d67
>>>>>> 88042d5e3da8
>>>>>> [1.835765]  0001  8000
>>>>>> 
>>>>>> [1.835777] Call Trace:
>>>>>> [1.835789]  [] dump_stack+0x45/0x57
>>>>>> [1.835799]  [] warn_slowpath_common+0x97/0xe0
>>>>>> [1.835806]  [] warn_slowpath_null+0x1a/0x20
>>>>>> [1.835813]  [] setup_local_APIC+0x284/0x340
>>>>>> [1.835824]  [] apic_bsp_setup+0x5b/0xb0
>>>>>> [1.835832]  []
>>>>>> native_smp_prepare_cpus+0x23b/0x295
>>>>>> [1.835842]  [] kernel_init_freeable+0xc7/0x20f
>>>>>> [1.835853]  [] ? rest_init+0x80/0x80
>>>>>> [1.835860]  [] kernel_init+0xe/0xf0
>>>>>> [1.835870]  [] ret_from_fork+0x42/0x70
>>>>>> [1.835877]  [] ? rest_init+0x80/0x80
>>>>>> [1.835891] ---[ end trace bdbe630a8de2832c ]---
>>>>>> [1.837613] Spurious LAPIC timer interrupt on cpu 0
>>>>>> [1.837957] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
>>>>>> [1.939574] smpboot: CPU0: Intel Westmere E56xx/L56xx/X56xx
>>>>>> (Nehalem-C) (fam: 06, model: 2c, stepping: 01)
>>>>>> [1.939630] Performance Events: unsupported p6 CPU model 44 no PMU
>>>>>> driver, software events only.
>>>>>> [1.950868] KVM setup paravirtual spinlock
>>>>>>
>>>>>> Greets,
>>>>>> Stefan
>>>>>> -- 
>>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>>>> the body of a message to majord...@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> -- 
>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>> the body of a message to majord...@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>
>>>
> 
> 



Re: [Qemu-devel] trace in arch/x86/kernel/apic/apic.c:1309 setup_local_APIC

2016-01-26 Thread Stefan Priebe - Profihost AG

Am 26.01.2016 um 11:13 schrieb Yang Zhang:
> On 2016/1/26 15:22, Stefan Priebe - Profihost AG wrote:
>> Hi,
>>
>> Am 26.01.2016 um 02:46 schrieb Han, Huaitong:
>>> What is the host kernel version and host dmesg information? And does
>>> the problem exist when you use latest kernel and QEMU to replace old
>>> binary file?
>>
>> Guest and Host is both 4.1.15. You mean the complete dmesg output from
>> host?
>>
>> What do you mean with replace old binary file? I haven't tested Kernel
>> 4.4 as we use 4.1 as it is a long term stable kernel release.
> 
> Have you seen this before? I mean use the old KVM like 3.10?

Guest or host? To test with a guest would be quite easy. Downgrading the
host is very difficult not sure if the hw is supported.

> 
>>
>> Stefan
>>
>>> On Mon, 2016-01-25 at 14:51 +0100, Stefan Priebe - Profihost AG wrote:
>>>> Hi,
>>>>
>>>> while running qemu 2.4 on whestmere CPUs i'm pretty often getting
>>>> this
>>>> one while booting:
>>>> [0.811645] Switched APIC routing to physical x2apic.
>>>> [1.835678] [ cut here ]
>>>> [1.835704] WARNING: CPU: 0 PID: 1 at
>>>> arch/x86/kernel/apic/apic.c:1309 setup_local_APIC+0x284/0x340()
>>>> [1.835714] Modules linked in:
>>>> [1.835724] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.1.15+72-ph
>>>> #1
>>>> [1.835731] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>> BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
>>>> [1.835743]  b69ffcea 88042d5e3d68 b669c37b
>>>> 0918
>>>> [1.835754]   88042d5e3da8 b6080d67
>>>> 88042d5e3da8
>>>> [1.835765]  0001  8000
>>>> 
>>>> [1.835777] Call Trace:
>>>> [1.835789]  [] dump_stack+0x45/0x57
>>>> [1.835799]  [] warn_slowpath_common+0x97/0xe0
>>>> [1.835806]  [] warn_slowpath_null+0x1a/0x20
>>>> [1.835813]  [] setup_local_APIC+0x284/0x340
>>>> [1.835824]  [] apic_bsp_setup+0x5b/0xb0
>>>> [1.835832]  []
>>>> native_smp_prepare_cpus+0x23b/0x295
>>>> [1.835842]  [] kernel_init_freeable+0xc7/0x20f
>>>> [1.835853]  [] ? rest_init+0x80/0x80
>>>> [1.835860]  [] kernel_init+0xe/0xf0
>>>> [1.835870]  [] ret_from_fork+0x42/0x70
>>>> [1.835877]  [] ? rest_init+0x80/0x80
>>>> [1.835891] ---[ end trace bdbe630a8de2832c ]---
>>>> [1.837613] Spurious LAPIC timer interrupt on cpu 0
>>>> [1.837957] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
>>>> [1.939574] smpboot: CPU0: Intel Westmere E56xx/L56xx/X56xx
>>>> (Nehalem-C) (fam: 06, model: 2c, stepping: 01)
>>>> [1.939630] Performance Events: unsupported p6 CPU model 44 no PMU
>>>> driver, software events only.
>>>> [1.950868] KVM setup paravirtual spinlock
>>>>
>>>> Greets,
>>>> Stefan
>>>> -- 
>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>> the body of a message to majord...@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> 



Re: [Qemu-devel] trace in arch/x86/kernel/apic/apic.c:1309 setup_local_APIC

2016-01-25 Thread Stefan Priebe - Profihost AG
Hi,

Am 26.01.2016 um 02:46 schrieb Han, Huaitong:
> What is the host kernel version and host dmesg information? And does
> the problem exist when you use latest kernel and QEMU to replace old
> binary file?

Guest and Host is both 4.1.15. You mean the complete dmesg output from host?

What do you mean with replace old binary file? I haven't tested Kernel
4.4 as we use 4.1 as it is a long term stable kernel release.

Stefan

> On Mon, 2016-01-25 at 14:51 +0100, Stefan Priebe - Profihost AG wrote:
>> Hi,
>>
>> while running qemu 2.4 on whestmere CPUs i'm pretty often getting
>> this
>> one while booting:
>> [0.811645] Switched APIC routing to physical x2apic.
>> [1.835678] [ cut here ]
>> [1.835704] WARNING: CPU: 0 PID: 1 at
>> arch/x86/kernel/apic/apic.c:1309 setup_local_APIC+0x284/0x340()
>> [1.835714] Modules linked in:
>> [1.835724] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.1.15+72-ph
>> #1
>> [1.835731] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>> BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
>> [1.835743]  b69ffcea 88042d5e3d68 b669c37b
>> 0918
>> [1.835754]   88042d5e3da8 b6080d67
>> 88042d5e3da8
>> [1.835765]  0001  8000
>> 
>> [1.835777] Call Trace:
>> [1.835789]  [] dump_stack+0x45/0x57
>> [1.835799]  [] warn_slowpath_common+0x97/0xe0
>> [1.835806]  [] warn_slowpath_null+0x1a/0x20
>> [1.835813]  [] setup_local_APIC+0x284/0x340
>> [1.835824]  [] apic_bsp_setup+0x5b/0xb0
>> [1.835832]  []
>> native_smp_prepare_cpus+0x23b/0x295
>> [1.835842]  [] kernel_init_freeable+0xc7/0x20f
>> [1.835853]  [] ? rest_init+0x80/0x80
>> [1.835860]  [] kernel_init+0xe/0xf0
>> [1.835870]  [] ret_from_fork+0x42/0x70
>> [1.835877]  [] ? rest_init+0x80/0x80
>> [1.835891] ---[ end trace bdbe630a8de2832c ]---
>> [1.837613] Spurious LAPIC timer interrupt on cpu 0
>> [1.837957] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
>> [1.939574] smpboot: CPU0: Intel Westmere E56xx/L56xx/X56xx
>> (Nehalem-C) (fam: 06, model: 2c, stepping: 01)
>> [1.939630] Performance Events: unsupported p6 CPU model 44 no PMU
>> driver, software events only.
>> [1.950868] KVM setup paravirtual spinlock
>>
>> Greets,
>> Stefan
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Qemu-devel] trace in arch/x86/kernel/apic/apic.c:1309 setup_local_APIC

2016-01-25 Thread Stefan Priebe - Profihost AG
Hi,

while running qemu 2.4 on whestmere CPUs i'm pretty often getting this
one while booting:
[0.811645] Switched APIC routing to physical x2apic.
[1.835678] [ cut here ]
[1.835704] WARNING: CPU: 0 PID: 1 at
arch/x86/kernel/apic/apic.c:1309 setup_local_APIC+0x284/0x340()
[1.835714] Modules linked in:
[1.835724] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.1.15+72-ph #1
[1.835731] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
[1.835743]  b69ffcea 88042d5e3d68 b669c37b
0918
[1.835754]   88042d5e3da8 b6080d67
88042d5e3da8
[1.835765]  0001  8000

[1.835777] Call Trace:
[1.835789]  [] dump_stack+0x45/0x57
[1.835799]  [] warn_slowpath_common+0x97/0xe0
[1.835806]  [] warn_slowpath_null+0x1a/0x20
[1.835813]  [] setup_local_APIC+0x284/0x340
[1.835824]  [] apic_bsp_setup+0x5b/0xb0
[1.835832]  [] native_smp_prepare_cpus+0x23b/0x295
[1.835842]  [] kernel_init_freeable+0xc7/0x20f
[1.835853]  [] ? rest_init+0x80/0x80
[1.835860]  [] kernel_init+0xe/0xf0
[1.835870]  [] ret_from_fork+0x42/0x70
[1.835877]  [] ? rest_init+0x80/0x80
[1.835891] ---[ end trace bdbe630a8de2832c ]---
[1.837613] Spurious LAPIC timer interrupt on cpu 0
[1.837957] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[1.939574] smpboot: CPU0: Intel Westmere E56xx/L56xx/X56xx
(Nehalem-C) (fam: 06, model: 2c, stepping: 01)
[1.939630] Performance Events: unsupported p6 CPU model 44 no PMU
driver, software events only.
[1.950868] KVM setup paravirtual spinlock

Greets,
Stefan



Re: [Qemu-devel] qemu : rbd block driver internal snapshot and vm_stop is hanging forever

2015-11-09 Thread Stefan Priebe - Profihost AG

> - Original Message -
>> From: "Alexandre DERUMIER" 
>> To: "ceph-devel" 
>> Cc: "qemu-devel" , jdur...@redhat.com
>> Sent: Monday, November 9, 2015 5:48:45 AM
>> Subject: Re: [Qemu-devel] qemu : rbd block driver internal snapshot and 
>> vm_stop is hanging forever
>>
>> adding to ceph.conf
>>
>> [client]
>> rbd_non_blocking_aio = false
>>
>>
>> fix the problem for me (with rbd_cache=false)
>>
>>
>> (@cc jdur...@redhat.com)

+1 same to me.

Stefan

>>
>>
>>
>> - Mail original -
>> De: "Denis V. Lunev" 
>> À: "aderumier" , "ceph-devel"
>> , "qemu-devel" 
>> Envoyé: Lundi 9 Novembre 2015 08:22:34
>> Objet: Re: [Qemu-devel] qemu : rbd block driver internal snapshot and vm_stop
>> is hanging forever
>>
>> On 11/09/2015 10:19 AM, Denis V. Lunev wrote:
>>> On 11/09/2015 06:10 AM, Alexandre DERUMIER wrote:
 Hi,

 with qemu (2.4.1), if I do an internal snapshot of an rbd device,
 then I pause the vm with vm_stop,

 the qemu process is hanging forever


 monitor commands to reproduce:


 # snapshot_blkdev_internal drive-virtio0 yoursnapname
 # stop




 I don't see this with qcow2 or sheepdog block driver for example.


 Regards,

 Alexandre

>>> this could look like the problem I have recenty trying to
>>> fix with dataplane enabled. Patch series is named as
>>>
>>> [PATCH for 2.5 v6 0/10] dataplane snapshot fixes
>>>
>>> Den
>>
>> anyway, even if above will not help, can you collect gdb
>> traces from all threads in QEMU process. May be I'll be
>> able to give a hit.
>>
>> Den
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



Re: [Qemu-devel] guest: NO_HZ FULL will not work with unstable sched clock

2015-08-15 Thread Stefan Priebe

same with kvm clock:

[0.195073] Switched to clocksource kvm-clock
[0.195509] [ cut here ]
[0.195509] WARNING: CPU: 1 PID: 1 at kernel/time/tick-sched.c:192 
can_stop_full_tick+0x1c1/0x200()

[0.195515] NO_HZ FULL will not work with unstable sched clock
[0.195518] Modules linked in:
[0.195531] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.1.5+11-ph #1
[0.195535] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), 
BIOS rel-1.8.1-0-g4adadbd-20150316_085822-nilsson.home.kraxel.org 04/01/2014
[0.195544]  819fc200 88023fc43e88 8162dd36 
121a
[0.195551]  88023fc43ed8 88023fc43ec8 8107dcb7 
88023fc43ea8
[0.195558]   0001 81e667c0 
81cf1540

[0.195564] Call Trace:
[0.195568][] dump_stack+0x45/0x57
[0.195591]  [] warn_slowpath_common+0x97/0xe0
[0.195596]  [] warn_slowpath_fmt+0x46/0x50
[0.195600]  [] can_stop_full_tick+0x1c1/0x200
[0.195604]  [] tick_nohz_irq_exit+0x97/0xd0
[0.195609]  [] irq_exit+0xcd/0x130
[0.195620]  [] smp_apic_timer_interrupt+0x4a/0x60
[0.195627]  [] apic_timer_interrupt+0x6e/0x80
[0.195631][] ? 
trace_event_enum_update+0x1c7/0x3b0

[0.195644]  [] ? trace_event_enum_update+0x20c/0x3b0
[0.195649]  [] ? stop_trace_on_warning+0x3f/0x3f
[0.195654]  [] tracer_init_tracefs+0x11a/0x23b
[0.195661]  [] do_one_initcall+0xd8/0x210
[0.195669]  [] ? parse_args+0x1b0/0x450
[0.195676]  [] ? __wake_up+0x53/0x70
[0.195684]  [] kernel_init_freeable+0x178/0x20f
[0.195689]  [] ? do_early_param+0x8c/0x8c
[0.195700]  [] ? create_pipe_files+0x1f2/0x200
[0.195706]  [] ? rest_init+0x80/0x80
[0.195711]  [] kernel_init+0xe/0xf0
[0.195715]  [] ret_from_fork+0x42/0x70
[0.195719]  [] ? rest_init+0x80/0x80
[0.195729] ---[ end trace cf665146248feec1 ]---

Stefan

Am 15.08.2015 um 20:44 schrieb Stefan Priebe:

Hi,

while switching to a FULL tickless kernel i detected that all our VMs
produce the following stack trace while running under qemu 2.3.0.

[0.195160] HPET: 3 timers in total, 0 timers will be used for
per-cpu timer
[0.195181] hpet0: at MMIO 0xfed0, IRQs 2, 8, 0
[0.195188] hpet0: 3 comparators, 64-bit 100.00 MHz counter
[0.200119] Switched to clocksource hpet
[0.201164] [ cut here ]
[0.201193] WARNING: CPU: 1 PID: 1 at kernel/time/tick-sched.c:192
can_stop_full_tick+0x1c1/0x200()
[0.201199] NO_HZ FULL will not work with unstable sched clock
[0.201203] Modules linked in:
[0.201214] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 3.18.20+55-ph #1
[0.201219] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS rel-1.8.1-0-g4adadbd-20150316_085822-nilsson.home.kraxel.org
04/01/2014
[0.201226]  0009 88023fc43e88 815f5889
08df
[0.201230]  88023fc43ed8 88023fc43ec8 8107433c
88023fc43ea8
[0.201233]   0001 880236c41300
cbbde78b
[0.201237] Call Trace:
[0.201240][] dump_stack+0x46/0x58
[0.201260]  [] warn_slowpath_common+0x8c/0xc0
[0.201264]  [] warn_slowpath_fmt+0x46/0x50
[0.201270]  [] can_stop_full_tick+0x1c1/0x200
[0.201274]  [] tick_nohz_irq_exit+0x97/0xd0
[0.201277]  [] irq_exit+0xcd/0x130
[0.201286]  [] smp_apic_timer_interrupt+0x4a/0x60
[0.201293]  [] apic_timer_interrupt+0x6a/0x70
[0.201296][] ? __d_lookup+0x55/0x160
[0.201309]  [] d_lookup+0x35/0x60
[0.201312]  [] lookup_dcache+0x3b/0xd0
[0.201315]  [] __lookup_hash+0x23/0x50
[0.201319]  [] lookup_one_len+0xce/0x120
[0.201328]  [] __create_file+0x8f/0x260
[0.201332]  [] debugfs_create_dir+0x1b/0x20
[0.201338]  [] event_create_dir+0x68/0x520
[0.201342]  [] event_trace_init+0x188/0x210
[0.201348]  [] ? trace_create_file+0x1b/0x50
[0.201352]  [] ? event_trace_enable+0x1a2/0x1a2
[0.201359]  [] do_one_initcall+0xd8/0x200
[0.201368]  [] ? parse_args+0x1f0/0x470
[0.201373]  [] ? __wake_up+0x53/0x70
[0.201378]  [] kernel_init_freeable+0x168/0x1ff
[0.201381]  [] ? do_early_param+0x8c/0x8c
[0.201389]  [] ? cryptomgr_probe+0x60/0xe0
[0.201398]  [] ? rest_init+0x80/0x80
[0.201401]  [] kernel_init+0xe/0xf0
[0.201405]  [] ret_from_fork+0x58/0x90
[0.201408]  [] ? rest_init+0x80/0x80
[0.201415] ---[ end trace a15ba3bc1d247b4c ]---

Stefan




[Qemu-devel] guest: NO_HZ FULL will not work with unstable sched clock

2015-08-15 Thread Stefan Priebe

Hi,

while switching to a FULL tickless kernel i detected that all our VMs
produce the following stack trace while running under qemu 2.3.0.

[0.195160] HPET: 3 timers in total, 0 timers will be used for
per-cpu timer
[0.195181] hpet0: at MMIO 0xfed0, IRQs 2, 8, 0
[0.195188] hpet0: 3 comparators, 64-bit 100.00 MHz counter
[0.200119] Switched to clocksource hpet
[0.201164] [ cut here ]
[0.201193] WARNING: CPU: 1 PID: 1 at kernel/time/tick-sched.c:192
can_stop_full_tick+0x1c1/0x200()
[0.201199] NO_HZ FULL will not work with unstable sched clock
[0.201203] Modules linked in:
[0.201214] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 3.18.20+55-ph #1
[0.201219] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS rel-1.8.1-0-g4adadbd-20150316_085822-nilsson.home.kraxel.org 04/01/2014
[0.201226]  0009 88023fc43e88 815f5889
08df
[0.201230]  88023fc43ed8 88023fc43ec8 8107433c
88023fc43ea8
[0.201233]   0001 880236c41300
cbbde78b
[0.201237] Call Trace:
[0.201240][] dump_stack+0x46/0x58
[0.201260]  [] warn_slowpath_common+0x8c/0xc0
[0.201264]  [] warn_slowpath_fmt+0x46/0x50
[0.201270]  [] can_stop_full_tick+0x1c1/0x200
[0.201274]  [] tick_nohz_irq_exit+0x97/0xd0
[0.201277]  [] irq_exit+0xcd/0x130
[0.201286]  [] smp_apic_timer_interrupt+0x4a/0x60
[0.201293]  [] apic_timer_interrupt+0x6a/0x70
[0.201296][] ? __d_lookup+0x55/0x160
[0.201309]  [] d_lookup+0x35/0x60
[0.201312]  [] lookup_dcache+0x3b/0xd0
[0.201315]  [] __lookup_hash+0x23/0x50
[0.201319]  [] lookup_one_len+0xce/0x120
[0.201328]  [] __create_file+0x8f/0x260
[0.201332]  [] debugfs_create_dir+0x1b/0x20
[0.201338]  [] event_create_dir+0x68/0x520
[0.201342]  [] event_trace_init+0x188/0x210
[0.201348]  [] ? trace_create_file+0x1b/0x50
[0.201352]  [] ? event_trace_enable+0x1a2/0x1a2
[0.201359]  [] do_one_initcall+0xd8/0x200
[0.201368]  [] ? parse_args+0x1f0/0x470
[0.201373]  [] ? __wake_up+0x53/0x70
[0.201378]  [] kernel_init_freeable+0x168/0x1ff
[0.201381]  [] ? do_early_param+0x8c/0x8c
[0.201389]  [] ? cryptomgr_probe+0x60/0xe0
[0.201398]  [] ? rest_init+0x80/0x80
[0.201401]  [] kernel_init+0xe/0xf0
[0.201405]  [] ret_from_fork+0x58/0x90
[0.201408]  [] ? rest_init+0x80/0x80
[0.201415] ---[ end trace a15ba3bc1d247b4c ]---

Stefan



Re: [Qemu-devel] [Qemu-stable] [PULL 0/3] Cve 2015 5154 patches

2015-07-27 Thread Stefan Priebe - Profihost AG

Am 27.07.2015 um 14:28 schrieb John Snow:
> 
> 
> On 07/27/2015 08:10 AM, Stefan Priebe - Profihost AG wrote:
>>
>> Am 27.07.2015 um 14:01 schrieb John Snow:
>>> The following changes since commit f793d97e454a56d17e404004867985622ca1a63b:
>>>
>>>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into 
>>> staging (2015-07-24 13:07:10 +0100)
>>>
>>> are available in the git repository at:
>>>
>>>   https://github.com/jnsnow/qemu.git tags/cve-2015-5154-pull-request
>>
>> Any details on this CVE? Is RCE possible? Only if IDE is used?
>>
>> Stefan
>>
> 
> It's a heap overflow. The most likely outcome is a segfault, but the
> guest is allowed to continue writing past the end of the PIO buffer at
> its leisure. This makes it similar to CVE-2015-3456.
> 
> This CVE can be mitigated unlike CVE-2015-3456 by just removing the
> CD-ROM drive until the patch can be applied.

Thanks. The seclist article explicitly references xen. So it does not
apply to qemu/kvm? Sorry for asking may be stupid questions.

Stefan

>>> for you to fetch changes up to cb72cba83021fa42719e73a5249c12096a4d1cfc:
>>>
>>>   ide: Clear DRQ after handling all expected accesses (2015-07-26 23:42:53 
>>> -0400)
>>>
>>> 
>>>
>>> 
>>>
>>> Kevin Wolf (3):
>>>   ide: Check array bounds before writing to io_buffer (CVE-2015-5154)
>>>   ide/atapi: Fix START STOP UNIT command completion
>>>   ide: Clear DRQ after handling all expected accesses
>>>
>>>  hw/ide/atapi.c |  1 +
>>>  hw/ide/core.c  | 32 
>>>  2 files changed, 29 insertions(+), 4 deletions(-)
>>>



Re: [Qemu-devel] [Qemu-stable] [PULL 0/3] Cve 2015 5154 patches

2015-07-27 Thread Stefan Priebe - Profihost AG

Am 27.07.2015 um 14:01 schrieb John Snow:
> The following changes since commit f793d97e454a56d17e404004867985622ca1a63b:
> 
>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into 
> staging (2015-07-24 13:07:10 +0100)
> 
> are available in the git repository at:
> 
>   https://github.com/jnsnow/qemu.git tags/cve-2015-5154-pull-request

Any details on this CVE? Is RCE possible? Only if IDE is used?

Stefan

> for you to fetch changes up to cb72cba83021fa42719e73a5249c12096a4d1cfc:
> 
>   ide: Clear DRQ after handling all expected accesses (2015-07-26 23:42:53 
> -0400)
> 
> 
> 
> 
> 
> Kevin Wolf (3):
>   ide: Check array bounds before writing to io_buffer (CVE-2015-5154)
>   ide/atapi: Fix START STOP UNIT command completion
>   ide: Clear DRQ after handling all expected accesses
> 
>  hw/ide/atapi.c |  1 +
>  hw/ide/core.c  | 32 
>  2 files changed, 29 insertions(+), 4 deletions(-)
> 



Re: [Qemu-devel] Query CPU model / type

2015-07-15 Thread Stefan Priebe



Am 15.07.2015 um 22:15 schrieb Andrey Korolyov:

On Wed, Jul 15, 2015 at 11:07 PM, Stefan Priebe  wrote:

Am 15.07.2015 um 13:32 schrieb Andrey Korolyov:


On Wed, Jul 15, 2015 at 2:20 PM, Stefan Priebe - Profihost AG
 wrote:


Hi,

is there a way to query the current cpu model / type of a running qemu
machine?

I mean host, kvm64, qemu64, ...

Stefan



I believe that the most proper one would be
'query-command-line-options'. The CPU definitions are not exposed
anywhere else in QMP except their list.



But this one only lists the possible arguments not the currently passed
ones?

Stefan


Sorry, my bad, wrong memories. So there is no way to retrieve
arguments via monitor.


and even no current cpu model ;-( pretty bad for migrations. So i'll 
have to hack the qemu code.


Stefan







Re: [Qemu-devel] Query CPU model / type

2015-07-15 Thread Stefan Priebe

Am 15.07.2015 um 13:32 schrieb Andrey Korolyov:

On Wed, Jul 15, 2015 at 2:20 PM, Stefan Priebe - Profihost AG
 wrote:

Hi,

is there a way to query the current cpu model / type of a running qemu
machine?

I mean host, kvm64, qemu64, ...

Stefan



I believe that the most proper one would be
'query-command-line-options'. The CPU definitions are not exposed
anywhere else in QMP except their list.


But this one only lists the possible arguments not the currently passed 
ones?


Stefan



[Qemu-devel] Query CPU model / type

2015-07-15 Thread Stefan Priebe - Profihost AG
Hi,

is there a way to query the current cpu model / type of a running qemu
machine?

I mean host, kvm64, qemu64, ...

Stefan



Re: [Qemu-devel] [Qemu-stable] [PATCH] fdc: force the fifo access to be in bounds of the allocated buffer

2015-05-13 Thread Stefan Priebe

Am 13.05.2015 um 21:05 schrieb Stefan Weil:

Am 13.05.2015 um 20:59 schrieb Stefan Priebe:


Am 13.05.2015 um 20:51 schrieb Stefan Weil:

Hi,

I just noticed this patch because my provider told me that my KVM based
server
needs a reboot because of a CVE (see this German news:
http://www.heise.de/newsticker/meldung/Venom-Schwachstelle-Aus-Hypervisor-ausbrechen-und-VMs-ausspionieren-2649614.html)



Isn't a live migration to a fixed version enough instead of a reboot?

Stefan


Good point. A live migration would be sufficient - if there are no bugs
in QEMU's live migration.


just migrating all our customer machines and wanted to be sure that live 
migration is enough.


Greets,
Stefan



Stefan




Re: [Qemu-devel] [Qemu-stable] [PATCH] fdc: force the fifo access to be in bounds of the allocated buffer

2015-05-13 Thread Stefan Priebe


Am 13.05.2015 um 21:04 schrieb John Snow:



On 05/13/2015 02:59 PM, Stefan Priebe wrote:


Am 13.05.2015 um 20:51 schrieb Stefan Weil:

Hi,

I just noticed this patch because my provider told me that my KVM based
server
needs a reboot because of a CVE (see this German news:
http://www.heise.de/newsticker/meldung/Venom-Schwachstelle-Aus-Hypervisor-ausbrechen-und-VMs-ausspionieren-2649614.html)



Isn't a live migration to a fixed version enough instead of a reboot?

Stefan




If your management API or host or whatever lets you migrate back to the
same host, or has another host they can migrate you to, yes.


i'm my host ;-) I was just interested if for whatever reason live 
migration is not enough and was wondering why someone wants to reboot a 
vm for something like this.


Thanks!

Stefan





Am 13.05.2015 um 16:33 schrieb John Snow:

From: Petr Matousek 

During processing of certain commands such as FD_CMD_READ_ID and
FD_CMD_DRIVE_SPECIFICATION_COMMAND the fifo memory access could
get out of bounds leading to memory corruption with values coming
from the guest.

Fix this by making sure that the index is always bounded by the
allocated memory.

This is CVE-2015-3456.

Signed-off-by: Petr Matousek 
Reviewed-by: John Snow 
Signed-off-by: John Snow 
---
   hw/block/fdc.c | 17 +++--
   1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index f72a392..d8a8edd 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -1497,7 +1497,7 @@ static uint32_t fdctrl_read_data(FDCtrl *fdctrl)
   {
   FDrive *cur_drv;
   uint32_t retval = 0;
-int pos;
+uint32_t pos;
   cur_drv = get_cur_drv(fdctrl);
   fdctrl->dsr &= ~FD_DSR_PWRDOWN;
@@ -1506,8 +1506,8 @@ static uint32_t fdctrl_read_data(FDCtrl *fdctrl)
   return 0;
   }
   pos = fdctrl->data_pos;
+pos %= FD_SECTOR_LEN;


I'd combine both statements and perhaps use fdctrl->fifo_size (even if
the resulting code will be slightly larger):

pos = fdctrl->data_pos % fdctrl->fifo_size;



   if (fdctrl->msr & FD_MSR_NONDMA) {
-pos %= FD_SECTOR_LEN;
   if (pos == 0) {
   if (fdctrl->data_pos != 0)
   if (!fdctrl_seek_to_next_sect(fdctrl, cur_drv)) {
@@ -1852,10 +1852,13 @@ static void fdctrl_handle_option(FDCtrl
*fdctrl, int direction)
   static void fdctrl_handle_drive_specification_command(FDCtrl
*fdctrl, int direction)
   {
   FDrive *cur_drv = get_cur_drv(fdctrl);
+uint32_t pos;
-if (fdctrl->fifo[fdctrl->data_pos - 1] & 0x80) {
+pos = fdctrl->data_pos - 1;
+pos %= FD_SECTOR_LEN;


Shorter (and more clear):

uint32_t pos = (fdctrl->data_pos - 1) % fdctrl->fifo_size;


+if (fdctrl->fifo[pos] & 0x80) {
   /* Command parameters done */
-if (fdctrl->fifo[fdctrl->data_pos - 1] & 0x40) {
+if (fdctrl->fifo[pos] & 0x40) {
   fdctrl->fifo[0] = fdctrl->fifo[1];
   fdctrl->fifo[2] = 0;
   fdctrl->fifo[3] = 0;
@@ -1955,7 +1958,7 @@ static uint8_t command_to_handler[256];
   static void fdctrl_write_data(FDCtrl *fdctrl, uint32_t value)
   {
   FDrive *cur_drv;
-int pos;
+uint32_t pos;
   /* Reset mode */
   if (!(fdctrl->dor & FD_DOR_nRESET)) {
@@ -2004,7 +2007,9 @@ static void fdctrl_write_data(FDCtrl *fdctrl,
uint32_t value)
   }
   FLOPPY_DPRINTF("%s: %02x\n", __func__, value);
-fdctrl->fifo[fdctrl->data_pos++] = value;
+pos = fdctrl->data_pos++;
+pos %= FD_SECTOR_LEN;
+fdctrl->fifo[pos] = value;
   if (fdctrl->data_pos == fdctrl->data_len) {
   /* We now have all parameters
* and will be able to treat the command


Not strictly related to this patch: The code which sets fifo_size could
also be improved.

  fdctrl->fifo = qemu_memalign(512, FD_SECTOR_LEN);
  fdctrl->fifo_size = 512;

The 2nd line should be

  fdctrl->fifo_size = FD_SECTOR_LEN;


As far as I see the original code can read or write illegal memory
locations in the address space of the QEMU process. It cannot (as it was
claimed) modify the code of the VM host because those memory is usually
write protected - at least if QEMU is running without KVM. If the code
which is generated for KVM is writable from anywhere in QEMU, we should
perhaps consider changing that.

Regards
Stefan










Re: [Qemu-devel] [Qemu-stable] [PATCH] fdc: force the fifo access to be in bounds of the allocated buffer

2015-05-13 Thread Stefan Priebe


Am 13.05.2015 um 20:51 schrieb Stefan Weil:

Hi,

I just noticed this patch because my provider told me that my KVM based
server
needs a reboot because of a CVE (see this German news:
http://www.heise.de/newsticker/meldung/Venom-Schwachstelle-Aus-Hypervisor-ausbrechen-und-VMs-ausspionieren-2649614.html)


Isn't a live migration to a fixed version enough instead of a reboot?

Stefan




Am 13.05.2015 um 16:33 schrieb John Snow:

From: Petr Matousek 

During processing of certain commands such as FD_CMD_READ_ID and
FD_CMD_DRIVE_SPECIFICATION_COMMAND the fifo memory access could
get out of bounds leading to memory corruption with values coming
from the guest.

Fix this by making sure that the index is always bounded by the
allocated memory.

This is CVE-2015-3456.

Signed-off-by: Petr Matousek 
Reviewed-by: John Snow 
Signed-off-by: John Snow 
---
  hw/block/fdc.c | 17 +++--
  1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index f72a392..d8a8edd 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -1497,7 +1497,7 @@ static uint32_t fdctrl_read_data(FDCtrl *fdctrl)
  {
  FDrive *cur_drv;
  uint32_t retval = 0;
-int pos;
+uint32_t pos;
  cur_drv = get_cur_drv(fdctrl);
  fdctrl->dsr &= ~FD_DSR_PWRDOWN;
@@ -1506,8 +1506,8 @@ static uint32_t fdctrl_read_data(FDCtrl *fdctrl)
  return 0;
  }
  pos = fdctrl->data_pos;
+pos %= FD_SECTOR_LEN;


I'd combine both statements and perhaps use fdctrl->fifo_size (even if
the resulting code will be slightly larger):

pos = fdctrl->data_pos % fdctrl->fifo_size;



  if (fdctrl->msr & FD_MSR_NONDMA) {
-pos %= FD_SECTOR_LEN;
  if (pos == 0) {
  if (fdctrl->data_pos != 0)
  if (!fdctrl_seek_to_next_sect(fdctrl, cur_drv)) {
@@ -1852,10 +1852,13 @@ static void fdctrl_handle_option(FDCtrl
*fdctrl, int direction)
  static void fdctrl_handle_drive_specification_command(FDCtrl
*fdctrl, int direction)
  {
  FDrive *cur_drv = get_cur_drv(fdctrl);
+uint32_t pos;
-if (fdctrl->fifo[fdctrl->data_pos - 1] & 0x80) {
+pos = fdctrl->data_pos - 1;
+pos %= FD_SECTOR_LEN;


Shorter (and more clear):

uint32_t pos = (fdctrl->data_pos - 1) % fdctrl->fifo_size;


+if (fdctrl->fifo[pos] & 0x80) {
  /* Command parameters done */
-if (fdctrl->fifo[fdctrl->data_pos - 1] & 0x40) {
+if (fdctrl->fifo[pos] & 0x40) {
  fdctrl->fifo[0] = fdctrl->fifo[1];
  fdctrl->fifo[2] = 0;
  fdctrl->fifo[3] = 0;
@@ -1955,7 +1958,7 @@ static uint8_t command_to_handler[256];
  static void fdctrl_write_data(FDCtrl *fdctrl, uint32_t value)
  {
  FDrive *cur_drv;
-int pos;
+uint32_t pos;
  /* Reset mode */
  if (!(fdctrl->dor & FD_DOR_nRESET)) {
@@ -2004,7 +2007,9 @@ static void fdctrl_write_data(FDCtrl *fdctrl,
uint32_t value)
  }
  FLOPPY_DPRINTF("%s: %02x\n", __func__, value);
-fdctrl->fifo[fdctrl->data_pos++] = value;
+pos = fdctrl->data_pos++;
+pos %= FD_SECTOR_LEN;
+fdctrl->fifo[pos] = value;
  if (fdctrl->data_pos == fdctrl->data_len) {
  /* We now have all parameters
   * and will be able to treat the command


Not strictly related to this patch: The code which sets fifo_size could
also be improved.

 fdctrl->fifo = qemu_memalign(512, FD_SECTOR_LEN);
 fdctrl->fifo_size = 512;

The 2nd line should be

 fdctrl->fifo_size = FD_SECTOR_LEN;


As far as I see the original code can read or write illegal memory
locations in the address space of the QEMU process. It cannot (as it was
claimed) modify the code of the VM host because those memory is usually
write protected - at least if QEMU is running without KVM. If the code
which is generated for KVM is writable from anywhere in QEMU, we should
perhaps consider changing that.

Regards
Stefan






Re: [Qemu-devel] Qemu 2.2.1 black screen of death in windows 2012 R2

2015-03-24 Thread Stefan Priebe

Hi,

it started to work again with virtio 100 instead of 94. No idea why it 
works with qemu 2.2.0.


Stefan

Am 24.03.2015 um 12:15 schrieb Stefan Priebe - Profihost AG:


Am 24.03.2015 um 11:45 schrieb Paolo Bonzini:



On 24/03/2015 11:39, Stefan Priebe - Profihost AG wrote:

after upgrading Qemu from 2.2.0 to 2.2.1

Windows 2012 R2 works after installing. But after applying 72 updates it
breaks with a black screen of death.


Can you bisect it?


Have to try might be possible.


Linking to this KB:
https://support.microsoft.com/en-us/kb/2939259


That KB, I think, refers to running a hypervisor (e.g. VMware
Workstation) _inside_ Windows.


Screenshot of the windows screen attached. This is just a blank windows
2012 R2 with virtio 94 drivers installed.

Stefan





Re: [Qemu-devel] Qemu 2.2.1 black screen of death in windows 2012 R2

2015-03-24 Thread Stefan Priebe - Profihost AG

Am 24.03.2015 um 11:45 schrieb Paolo Bonzini:
> 
> 
> On 24/03/2015 11:39, Stefan Priebe - Profihost AG wrote:
>> after upgrading Qemu from 2.2.0 to 2.2.1
>>
>> Windows 2012 R2 works after installing. But after applying 72 updates it
>> breaks with a black screen of death.
> 
> Can you bisect it?

Have to try might be possible.

>> Linking to this KB:
>> https://support.microsoft.com/en-us/kb/2939259
> 
> That KB, I think, refers to running a hypervisor (e.g. VMware
> Workstation) _inside_ Windows.

Screenshot of the windows screen attached. This is just a blank windows
2012 R2 with virtio 94 drivers installed.

Stefan


[Qemu-devel] Qemu 2.2.1 black screen of death in windows 2012 R2

2015-03-24 Thread Stefan Priebe - Profihost AG
Hi,

after upgrading Qemu from 2.2.0 to 2.2.1

Windows 2012 R2 works after installing. But after applying 72 updates it
breaks with a black screen of death.

Linking to this KB:
https://support.microsoft.com/en-us/kb/2939259

It works fine with Qemu 2.2.0

Greets,
Stefan



Re: [Qemu-devel] live migration fails after host kernel upgrade (3.12 => 3.18)

2015-03-23 Thread Stefan Priebe - Profihost AG
Hi,

Thanks.

I fixed it - there is already a patchseries in 4.0 to fix this. It will
be backported in 3.18.10 or 3.18.11.

Stefan

Am 23.03.2015 um 12:54 schrieb Stefan Hajnoczi:
> On Sun, Mar 15, 2015 at 09:59:25AM +0100, Stefan Priebe wrote:
>> after upgrading the host kernel from 3.12 to 3.18 live migration fails with
>> the following qemu output (guest running on a host with 3.12 => host with
>> 3.18):
>>
>> kvm: Features 0x30afffe3 unsupported. Allowed features: 0x79bfbbe7
>> qemu: warning: error while loading state for instance 0x0 of device
>> ':00:12.0/virtio-net'
>> kvm: load of migration failed: Operation not permitted
>>
>> But i can't
> 
> Hi Stefan,
> I haven't checked the exact feature bits but it might be the UFO feature
> problem.  The tun driver was changed to drop a feature, this broke live
> migration.
> 
> Take a look at
> https://lists.linuxfoundation.org/pipermail/virtualization/2015-February/029217.html
> 
> Stefan
> 



[Qemu-devel] live migration fails after host kernel upgrade (3.12 => 3.18)

2015-03-15 Thread Stefan Priebe

Hi,

after upgrading the host kernel from 3.12 to 3.18 live migration fails 
with the following qemu output (guest running on a host with 3.12 => 
host with 3.18):


kvm: Features 0x30afffe3 unsupported. Allowed features: 0x79bfbbe7
qemu: warning: error while loading state for instance 0x0 of device 
':00:12.0/virtio-net'

kvm: load of migration failed: Operation not permitted

But i can't

Greets,
Stefan



Re: [Qemu-devel] slow speed for virtio-scsi since qemu 2.2

2015-02-16 Thread Stefan Priebe


Am 16.02.2015 um 16:50 schrieb Andreas Färber:

Am 16.02.2015 um 16:41 schrieb Stefan Priebe - Profihost AG:

Am 16.02.2015 um 15:49 schrieb Paolo Bonzini :



On 16/02/2015 15:47, Stefan Priebe - Profihost AG wrote:


Could it be that this is a results of compiling qemu with --enable-debug
to get debugging symbols?


Yes.

*urg* my fault - sorry. Is there a way to get debugging symbols without
using enable-debug / getting slower performance?


Yes, just do nothing ("--enable-debug-info" is the default;
"--enable-debug" enables debug info _and_ turns off optimization).


If I do not enable-debug dh_strip does not extract any debugging symbols.


Maybe you forgot --disable-strip?


Thanks!

Stefan



Regards,
Andreas





Re: [Qemu-devel] slow speed for virtio-scsi since qemu 2.2

2015-02-16 Thread Stefan Priebe


Am 16.02.2015 um 16:49 schrieb Paolo Bonzini:



On 16/02/2015 16:41, Stefan Priebe - Profihost AG wrote:

Yes, just do nothing ("--enable-debug-info" is the default;
"--enable-debug" enables debug info _and_ turns off optimization).

If I do not enable-debug dh_strip does not extract any debugging symbols.


Then you need --disable-strip too.


Thanks!

Stefan



Paolo





Re: [Qemu-devel] slow speed for virtio-scsi since qemu 2.2

2015-02-16 Thread Stefan Priebe - Profihost AG

> Am 16.02.2015 um 15:49 schrieb Paolo Bonzini :
> 
> 
> 
> On 16/02/2015 15:47, Stefan Priebe - Profihost AG wrote:
>>>>>> 
>>>>>> Could it be that this is a results of compiling qemu with --enable-debug
>>>>>> to get debugging symbols?
>>>> 
>>>> Yes.
>> *urg* my fault - sorry. Is there a way to get debugging symbols without
>> using enable-debug / getting slower performance?
> 
> Yes, just do nothing ("--enable-debug-info" is the default;
> "--enable-debug" enables debug info _and_ turns off optimization).

If I do not enable-debug dh_strip does not extract any debugging symbols.

Stefan 

> 
> Paolo



Re: [Qemu-devel] slow speed for virtio-scsi since qemu 2.2

2015-02-16 Thread Stefan Priebe - Profihost AG
Hi,

> Am 16.02.2015 um 15:58 schrieb Andrey Korolyov :
> 
> On Mon, Feb 16, 2015 at 5:47 PM, Stefan Priebe - Profihost AG
>  wrote:
>> Hi,
>>> Am 16.02.2015 um 15:44 schrieb Paolo Bonzini:
>>> 
>>> 
>>>> On 16/02/2015 15:43, Stefan Priebe - Profihost AG wrote:
>>>> Hi,
>>>> 
>>>>> Am 16.02.2015 um 13:24 schrieb Paolo Bonzini:
>>>>> 
>>>>> 
>>>>>> On 15/02/2015 19:46, Stefan Priebe wrote:
>>>>>> 
>>>>>> while i get a constant random 4k i/o write speed of 20.000 iops with
>>>>>> qemu 2.1.0 or 2.1.3. I get jumping speeds with qemu 2.2 (jumping between
>>>>>> 500 iops and 15.000 iop/s).
>>>>>> 
>>>>>> If i use virtio instead of virtio-scsi speed is the same between 2.2 and
>>>>>> 2.1.
>>>>> 
>>>>> http://wiki.qemu.org/Contribute/ReportABug
>>>>> 
>>>>> What is your command line?
>>>> 
>>>> Could it be that this is a results of compiling qemu with --enable-debug
>>>> to get debugging symbols?
>>> 
>>> Yes.
>> 
>> *urg* my fault - sorry. Is there a way to get debugging symbols without
>> using enable-debug / getting slower performance?
> 
> Hi Stefan, splitdebug will help you there. Also this is a standard way
> to ship debugging symbols in most distribuitions, so I wonder if you
> may use just a generic build skeleton for this task...

I'm using enable-debug with dh_strip from Debian

Stefan




Re: [Qemu-devel] slow speed for virtio-scsi since qemu 2.2

2015-02-16 Thread Stefan Priebe - Profihost AG
Hi,
Am 16.02.2015 um 15:44 schrieb Paolo Bonzini:
> 
> 
> On 16/02/2015 15:43, Stefan Priebe - Profihost AG wrote:
>> Hi,
>>
>> Am 16.02.2015 um 13:24 schrieb Paolo Bonzini:
>>>
>>>
>>> On 15/02/2015 19:46, Stefan Priebe wrote:
>>>>
>>>> while i get a constant random 4k i/o write speed of 20.000 iops with
>>>> qemu 2.1.0 or 2.1.3. I get jumping speeds with qemu 2.2 (jumping between
>>>> 500 iops and 15.000 iop/s).
>>>>
>>>> If i use virtio instead of virtio-scsi speed is the same between 2.2 and
>>>> 2.1.
>>>
>>> http://wiki.qemu.org/Contribute/ReportABug
>>>
>>> What is your command line?
>>
>> Could it be that this is a results of compiling qemu with --enable-debug
>> to get debugging symbols?
> 
> Yes.

*urg* my fault - sorry. Is there a way to get debugging symbols without
using enable-debug / getting slower performance?

Greets,
Stefan



Re: [Qemu-devel] slow speed for virtio-scsi since qemu 2.2

2015-02-16 Thread Stefan Priebe - Profihost AG
Hi,

Am 16.02.2015 um 13:24 schrieb Paolo Bonzini:
> 
> 
> On 15/02/2015 19:46, Stefan Priebe wrote:
>>
>> while i get a constant random 4k i/o write speed of 20.000 iops with
>> qemu 2.1.0 or 2.1.3. I get jumping speeds with qemu 2.2 (jumping between
>> 500 iops and 15.000 iop/s).
>>
>> If i use virtio instead of virtio-scsi speed is the same between 2.2 and
>> 2.1.
> 
> http://wiki.qemu.org/Contribute/ReportABug
> 
> What is your command line?
> 
> Paolo
> 

Could it be that this is a results of compiling qemu with --enable-debug
to get debugging symbols?

Stefan



[Qemu-devel] slow speed for virtio-scsi since qemu 2.2

2015-02-15 Thread Stefan Priebe

Hi,

while i get a constant random 4k i/o write speed of 20.000 iops with 
qemu 2.1.0 or 2.1.3. I get jumping speeds with qemu 2.2 (jumping between 
500 iops and 15.000 iop/s).


If i use virtio instead of virtio-scsi speed is the same between 2.2 and 
2.1.


Is there a known regression?

Greets,
Stefan



[Qemu-devel] kvm segfault in iov_to_buf / virtio_scsi_parse_req

2014-10-18 Thread Stefan Priebe

Hi,

while migrating a bunch of VMs i saw multiple times segaults with qemu 
2.1.2.


Is this a known bug?

Full backtrace:
Program terminated with signal 11, Segmentation fault.
#0  0x7ff9c73bca90 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0  0x7ff9c73bca90 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x7ff9cde46b8e in iov_to_buf (iov=0x7ff9b805c0b0, iov_cnt=1, 
offset=0, buf=0x7ff9b8060118, bytes=51) at util/iov.c:49
#2  0x7ff9cdacdc56 in virtio_scsi_parse_req (req=0x7ff9b8054090, 
req_size=51, resp_size=108)

at qemu-kvm/hw/scsi/virtio-scsi.c:140
#3  0x7ff9cdace060 in virtio_scsi_load_request (f=0x7ff9b803b620, 
sreq=0x7ff9b8089550)

at qemu-kvm/hw/scsi/virtio-scsi.c:215
#4  0x7ff9cdcf5737 in get_scsi_requests (f=0x7ff9b803b620, 
pv=0x7ff9cfe4a440, size=0) at hw/scsi/scsi-bus.c:1875
#5  0x7ff9cdb76e0c in vmstate_load_state (f=0x7ff9b803b620, 
vmsd=0x7ff9ce24cac0, opaque=0x7ff9cfe4a440, version_id=1)

at vmstate.c:105
#6  0x7ff9cdb76de8 in vmstate_load_state (f=0x7ff9b803b620, 
vmsd=0x7ff9ce24c500, opaque=0x7ff9cfe4a440, version_id=1)

at vmstate.c:102
#7  0x7ff9cda935b6 in vmstate_load (f=0x7ff9b803b620, 
se=0x7ff9cfe95570, version_id=1)

at qemu-kvm/savevm.c:572
#8  0x7ff9cda94652 in qemu_loadvm_state (f=0x7ff9b803b620)
at qemu-kvm/savevm.c:951
#9  0x7ff9cdb748bd in process_incoming_migration_co 
(opaque=0x7ff9b803b620) at migration.c:97
#10 0x7ff9cdddb433 in coroutine_trampoline (i0=-1207852976, 
i1=32761) at coroutine-ucontext.c:118

#11 0x7ff9c72dbd10 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x7fff5e55a830 in ?? ()
#13 0x in ?? ()

Stefan



Re: [Qemu-devel] [PATCH] aio: fix qemu_bh_schedule() bh->ctx race condition

2014-06-03 Thread Stefan Priebe - Profihost AG
Tested-by: Stefan Priebe 

Am 03.06.2014 11:21, schrieb Stefan Hajnoczi:
> qemu_bh_schedule() is supposed to be thread-safe at least the first time
> it is called.  Unfortunately this is not quite true:
> 
>   bh->scheduled = 1;
>   aio_notify(bh->ctx);
> 
> Since another thread may run the BH callback once it has been scheduled,
> there is a race condition if the callback frees the BH before
> aio_notify(bh->ctx) has a chance to run.
> 
> Reported-by: Stefan Priebe 
> Signed-off-by: Stefan Hajnoczi 
> ---
>  async.c | 14 ++
>  1 file changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/async.c b/async.c
> index 6930185..5b6fe6b 100644
> --- a/async.c
> +++ b/async.c
> @@ -117,15 +117,21 @@ void qemu_bh_schedule_idle(QEMUBH *bh)
>  
>  void qemu_bh_schedule(QEMUBH *bh)
>  {
> +AioContext *ctx;
> +
>  if (bh->scheduled)
>  return;
> +ctx = bh->ctx;
>  bh->idle = 0;
> -/* Make sure that idle & any writes needed by the callback are done
> - * before the locations are read in the aio_bh_poll.
> +/* Make sure that:
> + * 1. idle & any writes needed by the callback are done before the
> + *locations are read in the aio_bh_poll.
> + * 2. ctx is loaded before scheduled is set and the callback has a chance
> + *to execute.
>   */
> -smp_wmb();
> +smp_mb();
>  bh->scheduled = 1;
> -aio_notify(bh->ctx);
> +aio_notify(ctx);
>  }
>  
>  
> 



Re: [Qemu-devel] qemu 2.0 segfaults in event notifier

2014-06-02 Thread Stefan Priebe

Am 02.06.2014 22:45, schrieb Paolo Bonzini:

Il 02/06/2014 21:32, Stefan Priebe ha scritto:


#0  0x7f69e421c43f in event_notifier_set (e=0x124) at
util/event_notifier-posix.c:97
#1  0x7f69e3e37afc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f69e3e37697 in qemu_bh_schedule (bh=0x7f5dac217f60) at
async.c:128
#3  0x7f69e3e8bc44 in rbd_finish_aiocb (c=0x7f5dac0c3f30,
rcb=0x7f5dafa50610) at block/rbd.c:585
#4  0x7f69e17bee44 in librbd::AioCompletion::complete() () from
/usr/lib/librbd.so.1
#5  0x7f69e17be832 in
librbd::AioCompletion::complete_request(CephContext*, long) () from
/usr/lib/librbd.so.1
#6  0x7f69e1c946ba in Context::complete(int) () from
/usr/lib/librados.so.2
#7  0x7f69e17f1e85 in ObjectCacher::C_WaitForWrite::finish(int) ()
from /usr/lib/librbd.so.1
#8  0x7f69e1c946ba in Context::complete(int) () from
/usr/lib/librados.so.2
#9  0x7f69e1d373c8 in Finisher::finisher_thread_entry() () from
/usr/lib/librados.so.2
#10 0x7f69dbd43b50 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#11 0x7f69dba8e13d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()


Can you also print qemu_aio_context?


(gdb) print  qemu_aio_context
$1 = (AioContext *) 0x7f69e68a4e00

(gdb) print *(AioContext*)0x7f69e68a4e00
$2 = {source = {callback_data = 0x0, callback_funcs = 0x0, source_funcs 
= 0x7f69e462d020, ref_count = 2, context = 0x7f69e68a5190,
priority = 0, flags = 1, source_id = 1, poll_fds = 0x7f69e686aea0, 
prev = 0x0, next = 0x7f69e743ccd0, name = 0x0, priv = 0x0},
  lock = {lock = {lock = {__data = {__lock = 0, __count = 0, __owner = 
0, __nusers = 0, __kind = 2, __spins = 0, __list = {
__prev = 0x0, __next = 0x0}}, __size = '\000' times>, "\002", '\000' , __align = 0}},
head = 0, tail = 0, cond = {cond = {__data = {__lock = 0, __futex = 
0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0,
  __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = 
'\000' , __align = 0}}, owner_thread = {
  thread = 0}, nesting = 0, cb = 0x7f69e3e37b4f , 
cb_opaque = 0x7f69e68a4e00}, aio_handlers = {
lh_first = 0x7f69e68a4f60}, walking_handlers = 0, bh_lock = {lock = 
{__data = {__lock = 0, __count = 0, __owner = 0,
__nusers = 0, __kind = 2, __spins = 0, __list = {__prev = 0x0, 
__next = 0x0}},
  __size = '\000' , "\002", '\000' times>, __align = 0}}, first_bh = 0x7f69e6b04d10,
  walking_bh = 0, notifier = {rfd = 4, wfd = 4}, pollfds = 
0x7f69e68a4630, thread_pool = 0x0, tlg = {tl = {0x7f69e68a4fa0,

  0x7f69e68a5010, 0x7f69e68a5080}}}


 Also print the backtrace of all
threads, using "thread apply all bt full".


http://pastebin.com/raw.php?i=uzcpN0zk

THanks,
Stefan



Re: [Qemu-devel] qemu 2.0 segfaults in event notifier

2014-06-02 Thread Stefan Priebe

Am 02.06.2014 15:40, schrieb Stefan Hajnoczi:

On Fri, May 30, 2014 at 04:10:39PM +0200, Stefan Priebe wrote:

even with
+From 271c0f68b4eae72691721243a1c37f46a3232d61 Mon Sep 17 00:00:00 2001
+From: Fam Zheng 
+Date: Wed, 21 May 2014 10:42:13 +0800
+Subject: [PATCH] aio: Fix use-after-free in cancellation path

applied i saw today segfault with the following backtrace:

Program terminated with signal 11, Segmentation fault.
#0  0x7f9dd633343f in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97
97  util/event_notifier-posix.c: No such file or directory.
(gdb) bt
#0  0x7f9dd633343f in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97
#1  0x7f9dd5f4eafc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f9dd5f4e697 in qemu_bh_schedule (bh=0x7f9b98eeeb30) at async.c:128
#3  0x7f9dd5fa2c44 in rbd_finish_aiocb (c=0x7f9dd9069ad0, 
rcb=0x7f9dd85f1770) at block/rbd.c:585


Hi Stefan,
Please print the QEMUBH:
(gdb) p *(QEMUBH*)0x7f9b98eeeb30


new trace:
(gdb) bt
#0  0x7f69e421c43f in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97

#1  0x7f69e3e37afc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f69e3e37697 in qemu_bh_schedule (bh=0x7f5dac217f60) at 
async.c:128
#3  0x7f69e3e8bc44 in rbd_finish_aiocb (c=0x7f5dac0c3f30, 
rcb=0x7f5dafa50610) at block/rbd.c:585
#4  0x7f69e17bee44 in librbd::AioCompletion::complete() () from 
/usr/lib/librbd.so.1
#5  0x7f69e17be832 in 
librbd::AioCompletion::complete_request(CephContext*, long) () from 
/usr/lib/librbd.so.1
#6  0x7f69e1c946ba in Context::complete(int) () from 
/usr/lib/librados.so.2
#7  0x7f69e17f1e85 in ObjectCacher::C_WaitForWrite::finish(int) () 
from /usr/lib/librbd.so.1
#8  0x7f69e1c946ba in Context::complete(int) () from 
/usr/lib/librados.so.2
#9  0x7f69e1d373c8 in Finisher::finisher_thread_entry() () from 
/usr/lib/librados.so.2
#10 0x7f69dbd43b50 in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0

#11 0x7f69dba8e13d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()

this i another core dump so address differ:
(gdb) p *(QEMUBH*)0x7f5dac217f60
$1 = {ctx = 0x0, cb = 0x7f69e3e8bb75 , opaque = 
0x7f5dafa50610, next = 0x7f69e6b04d10, scheduled = false,

  idle = false, deleted = true}


It would also be interesting to print out the qemu_aio_context->first_bh
linked list of QEMUBH structs to check whether 0x7f9b98eeeb30 is on the
list.


Do you mean just this:
(gdb) p *(QEMUBH*)qemu_aio_context->first_bh
$3 = {ctx = 0x7f69e68a4e00, cb = 0x7f69e41546a5 , 
opaque = 0x7f69e6b4a5e0, next = 0x7f69e6b4a570,

  scheduled = false, idle = false, deleted = false}


The aio_bh_new() and aio_bh_schedule() APIs are supposed to be
thread-safe.  In theory the rbd.c code is fine.  But maybe there is a
race condition somewhere.


rbd.c was fine with 1.7.0

Stefan



Re: [Qemu-devel] qemu 2.0 segfaults in event notifier

2014-06-02 Thread Stefan Priebe - Profihost AG

> Am 02.06.2014 um 15:40 schrieb Stefan Hajnoczi :
> 
>> On Fri, May 30, 2014 at 04:10:39PM +0200, Stefan Priebe wrote:
>> even with
>> +From 271c0f68b4eae72691721243a1c37f46a3232d61 Mon Sep 17 00:00:00 2001
>> +From: Fam Zheng 
>> +Date: Wed, 21 May 2014 10:42:13 +0800
>> +Subject: [PATCH] aio: Fix use-after-free in cancellation path
>> 
>> applied i saw today segfault with the following backtrace:
>> 
>> Program terminated with signal 11, Segmentation fault.
>> #0  0x7f9dd633343f in event_notifier_set (e=0x124) at 
>> util/event_notifier-posix.c:97
>> 97  util/event_notifier-posix.c: No such file or directory.
>> (gdb) bt
>> #0  0x7f9dd633343f in event_notifier_set (e=0x124) at 
>> util/event_notifier-posix.c:97
>> #1  0x7f9dd5f4eafc in aio_notify (ctx=0x0) at async.c:246
>> #2  0x7f9dd5f4e697 in qemu_bh_schedule (bh=0x7f9b98eeeb30) at async.c:128
>> #3  0x7f9dd5fa2c44 in rbd_finish_aiocb (c=0x7f9dd9069ad0, 
>> rcb=0x7f9dd85f1770) at block/rbd.c:585
> 
> Hi Stefan,
> Please print the QEMUBH:
> (gdb) p *(QEMUBH*)0x7f9b98eeeb30
> 
> It would also be interesting to print out the qemu_aio_context->first_bh
> linked list of QEMUBH structs to check whether 0x7f9b98eeeb30 is on the
> list.
> 
> The aio_bh_new() and aio_bh_schedule() APIs are supposed to be
> thread-safe.  In theory the rbd.c code is fine.  But maybe there is a
> race condition somewhere.
> 
> If you want to debug interactively, ping me on #qemu on irc.oftc.net.

Hi,

that would be great what's your username? On trip right now. Will be on irc in 
4-5 hours or tomorrow in 16 hours.

Greets,
Stefan


[Qemu-devel] qemu 2.0 segfaults in event notifier

2014-05-30 Thread Stefan Priebe
Hi,

even with
+From 271c0f68b4eae72691721243a1c37f46a3232d61 Mon Sep 17 00:00:00 2001
+From: Fam Zheng 
+Date: Wed, 21 May 2014 10:42:13 +0800
+Subject: [PATCH] aio: Fix use-after-free in cancellation path

applied i saw today segfault with the following backtrace:

Program terminated with signal 11, Segmentation fault.
#0  0x7f9dd633343f in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97
97  util/event_notifier-posix.c: No such file or directory.
(gdb) bt
#0  0x7f9dd633343f in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97
#1  0x7f9dd5f4eafc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f9dd5f4e697 in qemu_bh_schedule (bh=0x7f9b98eeeb30) at async.c:128
#3  0x7f9dd5fa2c44 in rbd_finish_aiocb (c=0x7f9dd9069ad0, 
rcb=0x7f9dd85f1770) at block/rbd.c:585
#4  0x7f9dd38d5e44 in librbd::AioCompletion::complete() () from 
/usr/lib/librbd.so.1
#5  0x7f9dd38d5832 in librbd::AioCompletion::complete_request(CephContext*, 
long) () from /usr/lib/librbd.so.1
#6  0x7f9dd3dab6ba in Context::complete(int) () from /usr/lib/librados.so.2
#7  0x7f9dd3908e85 in ObjectCacher::C_WaitForWrite::finish(int) () from 
/usr/lib/librbd.so.1
#8  0x7f9dd3dab6ba in Context::complete(int) () from /usr/lib/librados.so.2
#9  0x7f9dd3e4e3c8 in Finisher::finisher_thread_entry() () from 
/usr/lib/librados.so.2
#10 0x7f9dcde5ab50 in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#11 0x7f9dcdba513d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()


Am 28.05.2014 21:44, schrieb Stefan Priebe:
> is this:
> commit 271c0f68b4eae72691721243a1c37f46a3232d61
> Author: Fam Zheng 
> Date:   Wed May 21 10:42:13 2014 +0800
> 
> aio: Fix use-after-free in cancellation path
> 
> Stefan
> 
> Am 28.05.2014 21:40, schrieb Stefan Priebe:
>> Hello,
>> 
>> i mean since using qemu 2.0 i've now seen several times the following
>> segfault:
>> (gdb) bt
>> #0  0x7f2af1196433 in event_notifier_set (e=0x124) at
>> util/event_notifier-posix.c:97
>> #1  0x7f2af0db1afc in aio_notify (ctx=0x0) at async.c:246
>> #2  0x7f2af0db1697 in qemu_bh_schedule (bh=0x7f2ad401bec0) at
>> async.c:128
>> #3  0x7f2af0e05c44 in rbd_finish_aiocb (c=0x7f2ad5ec4590,
>> rcb=0x7f2ad63c5df0) at block/rbd.c:585
>> #4  0x7f2aee738e44 in librbd::AioCompletion::complete() () from
>> /usr/lib/librbd.so.1
>> #5  0x7f2aee738832 in
>> librbd::AioCompletion::complete_request(CephContext*, long) () from
>> /usr/lib/librbd.so.1
>> #6  0x7f2aeec0e6ba in Context::complete(int) () from
>> /usr/lib/librados.so.2
>> #7  0x7f2aee76be85 in ObjectCacher::C_WaitForWrite::finish(int) ()
>> from /usr/lib/librbd.so.1
>> #8  0x7f2aeec0e6ba in Context::complete(int) () from
>> /usr/lib/librados.so.2
>> #9  0x7f2aeecb13c8 in Finisher::finisher_thread_entry() () from
>> /usr/lib/librados.so.2
>> #10 0x7f2ae8cbdb50 in start_thread () from
>> /lib/x86_64-linux-gnu/libpthread.so.0
>> #11 0x7f2ae8a080ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
>> #12 0x in ?? ()
>> (gdb)
>> 
>> 
>> from another VM:
>> #0  0x7f89565ec433 in event_notifier_set (e=0x124) at
>> util/event_notifier-posix.c:97
>> #1  0x7f8956207afc in aio_notify (ctx=0x0) at async.c:246
>> #2  0x7f8956207697 in qemu_bh_schedule (bh=0x7f882dd6d340) at
>> async.c:128
>> #3  0x7f895625bc44 in rbd_finish_aiocb (c=0x7f882d4c34a0,
>> rcb=0x7f882c0ae350) at block/rbd.c:585
>> #4  0x7f8953b8ee44 in librbd::AioCompletion::complete() () from
>> /usr/lib/librbd.so.1
>> #5  0x7f8953b8e832 in
>> librbd::AioCompletion::complete_request(CephContext*, long) () from
>> /usr/lib/librbd.so.1
>> #6  0x7f89540646ba in Context::complete(int) () from
>> /usr/lib/librados.so.2
>> #7  0x7f8953bc1e85 in ObjectCacher::C_WaitForWrite::finish(int) ()
>> from /usr/lib/librbd.so.1
>> #8  0x7f89540646ba in Context::complete(int) () from
>> /usr/lib/librados.so.2
>> #9  0x7f89541073c8 in Finisher::finisher_thread_entry() () from
>> /usr/lib/librados.so.2
>> #10 0x7f894e113b50 in start_thread () from
>> /lib/x86_64-linux-gnu/libpthread.so.0
>> #11 0x7f894de5e0ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
>> #12 0x in ?? ()
>> 
>> Stefan
>> 



Re: [Qemu-devel] [Qemu-stable] qemu segfault in librbd / aio_notify

2014-05-30 Thread Stefan Priebe

Hi,

even with
+From 271c0f68b4eae72691721243a1c37f46a3232d61 Mon Sep 17 00:00:00 2001
+From: Fam Zheng 
+Date: Wed, 21 May 2014 10:42:13 +0800
+Subject: [PATCH] aio: Fix use-after-free in cancellation path

applied i saw today segfault with the following backtrace:

Program terminated with signal 11, Segmentation fault.
#0  0x7f9dd633343f in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97

97  util/event_notifier-posix.c: No such file or directory.
(gdb) bt
#0  0x7f9dd633343f in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97

#1  0x7f9dd5f4eafc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f9dd5f4e697 in qemu_bh_schedule (bh=0x7f9b98eeeb30) at 
async.c:128
#3  0x7f9dd5fa2c44 in rbd_finish_aiocb (c=0x7f9dd9069ad0, 
rcb=0x7f9dd85f1770) at block/rbd.c:585
#4  0x7f9dd38d5e44 in librbd::AioCompletion::complete() () from 
/usr/lib/librbd.so.1
#5  0x7f9dd38d5832 in 
librbd::AioCompletion::complete_request(CephContext*, long) () from 
/usr/lib/librbd.so.1
#6  0x7f9dd3dab6ba in Context::complete(int) () from 
/usr/lib/librados.so.2
#7  0x7f9dd3908e85 in ObjectCacher::C_WaitForWrite::finish(int) () 
from /usr/lib/librbd.so.1
#8  0x7f9dd3dab6ba in Context::complete(int) () from 
/usr/lib/librados.so.2
#9  0x7f9dd3e4e3c8 in Finisher::finisher_thread_entry() () from 
/usr/lib/librados.so.2
#10 0x7f9dcde5ab50 in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0

#11 0x7f9dcdba513d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()


Am 28.05.2014 21:44, schrieb Stefan Priebe:

is this:
commit 271c0f68b4eae72691721243a1c37f46a3232d61
Author: Fam Zheng 
Date:   Wed May 21 10:42:13 2014 +0800

 aio: Fix use-after-free in cancellation path

Stefan

Am 28.05.2014 21:40, schrieb Stefan Priebe:

Hello,

i mean since using qemu 2.0 i've now seen several times the following
segfault:
(gdb) bt
#0  0x7f2af1196433 in event_notifier_set (e=0x124) at
util/event_notifier-posix.c:97
#1  0x7f2af0db1afc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f2af0db1697 in qemu_bh_schedule (bh=0x7f2ad401bec0) at
async.c:128
#3  0x7f2af0e05c44 in rbd_finish_aiocb (c=0x7f2ad5ec4590,
rcb=0x7f2ad63c5df0) at block/rbd.c:585
#4  0x7f2aee738e44 in librbd::AioCompletion::complete() () from
/usr/lib/librbd.so.1
#5  0x7f2aee738832 in
librbd::AioCompletion::complete_request(CephContext*, long) () from
/usr/lib/librbd.so.1
#6  0x7f2aeec0e6ba in Context::complete(int) () from
/usr/lib/librados.so.2
#7  0x7f2aee76be85 in ObjectCacher::C_WaitForWrite::finish(int) ()
from /usr/lib/librbd.so.1
#8  0x7f2aeec0e6ba in Context::complete(int) () from
/usr/lib/librados.so.2
#9  0x7f2aeecb13c8 in Finisher::finisher_thread_entry() () from
/usr/lib/librados.so.2
#10 0x7f2ae8cbdb50 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#11 0x7f2ae8a080ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()
(gdb)


from another VM:
#0  0x7f89565ec433 in event_notifier_set (e=0x124) at
util/event_notifier-posix.c:97
#1  0x7f8956207afc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f8956207697 in qemu_bh_schedule (bh=0x7f882dd6d340) at
async.c:128
#3  0x7f895625bc44 in rbd_finish_aiocb (c=0x7f882d4c34a0,
rcb=0x7f882c0ae350) at block/rbd.c:585
#4  0x7f8953b8ee44 in librbd::AioCompletion::complete() () from
/usr/lib/librbd.so.1
#5  0x7f8953b8e832 in
librbd::AioCompletion::complete_request(CephContext*, long) () from
/usr/lib/librbd.so.1
#6  0x7f89540646ba in Context::complete(int) () from
/usr/lib/librados.so.2
#7  0x7f8953bc1e85 in ObjectCacher::C_WaitForWrite::finish(int) ()
from /usr/lib/librbd.so.1
#8  0x7f89540646ba in Context::complete(int) () from
/usr/lib/librados.so.2
#9  0x7f89541073c8 in Finisher::finisher_thread_entry() () from
/usr/lib/librados.so.2
#10 0x7f894e113b50 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#11 0x7f894de5e0ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()

Stefan





Re: [Qemu-devel] [Qemu-stable] qemu segfault in librbd / aio_notify

2014-05-28 Thread Stefan Priebe

is this:
commit 271c0f68b4eae72691721243a1c37f46a3232d61
Author: Fam Zheng 
Date:   Wed May 21 10:42:13 2014 +0800

aio: Fix use-after-free in cancellation path

Stefan

Am 28.05.2014 21:40, schrieb Stefan Priebe:

Hello,

i mean since using qemu 2.0 i've now seen several times the following
segfault:
(gdb) bt
#0  0x7f2af1196433 in event_notifier_set (e=0x124) at
util/event_notifier-posix.c:97
#1  0x7f2af0db1afc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f2af0db1697 in qemu_bh_schedule (bh=0x7f2ad401bec0) at
async.c:128
#3  0x7f2af0e05c44 in rbd_finish_aiocb (c=0x7f2ad5ec4590,
rcb=0x7f2ad63c5df0) at block/rbd.c:585
#4  0x7f2aee738e44 in librbd::AioCompletion::complete() () from
/usr/lib/librbd.so.1
#5  0x7f2aee738832 in
librbd::AioCompletion::complete_request(CephContext*, long) () from
/usr/lib/librbd.so.1
#6  0x7f2aeec0e6ba in Context::complete(int) () from
/usr/lib/librados.so.2
#7  0x7f2aee76be85 in ObjectCacher::C_WaitForWrite::finish(int) ()
from /usr/lib/librbd.so.1
#8  0x7f2aeec0e6ba in Context::complete(int) () from
/usr/lib/librados.so.2
#9  0x7f2aeecb13c8 in Finisher::finisher_thread_entry() () from
/usr/lib/librados.so.2
#10 0x7f2ae8cbdb50 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#11 0x7f2ae8a080ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()
(gdb)


from another VM:
#0  0x7f89565ec433 in event_notifier_set (e=0x124) at
util/event_notifier-posix.c:97
#1  0x7f8956207afc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f8956207697 in qemu_bh_schedule (bh=0x7f882dd6d340) at
async.c:128
#3  0x7f895625bc44 in rbd_finish_aiocb (c=0x7f882d4c34a0,
rcb=0x7f882c0ae350) at block/rbd.c:585
#4  0x7f8953b8ee44 in librbd::AioCompletion::complete() () from
/usr/lib/librbd.so.1
#5  0x7f8953b8e832 in
librbd::AioCompletion::complete_request(CephContext*, long) () from
/usr/lib/librbd.so.1
#6  0x7f89540646ba in Context::complete(int) () from
/usr/lib/librados.so.2
#7  0x7f8953bc1e85 in ObjectCacher::C_WaitForWrite::finish(int) ()
from /usr/lib/librbd.so.1
#8  0x7f89540646ba in Context::complete(int) () from
/usr/lib/librados.so.2
#9  0x7f89541073c8 in Finisher::finisher_thread_entry() () from
/usr/lib/librados.so.2
#10 0x7f894e113b50 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#11 0x7f894de5e0ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()

Stefan





[Qemu-devel] qemu segfault in librbd / aio_notify

2014-05-28 Thread Stefan Priebe

Hello,

i mean since using qemu 2.0 i've now seen several times the following 
segfault:

(gdb) bt
#0  0x7f2af1196433 in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97

#1  0x7f2af0db1afc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f2af0db1697 in qemu_bh_schedule (bh=0x7f2ad401bec0) at 
async.c:128
#3  0x7f2af0e05c44 in rbd_finish_aiocb (c=0x7f2ad5ec4590, 
rcb=0x7f2ad63c5df0) at block/rbd.c:585
#4  0x7f2aee738e44 in librbd::AioCompletion::complete() () from 
/usr/lib/librbd.so.1
#5  0x7f2aee738832 in 
librbd::AioCompletion::complete_request(CephContext*, long) () from 
/usr/lib/librbd.so.1
#6  0x7f2aeec0e6ba in Context::complete(int) () from 
/usr/lib/librados.so.2
#7  0x7f2aee76be85 in ObjectCacher::C_WaitForWrite::finish(int) () 
from /usr/lib/librbd.so.1
#8  0x7f2aeec0e6ba in Context::complete(int) () from 
/usr/lib/librados.so.2
#9  0x7f2aeecb13c8 in Finisher::finisher_thread_entry() () from 
/usr/lib/librados.so.2
#10 0x7f2ae8cbdb50 in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0

#11 0x7f2ae8a080ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()
(gdb)


from another VM:
#0  0x7f89565ec433 in event_notifier_set (e=0x124) at 
util/event_notifier-posix.c:97

#1  0x7f8956207afc in aio_notify (ctx=0x0) at async.c:246
#2  0x7f8956207697 in qemu_bh_schedule (bh=0x7f882dd6d340) at 
async.c:128
#3  0x7f895625bc44 in rbd_finish_aiocb (c=0x7f882d4c34a0, 
rcb=0x7f882c0ae350) at block/rbd.c:585
#4  0x7f8953b8ee44 in librbd::AioCompletion::complete() () from 
/usr/lib/librbd.so.1
#5  0x7f8953b8e832 in 
librbd::AioCompletion::complete_request(CephContext*, long) () from 
/usr/lib/librbd.so.1
#6  0x7f89540646ba in Context::complete(int) () from 
/usr/lib/librados.so.2
#7  0x7f8953bc1e85 in ObjectCacher::C_WaitForWrite::finish(int) () 
from /usr/lib/librbd.so.1
#8  0x7f89540646ba in Context::complete(int) () from 
/usr/lib/librados.so.2
#9  0x7f89541073c8 in Finisher::finisher_thread_entry() () from 
/usr/lib/librados.so.2
#10 0x7f894e113b50 in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0

#11 0x7f894de5e0ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x in ?? ()

Stefan



Re: [Qemu-devel] Migration from older Qemu to Qemu 2.0.0 does not work

2014-05-14 Thread Stefan Priebe - Profihost AG

Am 14.05.2014 11:00, schrieb Paolo Bonzini:
> Il 14/05/2014 10:38, Stefan Priebe - Profihost AG ha scritto:
>> Currently it has the same as i already updated the package there too.
>>
>> So you mean i had done a mistake compiling the old package - so it had
>> wrong sizes?
> 
> Yes, probably.
> 
> Can you do an "info mtree" for a machine that's running on the source?
> 
> Paolo

here it is:
# info mtree
memory
-7ffe (prio 0, RW): system
  -dfff (prio 0, RW): alias ram-below-4g
@pc.ram -dfff
  000a-000b (prio 1, RW): alias smram-region
@pci 000a-000b
  000c-000c3fff (prio 1, R-): alias pam-rom @pc.ram
000c-000c3fff
  000c4000-000c7fff (prio 1, R-): alias pam-rom @pc.ram
000c4000-000c7fff
  000c8000-000cbfff (prio 1, R-): alias pam-rom @pc.ram
000c8000-000cbfff
  000ca000-000ccfff (prio 1000, RW): alias kvmvapic-rom
@pc.ram 000ca000-000ccfff
  000cc000-000c (prio 1, R-): alias pam-rom @pc.ram
000cc000-000c
  000d-000d3fff (prio 1, RW): alias pam-ram @pc.ram
000d-000d3fff
  000d4000-000d7fff (prio 1, RW): alias pam-ram @pc.ram
000d4000-000d7fff
  000d8000-000dbfff (prio 1, RW): alias pam-ram @pc.ram
000d8000-000dbfff
  000dc000-000d (prio 1, RW): alias pam-ram @pc.ram
000dc000-000d
  000e-000e3fff (prio 1, RW): alias pam-ram @pc.ram
000e-000e3fff
  000e4000-000e7fff (prio 1, RW): alias pam-ram @pc.ram
000e4000-000e7fff
  000e8000-000ebfff (prio 1, RW): alias pam-ram @pc.ram
000e8000-000ebfff
  000ec000-000e (prio 1, RW): alias pam-ram @pc.ram
000ec000-000e
  000f-000f (prio 1, R-): alias pam-rom @pc.ram
000f-000f
  e000- (prio 0, RW): alias pci-hole @pci
e000-
  fec0-fec00fff (prio 0, RW): kvm-ioapic
  fed0-fed003ff (prio 0, RW): hpet
  fee0-feef (prio 4096, RW): icc-apic-container
fee0-feef (prio 0, RW): kvm-apic-msi
  0001-00021fff (prio 0, RW): alias ram-above-4g
@pc.ram e000-0001
  00022000-40021fff (prio 0, RW): alias pci-hole64 @pci
00022000-40021fff
I/O
- (prio 0, RW): io
  -0007 (prio 0, RW): dma-chan
  0008-000f (prio 0, RW): dma-cont
  0020-0021 (prio 0, RW): kvm-pic
  0040-0043 (prio 0, RW): kvm-pit
  0060-0060 (prio 0, RW): i8042-data
  0061-0061 (prio 0, RW): elcr
  0064-0064 (prio 0, RW): i8042-cmd
  0070-0071 (prio 0, RW): rtc
  007e-007f (prio 0, RW): kvmvapic
  0080-0080 (prio 0, RW): ioport80
  0081-0083 (prio 0, RW): alias dma-page
@dma-page 0081-0083
  0087-0087 (prio 0, RW): alias dma-page
@dma-page 0087-0087
  0089-008b (prio 0, RW): alias dma-page
@dma-page 0089-008b
  008f-008f (prio 0, RW): alias dma-page
@dma-page 008f-008f
  0092-0092 (prio 0, RW): port92
  00a0-00a1 (prio 0, RW): kvm-pic
  00b2-00b3 (prio 0, RW): apm-io
  00c0-00cf (prio 0, RW): dma-chan
  00d0-00df (prio 0, RW): dma-cont
  00f0-00f0 (prio 0, RW): ioportF0
  0170-0177 (prio 0, RW): alias ide @ide
0170-0177
  01f0-01f7 (prio 0, RW): alias ide @ide
01f0-01f7
  0376-0376 (prio 0, RW): alias ide @ide
0376-0376
  03b0-03df (prio 0, RW): cirrus-io
  03f1-03f5 (prio 0, RW): alias fdc @fdc
03f1-03f5
  03f6-03f6 (prio 0, RW): alias ide @ide
03f6-03f6
  03f7-03f7 (prio 0, RW): alias fdc @fdc
03f7-03f7
  04d0-04d0 (prio 0, RW): kvm-elcr
  04d1-04d1 (prio 0, R

Re: [Qemu-devel] Migration from older Qemu to Qemu 2.0.0 does not work

2014-05-14 Thread Stefan Priebe - Profihost AG
Am 14.05.2014 10:36, schrieb Paolo Bonzini:
> Il 14/05/2014 10:29, Stefan Priebe - Profihost AG ha scritto:
>> Hi,
>>
>> i compile qemu on my own.
>>
>> I have the rom files under /usr/share/kvm and they look like this:
>> ls -la /usr/share/kvm/*.rom
>> -rw-r--r-- 1 root root 173568 May 14 09:39 /usr/share/kvm/efi-e1000.rom
>> -rw-r--r-- 1 root root 174592 May 14 09:39
>> /usr/share/kvm/efi-eepro100.rom
>> -rw-r--r-- 1 root root 173056 May 14 09:39
>> /usr/share/kvm/efi-ne2k_pci.rom
>> -rw-r--r-- 1 root root 173056 May 14 09:39 /usr/share/kvm/efi-pcnet.rom
>> -rw-r--r-- 1 root root 176640 May 14 09:39 /usr/share/kvm/efi-rtl8139.rom
>> -rw-r--r-- 1 root root 171008 May 14 09:39 /usr/share/kvm/efi-virtio.rom
>> -rw-r--r-- 1 root root  67072 May 14 09:39 /usr/share/kvm/pxe-e1000.rom
>> -rw-r--r-- 1 root root  61440 May 14 09:39
>> /usr/share/kvm/pxe-eepro100.rom
>> -rw-r--r-- 1 root root  61440 May 14 09:39
>> /usr/share/kvm/pxe-ne2k_pci.rom
>> -rw-r--r-- 1 root root  61440 May 14 09:39 /usr/share/kvm/pxe-pcnet.rom
>> -rw-r--r-- 1 root root  61440 May 14 09:39 /usr/share/kvm/pxe-rtl8139.rom
>> -rw-r--r-- 1 root root  60416 May 14 09:39 /usr/share/kvm/pxe-virtio.rom
>>
>> currently i don't know what's wrong.
> 
> What about the source machine?

Currently it has the same as i already updated the package there too.

So you mean i had done a mistake compiling the old package - so it had
wrong sizes?

Greets.
Stefan



Re: [Qemu-devel] Migration from older Qemu to Qemu 2.0.0 does not work

2014-05-14 Thread Stefan Priebe - Profihost AG
Am 14.05.2014 10:11, schrieb Paolo Bonzini:
> Il 14/05/2014 09:17, Stefan Priebe - Profihost AG ha scritto:
>> i now was able to catch the error.
>>
>> It is:
>> Length mismatch: :00:12.0/virtio-net-pci.rom: 4 in != 1
>> qemu: warning: error while loading state for instance 0x0 of device 'ram'
>> load of migration failed
> 
> This is a bug of your distribution.  The file sizes for ROMs should
> never change.  In particular, if you round the sizes up to the next
> power of 2 you should always have:
> 
> 128k for bios.bin
> 256k for bios-256k.bin
> 64k for pxe-*.rom
> 256k for efi-*.rom
> 64k for vgabios-*
> 
> Unfortunately, most distribution get pxe-*.rom sizes wrong, because at
> some point iPXE grew more features and didn't fit in 64k anymore with
> the default configuration.  I know at least Fedora does.
> 
> The solution is to copy the binaries from the QEMU git repository
> (directory pc-bios/) to /usr/share/qemu.

Hi,

i compile qemu on my own.

I have the rom files under /usr/share/kvm and they look like this:
ls -la /usr/share/kvm/*.rom
-rw-r--r-- 1 root root 173568 May 14 09:39 /usr/share/kvm/efi-e1000.rom
-rw-r--r-- 1 root root 174592 May 14 09:39 /usr/share/kvm/efi-eepro100.rom
-rw-r--r-- 1 root root 173056 May 14 09:39 /usr/share/kvm/efi-ne2k_pci.rom
-rw-r--r-- 1 root root 173056 May 14 09:39 /usr/share/kvm/efi-pcnet.rom
-rw-r--r-- 1 root root 176640 May 14 09:39 /usr/share/kvm/efi-rtl8139.rom
-rw-r--r-- 1 root root 171008 May 14 09:39 /usr/share/kvm/efi-virtio.rom
-rw-r--r-- 1 root root  67072 May 14 09:39 /usr/share/kvm/pxe-e1000.rom
-rw-r--r-- 1 root root  61440 May 14 09:39 /usr/share/kvm/pxe-eepro100.rom
-rw-r--r-- 1 root root  61440 May 14 09:39 /usr/share/kvm/pxe-ne2k_pci.rom
-rw-r--r-- 1 root root  61440 May 14 09:39 /usr/share/kvm/pxe-pcnet.rom
-rw-r--r-- 1 root root  61440 May 14 09:39 /usr/share/kvm/pxe-rtl8139.rom
-rw-r--r-- 1 root root  60416 May 14 09:39 /usr/share/kvm/pxe-virtio.rom

currently i don't know what's wrong.

Stefan



Re: [Qemu-devel] Migration from older Qemu to Qemu 2.0.0 does not work

2014-05-14 Thread Stefan Priebe - Profihost AG
Hi,

i now was able to catch the error.

It is:
Length mismatch: :00:12.0/virtio-net-pci.rom: 4 in != 1
qemu: warning: error while loading state for instance 0x0 of device 'ram'
load of migration failed

Stefan

Am 09.05.2014 19:05, schrieb Paolo Bonzini:
> Il 09/05/2014 15:13, Stefan Priebe - Profihost AG ha scritto:
>>
>> I see no output at the monitor of Qemu 2.0.
>>
>> # migrate -d tcp:a.b.c.d:6000
>>
>> # info migrate
>> capabilities: xbzrle: off x-rdma-pin-all: off auto-converge: on
>> zero-blocks: off
>> Migration status: failed
>> total time: 0 milliseconds
>>
>> The target machine is still running at this point with no output.
> 
> Anything on its stdout?
> 
> Another test you could do, in addition to changing the devices, is this:
> 
> 1) try with commit 6141f3bd6904df7cf9519c6444a14a608b9874c4 on the
> destination (the next one caused a migration problem that was fixed
> later).  If it passes, go to step 1a.  If it fails, go to step 2.
> 
> 1a) try with commit c01a71c1a56fa27f43449ff59e5d03b2483658a2.  If it
> passes, go to step 1b.  If it fails, report it here.
> 
> 1b) bisect between v2.0.0 (known-bad) and
> c01a71c1a56fa27f43449ff59e5d03b2483658a2 (known-good) to find the
> culprit.  Report results.
> 
> 2) change the source to v1.7.0 and bisect between v1.7.0 and
> 6141f3bd6904df7cf9519c6444a14a608b9874c4.  Report results.
> 
> Thanks!
> 
> Paolo



Re: [Qemu-devel] Migration from older Qemu to Qemu 2.0.0 does not work

2014-05-09 Thread Stefan Priebe

)

Am 09.05.2014 19:05, schrieb Paolo Bonzini:

Il 09/05/2014 15:13, Stefan Priebe - Profihost AG ha scritto:


I see no output at the monitor of Qemu 2.0.

# migrate -d tcp:a.b.c.d:6000

# info migrate
capabilities: xbzrle: off x-rdma-pin-all: off auto-converge: on
zero-blocks: off
Migration status: failed
total time: 0 milliseconds

The target machine is still running at this point with no output.


Anything on its stdout?


Sadly no.

But far more complex than i thought. I'm seeing this problem only on 
machines originally started under qemu 1.5 ten migrated to qemu 1.7 with 
machine type=pc440fc-1.5 . Only those do not work. But they also do not 
work migrating them to qemu 1.7.1.


So let's ignore the whole fact and thanks for your bisect ideas. I will 
just shut them down all and start them with a fresh qemu 2.0.



Another test you could do, in addition to changing the devices, is this:

1) try with commit 6141f3bd6904df7cf9519c6444a14a608b9874c4 on the
destination (the next one caused a migration problem that was fixed
later).  If it passes, go to step 1a.  If it fails, go to step 2.

1a) try with commit c01a71c1a56fa27f43449ff59e5d03b2483658a2.  If it
passes, go to step 1b.  If it fails, report it here.

1b) bisect between v2.0.0 (known-bad) and
c01a71c1a56fa27f43449ff59e5d03b2483658a2 (known-good) to find the
culprit.  Report results.

2) change the source to v1.7.0 and bisect between v1.7.0 and
6141f3bd6904df7cf9519c6444a14a608b9874c4.  Report results.

Thanks!

Paolo




Re: [Qemu-devel] Migration from older Qemu to Qemu 2.0.0 does not work

2014-05-09 Thread Stefan Priebe


Am 09.05.2014 18:29, schrieb Dr. David Alan Gilbert:

* Stefan Priebe - Profihost AG (s.pri...@profihost.ag) wrote:



Am 09.05.2014 um 15:41 schrieb "Dr. David Alan Gilbert" :

* Stefan Priebe - Profihost AG (s.pri...@profihost.ag) wrote:

Hello list,

i was trying to migrate older Qemu (1.5 and 1.7.2) to a machine running
Qemu 2.0.

I started the target machine with:

-machine type=pc-i440fx-1.5 / -machine type=pc-i440fx-1.7


I'd expect you to have to run with the same machine type on both sides.
I ran some simple virt-test migrate tests (just the basic one) and got
v1.5.3->v1.6.2
v1.5.3->v1.7.1
v1.5.3->v2.0.0-rc1
   working for most machine types, pc-i440fx-1.5 passed with all of those.
Note that's only the simplest test.


My test involved USB Controller virtio Network cards and rbd virtio-scsi drives.


That probably makes things more interesting :-) I'd start with the simplest
config you can find and add each one of those until it breaks.


Can you give me your simple start line so I could test if this works for me 
while adding some more arguments.


I've got a slightly hacked up libvirt, but I'm doing

   ./run -t qemu --machine-type=$MACTYPE -g Linux.JeOS.19.x86_64.i440fx 
--qemu-bin=$SRC --qemu-dst-bin=$DST --qemu_sandbox=off --tests 
migrate.default.tcp --disk-bus=ide

and looking at the logs we have:

/opt/qemu/v1.5.3/bin/qemu-system-x86_64 \
 -S  \
 -name 'virt-tests-vm1'  \
 -sandbox off  \
 -M pc-i440fx-1.5  \
 -nodefaults  \
 -vga std  \
 -chardev 
socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20140415-104517-rAfzfDef,server,nowait
 \
 -mon chardev=hmp_id_hmp1,mode=readline  \
 -chardev 
socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140415-104517-rAfzfDef,server,nowait
 \
 -device isa-serial,chardev=serial_id_serial0  \
 -chardev 
socket,id=seabioslog_id_20140415-104517-rAfzfDef,path=/tmp/seabios-20140415-104517-rAfzfDef,server,nowait
 \
 -device 
isa-debugcon,chardev=seabioslog_id_20140415-104517-rAfzfDef,iobase=0x402 \
 -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=03 \
 -drive 
id=drive_image1,if=none,file=/home/dgilbert/virt-test/shared/data/images/jeos-19-64.qcow2
 \
 -device 
virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=04 \
 -device 
virtio-net-pci,mac=9a:18:19:1a:1b:1c,id=idz0uWFP,vectors=4,netdev=idMwmdMc,bus=pci.0,addr=05
  \
 -netdev user,id=idMwmdMc,hostfwd=tcp::5000-:22  \
 -m 1024  \
 -smp 2,maxcpus=2,cores=1,threads=1,sockets=2  \
 -cpu 'SandyBridge' \
 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
 -vnc :1  \
 -rtc base=utc,clock=host,driftfix=none  \
 -boot order=cdn,once=c,menu=off \
 -enable-kvm

Dave


OK biggest bummer it works fine with all freshly started VMs. No idea why.

So i'll try with paolos idea migrating old VMs.




Thanks!

Stefan



Dave
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK





Re: [Qemu-devel] Migration from older Qemu to Qemu 2.0.0 does not work

2014-05-09 Thread Stefan Priebe - Profihost AG

> Am 09.05.2014 um 15:41 schrieb "Dr. David Alan Gilbert" :
> 
> * Stefan Priebe - Profihost AG (s.pri...@profihost.ag) wrote:
>> Hello list,
>> 
>> i was trying to migrate older Qemu (1.5 and 1.7.2) to a machine running
>> Qemu 2.0.
>> 
>> I started the target machine with:
>> 
>> -machine type=pc-i440fx-1.5 / -machine type=pc-i440fx-1.7
> 
> I'd expect you to have to run with the same machine type on both sides.
> I ran some simple virt-test migrate tests (just the basic one) and got
> v1.5.3->v1.6.2
> v1.5.3->v1.7.1
> v1.5.3->v2.0.0-rc1
>   working for most machine types, pc-i440fx-1.5 passed with all of those.
> Note that's only the simplest test.

My test involved USB Controller virtio Network cards and rbd virtio-scsi drives.

Can you give me your simple start line so I could test if this works for me 
while adding some more arguments.

Thanks!

Stefan

> 
> Dave
> --
> Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK



[Qemu-devel] Migration from older Qemu to Qemu 2.0.0 does not work

2014-05-09 Thread Stefan Priebe - Profihost AG
Hello list,

i was trying to migrate older Qemu (1.5 and 1.7.2) to a machine running
Qemu 2.0.

I started the target machine with:

-machine type=pc-i440fx-1.5 / -machine type=pc-i440fx-1.7

But the migration simply fails. Migrating Qemu 2.0 to Qemu 2.0 succeeds.

I see no output at the monitor of Qemu 2.0.

# migrate -d tcp:a.b.c.d:6000

# info migrate
capabilities: xbzrle: off x-rdma-pin-all: off auto-converge: on
zero-blocks: off
Migration status: failed
total time: 0 milliseconds

The target machine is still running at this point with no output.

Stefan



Re: [Qemu-devel] [PATCH 00/35] pc: ACPI memory hotplug

2014-05-07 Thread Stefan Priebe - Profihost AG
Hello Igor,

while testing your patchset i came to a very stupid problem.

I wanted to test migration and it cames out that the migration works
fine after plugging in memory only if i run the target VM without the
-daemonize option.

If i enable the -daemonize option the target vm tries to read from non
readable memory.

proc maps shows:
7f9334021000-7f933800 ---p  00:00 0

where it tries to read from.

Also the memory layout is different in daemonize mode than in non
daemonize mode.

Stefan

Am 04.04.2014 15:36, schrieb Igor Mammedov:
> What's new since v7:
> 
> * Per Andreas' suggestion dropped DIMMBus concept.
> 
> * Added hotplug binding for bus-less devices
> 
> * DIMM device is split to backend and frontend. Therefore following
>   command/options were added for supporting it:
> 
>   For memory-ram backend:
>   CLI: -object-add memory-ram,
>   with options: 'id' and 'size'
>   For dimm frontend:
>   option "size" became readonly, pulling it's size from attached backend
>   added option "memdev" for specifying backend by 'id'
> 
> * dropped support for 32 bit guests
> 
> * failed hotplug action doesn't consume 1 slot anymore
> 
> * vaious fixes adressing reviewer's comments most of them in ACPI part
> ---
> 
> This series allows to hotplug 'arbitrary' DIMM devices specifying size,
> NUMA node mapping (guest side), slot and address where to map it, at runtime.
> 
> Due to ACPI limitation there is need to specify a number of possible
> DIMM devices. For this task -m option was extended to support
> following format:
> 
>   -m [mem=]RamSize[,slots=N,maxmem=M]
> 
> To allow memory hotplug user must specify a pair of additional parameters:
> 'slots' - number of possible increments
> 'maxmem' - max possible total memory size QEMU is allowed to use,
>including RamSize.
> 
> minimal monitor command syntax to hotplug DIMM device:
> 
>   object_add memory-ram,id=memX,size=1G
>   device_add dimm,id=dimmX,memdev=memX
> 
> DIMM device provides following properties that could be used with
> device_add / -device to alter default behavior:
> 
>   id- unique string identifying device [mandatory]
>   slot  - number in range [0-slots) [optional], if not specified
>   the first free slot is used
>   node  - NUMA node id [optional] (default: 0)
>   size  - amount of memory to add, readonly derived from backing memdev
>   start - guest's physical address where to plug DIMM [optional],
>   if not specified the first gap in hotplug memory region
>   that fits DIMM is used
> 
>  -device option could be used for adding potentially hotunplugable DIMMs
> and also for specifying hotplugged DIMMs in migration case.
> 
> Tested guests:
>  - RHEL 6x64
>  - Windows 2012DCx64
>  - Windows 2008DCx64
> 
> Known limitations/bugs/TODOs:
>  - hot-remove is not supported, yet
>  - max number of supported DIMM devices 255 (due to ACPI object name
>limit), could be increased creating several containers and putting
>DIMMs there. (exercise for future) 
>  - e820 table doesn't include DIMM devices added with -device /
>(or after reboot devices added with device_add)
>  - Windows 2008 remembers DIMM configuration, so if DIMM with other
>start/size is added into the same slot, it refuses to use it insisting
>on old mapping.
> 
> QEMU git tree for testing is available at:
>   https://github.com/imammedo/qemu/commits/memory-hotplug-v8
> 
> Example QEMU cmd line:
>   qemu-system-x86_64 -enable-kvm -monitor unix:/tmp/mon,server,nowait \ 
>  -m 4096,slots=4,maxmem=8G guest.img
> 
> PS:
>   Windows guest requires SRAT table for hotplug to work so add an extra 
> option:
>-numa node
>   to QEMU command line.
> 
> 
> Igor Mammedov (34):
>   vl: convert -m to QemuOpts
>   object_add: allow completion handler to get canonical path
>   add memdev backend infrastructure
>   vl.c: extend -m option to support options for memory hotplug
>   add pc-{i440fx,q35}-2.1 machine types
>   pc: create custom generic PC machine type
>   qdev: hotplug for buss-less devices
>   qdev: expose DeviceState.hotplugged field as a property
>   dimm: implement dimm device abstraction
>   memory: add memory_region_is_mapped() API
>   dimm: do not allow to set already busy memdev
>   pc: initialize memory hotplug address space
>   pc: exit QEMU if slots > 256
>   pc: add 'etc/reserved-memory-end' fw_cfg interface for SeaBIOS
>   pc: add memory hotplug handler to PC_MACHINE
>   dimm: add busy address check and address auto-allocation
>   dimm: add busy slot check and slot auto-allocation
>   acpi: rename cpu_hotplug_defs.h to acpi_defs.h
>   acpi: memory hotplug ACPI hardware implementation
>   trace: add acpi memory hotplug IO region events
>   trace: add DIMM slot & address allocation for target-i386
>   acpi:piix4: make plug/unlug callbacks generic
>   acpi:piix4: add memory hotplug handling
>   pc: ich9 lpc: make it work with global/compat properties
>   acpi:ich9: add m

[Qemu-devel] multiple USB Serial devices qemu load

2014-03-26 Thread Stefan Priebe - Profihost AG
Hello,

i've 4 USB serial devices and one HID device i want to pass to a guest.

The passing itself works fine but while the guest has 0 load or cpu
usage the qemu process itself has around 40% cpu usage on a single 3,2
ghz E3 xeon.

I already tried xhci but it doesn't change anything. Also the latency of
the usb devices is very high and fluctuate. Qemu Version 1.7.1

Inside the guest:
# lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
|__ Port 1: Dev 2, If 0, Class=HID, Driver=usbhid, 480M
|__ Port 2: Dev 3, If 0, Class=HID, Driver=usbfs, 12M
|__ Port 3: Dev 4, If 0, Class=vend., Driver=ftdi_sio, 12M
|__ Port 4: Dev 5, If 0, Class=hub, Driver=hub/8p, 12M
|__ Port 1: Dev 6, If 0, Class=vend., Driver=ftdi_sio, 12M
|__ Port 2: Dev 7, If 0, Class=comm., Driver=cdc_acm, 12M
|__ Port 2: Dev 7, If 1, Class=data, Driver=cdc_acm, 12M
|__ Port 3: Dev 8, If 0, Class=vend., Driver=ftdi_sio, 12M

Qremu command line:

qemu -pidfile /var/run/qemu-server/102.pid -daemonize -name test -smp
sockets=1,cores=1 -nodefaults -boot
menu=on,strict=on,reboot-timeout=1000 -vga cirrus -cpu
kvm64,+lahf_lm,+x2apic,+sep -k de -m 1024 -device
nec-usb-xhci,id=xhci,bus=pci.0,addr=0x1.0x2 -device
usb-tablet,id=tablet,bus=xhci.0,port=1 -device
usb-host,hostbus=2,hostport=1.2 -device usb-host,hostbus=2,hostport=1.3
-device usb-host,hostbus=2,hostport=1.4 -device
usb-host,hostbus=2,hostport=1.1.1 -device
usb-host,hostbus=2,hostport=1.1.4 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -device
virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5 -drive
file=/ssdstor/images/102/vm-102-disk-1.qcow2,if=none,id=drive-scsi0,format=qcow2,cache=writeback,discard=on,aio=native
-device
scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0
-netdev
type=tap,id=net0,ifname=tap102i0,script=/var/lib/qemu-server/pve-bridge,vhost=on
-device
virtio-net-pci,mac=3A:53:02:E3:76:59,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300

Does anybody have an idea how to lower the CPU usage?

Greets,
Stefan



Re: [Qemu-devel] memory allocation of migration changed?

2014-03-12 Thread Stefan Priebe


Am 24.02.2014 17:13, schrieb Eric Blake:

On 02/24/2014 08:00 AM, Stefan Hajnoczi wrote:


What is the right way to check for enough free memory and memory
usage of a specific vm?


I would approach it in terms of guest RAM allocation plus QEMU overhead:

   host_ram >= num_guests * guest_ram_size + num_guests * qemu_overhead

The qemu_overhead is the question mark.  It depends on the QEMU features
the guests have enabled and number of devices.

QEMU also does not have a strict policy on predictable memory
consumption, which makes it hard to give a formula for it.


In fact, at one point libvirt tried to put an automatic cap on the
memory usable by qemu by multiplying RAM size and accounting for a
margin of overhead, but no matter what heuristics we tried, we still got
complaints from users that their guests were killed when they ran out of
memory, and so we ended up reverting the automatic limits from libvirt.
  (You can still enforce a limit as an end user, although the libvirt
documentation no longer recommends attempting that, for as long as the
qemu allocation remains unpredictable.)



Might something like:
vm.overcommit_ratio=100
vm.overcommit_memory=2

help? So that the migration might not happen but that's better than 
killing a random process?


Stefan



Re: [Qemu-devel] memory allocation of migration changed?

2014-02-14 Thread Stefan Priebe


Am 14.02.2014 16:03, schrieb Stefan Hajnoczi:

On Tue, Feb 11, 2014 at 07:30:54PM +0100, Stefan Priebe wrote:

Am 11.02.2014 16:44, schrieb Stefan Hajnoczi:

On Tue, Feb 11, 2014 at 3:54 PM, Stefan Priebe - Profihost AG
 wrote:

in the past (Qemu 1.5) a migration failed if there was not enogh memory
on the target host available directly at the beginning.

Now with Qemu 1.7 i've seen succeeded migrations but the kernel OOM
memory killer killing qemu processes. So the migration seems to takes
place without having anough memory on the target machine?


How much memory is the guest configured with?  How much memory does
the host have?


Guest: 48GB
Host: 192GB


I wonder if there are zero pages that can be migrated almost "for
free" and the destination host doesn't touch.  When they are touched
for the first time after migration handover, they need to be allocated
on the destination host.  This can lead to OOM if you overcommitted
memory.


In the past the migration failed immediatly with exit code 255.


Can you reproduce the OOM reliably?  It should be possible to debug it
and figure out whether it's just bad luck or a true regression.


So there is no known patch changing this behaviour?

What is about those?
fc1c4a5d32e15a4c40c47945da85ef9c1e0c1b54
211ea74022f51164a7729030b28eec90b6c99a08
f1c72795af573b24a7da5eb52375c9aba8a37972


Yes, that's what I was referring to when I mentioned zero pages.

The problem might just be that the destination host didn't have enough
free memory.  Migration succeeded due to memory overcommit on the host,
but quickly ran out of memory after handover.  The quick answer there is
to reconsider your overcommitting memory and also checking memory
availability before live migrating.


Yes that makes sense in the past it was just working due to a failing 
qemu process.


What is the right way to check for enough free memory and memory usage 
of a specific vm?


Stefan






Re: [Qemu-devel] memory allocation of migration changed?

2014-02-14 Thread Stefan Priebe

Am 14.02.2014 15:59, schrieb Stefan Hajnoczi:

On Tue, Feb 11, 2014 at 07:32:46PM +0100, Stefan Priebe wrote:

Am 11.02.2014 17:22, schrieb Peter Lieven:




Am 11.02.2014 um 16:44 schrieb Stefan Hajnoczi :

On Tue, Feb 11, 2014 at 3:54 PM, Stefan Priebe - Profihost AG
 wrote:

in the past (Qemu 1.5) a migration failed if there was not enogh memory
on the target host available directly at the beginning.

Now with Qemu 1.7 i've seen succeeded migrations but the kernel OOM
memory killer killing qemu processes. So the migration seems to takes
place without having anough memory on the target machine?


How much memory is the guest configured with?  How much memory does
the host have?

I wonder if there are zero pages that can be migrated almost "for
free" and the destination host doesn't touch.  When they are touched
for the first time after migration handover, they need to be allocated
on the destination host.  This can lead to OOM if you overcommitted
memory.

Can you reproduce the OOM reliably?  It should be possible to debug it
and figure out whether it's just bad luck or a true regression.

Stefan


Kernel Version would also be interesting as well as thp and ksm settings.


Kernel Host: 3.10.26

What's thp / ksm? how to get those settings?


Transparent Huge Pages

# cat /sys/kernel/mm/transparent_hugepage/enabled

Kernel Samepage Merging

# cat /sys/kernel/mm/ksm/run



# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never


# cat /sys/kernel/mm/ksm/run
1




Stefan





Re: [Qemu-devel] [pve-devel] QEMU LIve Migration - swap_free: Bad swap file entry

2014-02-13 Thread Stefan Priebe


Am 13.02.2014 21:06, schrieb Dr. David Alan Gilbert:

* Stefan Priebe (s.pri...@profihost.ag) wrote:

Am 10.02.2014 17:07, schrieb Dr. David Alan Gilbert:

* Stefan Priebe (s.pri...@profihost.ag) wrote:

i could fix it by explicitly disable xbzrle - it seems its
automatically on if i do not set the migration caps to false.

So it seems to be a xbzrle bug.


Stefan can you give me some more info on your hardware and
migration setup;   that stressapptest (which is a really nice
find!) really batters the memory and it means the migration
isn't converging for me, so I'm curious what your setup is.


That one is devlopment by google and known to me since a few years.
Google has detected that memtest and co are not good enough to
stress test memory.


Hi Stefan,
   I've just posted a patch to qemu-devel that fixes two bugs that
we found; I've only tried a small stressapptest run and it seems
to survive with them (where it didn't before);  you might like to try
it if you're up for rebuilding qemu.

It's the one entitled ' [PATCH] Fix two XBZRLE corruption issues'

I'll try and get a larger run done myself, but I'd be interested to
hear if it fixes it for you (or anyone else who hit the problem).


Yes works fine - now no crash but it's sower than without XBZRLE ;-)

Without XBZRLE: i needed migrate_downtime 4 around 60s
With XBZRLE: i needed migrate_downtime 16 and 240s



Dave
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK





  1   2   3   >