On 08/02/2013 01:22 PM, Wei ZHOU wrote:
Wido,
You applied the libvirt patch on your production system, and this issue
disappeared, right?
Yes, I applied that patch and rebuild libvirt. The issue never came back.
If so, that is good.
I expect the redhat community can accep the patch (or the v2
https://www.redhat.com/archives/libvir-list/2013-July/msg00639.html) ASAP.
I've been watching the thread, but it seems kind of dead.
Wido
-Wei
2013/8/2 Wido den Hollander <w...@widodh.nl>
On 08/02/2013 01:55 AM, Edison Su wrote:
Hi Wei, regarding to the bug CLOUDSTACK-2729, I removed storage.refresh
during getStoragePool in LibvirtStorageAdaptor, but the issue still
happened in BVT.
I am thinking add file lock on primary storage, seems you already have
the patch, could you share the patch with us?
Fyi, I fixed this by patching the libvirt on our production systems rather
then fixing the CloudStack agent.
It's just one very small patch: https://bugzilla.redhat.com/**
show_bug.cgi?id=977706<https://bugzilla.redhat.com/show_bug.cgi?id=977706>
https://www.redhat.com/**archives/libvir-list/2013-**July/msg00635.html<https://www.redhat.com/archives/libvir-list/2013-July/msg00635.html>
Wido
-----Original Message-----
From: Wei ZHOU [mailto:ustcweiz...@gmail.com]
Sent: Tuesday, July 16, 2013 3:35 AM
To: dev@cloudstack.apache.org
Subject: Re: How to fix libvirt storage pool refresh issue?
I agree with Wido.
Moreover, the file lock will cause performane degrade of VM deployment.
-Wei
2013/7/16 Wido den Hollander <w...@widodh.nl>
On 07/16/2013 12:27 AM, Marcus Sorensen wrote:
I'm ok with a symptom fix on our end, if the root cause is in
Libvirt we can't do much about that. This is the sort of patch that
tends to get pulled into the regular update cycle of the
distributions, so unless there's more to it and it's not a good fix I
imagine we will see it come through without having to wait for the
next point releases. We still have to support existing users who
might not be running the latest, though, so the symptom fix is
probably ok as a temporary measure.
I'm ok with not calling storagePoolRefresh every time we want a
capacity update, since that's also kind of I/O intensive for larger
storage
arrays.
However, we should make sure we have a GOOD comment in the code
about
this "fix", since that's the reason I initially removed the old code
which invoked "df".
I'll see if I can get this libvirt patch into Ubuntu when it hits
libvirt upstream, since this bug is really annoying.
Wido
On Mon, Jul 15, 2013 at 3:42 PM, Edison Su <edison...@citrix.com>
wrote:
There is a serious issue on
KVM(https://issues.apache.org/****<https://issues.apache.org/**>
jira/browse/CLOUDSTACK-
2729<https://issues.apache.**org/jira/browse/CLOUDSTACK-**2729<https://issues.apache.org/jira/browse/CLOUDSTACK-2729>
):
a libvirt storage pool can disappear on KVM host, it's easy to be
reproduced in our internal QA environment.
Wei found the root cause, is on the libvirt:
"
This is a libvirt issue. I created a ticket for it.
https://bugzilla.redhat.com/****show_bug.cgi?id=977706<https://bugzilla.redhat.com/**show_bug.cgi?id=977706>
<https:/**/bugzill <https://bugzill>
a.redhat.com/show_bug.cgi?id=**977706<http://a.redhat.com/show_bug.cgi?id=977706>
The patch is very simple.
https://www.redhat.com/****archives/libvir-list/2013-****
July/msg00635.h<https://www.redhat.com/**archives/libvir-list/2013-**July/msg00635.h>
tml<https://www.redhat.com/**archives/libvir-list/2013-**
July/msg00635.h<https://www.redhat.com/archives/libvir-list/2013-July/msg00635.h>
tml>
"
But it's also introduced by CloudStack, as cloudstack will call
libvirt storage pool refresh method each time when access the
storage pool. The code is added by commit:
2ffc9907f7b0d371737e39b7649f7a****f23026f5cf,
about less than one year ago.
As Wei suggested, we can call storage pool refresh only if needed,
it will mitigate the issue(It's behavior I did on cloudstack
pre-4.0), but it's only treat the symptom, not the cause.
Or add a cluster wide lock, only one guy can access storage pool at
one time, we can add a file lock on NFS primary storage.
Any idea/feedback on how to fix this KVM issue?
