Re: [libvirt] [Xen-devel] [PATCH 0/3] libxl: domain destroy fixes
Jim Fehlig wrote: > Konrad Rzeszutek Wilk wrote: > >> On Wed, Mar 25, 2015 at 02:08:33PM -0600, Jim Fehlig wrote: >> >> >>> This small series of patches fixes some issues wrt domain destroy in >>> the libxl driver. The primary motivation for this work is to >>> prevent locking the virDomainObj during long running destroy operations >>> on large memory domains. >>> >>> Patch 1 moves job acquisition from libxlDomainStart to it's callers so >>> they have more control over when the job is acquired. Patch 2 fixes a >>> few spots where we never acquired a job during domain destroy. Patch 3 >>> contains the interesting change, where the virDomainObj is unlocked >>> during the long-running destroy operation. >>> >>> This series wraps up my work to improve parallel OpenStack Tempest runs >>> against the libxl driver. With libvirt.git master + this series + a >>> patched libxl [1], I've successfully run a reproducer that was hitting >>> the same issues encountered by Tempest. >>> >>> [1] libxl commits from xen.git: 93699882d, f1335f0d, 4783c99a, 1c91d6fba, >>> and 188e9c54. I'll contact the stable branch maintainers and ask them >>> to include these commits in the next Xen 4.4.x and 4.5.x releases. >>> >>> Jim Fehlig (3): >>> libxl: Move job acquisition in libxlDomainStart to callers >>> libxl: acquire a job when destroying a domain >>> libxl: drop virDomainObj lock when destroying a domain >>> >>> >> I am no expert at this- but I dug through the code to understand how >> the job and locking is done and now I am more comfortable with it. >> >> Since I am new to this I went through all of the the callsites (which used >> the job now) from the driver to make sure that there are no chained calls >> (one function calling another which also uses a mutex or job locking). >> >> I only found one culprit (libxlDomainAutoCoreDump being called from >> libxlDomainShutdownThread). >> >> Reviewed-by: Konrad Rzeszutek Wilk >> >> > > Thanks for taking time to review the series and familiarize yourself > with the libvirt libxl code! As mentioned, I squashed in your > libxlDomainAutoCoreDump fix in 2/3. > > Do any other libvirt devs have time for a quick review? I'd like to > push this series, and the dom0 ballooning fix, for 1.2.14 if there are > no objections. The latter was reviewed by Stefano Stabellini before the > freeze > > https://www.redhat.com/archives/libvir-list/2015-March/msg01248.html > Ping! I'd like to include these fixes for 1.2.14. The patches have been "Reviewed-by" Konrad and Stefano. Anthony also responded today that his OpenStack Tempest runs are much happier https://www.redhat.com/archives/libvir-list/2015-March/msg01540.html Regards, Jim -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Xen-devel] [PATCH 0/3] libxl: domain destroy fixes
On Wed, Mar 25, 2015 at 02:08:33PM -0600, Jim Fehlig wrote: > This small series of patches fixes some issues wrt domain destroy in > the libxl driver. The primary motivation for this work is to > prevent locking the virDomainObj during long running destroy operations > on large memory domains. > > Patch 1 moves job acquisition from libxlDomainStart to it's callers so > they have more control over when the job is acquired. Patch 2 fixes a > few spots where we never acquired a job during domain destroy. Patch 3 > contains the interesting change, where the virDomainObj is unlocked > during the long-running destroy operation. > > This series wraps up my work to improve parallel OpenStack Tempest runs > against the libxl driver. With libvirt.git master + this series + a > patched libxl [1], I've successfully run a reproducer that was hitting > the same issues encountered by Tempest. > > [1] libxl commits from xen.git: 93699882d, f1335f0d, 4783c99a, 1c91d6fba, > and 188e9c54. I'll contact the stable branch maintainers and ask them > to include these commits in the next Xen 4.4.x and 4.5.x releases. Hi, I gave a try to this series with OpenStack Tempest. And it is much better. Thanks! -- Anthony PERARD -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Xen-devel] [PATCH 0/3] libxl: domain destroy fixes
Konrad Rzeszutek Wilk wrote: > On Wed, Mar 25, 2015 at 02:08:33PM -0600, Jim Fehlig wrote: > >> This small series of patches fixes some issues wrt domain destroy in >> the libxl driver. The primary motivation for this work is to >> prevent locking the virDomainObj during long running destroy operations >> on large memory domains. >> >> Patch 1 moves job acquisition from libxlDomainStart to it's callers so >> they have more control over when the job is acquired. Patch 2 fixes a >> few spots where we never acquired a job during domain destroy. Patch 3 >> contains the interesting change, where the virDomainObj is unlocked >> during the long-running destroy operation. >> >> This series wraps up my work to improve parallel OpenStack Tempest runs >> against the libxl driver. With libvirt.git master + this series + a >> patched libxl [1], I've successfully run a reproducer that was hitting >> the same issues encountered by Tempest. >> >> [1] libxl commits from xen.git: 93699882d, f1335f0d, 4783c99a, 1c91d6fba, >> and 188e9c54. I'll contact the stable branch maintainers and ask them >> to include these commits in the next Xen 4.4.x and 4.5.x releases. >> >> Jim Fehlig (3): >> libxl: Move job acquisition in libxlDomainStart to callers >> libxl: acquire a job when destroying a domain >> libxl: drop virDomainObj lock when destroying a domain >> > > I am no expert at this- but I dug through the code to understand how > the job and locking is done and now I am more comfortable with it. > > Since I am new to this I went through all of the the callsites (which used > the job now) from the driver to make sure that there are no chained calls > (one function calling another which also uses a mutex or job locking). > > I only found one culprit (libxlDomainAutoCoreDump being called from > libxlDomainShutdownThread). > > Reviewed-by: Konrad Rzeszutek Wilk > Thanks for taking time to review the series and familiarize yourself with the libvirt libxl code! As mentioned, I squashed in your libxlDomainAutoCoreDump fix in 2/3. Do any other libvirt devs have time for a quick review? I'd like to push this series, and the dom0 ballooning fix, for 1.2.14 if there are no objections. The latter was reviewed by Stefano Stabellini before the freeze https://www.redhat.com/archives/libvir-list/2015-March/msg01248.html Regards, Jim -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Xen-devel] [PATCH 0/3] libxl: domain destroy fixes
On Wed, Mar 25, 2015 at 02:08:33PM -0600, Jim Fehlig wrote: > This small series of patches fixes some issues wrt domain destroy in > the libxl driver. The primary motivation for this work is to > prevent locking the virDomainObj during long running destroy operations > on large memory domains. > > Patch 1 moves job acquisition from libxlDomainStart to it's callers so > they have more control over when the job is acquired. Patch 2 fixes a > few spots where we never acquired a job during domain destroy. Patch 3 > contains the interesting change, where the virDomainObj is unlocked > during the long-running destroy operation. > > This series wraps up my work to improve parallel OpenStack Tempest runs > against the libxl driver. With libvirt.git master + this series + a > patched libxl [1], I've successfully run a reproducer that was hitting > the same issues encountered by Tempest. > > [1] libxl commits from xen.git: 93699882d, f1335f0d, 4783c99a, 1c91d6fba, > and 188e9c54. I'll contact the stable branch maintainers and ask them > to include these commits in the next Xen 4.4.x and 4.5.x releases. > > Jim Fehlig (3): > libxl: Move job acquisition in libxlDomainStart to callers > libxl: acquire a job when destroying a domain > libxl: drop virDomainObj lock when destroying a domain I am no expert at this- but I dug through the code to understand how the job and locking is done and now I am more comfortable with it. Since I am new to this I went through all of the the callsites (which used the job now) from the driver to make sure that there are no chained calls (one function calling another which also uses a mutex or job locking). I only found one culprit (libxlDomainAutoCoreDump being called from libxlDomainShutdownThread). Reviewed-by: Konrad Rzeszutek Wilk > > src/libxl/libxl_domain.c | 77 +++ > src/libxl/libxl_domain.h | 4 --- > src/libxl/libxl_driver.c | 78 > > 3 files changed, 89 insertions(+), 70 deletions(-) > > -- > 1.8.4.5 > > > ___ > Xen-devel mailing list > xen-de...@lists.xen.org > http://lists.xen.org/xen-devel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
