[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-10-05 Thread Launchpad Bug Tracker
This bug was fixed in the package libvirt - 1.2.2-0ubuntu13.1.23 --- libvirt (1.2.2-0ubuntu13.1.23) trusty; urgency=medium * d/libvirt-bin.init, d/libvirt-bin.upstart: fix waiting for the libvirt socket (LP: #1571209) - avoid timing out on slow systems (only stop when servic

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-28 Thread ChristianEhrhardt
I tested again the same way I did with the ppa, and it works for me with the fix that is in proposed. So the behavior should be more reliable now for those "real" cases affected by slow systems that open up the window for the former issue to happen. ** Tags removed: verification-needed verificat

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-21 Thread Brian Murray
Hello guessi, or anyone else affected, Accepted libvirt into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/1.2.2-0ubuntu13.1.23 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See htt

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-13 Thread ChristianEhrhardt
Uploaded to trusty-unapproved for SRU Team review. Please note: there is another upload [1] currently passing through SRU processing. We want [1] to completely pass before the new upload [2] is accepted. The former one is all complete and just needs a few days more in proposed. So if all goes ri

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-13 Thread ChristianEhrhardt
The MP review seems to get close to approval, added an SRU Template here. ** Description changed: + [Impact] + + * Libvirt service reports to be ready, but it has not spawned the libvirt +socket yet. Depending services fail. There was an SRU (#1455608) meant +to fix that but it has ma

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-07 Thread ChristianEhrhardt
MP open and linked here -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retries too short for a busy system boot To manage notifications about this bug go to: https://bu

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-07 Thread Launchpad Bug Tracker
** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/libvirt/+git/libvirt/+merge/330359 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retrie

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-07 Thread ChristianEhrhardt
Fix seems to work, I see it waiting for 1+2+4+8 second overall before declaring the job ready by detecting the socket. Also the extended test adapting unix_sock_path in /etc/libvirt/libvirtd.conf e.g. to "/var/run" worked (which would have failed before). Libvirt comes up and is considered up corr

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-07 Thread guessi
@paelzer, glad to see you reproduce the problem successfully, and simulate by creating nested KVM is better the my test, thanks here I would like to add more history for this bug report, * it is running under KVM with multiple instance * it is ungracefully shut-down (could be simulated by virsh i

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-07 Thread guessi
forget to mention, the RAID system I mentioned above, it is talking about software-raid system, there's no raid card on server hope these info would help :) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bu

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-07 Thread ChristianEhrhardt
Backport of what was applied to zesty plus the fixes discussed here is building in ppa [1] * d/libvirt-bin.init, d/libvirt-bin.upstart: fix waiting for the libvirt socket (LP: #1571209) - avoid timing out on slow sy

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-07 Thread ChristianEhrhardt
Thank you Guessi for your summary once more. I tried to get closer to your case. # Install a Trusty system with Raid-1, that has space for many guests (a lot of memory) but is slow (1 cpu) $ wget http://releases.ubuntu.com/14.04/ubuntu-14.04.5-server-amd64.iso $ qemu-img create -f qcow2 trusty-di

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
Yeah, nobody minds yakkety anymore (yes - it is that much time that passed :-/) and on the others there is no change yet (and systemd by default). I'll go into another repro session closer to the case you had adding: - reboots - unclean shutdown - maybe a raid or something similar to stress this e

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
Acitivty-Ball moved to my side of the field, setting triaged to make it clear that we do not wait for the reporter - even though we havn't reached "confirmed" yet in a strict sense. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread guessi
@paelzer, by the way, the final patch that merged into Yakkety is "sleep 0.5" see "diff from 1.3.4-1ubuntu3 to 1.3.4-1ubuntu4 (1.6 KiB)" - https://launchpad.net/ubuntu/+source/libvirt/1.3.4-1ubuntu4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscrib

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread guessi
@paelzer, glad to see people start re-work on this issue, but it's been 16+ months old, I can only tell the scenario I remember, I think it would not happened if it is already booted, it is only happened *at boot time*, as I describe in originally message, key points are, * it is running with >1

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
Slowed down further (lmited to 2 cpus), startup of the guests takes minutes now. Still no issue to be seen. The socket is around and working, while in background the guests start slowly but surely. The test is now: # cat test-restart.sh #!/bin/bash set -uxe #starts off date ls -laF /var/run/libv

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
My Trusty system wan't as slow, so I used 100 guests on autostart. But that seemed to work as well (test adapted for upstart): ./test-restart.sh + date Wed Sep 6 09:34:58 UTC 2017 + sudo service libvirt-bin start libvirt-bin start/running, process 19805 + date Wed Sep 6 09:35:00 UTC 2017 + /bi

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
That leaves to fix Trusty: - do verifications on Trusty before working on SRU (we need working tests steps for SRU anyway) If confirmed: - backport supposed fix from guessi (thanks for all your work btw and sorry this fell out of sight) - adapt to not print messages too often (adaptive sleep) - h

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
Changed the order of status and list to get it even better while testing on Xenial. Also the system was even slower which is good I think (for the test). So things are good on Xenial onward (with systemd). $ sudo ./test-restart.sh + date Mi 6. Sep 09:03:35 UTC 2017 + sudo systemctl start libvir

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
To check on the systemd case I picked a rather slow system and spawned 30 guests. I set all those to autostart and then shut them down. Starting them sequentially as well as concurrently takes about ~20 seconds. /var/run/libvirt/libvirt-sock is gone after the service is stopped. With the follow

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
Found while testing: BTW - the base path of the socket is configurable, so the fixes would need to consider "unix_sock_dir" from /etc/libvirt/libvirtd.conf to be correct. If anyone changes that in the config the service will become "unstartable" as it always would consider it not up and break it.

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2017-09-06 Thread ChristianEhrhardt
This bug was dead for quite some time, but the issue still exists for Trusty (were we could take the supposed change) and we have to check the systemd cases in >=Xenial. Adapting tasks and will come back once I tested on >=Xenial. ** Also affects: libvirt (Ubuntu Zesty) Importance: Undecided

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-09-30 Thread Jon Grimm
IRC chat with hallyn is that he'd like to hand this off to someone else to SRU, but is a willing sponsor if needed. ** Changed in: libvirt (Ubuntu Wily) Status: New => Won't Fix -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. h

Re: [Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-05-26 Thread Serge Hallyn
Yes, I suppose xenial we could do right now, but I'm waiting on another package to clear trusty-proposed. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retries too shor

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-05-26 Thread guessi
@serge, great thanks! looking for LTS (trusty/xenial). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retries too short for a busy system boot To manage notifications a

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-05-23 Thread Launchpad Bug Tracker
This bug was fixed in the package libvirt - 1.3.4-1ubuntu4 --- libvirt (1.3.4-1ubuntu4) yakkety; urgency=medium * Re-enable the upstart job by renaming the file. * Include patchby @guessi to continally wait for libvirtd to start when using sysvinit or upstart. (LP: #1571209)

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-05-23 Thread Serge Hallyn
Sorry, working on it for yakkety now. ** Changed in: libvirt (Ubuntu) Assignee: Edward Hope-Morley (hopem) => Serge Hallyn (serge-hallyn) ** Changed in: libvirt (Ubuntu) Status: Confirmed => In Progress ** Changed in: libvirt (Ubuntu Precise) Importance: Undecided => High ** Chan

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-05-22 Thread guessi
@serge, is there any milestone to fix this rare case problem? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retries too short for a busy system boot To manage notifica

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-25 Thread guessi
thanks, I'll keep track on this issue. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retries too short for a busy system boot To manage notifications about this bug go

Re: [Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-25 Thread Serge Hallyn
Hi, I'll just drop it to 1s, thanks. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retries too short for a busy system boot To manage notifications about this bug go

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-25 Thread guessi
@serge, sorry for the late reply, we live in different timezone :p I would prefer decrease the delay time down to 1s, but keep it wait infinitely. according my original post (boot up with degraded RAID, 5s -> 10s), it takes up to 10s in my case, so I don't think increasing delay from 0.5s to 2s

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-25 Thread Serge Hallyn
Heh. So the importance of this patch is underscored by the fact that this has basically turned every 'start libvirt-bin' into a sleep 2s. So I guess we should drop the number. Or perhaps we should increment the sleep time on every iteration? start at 0.5 second and go up to a 2s sleep at the en

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-25 Thread Serge Hallyn
** Also affects: libvirt (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: libvirt (Ubuntu Wily) Importance: Undecided Status: New ** Also affects: libvirt (Ubuntu Trusty) Importance: Undecided Status: New ** Also affects: libvirt (Ubuntu Precise)

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-20 Thread guessi
** Patch added: "libvirt-bin-wily.diff" https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1571209/+attachment/4641121/+files/libvirt-bin-wily.diff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-20 Thread guessi
** Patch added: "libvirt-bin-xenial.diff" https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1571209/+attachment/4641122/+files/libvirt-bin-xenial.diff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-20 Thread guessi
** Patch added: "libvirt-bin-precise.diff" https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1571209/+attachment/4641119/+files/libvirt-bin-precise.diff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.ne

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-20 Thread guessi
@serge, to avoid emit too much log, I've change the delay time back to "2". @serge, @hopem, patch files above for precise/trusty/vivid/wily/xenial have been uploaded, should work as expected, but still, please have time to review them. thanks !!! ** Tags added: precise vivid wily -- You recei

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-20 Thread guessi
** Patch added: "libvirt-bin-vivid.diff" https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1571209/+attachment/4641120/+files/libvirt-bin-vivid.diff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bu

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-20 Thread guessi
this is the patch file from #4, but keep sleep wait time "2" https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1571209/comments/4 ** Patch added: "trusty patch" https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1571209/+attachment/4641118/+files/libvirt-bin-trusty.diff -- You rece

Re: [Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-20 Thread Serge Hallyn
Quoting guessi (gue...@gmail.com): > @serge, > > no, I don't, > it's simply shortened the waiting time for service back or down. Ok - the shorter time itself is probably fine, but printing a message every 0.2 seconds seemed a bit much. On the other hand I don't want to complicate the logic to on

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-19 Thread guessi
@serge, no, I don't, it's simply shortened the waiting time for service back or down. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retries too short for a busy system

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-19 Thread Serge Hallyn
Thanks, @guessi, that patch looks great. The only thing i wonder is whether sleep 0.5s is a bit too short. I assume you had a rationale for shortening the sleep time? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launc

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-19 Thread guessi
@serge, @hopem, I'd like to propose a new approach, wait infinitely until the sockfile is ready, and allow it to exit if there's an interrupt of stop/restart/force-stop event. * excluding systemd solution, sorry I'm not familiar with systemd. it is inspired by the following PR from Docker project

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-18 Thread Serge Hallyn
This is the very definition of racy. It appears to me that the point of the post-start is to make sure libvirt is ready, no matter how long it takes. If libvirt actually fails to start, upstart should catch that and mark it failed and abort post-start, is that right? If not, then we should compl

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-16 Thread guessi
** Description changed: [ problem description ] - sockfile_check_retries is first introduced by #1455608, for preventing the failure case of sockfile not ready, - but it was default to a hard-coded value "5", it might be too short for a busy system boot, + sockfile_check_retries is first i

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-16 Thread guessi
** Tags added: trusty xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1571209 Title: Sockfile check retries too short for a busy system boot To manage notifications about this bug go to: https

[Bug 1571209] Re: Sockfile check retries too short for a busy system boot

2016-04-16 Thread Ubuntu Foundations Team Bug Bot
The attachment "allow-change-sockfile_check_retries.diff" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu- reviewers, unsubscribe the team. [This is an automated message performed by a Launchpad u