[Bug 711635] Re: init: post-start can cause respawn to hang
lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as Won't Fix. ** Changed in: mysql-dfsg-5.1 (Ubuntu Lucid) Status: Triaged = Won't Fix -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
eol reached https://wiki.ubuntu.com/Releases ** Changed in: mysql-5.1 (Ubuntu Oneiric) Status: Triaged = Invalid ** Changed in: mysql-5.1 (Ubuntu Natty) Status: Triaged = Invalid -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Also affects: upstart (Ubuntu) Importance: Undecided Status: New ** Changed in: upstart (Ubuntu) Importance: Undecided = Medium ** Changed in: upstart (Ubuntu) Status: New = Triaged ** Changed in: upstart (Ubuntu) Assignee: (unassigned) = James Hunt (jamesodhunt) ** Changed in: upstart (Ubuntu Lucid) Status: New = Won't Fix -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Changed in: upstart (Ubuntu Natty) Status: New = Won't Fix ** Changed in: upstart (Ubuntu Oneiric) Status: New = Won't Fix -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Changed in: upstart (Ubuntu Precise) Importance: Undecided = Medium ** Changed in: upstart (Ubuntu Precise) Status: New = Triaged -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
I've rejected the lucid upload here; 5.1.62 has been uploaded by the security team to -proposed, so this will need to be rebased on that. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Description changed: + == SRU Justification == + + = IMPACT = + + Users who have inadvertently broken their mysql configuration will be + met with a hung package upgrade process on any updates, and no easy way + to see that fact. + + = TEST CASE: = + + 1. sudo apt-get install mysql-server + 2. edit /etc/mysql/my.cnf and add an invalid variable to the [mysqld] section + 3. sudo apt-get --reinstall install mysql-server - affected system will hang forever on the 'start' of mysql + 4. in another terminal, verify status of mysql job is 'start/respawn' + 5. in that second terminal, run 'sudo stop mysql' + 6. first terminal should now be unlocked + 7. install updated package + 8. install should error out on starting mysql, but not hang, giving the user an opportunity to see their broken /etc/mysql/my.cnf. + + = Dev Fix = + + This was fixed in the 5.5 packages by making sure that in the post-start + if the job is in 'respawn' that the post-start is aborted immediately. + + = REGRESSION POTENTIAL = + + Its possible that this code will cause some systems to error out where + they would keep retrying forever. This might be particularly problematic + in systems that use configuration management like puppet or chef, where + they would boot the system, and then run the config management to apply + the correct configs. + + + I believe that it is looking for some input when updating mysql. The updating just stops when mysql is being updated. There is no dialog box to be found. This has happened to me several times on different computers. ProblemType: Package DistroRelease: Ubuntu 10.04 Package: mysql-server-5.1 5.1.41-3ubuntu12.9 ProcVersionSignature: Ubuntu 2.6.32-27.49-generic 2.6.32.26+drm33.12 Uname: Linux 2.6.32-27-generic x86_64 NonfreeKernelModules: wl nvidia Architecture: amd64 Date: Tue Feb 1 18:32:38 2011 ErrorMessage: subprocess installed post-installation script killed by signal (Terminated) InstallationMedia: Ubuntu 9.10 Karmic Koala - Release amd64 (20091027) MySQLConf.etc.mysql.conf.d.mysqld.safe.syslog.cnf: - [mysqld_safe] - syslog + [mysqld_safe] + syslog MySQLVarLibDirListing: ['mysql', 'debian-5.1.flag'] SourcePackage: mysql-dfsg-5.1 Title: package mysql-server-5.1 5.1.41-3ubuntu12.9 failed to install/upgrade: subprocess installed post-installation script killed by signal (Terminated) mtime.conffile..etc.apparmor.d.usr.sbin.mysqld: 2010-03-06T11:30:30.842210 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: mysql-5.5 (Ubuntu Lucid) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: mysql-5.5 (Ubuntu Natty) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: mysql-5.5 (Ubuntu Oneiric) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Changed in: mysql-5.5 (Ubuntu Lucid) Status: Confirmed = Invalid ** Changed in: mysql-5.5 (Ubuntu Natty) Status: Confirmed = Invalid ** Changed in: mysql-5.5 (Ubuntu Oneiric) Status: Confirmed = Invalid ** Description changed: == SRU Justification == = IMPACT = Users who have inadvertently broken their mysql configuration will be met with a hung package upgrade process on any updates, and no easy way to see that fact. = TEST CASE: = 1. sudo apt-get install mysql-server 2. edit /etc/mysql/my.cnf and add an invalid variable to the [mysqld] section 3. sudo apt-get --reinstall install mysql-server - affected system will hang forever on the 'start' of mysql 4. in another terminal, verify status of mysql job is 'start/respawn' 5. in that second terminal, run 'sudo stop mysql' 6. first terminal should now be unlocked 7. install updated package - 8. install should error out on starting mysql, but not hang, giving the user an opportunity to see their broken /etc/mysql/my.cnf. + 8. install should error out or move on quietly after trying to start mysql, but not hang indefinitely. = Dev Fix = This was fixed in the 5.5 packages by making sure that in the post-start if the job is in 'respawn' that the post-start is aborted immediately. = REGRESSION POTENTIAL = Its possible that this code will cause some systems to error out where they would keep retrying forever. This might be particularly problematic in systems that use configuration management like puppet or chef, where they would boot the system, and then run the config management to apply the correct configs. I believe that it is looking for some input when updating mysql. The updating just stops when mysql is being updated. There is no dialog box to be found. This has happened to me several times on different computers. ProblemType: Package DistroRelease: Ubuntu 10.04 Package: mysql-server-5.1 5.1.41-3ubuntu12.9 ProcVersionSignature: Ubuntu 2.6.32-27.49-generic 2.6.32.26+drm33.12 Uname: Linux 2.6.32-27-generic x86_64 NonfreeKernelModules: wl nvidia Architecture: amd64 Date: Tue Feb 1 18:32:38 2011 ErrorMessage: subprocess installed post-installation script killed by signal (Terminated) InstallationMedia: Ubuntu 9.10 Karmic Koala - Release amd64 (20091027) MySQLConf.etc.mysql.conf.d.mysqld.safe.syslog.cnf: [mysqld_safe] syslog MySQLVarLibDirListing: ['mysql', 'debian-5.1.flag'] SourcePackage: mysql-dfsg-5.1 Title: package mysql-server-5.1 5.1.41-3ubuntu12.9 failed to install/upgrade: subprocess installed post-installation script killed by signal (Terminated) mtime.conffile..etc.apparmor.d.usr.sbin.mysqld: 2010-03-06T11:30:30.842210 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
This bug was fixed in the package mysql-5.5 - 5.5.22-0ubuntu1 --- mysql-5.5 (5.5.22-0ubuntu1) precise; urgency=low * New upstream release (LP: #965523) * d/mysql-server-5.5.mysql.upstart: Add respawn limit and code to exit ping loop if mysqld needs to be respawned. (LP: #711635) * New upstream bugfix release. * Refreshed patches * d/control: Build-Dep on dh-apparmor (LP: #948481) * d/p/fix-mysqldump-test.patch: Fixes unreliable test introduced in new upstream version. http://bugs.mysql.com/bug.php?id=64401 * d/control: Re-add empty meta-package for mysql-testsuite so users will get the 5.5 version on upgrade and when installing mysql-testsuite. (LP: #959683) -- Clint Byrum cl...@ubuntu.com Thu, 29 Mar 2012 17:31:30 -0700 ** Changed in: mysql-5.5 (Ubuntu Precise) Status: Fix Committed = Fix Released ** Bug watch added: MySQL Bug System #64401 http://bugs.mysql.com/bug.php?id=64401 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Tags added: precise rls-mgr-p-tracking -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Also affects: mysql-5.1 (Ubuntu Lucid) Importance: Undecided Status: New ** Also affects: mysql-5.5 (Ubuntu Lucid) Importance: Undecided Status: New ** Also affects: mysql-dfsg-5.1 (Ubuntu Lucid) Importance: Undecided Status: New ** Also affects: mysql-5.1 (Ubuntu Natty) Importance: Undecided Status: New ** Also affects: mysql-5.5 (Ubuntu Natty) Importance: Undecided Status: New ** Also affects: mysql-dfsg-5.1 (Ubuntu Natty) Importance: Undecided Status: New ** Also affects: mysql-5.1 (Ubuntu Oneiric) Importance: Undecided Status: New ** Also affects: mysql-5.5 (Ubuntu Oneiric) Importance: Undecided Status: New ** Also affects: mysql-dfsg-5.1 (Ubuntu Oneiric) Importance: Undecided Status: New ** Changed in: mysql-5.1 (Ubuntu Lucid) Status: New = Invalid ** Also affects: mysql-5.1 (Ubuntu Precise) Importance: High Status: Triaged ** Also affects: mysql-5.5 (Ubuntu Precise) Importance: High Assignee: Clint Byrum (clint-fewbar) Status: In Progress ** Also affects: mysql-dfsg-5.1 (Ubuntu Precise) Importance: High Status: Triaged ** Changed in: mysql-5.1 (Ubuntu Precise) Status: Triaged = Invalid ** Changed in: mysql-5.5 (Ubuntu Precise) Status: In Progress = Fix Committed ** Changed in: mysql-dfsg-5.1 (Ubuntu Precise) Status: Triaged = Invalid ** Changed in: mysql-dfsg-5.1 (Ubuntu Lucid) Importance: Undecided = High ** Changed in: mysql-dfsg-5.1 (Ubuntu Lucid) Status: New = Triaged ** Changed in: mysql-5.1 (Ubuntu Oneiric) Importance: Undecided = High ** Changed in: mysql-5.1 (Ubuntu Oneiric) Status: New = Triaged ** Changed in: mysql-5.1 (Ubuntu Natty) Importance: Undecided = High ** Changed in: mysql-5.1 (Ubuntu Natty) Status: New = Triaged ** Changed in: mysql-dfsg-5.1 (Ubuntu Natty) Status: New = Invalid ** Changed in: mysql-dfsg-5.1 (Ubuntu Oneiric) Status: New = Invalid -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Also affects: mysql-5.1 (Ubuntu) Importance: Undecided Status: New ** Also affects: mysql-5.5 (Ubuntu) Importance: Undecided Status: New ** Changed in: mysql-5.1 (Ubuntu) Status: New = Triaged ** Changed in: mysql-5.5 (Ubuntu) Status: New = Triaged ** Changed in: mysql-5.1 (Ubuntu) Importance: Undecided = Low ** Changed in: mysql-5.5 (Ubuntu) Importance: Undecided = Low -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
I spent some time on this today, and I think that upstart's handling of respawn + post-start is probably making jobs harder than they need to be. In the mysql case, we need to exit the mysqladmin ping loop if the process goes into 'respawn', so that the process can in fact be respawned... otherwise it waits. This also causes the usual respawn limit to not be hit because we're respawning 1 time every 30 seconds. In fact, just the lag of forking and running mysqladmin ping is enough to prevent us from restarting 10 times in 5 seconds, which is the default limit. So IMO, upstart should have a way to respawn independent of post-start. But that is a much bigger change and needs more thought. For jobs with a post-start, they need to make sure the post-start is reactive to the respawn status by exitting when it is detected, and considering this in any respawn limits. So, I'll be uploading a fix to precise's mysql-5.5 package which will fail with 2 respawns in 5 seconds, and exits the mysqladmin loop if status is respawn. This causes the job to report a failure to start if things are truly broken, which is what we want. It will also cause respawn to give up faster if mysqld exits, but I think that is fine given that it is a large database daemon and probably isn't going to respond well to the rapid respawning. ** Changed in: mysql-5.5 (Ubuntu) Status: Triaged = In Progress ** Changed in: mysql-5.5 (Ubuntu) Assignee: (unassigned) = Clint Byrum (clint-fewbar) ** Changed in: mysql-5.5 (Ubuntu) Importance: Low = High ** Changed in: mysql-5.1 (Ubuntu) Importance: Low = High ** Changed in: mysql-dfsg-5.1 (Ubuntu) Importance: Low = High -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
One final thought, I'm raising the importance to High. Failing to detect and report a serious problem in this sort of daemon seems like an important problem! -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Branch linked: lp:ubuntu/mysql-5.5 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-5.1 in Ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang To manage notifications about this bug go to: https://bugs.launchpad.net/upstart/+bug/711635/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 711635] Re: init: post-start can cause respawn to hang
** Summary changed: - mysql update fails waiting for upstart job to respawn + init: post-start can cause respawn to hang ** Changed in: upstart Status: New = Triaged ** Changed in: upstart Importance: Undecided = Medium -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to mysql-dfsg-5.1 in ubuntu. https://bugs.launchpad.net/bugs/711635 Title: init: post-start can cause respawn to hang -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
