[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
** Attachment added: "juju-crashdump-openstack-2018-10-11-13.29.52.tar.gz" https://bugs.launchpad.net/ubuntu/+source/open-iscsi/+bug/1651497/+attachment/5200440/+files/juju-crashdump-openstack-2018-10-11-13.29.52.tar.gz ** Changed in: open-iscsi (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
Just saw this one again after a while. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
If we were to change the behavior of open-iscsi to be socket activated, then that should happen under bug 1755858. ** Description changed: I am getting a transient error installing open-iscsi in a xenial lxd container. Sometimes it works fine, but occasionally, it fails like this (apt-get -f install usually fixes it up). As you can see in the syslog, the daemon fails to start, but then later starts fine. I think it might have something to do with the refresh cycle that is going on at the same time the iscsi daemon is starting, but I'm not really sure: Easier reading: http://paste.ubuntu.com/23659441/ Error installing iscsi: 017-12-20 12:33:54 INFO install Setting up open-iscsi (2.0.873+git0.3b4b4500-14ubuntu3.3) ... 2016-12-20 12:33:55 INFO install Job for iscsid.service failed because a configured resource limit was exceeded. See "systemctl status iscsid.service" and "journalctl -xe" for details. 2016-12-20 12:33:55 INFO install invoke-rc.d: initscript iscsid, action "restart" failed. 2016-12-20 12:33:55 INFO install dpkg: error processing package open-iscsi (--configure): 2016-12-20 12:33:55 INFO install subprocess installed post-installation script returned error exit status 1 - Syslog, from the end of cloud-init until the shutdown of the system: - Dec 20 12:33:24 juju-e08f21-0-lxd-2 cloud-init[387]: Cloud-init v. 0.7.8 finished at Tue, 20 Dec 2016 12:33:24 +. Datasource DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]. Up 76.0 seconds Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: cloud-final.service: Main process exited, code=exited, status=1/FAILURE Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: Failed to start Execute cloud user/final scripts. Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: cloud-final.service: Unit entered failed state. Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: cloud-final.service: Failed with result 'exit-code'. Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: Reached target Cloud-init target. Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: Startup finished in 1min 15.898s. Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-bridge.service: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Starting LXD - network bridge... Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Started LXD - network bridge. Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd.service: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Starting LXD - main daemon... Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Reloading. Dec 20 12:33:25 juju-e08f21-0-lxd-2 lxd[8960]: t=2016-12-20T12:33:25+ lvl=warn msg="Per-container AppArmor profiles are disabled because LXD is running in an unprivileged container." Dec 20 12:33:25 juju-e08f21-0-lxd-2 lxd[8960]: t=2016-12-20T12:33:25+ lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored." Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 9h 57min 27.662713s random time. Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 5h 57min 51.806386s random time. Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 1h 24min 59.817093s random time. Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-unit-postgresql-0.service: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Started juju unit agent for postgresql/0. Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /user.slice: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /init.scope: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ssh.service: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-null.mount: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-fuse.mount: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-bridge.service: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-.lxd\x2dmounts.mount: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-full.mount: Operation not permitted Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-update-utmp.service: Operation not pe
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
2016-12-20 12:33:55 INFO install Job for iscsid.service failed because a configured resource limit was exceeded. See "systemctl status ^ if that is the source of the issue, then this is "working as designed". If the container had some memory or other constraints and open-iscsi failed to restart as a result, then there isn't really much that can be done here. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
I also cannot repro this failure. I can trigger the upgrade, and use a script similar to what smoser had, but each time, the upgrade works fine. I went through 250 iterations of the script without failure. Could someone please attach a reproducer, or give access to a live system that has just shown the failure? ** Changed in: open-iscsi (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
I cannot seem to re-create this, so I'm not really interested in SRU'ing a fix that we think might solve the problem without the ability to verify that. Here is what I'm doing to attempt to recreate http://paste.ubuntu.com/p/STVg62nGPb/ I'm through 432 containers 10 at a time right now on my system without any recreate. I'm not opposed to SRU'ing the change to start open-iscsi only when necessary, but that should be done on its own merit, with its own test cases and such. I'd suggest another bug for that rather than taking over this one. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
@Jason, I'm guessing that you ran some script to collect the logs that you collected (open-iscsi-errors.tar.gz). There is a lot of good information there, thank you. Please add collection of /etc/cloud/build.info to the script that collects. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
I had hoped to reproduce this so that I could be sure that I actually fixed it. What I did to attempt that was - download the 20180816 xenial daily build that had 2.0.873+git0.3b4b4500-14ubuntu3.4. - add to lxc lxc image import xenial-server-cloudimg-amd64-lxd.tar.xz xenial-server-cloudimg-amd64.squashfs --alias xenial-20180816 - start instance: lxc launch xenial-20180816 xtest - enter: lxc exec xtest /bin/bash - update, upgrade ... that did not reproduce. :-( Attached is a screen log. ** Attachment added: "info collected from 20180816 and upgrade" https://bugs.launchpad.net/ubuntu/+source/open-iscsi/+bug/1651497/+attachment/5183520/+files/data.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
Fixing this properly should mean that open-iscsi is socket-activated in stable releases the same as it currently is in cosmic. This should resolve the problems including on package upgrade since the postinst will no longer be trying to start the daemon that shouldn't be started. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1651497] Re: iscsid.service fails to start in container, results in failed dist-upgrade later on
** Changed in: open-iscsi (Ubuntu) Assignee: (unassigned) => Scott Moser (smoser) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1651497 Title: iscsid.service fails to start in container, results in failed dist- upgrade later on To manage notifications about this bug go to: https://bugs.launchpad.net/autopilot-log-analyser/+bug/1651497/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
