** Description changed: - When bringing up the Azure datasource in cloud-init.service, cloud-init - tries 'service start walinuxagent'. + http://pad.lv/1623570 + ==== Begin SRU Template ==== + [Impact] + On Azure (and only on Azure), cloud-init utilizes the walinuxagent + to do some of the instance initialization. A change to cloud-init + to support installation of packages on a systemd system (bug 1576692), + exposed an issue in the walinuxagent.service file. + + The end result is freshly booted instances on azure with the cloud-init + fix and the old version of walinuxagent would fail to import ssh keys + that were provided to the instance. + + [Test Case] + * Start an instance on Azure + * upgrade to cloud-init at 0.7.7-31-g65ace7b or better + this is in both xenial-proposed and yakkety. + + * clean up, so it looks like a first boot. + sudo sh -c 'rm -Rf /var/lib/cloud /var/log/cloud-init* /var/lib/waagent/*.crt' + * At this point a reboot would cause failure, and you would see WARN messages in the logs. + + * upgrade walinuxagent with a fix (2.1.5-0ubuntu2 in yakkety) + + * reboot + * ssh back in, + * There should be no WARN in /var/log/cloud-init.log + grep WARN /var/log/cloud-init.log + * There should exist a file /var/lib/waagent/*.crt + * walinuxagent.service should be running + systemctl status walinuxagent.service + + Note, that in event of failure you can still get into the system by + providing user-data that populates your ssh keys: + #!/bin/sh + ssh-import-id smoser + + [Regression Potential] + Changing order of things in boot is scary. This is no different. + This would in theory allow walinux-agent to start earlier in the boot. + In practice, however, it is started by cloud-init.service so it was + already running that early in the boot. + ==== End SRU Template ==== + + + When bringing up the Azure datasource in cloud-init.service, cloud-init tries 'service start walinuxagent'. That previously worked fine, and the agent would start and then would produce the certificate files that cloud-init needed (for ssh keys and things). I found this when testing SRU for 0.7.7-31-g65ace7b-0ubuntu1~16.04.1 but it is likely present also in 0.7.7-31-g65ace7b-0ubuntu1 (yakkety) Now, however we see a log like: Sep 14 14:53:18 smoser0914x [CLOUDINIT] DataSourceAzure.py[DEBUG]: Getting metadata via agent. hostname=smoser0914x cmd=['service', 'walinuxagent', 'start'] Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[DEBUG]: Running command hostname with allowed return codes [0] (shell=False, capture=True) Sep 14 14:53:18 smoser0914x [CLOUDINIT] DataSourceAzure.py[DEBUG]: invoking agent: ['service', 'walinuxagent', 'start'] Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[DEBUG]: Running command ['service', 'walinuxagent', 'start'] with allowed return codes [0] (shell=False, capture=True) Sep 14 14:53:18 smoser0914x [CLOUDINIT] util.py[WARNING]: agent command '['service', 'walinuxagent', 'start']' failed. Sep 14 14:53:19 smoser0914x [CLOUDINIT] util.py[DEBUG]: agent command '['service', 'walinuxagent', 'start']' failed. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 145, in get_metadata_from_agent invoke_agent(agent_cmd) File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 452, in invoke_agent util.subp(cmd, shell=(not isinstance(cmd, list))) File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1832, in subp cmd=args) cloudinit.util.ProcessExecutionError: Unexpected error while running command. Command: ['service', 'walinuxagent', 'start'] Exit code: 1 Reason: - Stdout: '' Stderr: " Failed to start walinuxagent.service: Transaction order is cyclic. See system logs for details. See system logs and 'systemctl status walinuxagent.service' for details I believe the relevant change is in 34a26f7f https://git.launchpad.net/cloud-init/commit/?id=34a26f7f59f2963691e36ca0476bec9fc9ccef63 That added multi-user.target to the list of After for cloud-init-final.service. Related bugs: - * bug 1576692: fully support package installation in systemd + * bug 1576692: fully support package installation in systemd
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1623570 Title: Azure: cannot start walinux agent (Transaction order is cyclic.) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1623570/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
