[Yahoo-eng-team] [Bug 1987393] Re: autopkgtest-buildvm-ubuntu-cloud hangs on ppc64el (P9)

[Paride Legovini]

** Also affects: cloud-init
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1987393

Title:
  autopkgtest-buildvm-ubuntu-cloud hangs on ppc64el (P9)

Status in Auto Package Testing:
  New
Status in cloud-init:
  New

Bug description:
  Trying (in different ways) to create autopkgtest VM images failed.
  e.g. with:
  sudo autopkgtest-buildvm-ubuntu-cloud -a ppc64el -r jammy -s 15G
  or:
  sudo autopkgtest-buildvm-ubuntu-cloud -r jammy -v --cloud-image-url 
http://cloud-images.ubuntu.com/daily/server -s 15G -a ppc64el
  or just native on ppc64el:
  sudo autopkgtest-buildvm-ubuntu-cloud -v

  Things look generally pretty promising, but eventually the image hangs / does 
not complete and the latest console messages are:
  ...
  [  OK  ] Finished Record Runlevel Change in UTMP.
  [FAILED] Failed to start Execute cloud user/final scripts.
  See 'systemctl status cloud-final.service' for details.
  [  OK  ] Reached target Cloud-init target.

  or:
  [  387.904399] cloud-init[1234]: Cloud-init v. 22.2-0ubuntu1~22.04.3 finished 
at Tue, 23 Aug 2022 12:23:09 +. Datasource DataSourceNoCloud 
[seed=/dev/vdb][dsmode=net].  Up 387.81 seconds
  [FAILED] Failed to start Execute cloud user/final scripts.
  See 'systemctl status cloud-final.service' for details.
  [  OK  ] Reached target Cloud-init target.

  Due to this it's currently not possible to use autopkgtests with VMs
  on ppc64el.

  (Might also be a cloud-init issue.)

To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-package-testing/+bug/1987393/+subscriptions


-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1940791] Re: sr0 not available causes cloud-init.target not run on focal cloud image

[Paride Legovini]

** Changed in: cloud-images
   Status: Expired => Incomplete

** Changed in: cloud-init
   Status: Expired => Incomplete

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1940791

Title:
  sr0 not available causes cloud-init.target not run on focal cloud
  image

Status in cloud-images:
  Incomplete
Status in cloud-init:
  Incomplete

Bug description:
  Focal image cloud-init generator reports:
  'cloud-init is enabled but no datasource found, disabling'

  looks to be related to ds-identify not finding the cdrom drive (and
  caching it) on first run. Not sure why /dev/sr0 would not be available
  early enough.

  cat /run/cloud-init/ds-identify.log
  ...
  ISO9660_DEVS=
  ...
  No ds found [mode=search, notfound=disabled]. Disabled cloud-init [1]
  [up 1.20s] returning 1
  root@ubuntu:~# /usr/lib/cloud-init/ds-identify --force
  [up 200.71s] ds-identify --force
  ...
  ISO9660_DEVS=/dev/sr0=cidata
  ...
  Found single datasource: NoCloud
  [up 200.79s] returning 0

  Booting https://cloud-images.ubuntu.com/focal/current/focal-server-
  cloudimg-amd64-disk-kvm.img as of Aug 22, 2021 in KVM (created with
  virt-install and libvirt) along with cloud-config ISO

  $ cat /tmp/cloud
  #cloud-config
  hostname: proxy1
  $ cloud-localds /tmp/test.iso /tmp/cloud

  cloud-init.target never reached and network doesn't come up (default
  behavior for cloud-init is eth0 DHCP). If I manually start `systemctl
  start cloud-init.target` then I get what I expected, but by then it is
  "too late" and I also have to kick systemd-networkd.

  cloud-init starts up as expected with the same environment when using
  Bionic (https://cloud-images.ubuntu.com/bionic/current/bionic-server-
  cloudimg-amd64.img)

  The focal image never touches cloud-init.target. Note that there is no
  reverse dependency in focal.

  root@ubuntu:~# systemctl list-dependencies --reverse cloud-init.target
  cloud-init.target

  Both images have default target of "graphical.target"

  There is mention of a "generator" and "detection" in the cloud-init
  docs. https://cloudinit.readthedocs.io/en/latest/topics/boot.html

  The generator appears to be what is adding the "wants" of cloud-init.target 
to multi-user.target
  from /lib/systemd/system-generators/cloud-init-generator:
  local target_name="multi-user.target" gen_d="$early_d"
  local link_path="$gen_d/${target_name}.wants/${CLOUD_TARGET_NAME}"

  Bionic:
  root@proxy1:~# systemctl get-default
  graphical.target
  root@proxy1:~#
  UNIT   LOAD   ACTIVE SUBDESCRIPTION
  basic.target   loaded active active Basic System
  cloud-config.targetloaded active active Cloud-config availability
  cloud-init.target  loaded active active Cloud-init target
  ...
  root@proxy1:~# systemctl list-dependencies --reverse cloud-init.target
  cloud-init.target
  ● └─multi-user.target
  ●   └─graphical.target
  root@proxy1:/etc/systemd/system# cat /run/cloud-init/cloud-init-generator.log
  /lib/systemd/system-generators/cloud-init-generator 
normal=/run/systemd/generator early=/run/systemd/generator.early 
late=/run/systemd/generator.late
  kernel command line (/proc/cmdline): 
BOOT_IMAGE=/boot/vmlinuz-4.15.0-154-generic root=LABEL=cloudimg-rootfs ro 
console=tty1 console=ttyS0
  kernel_cmdline found unset
  etc_file found unset
  default found enabled
  checking for datasource
  ds-identify rc=0
  ds-identify _RET=found
  enabled via 
/run/systemd/generator.early/multi-user.target.wants/cloud-init.target -> 
/lib/systemd/system/cloud-init.target

  Focal:
  root@ubuntu:~# systemctl get-default
  graphical.target
  root@ubuntu:~# systemctl list-units --type=target --all
    UNITLOAD   ACTIVE   SUB 
 >
    basic.targetloaded active   
activ>
    blockdev@dev-disk-by\x2dlabel-cloudimg\x2drootfs.target loaded inactive 
dead >
    blockdev@dev-disk-by\x2dlabel-UEFI.target   loaded inactive 
dead >
    blockdev@dev-loop0.target   loaded inactive 
dead >
    blockdev@dev-loop1.target   loaded inactive 
dead >
    blockdev@dev-loop2.target   loaded inactive 
dead >
    blockdev@dev-vda15.target   loaded inactive 
dead >
    cloud-config.target loaded inactive 
dead >
    cloud-init.target   loaded inactive 
dead >
  root@ubuntu:~# systemctl list-unit-files
  ...
  cloud-config.service   enabled enabled
  cloud-final.serviceenabled enabled
  cloud-init-local.service   enabled enabled
  cloud-init.service enabled enabled
  ...
  root@ubuntu:~# systemctl list-dependencies --reverse 

[Yahoo-eng-team] [Bug 1961620] Re: cloud-init can add users in wrong filesystem (race with `mount /home`)

[Paride Legovini]

Thanks for looking into this. I thought about pivoting into /target too,
that should save us from worrying about mounts at all, but requires
changes in how subiquity operates, and in general moves a bit away from
the idea that installs done from ISO should be treatable like cloud
instances, which is what allows us to use cloud-init on bare metal after
all. It a good guiding principle, as it helps convergence between bare
metal server systems and cloud instances.

On "not making cloud-init wait forever": I see your point, however in
the case of subiquity we're speaking of a freshly installed system,
which can't be in production. Between blocking boot and booting but
misconfiguring the system, I'm not sure the latter is better.

** Also affects: subiquity
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1961620

Title:
  cloud-init can add users in wrong filesystem (race with `mount /home`)

Status in cloud-init:
  Triaged
Status in subiquity:
  New

Bug description:
  When cloud-init is used to configure a new Ubuntu Server system
  installed from the ISO images, and /home is configured as a separate
  partition, there is a (slow) race between the user creation and /home
  being mounted. This can lead to the user $HOME being created in the
  wrong filesystem.

  Steps to reproduce:

  1. Prepare to install focal-live-server-amd64.iso in a VM.
 In my case I used one of the 20.04.4 dailies.

  2. Proceed with all-defaults but for storage. Configure the storage
 so / is in a dedicated partition, while /home in a an *encrypted*
 LVM volume. (The only purpose of encryption is to add delay in the
 /home mount, see the next point.)

  3. Finish the install and reboot. At the dm-crypt password prompt
 stop and wait a few minutes. At some point cloud-init will proceed
 creating the configured username, but /home is not mounted yet!
 The user's $HOME is now in the same filesystem as /.

  4. Enter the dm-crypt password. This will cause /home to be mounted
 from the encrypted volume, and this will shadow the actual $HOME.

  5. Login with the configured credentials and verify that $HOME is
 inaccessible.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1961620/+subscriptions


-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1961620] [NEW] cloud-init can add users in wrong filesystem (race with `mount /home`)

[Paride Legovini]

Public bug reported:

When cloud-init is used to configure a new Ubuntu Server system
installed from the ISO images, and /home is configured as a separate
partition, there is a (slow) race between the user creation and /home
being mounted. This can lead to the user $HOME being created in the
wrong filesystem.

Steps to reproduce:

1. Prepare to install focal-live-server-amd64.iso in a VM.
   In my case I used one of the 20.04.4 dailies.

2. Proceed with all-defaults but for storage. Configure the storage
   so / is in a dedicated partition, while /home in a an *encrypted*
   LVM volume. (The only purpose of encryption is to add delay in the
   /home mount, see the next point.)

3. Finish the install and reboot. At the dm-crypt password prompt
   stop and wait a few minutes. At some point cloud-init will proceed
   creating the configured username, but /home is not mounted yet!
   The user's $HOME is now in the same filesystem as /.

4. Enter the dm-crypt password. This will cause /home to be mounted
   from the encrypted volume, and this will shadow the actual $HOME.

5. Login with the configured credentials and verify that $HOME is
   inaccessible.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1961620

Title:
  cloud-init can add users in wrong filesystem (race with `mount /home`)

Status in cloud-init:
  New

Bug description:
  When cloud-init is used to configure a new Ubuntu Server system
  installed from the ISO images, and /home is configured as a separate
  partition, there is a (slow) race between the user creation and /home
  being mounted. This can lead to the user $HOME being created in the
  wrong filesystem.

  Steps to reproduce:

  1. Prepare to install focal-live-server-amd64.iso in a VM.
 In my case I used one of the 20.04.4 dailies.

  2. Proceed with all-defaults but for storage. Configure the storage
 so / is in a dedicated partition, while /home in a an *encrypted*
 LVM volume. (The only purpose of encryption is to add delay in the
 /home mount, see the next point.)

  3. Finish the install and reboot. At the dm-crypt password prompt
 stop and wait a few minutes. At some point cloud-init will proceed
 creating the configured username, but /home is not mounted yet!
 The user's $HOME is now in the same filesystem as /.

  4. Enter the dm-crypt password. This will cause /home to be mounted
 from the encrypted volume, and this will shadow the actual $HOME.

  5. Login with the configured credentials and verify that $HOME is
 inaccessible.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1961620/+subscriptions


-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1944414] [NEW] tox: tip-pylint fails with pylint >= 2.11.1

[Paride Legovini]

Public bug reported:

As usual we periodically need to:
 - Fix the issue with the newer linters
 - Update the pinned deps in tox.ini

Reproducer:
  - tox -e tip-pylint

Log excerpt:

* Module cloudinit.subp
cloudinit/subp.py:238: [W1514(unspecified-encoding), subp] Using open without 
explicitly specifying an encoding
* Module cloudinit.url_helper
cloudinit/url_helper.py:488: [W1514(unspecified-encoding), 
OauthUrlHelper.read_skew_file] Using open without explicitly specifying an 
encoding
cloudinit/url_helper.py:500: [W1514(unspecified-encoding), 
OauthUrlHelper.update_skew_file] Using open without explicitly specifying an 
encoding
* Module cloudinit.analyze.dump
cloudinit/analyze/dump.py:174: [W1514(unspecified-encoding), main] Using open 
without explicitly specifying an encoding
* Module cloudinit.analyze.__main__
cloudinit/analyze/__main__.py:218: [W1514(unspecified-encoding), configure_io] 
Using open without explicitly specifying an encoding
cloudinit/analyze/__main__.py:227: [W1514(unspecified-encoding), configure_io] 
Using open without explicitly specifying an encoding
* Module cloudinit.util
cloudinit/util.py:369: [W1514(unspecified-encoding), multi_log] Using open 
without explicitly specifying an encoding
cloudinit/util.py:1162: [W1514(unspecified-encoding), close_stdin] Using open 
without explicitly specifying an encoding
cloudinit/util.py:1975: [W1514(unspecified-encoding), write_file] Using open 
without explicitly specifying an encoding
* Module cloudinit.cmd.cloud_id
cloudinit/cmd/cloud_id.py:56: [W1514(unspecified-encoding), handle_args] Using 
open without explicitly specifying an encoding
* Module cloudinit.cmd.devel.make_mime
cloudinit/cmd/devel/make_mime.py:24: [W1514(unspecified-encoding), 
file_content_type] Using open without explicitly specifying an encoding
* Module cloudinit.cmd.main
cloudinit/cmd/main.py:226: [W1514(unspecified-encoding), 
purge_cache_on_python_version_change] Using open without explicitly specifying 
an encoding
* Module cloudinit.cmd.devel.render
cloudinit/cmd/devel/render.py:70: [W1514(unspecified-encoding), handle_args] 
Using open without explicitly specifying an encoding
* Module cloudinit.reporting.handlers
cloudinit/reporting/handlers.py:165: [W1514(unspecified-encoding), 
HyperVKvpReportingHandler._truncate_guest_pool_file] Using open without 
explicitly specifying an encoding
* Module cloudinit.net.eni
cloudinit/net/eni.py:179: [W1514(unspecified-encoding), _parse_deb_config_data] 
Using open without explicitly specifying an encoding
cloudinit/net/eni.py:190: [W1514(unspecified-encoding), _parse_deb_config_data] 
Using open without explicitly specifying an encoding
cloudinit/net/eni.py:281: [W1514(unspecified-encoding), parse_deb_config] Using 
open without explicitly specifying an encoding
* Module cloudinit.net.tests.test_dhcp
cloudinit/net/tests/test_dhcp.py:409: [W1514(unspecified-encoding), 
TestDHCPDiscoveryClean.test_dhcp_discovery_run_in_sandbox] Using open without 
explicitly specifying an encoding
cloudinit/net/tests/test_dhcp.py:455: [W1514(unspecified-encoding), 
TestDHCPDiscoveryClean.test_dhcp_discovery_outside_sandbox] Using open without 
explicitly specifying an encoding
* Module cloudinit.tests.helpers
cloudinit/tests/helpers.py:472: [W1514(unspecified-encoding), readResource] 
Using open without explicitly specifying an encoding
* Module cloudinit.tests.test_features
cloudinit/tests/test_features.py:31: [W1514(unspecified-encoding), 
create_override] Using open without explicitly specifying an encoding
* Module cloudinit.sources.DataSourceNoCloud
cloudinit/sources/DataSourceNoCloud.py:250: 
[E1135(unsupported-membership-test), _quick_read_instance_id] Value 'md' 
doesn't support membership test
cloudinit/sources/DataSourceNoCloud.py:251: [E1136(unsubscriptable-object), 
_quick_read_instance_id] Value 'md' is unsubscriptable
* Module cloudinit.sources.DataSourceOVF
cloudinit/sources/DataSourceOVF.py:755: [W1514(unspecified-encoding), 
setup_marker_files] Using open without explicitly specifying an encoding
* Module cloudinit.sources.DataSourceCloudStack
cloudinit/sources/DataSourceCloudStack.py:253: [W1514(unspecified-encoding), 
get_vr_address] Using open without explicitly specifying an encoding
* Module cloudinit.sources.DataSourceVMware
cloudinit/sources/DataSourceVMware.py:270: [W1514(unspecified-encoding), 
DataSourceVMware.get_instance_id] Using open without explicitly specifying an 
encoding
* Module cloudinit.sources.helpers.azure
cloudinit/sources/helpers/azure.py:262: [W1514(unspecified-encoding), 
get_last_log_byte_pushed_to_kvp_index] Using open without explicitly specifying 
an encoding
cloudinit/sources/helpers/azure.py:509: [W1514(unspecified-encoding), 
OpenSSLManager.generate_certificate] 

[Yahoo-eng-team] [Bug 1853376] Re: fix debian packaging warnings/errors

[Paride Legovini]

Lintian is now rather happy with the package (both source and binary),
with just one warning remaining:

W: cloud-init: command-with-path-in-maintainer-script postinst:296
/usr/sbin/grub-install

Explanation: https://lintian.debian.org/tags/command-with-path-in-
maintainer-script

I'll submit a PR for this, but given that there are no errors anymore I
think we can mark this as Fix Released.

** Changed in: cloud-init
   Status: Triaged => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1853376

Title:
  fix debian packaging warnings/errors

Status in cloud-init:
  Fix Released

Bug description:
  E: cloud-init source: untranslatable-debconf-templates cloud-init.templates: 6
  W: cloud-init source: missing-file-from-potfiles-in grub.templates
  W: cloud-init source: build-depends-on-obsolete-package build-depends: 
dh-systemd => use debhelper (>= 9.20160709)
  W: cloud-init source: timewarp-standards-version (2011-12-16 < 2014-09-17)
  W: cloud-init source: ancient-standards-version 3.9.6 (released 2014-09-17) 
(current is 4.4.1)
  W: cloud-init source: binary-nmu-debian-revision-in-source 
19.3-244-gbee7e918-1~bddeb~20.04.1
  W: cloud-init: binary-without-manpage usr/bin/cloud-id
  W: cloud-init: binary-without-manpage usr/bin/cloud-init
  W: cloud-init: binary-without-manpage usr/bin/cloud-init-per
  W: cloud-init: command-with-path-in-maintainer-script postinst:141 
/usr/sbin/grub-install
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-config.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-final.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init-local.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init.service cloud-init.target
  W: cloud-init: systemd-service-file-shutdown-problems 
lib/systemd/system/cloud-init.service
  N: 1 tag overridden (1 error)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1853376/+subscriptions


-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1831632] Re: "Error: retrieving gpg key timed out." during integration test setup

[Paride Legovini]

This was merged long ago.

** Changed in: cloud-init
 Assignee: Paride Legovini (paride) => (unassigned)

** Changed in: cloud-init
   Status: Triaged => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1831632

Title:
  "Error: retrieving gpg key timed out." during integration test setup

Status in cloud-init:
  Fix Released

Bug description:
  Looking at the code, I believe the error is produced by the call to
  `add-apt-repository` in download() in daily_deb.sh (from server-test-
  scripts).

  This could probably be improved by retrying the deb creation step a
  couple of times, as this may have been caused by a temporary network
  glitch.

  (Full console text attached, as jenkins.u.c isn't publicly-accessible
  ATM.)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1831632/+subscriptions


-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1916629] [NEW] pytests: tests using apt-key fail on Xenial

[Paride Legovini]

Public bug reported:

Sample error:



def test_keyserver(self, class_client: IntegrationInstance):
"""Test the apt keyserver functionality.

Ported from
tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.py
"""
test_keyserver_contents = class_client.read_from_file(
'/etc/apt/sources.list.d/test_keyserver.list'
)

assert (
'http://ppa.launchpad.net/cloud-init-raharper/curtin-dev/ubuntu'
) in test_keyserver_contents

keys = class_client.execute('apt-key finger')
>   assert TEST_KEYSERVER_KEY in keys
E   AssertionError: assert 'pub   rsa1024 2013-12-09 [SC]\n  7260 0DB1 
5B8E 4C8B 1964  B868 038A CC97 C660 A937\nuid   [ unknown] Launchpad 
PPA for Ryan Harper\n' in '/etc/apt/trusted.gpg\n\npub   
1024D/437D05B5 2004-09-12\n  Key fingerprint = 6302 39CC 130E 1...erprint = 
3552 C902 B4DD F7BD 3842  1821 015D 28D7 4416 14D8\nuid  
Launchpad PPA for simplestreams-dev'

class_client = 
keys   = '/etc/apt/trusted.gpg\n\npub   1024D/437D05B5 
2004-09-12\n  Key fingerprint = 6302 39CC 130E 1...erprint = 3552 C902 B4DD 
F7BD 3842  1821 015D 28D7 4416 14D8\nuid  Launchpad PPA for 
simplestreams-dev'
self   = 
test_keyserver_contents = 'deb 
http://ppa.launchpad.net/cloud-init-raharper/curtin-dev/ubuntu xenial main'



PR: https://github.com/canonical/cloud-init/pull/823

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1916629

Title:
  pytests: tests using apt-key fail on Xenial

Status in cloud-init:
  New

Bug description:
  Sample error:

  

  def test_keyserver(self, class_client: IntegrationInstance):
  """Test the apt keyserver functionality.
  
  Ported from
  tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.py
  """
  test_keyserver_contents = class_client.read_from_file(
  '/etc/apt/sources.list.d/test_keyserver.list'
  )
  
  assert (
  'http://ppa.launchpad.net/cloud-init-raharper/curtin-dev/ubuntu'
  ) in test_keyserver_contents
  
  keys = class_client.execute('apt-key finger')
  >   assert TEST_KEYSERVER_KEY in keys
  E   AssertionError: assert 'pub   rsa1024 2013-12-09 [SC]\n  7260 
0DB1 5B8E 4C8B 1964  B868 038A CC97 C660 A937\nuid   [ unknown] 
Launchpad PPA for Ryan Harper\n' in 
'/etc/apt/trusted.gpg\n\npub   1024D/437D05B5 2004-09-12\n  
Key fingerprint = 6302 39CC 130E 1...erprint = 3552 C902 B4DD F7BD 3842  
1821 015D 28D7 4416 14D8\nuid  Launchpad PPA for 
simplestreams-dev'

  class_client = 
  keys   = '/etc/apt/trusted.gpg\n\npub   
1024D/437D05B5 2004-09-12\n  Key fingerprint = 6302 39CC 130E 1...erprint = 
3552 C902 B4DD F7BD 3842  1821 015D 28D7 4416 14D8\nuid  
Launchpad PPA for simplestreams-dev'
  self   = 
  test_keyserver_contents = 'deb 
http://ppa.launchpad.net/cloud-init-raharper/curtin-dev/ubuntu xenial main'

  

  PR: https://github.com/canonical/cloud-init/pull/823

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1916629/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1896604] Re: Groovy kernel (5.8.0-1004-aws) creates broken /dev/console on i3.metal instances

[Paride Legovini]

** Changed in: cloud-init
   Status: Triaged => Fix Released

** Changed in: cloud-images
   Status: New => Fix Committed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1896604

Title:
  Groovy kernel (5.8.0-1004-aws) creates broken /dev/console on i3.metal
  instances

Status in cloud-images:
  Fix Committed
Status in cloud-init:
  Fix Released
Status in linux-aws package in Ubuntu:
  Fix Released

Bug description:
  [Impact]

  Starting with kernel 5.8 the default nr_uarts has been changed from 4
  to 2 for amd64, but this seems to affect i3.metal instances in AWS,
  because ttyS0 is now remapped to ttyS4 and this is breaking tools like
  cloud-init (and probably something else).

  [Test case]

  # echo > /dev/console
  bash: echo: write error: Input/output error

  [Fix]

  Setting nr_uarts=4 by default (via CONFIG_SERIAL_8250_RUNTIME_UARTS)
  restores the previous behavior and writing to /dev/console works
  without returning any error.

  [Regression potential]

  Minimal. Restores the old behavior used in 5.4 (that shouldn't have
  changed in the first place).

  [Original bug report]

  Hi,

  When running Groovy daily images on i3.metal instances a broken
  /dev/console is created. The char device appears to be writable but
  writing to it causes an Input/output error. This is breaking cloud-
  init, as it tries to log to /dev/console, and is likely to break other
  programs.

  On Focal:

  root@ip-172-31-24-163:~# ls -l /dev/console
  crw--- 1 root root 5, 1 Sep 21 16:07 /dev/console
  root@ip-172-31-24-163:~# echo x > /dev/console
  root@ip-172-31-24-163:~#

  On Groovy:

  root@ip-172-31-20-184:~# ls -l /dev/console
  crw--w 1 root tty 5, 1 Sep 21 16:03 /dev/console
  root@ip-172-31-20-184:~# echo x > /dev/console
  bash: echo: write error: Input/output error

  The Groovy kernel log has a

  [ 3.561696] fbcon: Taking over console

  line in it, which is not present in the Focal kernel log
  (5.4.0-1024-aws). Perhaps fbcon should be prevented from taking over
  console?

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/1896604/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1896604] Re: Groovy kernel (5.8.0-1004-aws) creates broken /dev/console on i3.metal instances

[Paride Legovini]

Thanks Andrea for looking into this.

Added a cloud-init task for tracking.

** Also affects: cloud-init (Ubuntu)
   Importance: Undecided
   Status: New

** No longer affects: cloud-init (Ubuntu)

** Also affects: cloud-init
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: New => Triaged

** Changed in: cloud-init
 Assignee: (unassigned) => Paride Legovini (paride)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1896604

Title:
  Groovy kernel (5.8.0-1004-aws) creates broken /dev/console on i3.metal
  instances

Status in cloud-images:
  New
Status in cloud-init:
  Triaged
Status in linux-aws package in Ubuntu:
  New

Bug description:
  Hi,

  When running Groovy daily images on i3.metal instances a broken
  /dev/console is created. The char device appears to be writable but
  writing to it causes an Input/output error. This is breaking cloud-
  init, as it tries to log to /dev/console, and is likely to break other
  programs.

  On Focal:

  root@ip-172-31-24-163:~# ls -l /dev/console
  crw--- 1 root root 5, 1 Sep 21 16:07 /dev/console
  root@ip-172-31-24-163:~# echo x > /dev/console
  root@ip-172-31-24-163:~#

  On Groovy:

  root@ip-172-31-20-184:~# ls -l /dev/console
  crw--w 1 root tty 5, 1 Sep 21 16:03 /dev/console
  root@ip-172-31-20-184:~# echo x > /dev/console
  bash: echo: write error: Input/output error

  The Groovy kernel log has a

  [ 3.561696] fbcon: Taking over console

  line in it, which is not present in the Focal kernel log
  (5.4.0-1024-aws). Perhaps fbcon should be prevented from taking over
  console?

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/1896604/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1776958] Re: error creating lxdbr0.

[Paride Legovini]

** Changed in: cloud-init (Ubuntu)
   Status: Fix Released => New

** Changed in: cloud-init
 Assignee: (unassigned) => Paride Legovini (paride)

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1776958

Title:
  error creating lxdbr0.

Status in cloud-init:
  Triaged
Status in cloud-init package in Ubuntu:
  New

Bug description:
  $ cat > my.yaml <) failed
   cloudinit.util.ProcessExecutionError: Unexpected error while running command.
   Stderr: Error: The network already exists

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1776958/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1776958] Re: error creating lxdbr0.

[Paride Legovini]

This is happening again with Focal and Groovy:

tox -e citest -- run --os-name=focal --platform=lxd --preserve-data
--data-dir=results --verbose --deb=cloud-
init_20.2-134-g747723a4-1\~bddeb_all.deb --test-
config=tests/cloud_tests/testcases/modules/lxd_bridge.yaml

=

2020-08-21 15:56:11,115 - subp.py[DEBUG]: Running command ['lxd', 'waitready', 
'--timeout=300'] with allowed return codes [0] (shell=False, capture=True)
2020-08-21 15:56:20,165 - subp.py[DEBUG]: Running command ['lxd', 'init', 
'--auto', '--storage-backend=dir'] with allowed return codes [0] (shell=False, 
capture=True)
2020-08-21 15:56:23,684 - subp.py[DEBUG]: Running command ['lxc', 'network', 
'delete', 'lxdbr0', '--force-local'] with allowed return codes [0] 
(shell=False, capture=True)
2020-08-21 15:56:24,413 - cc_lxd.py[DEBUG]: Deletion of lxd network 'lxdbr0' 
failed. Assuming it did not exist.
2020-08-21 15:56:24,413 - subp.py[DEBUG]: Running command ['lxc', 'profile', 
'device', 'remove', 'default', 'eth0', '--force-local'] with allowed return 
codes [0] (shell=False, capture=True)
2020-08-21 15:56:24,491 - cc_lxd.py[DEBUG]: Removal of device 'eth0' from 
profile 'default' succeeded.
2020-08-21 15:56:24,491 - cc_lxd.py[DEBUG]: Creating lxd bridge: network create 
lxdbr0 ipv4.address=10.100.100.1/24 
ipv4.dhcp.ranges=10.100.100.100-10.100.100.200 ipv6.address=none dns.domain=lxd 
   
2020-08-21 15:56:24,492 - subp.py[DEBUG]: Running command ['lxc', 'network', 
'create', 'lxdbr0', 'ipv4.address=10.100.100.1/24', 
'ipv4.dhcp.ranges=10.100.100.100-10.100.100.200', 'ipv6.address=none', 
'dns.domain=lxd', '--force-local'] with allowed return codes [0] (shell=False, 
capture=True)
2020-08-21 15:56:24,569 - handlers.py[DEBUG]: finish: modules-final/config-lxd: 
FAIL: running config-lxd with frequency once-per-instance
2020-08-21 15:56:24,569 - util.py[WARNING]: Running module lxd () failed
2020-08-21 15:56:24,569 - util.py[DEBUG]: Running module lxd () failed
Traceback (most recent call last):  
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 848, in 
_run_modules
ran, _r = cc.run(run_name, mod.handle, func_args,   
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 54, in run 
return self._runners.run(name, functor, args, freq, clear_on_fail)  
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 185, in run  
results = functor(*args)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_lxd.py", line 152, 
in handle
_lxc(cmd_create)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_lxd.py", line 268, 
in _lxc
subp.subp(['lxc'] + list(cmd) + ["--force-local"], update_env=env)  
  File "/usr/lib/python3/dist-packages/cloudinit/subp.py", line 290, in subp
raise ProcessExecutionError(stdout=out, stderr=err, 
cloudinit.subp.ProcessExecutionError: Unexpected error while running command.   
Command: ['lxc', 'network', 'create', 'lxdbr0', 'ipv4.address=10.100.100.1/24', 
'ipv4.dhcp.ranges=10.100.100.100-10.100.100.200', 'ipv6.address=none', 
'dns.domain=lxd', '--force-local']
Exit code: 1
Reason: -   
Stdout:-
Stderr: Error: The network already exists

=

So the cleanup code runs, but for some reason fails to delete the
bridge. I tried running:

lxd init --auto --storage-backend=dir ; lxc network delete lxdbr0
--force-local

in a fresh Focal LXD container and it works (deletes lxdbr0), so I can't
tell where the problem is yet.

** Changed in: cloud-init
   Status: Fix Released => New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1776958

Title:
  error creating lxdbr0.

Status in cloud-init:
  New
Status in cloud-init package in Ubuntu:
  Fix Released

Bug description:
  $ cat > my.yaml <) failed
   cloudinit.util.ProcessExecutionError: Unexpected error while running command.
   Stderr: Error: The network already exists

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1776958/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1892493] [NEW] apt-key is deprecated and will be removed from Debian and Ubuntu

[Paride Legovini]

Public bug reported:

Use of apt-key is deprecated and it will last be available in Debian 11
and Ubuntu 22.04, see apt-key(8).

Cloud-init uses `apt-key add`, which can be replaced by dropping the
pubkeys to be trusted in the /etc/apt/trusted.gpg.d/ directory. The
`apt-key list` and `apt-key finger` commands can be replaced by
something like:

gpg --with-fingerprint --no-default-keyring --list-keys --keyring
/etc/apt/trusted.gpg.d/key1 --keyring /etc/apt/trusted.gpg.d/key2
[--keyring ...]

which should produce the same result.

** Affects: cloud-init
 Importance: Low
 Status: Triaged

** Changed in: cloud-init
   Status: New => Triaged

** Changed in: cloud-init
   Importance: Undecided => Low

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1892493

Title:
  apt-key is deprecated and will be removed from Debian and Ubuntu

Status in cloud-init:
  Triaged

Bug description:
  Use of apt-key is deprecated and it will last be available in Debian
  11 and Ubuntu 22.04, see apt-key(8).

  Cloud-init uses `apt-key add`, which can be replaced by dropping the
  pubkeys to be trusted in the /etc/apt/trusted.gpg.d/ directory. The
  `apt-key list` and `apt-key finger` commands can be replaced by
  something like:

  gpg --with-fingerprint --no-default-keyring --list-keys --keyring
  /etc/apt/trusted.gpg.d/key1 --keyring /etc/apt/trusted.gpg.d/key2
  [--keyring ...]

  which should produce the same result.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1892493/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1876139] Re: Groovy cloud-images failing during growpart

[Paride Legovini]

** Changed in: cloud-utils
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1876139

Title:
  Groovy cloud-images failing during growpart

Status in cloud-images:
  Invalid
Status in cloud-init:
  Invalid
Status in cloud-utils:
  Fix Released
Status in cloud-utils package in Ubuntu:
  Fix Released

Bug description:
  Was running on Azure, but I expect this happens on all cloud images.
  We did not see our disk grow as expected on first boot.

  Took a look at /var/log/cloud-init and saw the following:

  2020-04-30 16:04:46,837 - util.py[WARNING]: Failed growpart --dry-run for 
(/dev/sda, 1)
  2020-04-30 16:04:46,837 - util.py[DEBUG]: Failed growpart --dry-run for 
(/dev/sda, 1)
  Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/config/cc_growpart.py", line 
145, in resize
  util.subp(["growpart", '--dry-run', diskdev, partnum])
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2084, in subp
  raise ProcessExecutionError(stdout=out, stderr=err,
  cloudinit.util.ProcessExecutionError: Unexpected error while running command.
  Command: ['growpart', '--dry-run', '/dev/sda', '1']
  Exit code: 2
  Reason: -
  Stdout: FAILED: sfdisk not found

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/1876139/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1886428] Re: Cloud-Init on RHEL8 cannot parse #cloud-config user-data

[Paride Legovini]

Hi Jason,

Thanks for taking the time to report this issue and to follow-up with
your findings.

As apparently this isn't a bug in cloud-init I'm setting the status of
this report to Invalid. Should you not agree please don't hesitate to
change its status back to New, explain why you still believe it's a bug
in cloud-init and we'll look at it again.


** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1886428

Title:
  Cloud-Init on RHEL8 cannot parse #cloud-config user-data

Status in cloud-init:
  Invalid

Bug description:
  When using a "valid" #cloud-config yaml file passed in via the user-
  data service. Even though the file is valid, correct, and accessible
  via the meta-data URL - cloud-config on RHEL8 will not read it/read it
  incorrectly and therefore not process that file.

  Passing the exact same #cloud-config file into a RHEL7 image causes it
  to work just fine.

  Technical Specs:
  Working Server:
  RHEL 7.8
  [root@rhel7-test-hen ~]# uname -r
  3.10.0-1127.13.1.el7.x86_64
  [root@rhel7-test-hen ~]# rpm -qa | grep cloud-init
  cloud-init-18.5-3.el7.x86_64

  Non-Working Server:
  RHEL 8.2
  [root@rhel8-base ~]# uname -r
  4.18.0-193.6.3.el8_2.x86_64
  [root@rhel8-base ~]# rpm -qa | grep cloud-init
  cloud-init-18.5-12.el8_2.2.noarch

  Pastebin with specifics/commands/output: https://pastebin.com/1igasWkr

  The pastebin has details/info to reproduce this, but i can reproduce
  it 100% consistently by attaching the same user-data (see pastebin)
  via user-data. I can see it, pull it, hash it, its the same, its
  accessible, and its usable. RHEL8 doesn't parse/proces it, RHEL7 does.

  As a last resort i just did the following in the RHEL8 server:
  - Confirmed it was up
  - Confirmed networking was fine/working
  - Confirmed i could curl the user-data file from metadata service correctly
  - rm -rf /var/lib/cloud/instance
  - cloud-init init
  - cloud-init start

  After doing this:
  - /var/lib/cloud/instance/user-data.txt file was recreated (new date stamp) - 
but still incorrect (matches pastebin data)
  - /var/lib/cloud/instance/user-data.txt.I file was recreated (new date stamp) 
- but still incorrect (matches pastebin data)

  [root@rhel8-base log]# ls -al /var/lib/cloud/instance/ | grep user-data
  -rw---. 1 root root0 Jul  6 09:17 user-data.txt
  -rw---. 1 root root  308 Jul  6 09:17 user-data.txt.i

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1886428/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1118815] Re: Remove python-oauth from the archive

[Paride Legovini]

I updated some tasks:

- identicurse: removed from Groovy => Fix Released
- jsonbot: removed from Bionic => Fix Released
- python-django-piston: removed from Eoan => Fix Released
- turpial: removed from Xenial => Fix Releaed
- tweepy: dependency dropped in Xenial => Fix Released
- u1db: pkg dropped in Focal => Fix Released
- gwibber: removed from Xenial => Fix Released
- pyjuju/juju: removed from Bionic => Fix Released

According to

https://bugs.launchpad.net/ubuntu/+source/python-oauth/+bug/1883875

MAAS is not a blocker for the removal of python-oauth from Groovy. I
think it should be Fix Released too.

I am not familiar with the OpenStack hacluster charm.

** Changed in: identicurse (Ubuntu)
   Status: Confirmed => Fix Released

** Changed in: jsonbot (Ubuntu)
   Status: Confirmed => Fix Released

** Changed in: python-django-piston (Ubuntu)
   Status: Triaged => Fix Released

** Changed in: turpial (Ubuntu)
   Status: Confirmed => Fix Released

** Changed in: tweepy (Ubuntu)
   Status: Confirmed => Fix Released

** Changed in: u1db
   Status: New => Fix Released

** Changed in: gwibber
   Status: New => Fix Released

** Changed in: pyjuju
   Status: Triaged => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1118815

Title:
  Remove python-oauth from the archive

Status in OpenStack hacluster charm:
  New
Status in cloud-init:
  Fix Released
Status in Gwibber:
  Fix Released
Status in MAAS:
  Invalid
Status in pyjuju:
  Fix Released
Status in U1DB:
  Fix Released
Status in identicurse package in Ubuntu:
  Fix Released
Status in jsonbot package in Ubuntu:
  Fix Released
Status in python-django-piston package in Ubuntu:
  Fix Released
Status in python-oauth package in Ubuntu:
  Confirmed
Status in turpial package in Ubuntu:
  Fix Released
Status in tweepy package in Ubuntu:
  Fix Released

Bug description:
  This bug tracks the removal of python-oauth from the archive.  (See
  also
  https://blueprints.launchpad.net/ubuntu/+spec/foundations-r-python3-oauth
  for additional details).

  There are several very good reasons to remove python-oauth and port
  all reverse depends to python-oauthlib.

   * upstream oauth has been abandoned since 2009
   * upstream oauth only supports OAuth 1 (and probably not even the RFC 5849 
standard)
   * upstream oauth only supports Python 2
   * upstream oauthlib is actively maintained
   * upstream oauthlib supports Python 2 and Python 3
   * upstream oauthlib supports RFC 5849 and the OAuth2 spec draft

  As of yet, we cannot remove python-oauth because of existing reverse
  dependencies.  I'll add each of those as bug tasks to this one for
  tracking purposes.  When the time comes, I'll subscribe ~ubuntu-
  archive to the bug to do the final deed.

  It will need to be blacklisted from Debian sync too.

  In the meantime, *please* don't write any new code using python-oauth!
  Use python-oauthlib.

  http://pypi.python.org/pypi/oauthlib

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-hacluster/+bug/1118815/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1883180] Re: autoinstall with nocloud does not seed password when identity is interactive

[Paride Legovini]

Hi and thanks for filing this bug report.

I believe this is a bug in subiquity (the Ubuntu Server installer)
rather than a bug in cloud-init. I am reassigning it.

** Also affects: subiquity
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: New => Incomplete

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1883180

Title:
  autoinstall with nocloud does not seed password when identity is
  interactive

Status in cloud-init:
  Invalid
Status in subiquity:
  New

Bug description:
  
  I am using the [autoinstall 
process](https://ubuntu.com/server/docs/install/autoinstall) with nocloud 
provider.  I want to manually enter the hostname, so have set the identity 
section to interactive.  However, when I do this, the password section is blank 
instead of pre-seeded with the value provided.  The other identity sections 
(hostname, username, realname) are preseeded as expected, when the interactive 
section comes up.  Only the pw is missing.

  1. Tell us your cloud provider: `ds=nocloud-net`
  2. Any appropriate cloud-init configuration you can provide us:

  *user-data*
  ```
  #cloud-config
  autoinstall:
version: 1
interactive-sections:
  - identity
identity:
  hostname: change-me
  password: $6$xnH...
  username: admin
  realname: Admin
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1883180/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1880569] Re: bddep misses dependencies introduced by debian/patches

[Paride Legovini]

As per IRC discussion with @OddBloke, this whole problem will go away
with the next Focal SRU. We'll stand the bddeb failures up to that
point, they're not worth a non-trivial fix.

** Changed in: cloud-init
   Status: New => Won't Fix

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1880569

Title:
  bddep misses dependencies introduced by debian/patches

Status in cloud-init:
  Won't Fix

Bug description:
  When the packages/bddeb script is called from the ubuntu/focal branch
  it constructs a source package with missing build dependencies (pytest
  and pytest-cov). The package is therefore unsuitable for testing.

  The problem is due to the fact that the missing dependencies are
  introduced in test-requirements.txt by a patch in debian/patches.
  bddeb runs tools/read-dependencies to construct the list of
  dependencies, however when read-dependencies is run the patch is not
  applied, so the additional dependencies are missed.

  One solution is: call

dpkg-source --before-build 

  right before calling read-dependencies. The command applies all the
  patches, so the dep can be found. The same command with --after-build
  can be used to unapply the patches.

  Problem: read-dependencies is not run from the *target* directory, but
  from the directory where the git repo has been cloned. Easiest
  solution: always pass the full path of the requirements file to read-
  dependencies via --requirements-file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1880569/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1880569] [NEW] bddep misses dependencies introduced by debian/patches

[Paride Legovini]

Public bug reported:

When the packages/bddeb script is called from the ubuntu/focal branch it
constructs a source package with missing build dependencies (pytest and
pytest-cov). The package is therefore unsuitable for testing.

The problem is due to the fact that the missing dependencies are
introduced in test-requirements.txt by a patch in debian/patches. bddeb
runs tools/read-dependencies to construct the list of dependencies,
however when read-dependencies is run the patch is not applied, so the
additional dependencies are missed.

One solution is: call

  dpkg-source --before-build 

right before calling read-dependencies. The command applies all the
patches, so the dep can be found. The same command with --after-build
can be used to unapply the patches.

Problem: read-dependencies is not run from the *target* directory, but
from the directory where the git repo has been cloned. Easiest solution:
always pass the full path of the requirements file to read-dependencies
via --requirements-file.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1880569

Title:
  bddep misses dependencies introduced by debian/patches

Status in cloud-init:
  New

Bug description:
  When the packages/bddeb script is called from the ubuntu/focal branch
  it constructs a source package with missing build dependencies (pytest
  and pytest-cov). The package is therefore unsuitable for testing.

  The problem is due to the fact that the missing dependencies are
  introduced in test-requirements.txt by a patch in debian/patches.
  bddeb runs tools/read-dependencies to construct the list of
  dependencies, however when read-dependencies is run the patch is not
  applied, so the additional dependencies are missed.

  One solution is: call

dpkg-source --before-build 

  right before calling read-dependencies. The command applies all the
  patches, so the dep can be found. The same command with --after-build
  can be used to unapply the patches.

  Problem: read-dependencies is not run from the *target* directory, but
  from the directory where the git repo has been cloned. Easiest
  solution: always pass the full path of the requirements file to read-
  dependencies via --requirements-file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1880569/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1879737] Re: azure datasource fails if dhcp is done on a bridge instead of on the physical interface

[Paride Legovini]

Reported as working on askubuntu by the person who originally posted the
question; I think everything is working as intended. I'm setting this to
Invalid, no need to tell you to swiftly set it back to New if anything
remains unclear.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1879737

Title:
  azure datasource fails if dhcp is done on a bridge instead of on the
  physical interface

Status in cloud-init:
  Invalid

Bug description:
  As reported in https://askubuntu.com/questions/1241624/create-netplan-
  bridge-at-azure-ubuntu it appears that configuring an Azure instance
  so that the primary interface is on a bridge instead of networking
  being configured directly on the primary interface, prevents the azure
  datasource from being used correctly.  The net result is that the
  instance's network doesn't work at all.

  It looks like cloud-init needs to look up the correct interface name
  based on mac rather than using eth0 unconditionally.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1879737/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1869967] Re: subiquity->cloud-init generates netplan yaml telling user not to edit it

[Paride Legovini]

I'm not marking this Fix Released for subiquity as the change has not
been released in all the subiquity channels yet.

** Changed in: cloud-init
   Status: New => Invalid

** Changed in: subiquity
   Status: New => Fix Committed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1869967

Title:
  subiquity->cloud-init generates netplan yaml telling user not to edit
  it

Status in cloud-init:
  Invalid
Status in subiquity:
  Fix Committed

Bug description:
  As seen in , users who install with subiquity end up
  with a /etc/cloud/cloud.cfg.d/50-curtin-networking.cfg that persists
  on the target system, plus an /etc/netplan/50-cloud-init.yaml that
  tells users not to edit it without taking steps to disable cloud-init.

  I don't think this is what we want.  I think a subiquity install
  should unambiguously treat cloud-init as a one-shot at installation,
  and leave the user afterwards with config files that can be directly
  edited without fear of cloud-init interfering; and the yaml files
  generated by cloud-init on subiquity installs should therefore also
  not include this scary language:

  # This file is generated from information provided by the datasource.  Changes
  # to it will not persist across an instance reboot.  To disable cloud-init's
  # network configuration capabilities, write a file
  # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
  # network: {config: disabled}

  But we need to figure out how to fix this between subiquity and cloud-
  init.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1869967/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1867151] [NEW] Can't run tests on CentOS 8 (missing py3 dependencies)

[Paride Legovini]

Public bug reported:

The cloud-init test environment can't be set up on CentOS 8 due to
missing Python 3 dependencies. The missing libraries are at least the
following:

 - contextlib2
 - httpretty
 - unittest2

Of these unittest2 is available in the PowerTools repository as
python3-unittest2, the other two are not. The PowerTools repository is
currently added to the test system by the test helper scripts.

For this reason the unit testing and CI testing is currently disabled on
CentOS 8.

** Affects: cloud-init
 Importance: Undecided
 Status: Triaged

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1867151

Title:
  Can't run tests on CentOS 8 (missing py3 dependencies)

Status in cloud-init:
  Triaged

Bug description:
  The cloud-init test environment can't be set up on CentOS 8 due to
  missing Python 3 dependencies. The missing libraries are at least the
  following:

   - contextlib2
   - httpretty
   - unittest2

  Of these unittest2 is available in the PowerTools repository as
  python3-unittest2, the other two are not. The PowerTools repository is
  currently added to the test system by the test helper scripts.

  For this reason the unit testing and CI testing is currently disabled
  on CentOS 8.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1867151/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1859891] Re: fail to start cloud init

[Paride Legovini]

Hello magnate, thanks for your report.

This looks like a local configuration problem, rather than a bug in
cloud-init. The cloud-init package (like all the packages in Ubuntu) do
not install things in /usr/local/, so the cloud-init file you are trying
to run does not come from the Ubuntu package. You probably tried to
install cloud-init manually, without using apt. With the installation in
a dirty state it's difficult to tell anything on what could be wrong.

Since we use this bug tracker to track bugs in cloud-init, rather than
configuration or setup problems, I'm marking this bug as Invalid.

If you believe that this is really a bug, then you may find it helpful
to read "How to report bugs effectively"
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html. We'd be grateful
if you would then provide a more complete description of the problem,
explain why you believe this is a bug in cloud-init rather than a
problem specific to your system, and then change the bug status back to
New.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1859891

Title:
  fail to start cloud init

Status in cloud-init:
  Invalid

Bug description:
  My ironic server with ubuntu  fails to start cloud init which is be installed 
by apt.
  I analyze the cloud-init.service and so on ,and find that  
ExecStart=/usr/bin/cloud-init init
  [Service]
  Type=oneshot
  ExecStart=/usr/bin/cloud-init init
  RemainAfterExit=yes
  TimeoutSec=0

  but,the apt installs  cloud init in dirrectory /usr/local/bin/cloud-
  init。

  hence,I take the following steps to slove the problem:

  ln -s /usr/local/bin/cloud-init /usr/bin/cloud-init

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1859891/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1841182] Re: cloud-init fails when rebooting EC2 i3.metal instances

[Paride Legovini]

** Changed in: cloud-init
   Status: Expired => Incomplete

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1841182

Title:
  cloud-init fails when rebooting EC2 i3.metal instances

Status in cloud-init:
  Incomplete

Bug description:
  In order to collect boot-speed metrics I deploy/reboot/terminate
  several EC2 instances per day. At a high level this is that the jobs
  do:

  1. Deploy an instance and wait for cloud-init
 to finish using `cloud-init status --wait`
  2. Collect and retrieve some logs via SSH/SFTP
  3. Reboot the instance using boto3's reboot()
  4. Collect some more logs
  5. Terminate the instance

  This works in a fairly reliable way, but on i3.metal instances the
  instance often fails to survive the reboot step. After a failed reboot
  the instance state appears as "running", but it's unreachable via SSH.

  By detaching the root volume and attaching it to another instance in
  the same availability zone I've been able to inspect the logs, and
  problem is a cloud-init failure. At a first glance of the logs it
  looks like cloud-init doesn't like /var/lib/cloud/data/set-hostname
  being empty:


  2019-08-23 11:31:27,585 - util.py[DEBUG]: Reading from 
/var/lib/cloud/data/set-hostname (quiet=False)
  2019-08-23 11:31:27,585 - util.py[DEBUG]: Read 0 bytes from 
/var/lib/cloud/data/set-hostname
  2019-08-23 11:31:27,585 - util.py[WARNING]: failed stage init-local   
  
  2019-08-23 11:31:27,586 - util.py[DEBUG]: failed stage init-local 
  
  Traceback (most recent call last):
  
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 653, in 
status_wrapper
  ret = functor(name, args) 
  
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 361, in 
main_init 
  _maybe_set_hostname(init, stage='local', retry_stage='network')   
  
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 709, in 
_maybe_set_hostname
  cc_set_hostname.handle('set-hostname', init.cfg, cloud, LOG, None)
  
File "/usr/lib/python3/dist-packages/cloudinit/config/cc_set_hostname.py", 
line 67, in handle
  prev_hostname = util.load_json(util.load_file(prev_fn))   
  
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1586, in 
load_json
  decoded = json.loads(decode_binary(text)) 
  
File "/usr/lib/python3.6/json/__init__.py", line 354, in loads  
  
  return _default_decoder.decode(s) 
  
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode  
  
  obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 
  
File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode  
  
  raise JSONDecodeError("Expecting value", s, err.value) from None  
  
  json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)   
  


  I'm not sure on where the actual problem is here. Is set-hostname
  supposed to always contain something? Should cloud-init be able to
  handle an empty set-hostname? Could the fact that the instance is
  rebooted shortly after being deployed affect this?

  The full logs are attached.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1841182/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1846511] [NEW] KVM and LXD integration testing failing on Eoan

[Paride Legovini]

Public bug reported:

Integration tests on Eoan are failing on the lxd and nocloud kvm
platforms. In both cases the test system is not reachable via SSH.

LXD log: https://paste.ubuntu.com/p/r2hwqZQfm2/

KVM log: https://paste.ubuntu.com/p/W3N22yBxKN/

This failure mode began with the test run of September 26. On the same
system the Disco test run succeeds.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1846511

Title:
  KVM and LXD integration testing failing on Eoan

Status in cloud-init:
  New

Bug description:
  Integration tests on Eoan are failing on the lxd and nocloud kvm
  platforms. In both cases the test system is not reachable via SSH.

  LXD log: https://paste.ubuntu.com/p/r2hwqZQfm2/

  KVM log: https://paste.ubuntu.com/p/W3N22yBxKN/

  This failure mode began with the test run of September 26. On the same
  system the Disco test run succeeds.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1846511/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1845489] [NEW] Variable self-assignments causing pylint W0127 (self-assigning-variable)

[Paride Legovini]

Public bug reported:

The newly released pylint 2.4.0 detects variable self-assignments and
throws a W0127 when one is found. There are four of these self-
assignments in cloud-init, all in this form:

@skipUnlessJsonSchema()
class TestSchema(CiTestCase, SchemaTestCaseMixin):

with_logs = True
schema = schema
^^^

and all introduced by this commit:

https://git.launchpad.net/cloud-
init/commit/cloudinit/config/tests/test_snap.py?id=5037252ca5

I'd drop them, but it seems they have been introduced on purpose. Is
there a good reason to keep them?

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1845489

Title:
  Variable self-assignments causing pylint W0127 (self-assigning-
  variable)

Status in cloud-init:
  New

Bug description:
  The newly released pylint 2.4.0 detects variable self-assignments and
  throws a W0127 when one is found. There are four of these self-
  assignments in cloud-init, all in this form:

  @skipUnlessJsonSchema()
  class TestSchema(CiTestCase, SchemaTestCaseMixin):

  with_logs = True
  schema = schema
  ^^^

  and all introduced by this commit:

  https://git.launchpad.net/cloud-
  init/commit/cloudinit/config/tests/test_snap.py?id=5037252ca5

  I'd drop them, but it seems they have been introduced on purpose. Is
  there a good reason to keep them?

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1845489/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1841090] Re: dhclient exit hook could be hiding non-zero return code

[Paride Legovini]

Thanks for reaching back. Bugs are closed by setting them to a
"terminal" state, in this case "Invalid" is the most appropriate one.

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1841090

Title:
  dhclient exit hook could be hiding non-zero return code

Status in cloud-init:
  Invalid

Bug description:
  I'm seeing an empty dhcp lease file. Digging through the code we are
  passing the -1 opt to dhclient in cloudinit.net.dhcp.dhcp_discovery.
  This would allow a non-zero return code if dhclient has an issue.

  I see though that there is an exit hook for dhclient:
  https://git.launchpad.net/cloud-init/tree/tools/hook-dhclient. It
  seems as though this hook could be hiding a non-zero return code from
  dhclient main lease implementation (unless $reason is in one of those
  options _and_ `cloud-init dhclient-hook` fails).

  This does not seem like the desired behavior, as we get beyond the
  dhclient call then only to fail on a zero-byte dhcp lease file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1841090/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1841631] [NEW] cloud-init.service has no timeout

[Paride Legovini]

Public bug reported:

cloud-init's systemd service file (cloud-init.service) currently has not
timeout; it explicitly sets:

  TimeoutSec=0

which, while not explicitly documented in systemd-service(5), appears to
disable the timeout logic. This may cause the boot process to get stuck,
while the system could possibly reach a useful state even if cloud-init
failed.

We hit this on a headless server: cloud-init got stuck and this
prevented systemd to reach the point where systemd-getty-generator would
have spawned a getty on the serial line, allowing the server to be
accessed remotely. As a result we've got locked out of the server, which
had to re rebooted in single-user mode. A cloud-init timeout would have
prevented this.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1841631

Title:
  cloud-init.service has no timeout

Status in cloud-init:
  New

Bug description:
  cloud-init's systemd service file (cloud-init.service) currently has
  not timeout; it explicitly sets:

TimeoutSec=0

  which, while not explicitly documented in systemd-service(5), appears
  to disable the timeout logic. This may cause the boot process to get
  stuck, while the system could possibly reach a useful state even if
  cloud-init failed.

  We hit this on a headless server: cloud-init got stuck and this
  prevented systemd to reach the point where systemd-getty-generator
  would have spawned a getty on the serial line, allowing the server to
  be accessed remotely. As a result we've got locked out of the server,
  which had to re rebooted in single-user mode. A cloud-init timeout
  would have prevented this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1841631/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1841182] [NEW] cloud-init fails when rebooting EC2 i3.metal instances

[Paride Legovini]

Public bug reported:

In order to collect boot-speed metrics I deploy/reboot/terminate several
EC2 instances per day. At a high level this is that the jobs do:

1. Deploy an instance and wait for cloud-init
   to finish using `cloud-init status --wait`
2. Collect and retrieve some logs via SSH/SFTP
3. Reboot the instance using boto3's reboot()
4. Collect some more logs
5. Terminate the instance

This works in a fairly reliable way, but on i3.metal instances the
instance often fails to survive the reboot step. After a failed reboot
the instance state appears as "running", but it's unreachable via SSH.

By detaching the root volume and attaching it to another instance in the
same availability zone I've been able to inspect the logs, and problem
is a cloud-init failure. At a first glance of the logs it looks like
cloud-init doesn't like /var/lib/cloud/data/set-hostname being empty:


2019-08-23 11:31:27,585 - util.py[DEBUG]: Reading from 
/var/lib/cloud/data/set-hostname (quiet=False)
2019-08-23 11:31:27,585 - util.py[DEBUG]: Read 0 bytes from 
/var/lib/cloud/data/set-hostname
2019-08-23 11:31:27,585 - util.py[WARNING]: failed stage init-local 

2019-08-23 11:31:27,586 - util.py[DEBUG]: failed stage init-local   

Traceback (most recent call last):  

  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 653, in 
status_wrapper
ret = functor(name, args)   

  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 361, in 
main_init 
_maybe_set_hostname(init, stage='local', retry_stage='network') 

  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 709, in 
_maybe_set_hostname
cc_set_hostname.handle('set-hostname', init.cfg, cloud, LOG, None)  

  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_set_hostname.py", 
line 67, in handle
prev_hostname = util.load_json(util.load_file(prev_fn)) 

  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1586, in 
load_json
decoded = json.loads(decode_binary(text))   

  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads

return _default_decoder.decode(s)   

  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode

obj, end = self.raw_decode(s, idx=_w(s, 0).end())   

  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode

raise JSONDecodeError("Expecting value", s, err.value) from None

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 



I'm not sure on where the actual problem is here. Is set-hostname
supposed to always contain something? Should cloud-init be able to
handle an empty set-hostname? Could the fact that the instance is
rebooted shortly after being deployed affect this?

The full logs are attached.

** Affects: cloud-init
 Importance: Undecided
 Status: New

** Attachment added: "cloud-init.tar.gz"
   
https://bugs.launchpad.net/bugs/1841182/+attachment/5284197/+files/cloud-init.tar.gz

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1841182

Title:
  cloud-init fails when rebooting EC2 i3.metal instances

Status in cloud-init:
  New

Bug description:
  In order to collect boot-speed metrics I deploy/reboot/terminate
  several EC2 instances per day. At a high level this is that the jobs
  do:

  1. Deploy an instance and wait for cloud-init
 to finish using `cloud-init status --wait`
  2. Collect and retrieve some logs via SSH/SFTP
  3. Reboot the instance using boto3's reboot()
  4. Collect some more logs
  5. Terminate the instance

  This works in a fairly reliable way, but on i3.metal instances the
  instance often fails to survive the reboot step. After a failed reboot
  the instance state appears as "running", but it's unreachable via SSH.

  By detaching the root volume and attaching it to another instance in
  the same availability zone I've been able to inspect the logs, and
  problem is a cloud-init failure. At a first glance of the logs it
  looks like cloud-init doesn't like /var/lib/cloud/data/set-hostname
  being empty:


  2019-08-23 11:31:27,585 - util.py[DEBUG]: Reading from 
/var/lib/cloud/data/set-hostname (quiet=False)
  2019-08-23 11:31:27,585 - util.py[DEBUG]: Read 0 bytes from 
/var/lib/cloud/data/set-hostname
  2019-08-23 11:31:27,585 - util.py[WARNING]: failed stage init-local   
  
  2019-08-23 11:31:27,586 - util.py[DEBUG]: failed stage init-local 
  
  Traceback (most recent call last):