[Yahoo-eng-team] [Bug 1927020] Re: cloudconfig not writing maas data source

[Ryan Harper]

I'm marking the cloud-init task invalid.  After getting config and logs
we confirmed that the cloud-init package in the Debian image does not
contain a recent handle_maas_preseed() that Ubuntu uses and curtin/MAAS
rely upon to install the MAAS datasource correctly.

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1927020

Title:
  cloudconfig not writing maas data source

Status in cloud-init:
  Invalid
Status in curtin:
  Invalid

Bug description:
  further background https://discourse.maas.io/t/debian10-fails-on-
  final-reboot/4486

  I'm deploying a debian 10 buster image using MAAS. No errors are
  reported in MAAS or curtin install but on the final boot cloud-init
  reports `Failed to load metadata and userdata`. checking the config
  for the machine in maas shows all the cloudconfig: to connect to maas
  but the files with oauth credentials etc dont seem to have been copied
  to the target.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1927020/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1927020] Re: cloudconfig not writing maas data source

[Ryan Harper]

I'm marking the curtin task invalid; after getting config and logs we
confirmed that the cloud-init package in the Debian image does not
contain a recent handle_maas_preseed() that Ubuntu uses and curtin/MAAS
rely upon to install the MAAS datasource correctly.

** Changed in: curtin
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1927020

Title:
  cloudconfig not writing maas data source

Status in cloud-init:
  Invalid
Status in curtin:
  Invalid

Bug description:
  further background https://discourse.maas.io/t/debian10-fails-on-
  final-reboot/4486

  I'm deploying a debian 10 buster image using MAAS. No errors are
  reported in MAAS or curtin install but on the final boot cloud-init
  reports `Failed to load metadata and userdata`. checking the config
  for the machine in maas shows all the cloudconfig: to connect to maas
  but the files with oauth credentials etc dont seem to have been copied
  to the target.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1927020/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1925454] Re: Update Ubuntu 20.04 default network-config by cloud-config

[Ryan Harper]

I'm marking this bug as Invalid.  cloud-init is working as expected for
NoCloud datasource.  Changes to the network-config during execution
can't affect cloud-inits behavior during boot with the NoCloud
datasource.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1925454

Title:
  Update Ubuntu 20.04 default network-config  by cloud-config

Status in cloud-init:
  Invalid

Bug description:
  This is a follow-up on the case https://bugs.launchpad.net/cloud-
  init/+bug/1924922

  I cannot update default Ubuntu network-config by writing user-data
  file in cloud-config.

  Attached are logs. Below is a snippet from the cloud-config

  #cloud-config
  write_files:
- path: /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
  permissions: '0644'
  content: |
network: 
  config: disabled
- path: /etc/netplan/50-cloud-init.yaml
  permissions: '0644'
  content: |
network:
  version: 2
  ethernets:
eth0:
  dhcp4: true
  addresses:
- 192.168.53.3/26
  gateway4: 192.168.53.62
  nameservers:
addresses:
  - 1.1.1.1
  - 8.8.8.8
  runcmd:
- [sudo, netplan, try]
- [sudo, netplan, generate]
- [sudo, netplan, apply]
  power_state:
mode: 'reboot'
message: 'reboot triggered by cloud-init'

  -

  Result is such that the hardcoded network-config ('optional': True, not 
waiting for DHCP server) is attached/generated with IP as, for an example - 
192.168.53.4, not the IP I really want 192.168.53.3. This leads to couple of 
problems, one as specified in bug/1924922, that ssh_import_id is not working 
(for the time is not sync up)
  Is there a way to overwrite this default with user-data, and not to go to the 
config partition and manually change it?

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1925454/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1924922] Re: ssh-import-id gives bad handshake

[Ryan Harper]

I'm marking this bug invalid, cloud-init is doing what it was told to do
and it appears looks like the user-data config relies on networking but
the network-config was marked optional.

Please move this bug state back to New if you have further questions or
issues.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1924922

Title:
  ssh-import-id gives bad handshake

Status in cloud-init:
  Invalid

Bug description:
  Ubuntu 20.04 server runs with user data as below,  yet it raises the
  error from launchpad.net

  2020-04-01 17:24:18,815 ERROR HTTPSConnectionPool(host='launchpad.net', 
port=443): Max retries exceeded with url: /~aniecki/+sshkeys (C
  aused by SSLError(SSLError("bad handshake: Error([('SSL routines', 
'tls_process_server_certificate', 'certificate verify failed')])")))
  Cloud-init v. 20.4.1-0ubuntu1~20.04.1 running 'modules:config' at Wed, 01 Apr 
2020 17:24:17 +. Up 42.20 seconds.

  Apparently this is the error because the host time is not yet synch
  up, but the ntp is also set there,

  
  #cloud-config

  ntp:
enabled: true
ntp_client: chrony

  users:
- name: kevo
  primary-group: users
  shell: /bin/bash
  sudo: ALL=(ALL) NOPASSWD:ALL
  groups: users
  ssh_import_id: aniecki
  lock_passwd: true
  ssh_pwauth: false

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1924922/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1893770] Re: Cloudinit resets network scripts to default configuration DHCP once Config Drive is removed after sometime

[Ryan Harper]

** Changed in: cloud-init
   Status: Expired => In Progress

** Changed in: cloud-init
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1893770

Title:
  Cloudinit resets network scripts to default configuration DHCP once
  Config Drive is removed after sometime

Status in cloud-init:
  In Progress

Bug description:
  Cloudinit version: 19.1
  Platform: OpenStack based.
  OS: RHEL, SUSE

  We use config drive(/dev/sr0) as a data source to configure network
  interfaces in the guest but configdrive is not always available and
  may be removed after couple of hours from the hypervisor.

  On a first boot cloudinit uses data provided in config drive and
  updates system level network scripts /etc/sysconfig/ifcfg-* (Static
  configuration of networks) files and also configures the interface in
  the guest.

  As long as the configdrive is available, reboots will relay on system
  scripts for the configuring network but once configdrive is removed,
  datasource becomes None meaning it neither system script nor config
  drive

  which makes cloud init to configure default network which is DHCP

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1893770/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1899487] Re: cloud-init hard codes MTU configuration at initial deploy time

[Ryan Harper]

" # curl http://169.254.169.254/openstack/2018-08-27/network_data.json
{"links": [{"id": "tapa035fb68-01", "vif_id": 
"a035fb68-010c-42e3-8da7-ea3c36a0d607", "type": "ovs", "mtu": 8942, 
"ethernet_mac_address": "fa:16:3e:31:26:f7"}], "networks": [{"id": "network0", 
"type": "ipv4_dhcp", "link": "tapa035fb68-01", "network_id": 
"b4ef84c0-1235-48a8-aaf7-03fab7ef5367"}], "services": []}"

How is cloud-init to know from this network-config.json that DHCP will
provide an MTU value?  How does it know that it should ignore the
provided MTU?  If DHCP is providing MTU, should network-config.json then
not provide the MTU value?


** Also affects: netplan.io (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: New => Incomplete

** Changed in: netplan.io (Ubuntu)
   Status: New => Incomplete

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1899487

Title:
  cloud-init hard codes MTU configuration at initial deploy time

Status in cloud-init:
  Incomplete
Status in netplan.io package in Ubuntu:
  Incomplete

Bug description:
  When using OpenStack cloud provider `cloud-init` will write out
  /etc/netplan/50-cloud-init.yaml at initial instance boot and not
  update it on subsequent boots of the instance.

  The OpenStack metadata service provides information about MTU for the
  network [0] and cloud-init takes this value and writes it into the
  netplan configuration [1].

  A side effect of configuring the MTU through netplan is that the
  `systemd-networkd` [Link] section [2] gets the MTUBytes value filled
  and this in turn makes `systemd-networkd` ignore the MTU value
  provided by DHCP [3][4].

  During the lifetime of a cloud events occur that will force a operator
  to reduce the MTU available to instances attached to its overlay
  networks. This may happen because of software imposed change of tunnel
  type (GRE -> VXLAN, VXLAN -> GENEVE) or change of topology or
  encapsulation in the physical network equipment.

  To maximize performance these clouds have configured their instances
  to use the maximum available MTU without leaving any headroom to
  account for such changes and the only way to move forward is to reduce
  the available MTU on the instances. We are facing a concrete challenge
  with this now where we have users wanting to migrate from VXLAN
  tunnels to GENEVE tunnels with 38 byte header size.

  0: # curl http://169.254.169.254/openstack/2018-08-27/network_data.json
  {"links": [{"id": "tapa035fb68-01", "vif_id": 
"a035fb68-010c-42e3-8da7-ea3c36a0d607", "type": "ovs", "mtu": 8942, 
"ethernet_mac_address": "fa:16:3e:31:26:f7"}], "networks": [{"id": "network0", 
"type": "ipv4_dhcp", "link": "tapa035fb68-01", "network_id": 
"b4ef84c0-1235-48a8-aaf7-03fab7ef5367"}], "services": []}

  1: # cat /etc/netplan/50-cloud-init.yaml 
  # This file is generated from information provided by the datasource.  Changes
  # to it will not persist across an instance reboot.  To disable cloud-init's
  # network configuration capabilities, write a file
  # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
  # network: {config: disabled}
  network:
  version: 2
  ethernets:
  ens2:
  dhcp4: true
  match:
  macaddress: fa:16:3e:31:26:f7
  mtu: 8950
  set-name: ens2

  2: # cat /run/systemd/network/10-netplan-ens2.link 
  [Match]
  MACAddress=fa:16:3e:31:26:f7

  [Link]
  Name=ens2
  WakeOnLan=off
  MTUBytes=8950

  3: # cat /run/systemd/network/10-netplan-ens2.network 
  [Match]
  MACAddress=fa:16:3e:31:26:f7
  Name=ens2

  [Link]
  MTUBytes=8950

  [Network]
  DHCP=ipv4
  LinkLocalAddressing=ipv6

  [DHCP]
  RouteMetric=100
  UseMTU=true

  4: Oct 12 13:30:18 canary-3 systemd-networkd[24084]:
  /run/systemd/network/10-netplan-ens2.network: MTUBytes= in [Link]
  section and UseMTU= in [DHCP] section are set. Disabling UseMTU=.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1899487/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1890803] Re: Groovy amd64 / arm64 / PowerPC deployment seems not working

[Ryan Harper]

@Sean  you're right...  the cloud-init fix was to handle the case there is *no* 
/etc/fstab ... 
so any other place that expects /etc/fstab to exist must also handle missing 
file.

Looking at overlayroot, it expects to be able to update/modify the
baked-in fstab, but it's not there.

https://git.launchpad.net/cloud-initramfs-tools/tree/overlayroot/scripts
/init-bottom/overlayroot#n313


** Also affects: cloud-initramfs-tools
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1890803

Title:
  Groovy amd64 / arm64 / PowerPC deployment seems not working

Status in cloud-init:
  Invalid
Status in cloud-initramfs-tools:
  New
Status in MAAS:
  Triaged
Status in ubuntu-kernel-tests:
  New

Bug description:
  All bare-metal deployment tasks have failed on our bare-metal maas
  server and the PowerMAAS / HyperMAAS (image synced on maas server).

  Deployment failed with:
    Installation was aborted.

  Need to further investigate this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1890803/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1890803] Re: Groovy amd64 / arm64 / PowerPC deployment seems not working

[Ryan Harper]

I'm marking cloud-init task invalid; by the time cloud-init runs, rootfs
should be rw using overlayfs backed by tmpfs.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1890803

Title:
  Groovy amd64 / arm64 / PowerPC deployment seems not working

Status in cloud-init:
  Invalid
Status in MAAS:
  Incomplete
Status in ubuntu-kernel-tests:
  New

Bug description:
  All bare-metal deployment tasks have failed on our bare-metal maas
  server and the PowerMAAS / HyperMAAS (image synced on maas server).

  Deployment failed with:
    Installation was aborted.

  Need to further investigate this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1890803/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1888298] Re: cc_timezone fails on Ubuntu Bionic and Xenial minimal

[Ryan Harper]

https://cloud-images.ubuntu.com/daily/server/minimal/daily/focal/current
/focal-minimal-cloudimg-amd64.manifest

Has tzdata ...

This looks like a cloudimg issue, not cloud-init.

https://cloud-
images.ubuntu.com/daily/server/minimal/daily/bionic/current/bionic-
minimal-cloudimg-amd64.manifest


** Also affects: cloud-images
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1888298

Title:
  cc_timezone fails on Ubuntu Bionic and Xenial minimal

Status in cloud-images:
  New
Status in cloud-init:
  Incomplete

Bug description:
  Summary
  ===
  On Ubuntu Bionic and Xenial minimal images, there is no tzdata package. As a 
result, when cloud-init tries to set the timezone it will fail and produce a 
stack trace.

  Expected Result
  ===
  No trace and no failure of the cloud-config.service :)

  Actual result
  ===
  2020-07-20 18:13:22,515 - util.py[DEBUG]: Running module timezone () failed
    File "/usr/lib/python3/dist-packages/cloudinit/config/cc_timezone.py", line 
47, in handle
  cloud.distro.set_timezone(timezone)
    File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 
165, in set_timezone
  distros.set_etc_timezone(tz=tz, tz_file=self._find_tz_file(tz))
  OSError: Invalid timezone America/Vancouver, no file found at 
/usr/share/zoneinfo/America/Vancouver

  Steps to reproduce
  ===
  $ wget 
https://cloud-images.ubuntu.com/daily/server/minimal/releases/bionic/release/ubuntu-18.04-minimal-cloudimg-amd64.img
  $ multipass launch file:///$(pwd)/ubuntu-18.04-minimal-cloudimg-amd64.img 
--name=bionic-minimal
  $ multipass exec bionic-minimal -- sudo systemctl list-units --failed 
--no-legend
  # note that cloud-config.service fails
  $ multipass exec bionic-minimal -- sudo cat /var/log/cloud-init.log | grep 
timezone

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/1888298/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1883907] Re: EC2 data source does not properly return the instance ID when cached data exists

[Ryan Harper]

I'm marking this invalid; if you find more information that would
indicate that cloud-init is not booting like a new instance after
capturing on Ec2, please reopen this bug and include updated
information.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1883907

Title:
  EC2 data source does not properly return the instance ID when cached
  data exists

Status in cloud-init:
  Invalid

Bug description:
  When creating an AMI from a running instance without cleaning the
  cloud-init cache an instance from the new AMI is no properly
  identified as a new instance and none of the PER_INSTANCE tasks will
  be executed.

  The problem is that the Ec2 data source will return the cached
  instance-id rather than the new instance ID.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1883907/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1881814] Re: OpenStack datasource detection fails during init on Hyper-V compute nodes

[Ryan Harper]

I'm marking the cloud-init task invalid; if you believe there is still a
bug in cloud-init, please re-open this task along with new information
explaining the bug.


** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1881814

Title:
  OpenStack datasource detection fails during init on Hyper-V compute
  nodes

Status in cloud-init:
  Invalid
Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  The OpenStack datasource detection fails on an Hyper-V compute (Nova)
  node (running on-premises, no cloud provider). It does not try to
  attempt to use the metdata endpoint and the instance fails to
  initialize properly unless another datasource is used (like
  ConfigDrive or EC2, the latter which gives a warning).

  The issue is in cloudinit/sources/DataSourceOpenStack.py, in the
  detect_openstack function. The DMI data does not match anything
  expected.

  In Hyper-V the following info is returned from DMI:

  grep "" /sys/class/dmi/id/*

  /sys/class/dmi/id/bios_date:05/23/2012
  /sys/class/dmi/id/bios_vendor:American Megatrends Inc.
  /sys/class/dmi/id/bios_version:090006
  /sys/class/dmi/id/board_name:Virtual Machine
  /sys/class/dmi/id/board_serial:8096-7783-9998-*snip*
  /sys/class/dmi/id/board_vendor:Microsoft Corporation
  /sys/class/dmi/id/board_version:7.0
  /sys/class/dmi/id/chassis_asset_tag:8096-7783-9998-*snip*
  /sys/class/dmi/id/chassis_serial:8096-7783-9998-*snip*
  /sys/class/dmi/id/chassis_type:3
  /sys/class/dmi/id/chassis_vendor:Microsoft Corporation
  /sys/class/dmi/id/chassis_version:7.0
  
/sys/class/dmi/id/modalias:dmi:bvnAmericanMegatrendsInc.:bvr090006:bd05/23/2012:svnMicrosoftCorporation:pnVirtualMachine:pvr7.0:rvnMicrosoftCorporation:rnVirtualMachine:rvr7.0:cvnMicrosoftCorporation:ct3:cvr7.0:
  grep: /sys/class/dmi/id/power: Is a directory
  /sys/class/dmi/id/product_name:Virtual Machine
  /sys/class/dmi/id/product_serial:8096-7783-9998-*snip*
  /sys/class/dmi/id/product_uuid:f41d9e0e-c208-*snip*
  /sys/class/dmi/id/product_version:7.0
  grep: /sys/class/dmi/id/subsystem: Is a directory
  /sys/class/dmi/id/sys_vendor:Microsoft Corporation
  
/sys/class/dmi/id/uevent:MODALIAS=dmi:bvnAmericanMegatrendsInc.:bvr090006:bd05/23/2012:svnMicrosoftCorporation:pnVirtualMachine:pvr7.0:rvnMicrosoftCorporation:rnVirtualMachine:rvr7.0:cvnMicrosoftCorporation:ct3:cvr7.0:

  cat /proc/1/environ

  HOME=/ init=/sbin/init NETWORK_SKIP_ENSLAVED= TERM=linux
  BOOT_IMAGE=/boot/vmlinuz-5.4.0-33-generic drop_caps=
  PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  PWD=/ rootmnt=/root

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1881814/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1881806] Re: cloud-init fails to initialize with virtual router

[Ryan Harper]

This issue has already been fixed in upstream cloud-init, see:

https://github.com/canonical/cloud-
init/commit/5352dd99eb2937b4eaaaf596b40ad7ca69d87f64

And

 https://bugs.launchpad.net/cloud-init/+bug/1639263

For downstream, you'll need RedHat/Centos to cherry pick this commit.

If possible, you can test with our daily rpm build for Centos8 using
this COPR repo:

https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/


** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1881806

Title:
  cloud-init fails to initialize with virtual router

Status in cloud-init:
  Invalid
Status in CentOS:
  Unknown

Bug description:
  Cloud-init fails to initialize with RHEL8.1 and CentOS8.1 (RPM cloud-
  init-18.5-7.el8_1.1.noarch) when routing to the VMs are provider by
  virtual router (vrouter). In this scenario, we are spawning a VM on
  OpenStack using Contrail as the network provider. With cloud-init-18.5
  version, we see the following exception during the VM boot phase
  causing init and final stages of cloud-init to fail.

  Impact: cloud-init fails to inject ssh-keys, grow root partition and
  other critical functions.

  Version cloud-init-18.2-6.el8.noarch of cloud-init did not have this
  issue.

  Reproducibility: always
   

  2020-06-02 19:17:15,641 - util.py[WARNING]: failed stage init
  failed run of stage init
  
  Traceback (most recent call last):
    File "/usr/lib/python3.6/site-packages/cloudinit/cmd/main.py", line 652, in 
status_wrapper
  ret = functor(name, args)
    File "/usr/lib/python3.6/site-packages/cloudinit/cmd/main.py", line 362, in 
main_init
  init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
    File "/usr/lib/python3.6/site-packages/cloudinit/stages.py", line 649, in 
apply_network_config
  netcfg, src = self._find_networking_config()
    File "/usr/lib/python3.6/site-packages/cloudinit/stages.py", line 636, in 
_find_networking_config
  if self.datasource and hasattr(self.datasource, 'network_config'):
    File 
"/usr/lib/python3.6/site-packages/cloudinit/sources/DataSourceOpenStack.py", 
line 115, in network_config
  self.network_json, known_macs=None)
    File 
"/usr/lib/python3.6/site-packages/cloudinit/sources/helpers/openstack.py", line 
645, in convert_net_json
  'Unknown network_data link type: %s' % link['type'])
  ValueError: Unknown network_data link type: vrouter

  The error seems to be coming from
  cloudinit/sources/helpers/openstack.py file which only expects 'link'
  type to be only of one of the following:

  valid_keys = {
  'physical': [
  'name',
  'type',
  'mac_address',
  'subnets',
  'params',
  'mtu',
  ],

  However, with Contrail, the network_config object that gets passed to
  the method convert_net_json looks something like the following object:

  {
    "services": [],
    "networks": [
  {
    "network_id": "f42c8806-c128-4ef8-af9e-d4a3856dde23",
    "type": "ipv4",
    "netmask": "255.255.255.192",
    "link": "tap722bcc7f-5a",
    "routes": [
  {
    "netmask": "0.0.0.0",
    "network": "0.0.0.0",
    "gateway": "10.140.145.126"
  }
    ],
    "ip_address": "10.140.145.110",
    "id": "network0"
  }
    ],
    "links": [
  {
    "type": "vrouter",
    "vif_id": "722bcc7f-5ae5-4564-ad43-ff9a5357dd36",
    "ethernet_mac_address": "02:72:2b:cc:7f:5a",
    "id": "tap722bcc7f-5a",
    "mtu": null
  }
    ]
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1881806/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1880177] Re: cloud init netplan fails to apply (v2 18.04 cloud image)

[Ryan Harper]

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1880177

Title:
  cloud init netplan fails to apply (v2 18.04 cloud image)

Status in cloud-init:
  Invalid

Bug description:
  Hi,
  I have been running up against this for a while and haven't found a work 
around no matter how much I change the format of the source yaml. 

  Environment:
   - vSphere deploying using Terraform and the VMWare Guest Info datasource 
(https://github.com/vmware/cloud-init-vmware-guestinfo)
   - Ubuntu 18.04 Cloud image
   - Cloud-init v. 19.4-33-gbb4131a2-0ubuntu1~18.04.1
   
  Issue:

  When cloud init tries to apply the network configuration (using v2
  syntax) it results in the following error from netplan:

  Stderr: /etc/netplan/50-cloud-init.yaml:10:13: Error in network definition: 
malformed address '10.1.400.140/24', must be X.X.X.X/NN or X:X:X:X:X:X:X:X/NN
  - 10.1.400.140/24

  It is obvious why the error is happening because when you check the
  resulting yaml file it is wrongly indented:

  # This file is generated from information provided by the datasource.  Changes
  # to it will not persist across an instance reboot.  To disable cloud-init's
  # network configuration capabilities, write a file
  # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
  # network: {config: disabled}
  network:
  ethernets:
  ens192:
  addresses:
  - 10.1.1.50/24
  dhcp4: false
  gateway4: null
  nameservers:
  addresses: []
  set-name: TEST
  ens224:
  addresses:
  - 10.2.2.14/24
  dhcp4: false
  gateway4: 10.2.2.254
  nameservers:
  addresses:
  - 10.2.2.254
  set-name: MGMT
  ens256:
  addresses:
  - 192.168.1.14/24
  dhcp4: false
  gateway4: 192.168.1.254
  nameservers:
  addresses:
  - 192.168.1.254
  set-name: TEST2
  version: 2

  However the input yaml from the VMware datasource looks correct:
  #vmware-rpctool "info-get guestinfo.metadata" | base64 -d
  network:
version: 2
ethernets:
  ens192:
dhcp4: false
set-name: TEST
addresses: [10.1.1.50/24]
gateway4: 
nameservers:
  addresses: []
  ens224:
dhcp4: false
set-name: MGMT
addresses: [10.2.2.14/24]
gateway4: 10.2.2.254
nameservers:
  addresses: [10.2.2.254]
  ens256:
dhcp4: false
set-name: TEST2
addresses: [192.168.1.14/24]
gateway4: 192.168.1.254
nameservers:
  addresses: [192.168.1.254]
  local-hostname: server.mgmt
  instance-id: server.mgmt

  And cloud init seems to interpret it correctly to json (from cloud-
  init.log):

  {
'ens192': {
  'dhcp4': False,
  'set-name': 'TEST',
  'addresses': [
'10.1.1.50/24'
  ],
  'gateway4': None,
  'nameservers': {
'addresses': [
  
]
  }
},
'ens224': {
  'dhcp4': False,
  'set-name': 'MGMT',
  'addresses': [
'10.2.2.14/24'
  ],
  'gateway4': '10.2.2.254',
  'nameservers': {
'addresses': [
  '10.2.2.254'
]
  }
},
'ens256': {
  'dhcp4': False,
  'set-name': 'TEST2',
  'addresses': [
'192.168.1.14/24'
  ],
  'gateway4': '192.168.1.254',
  'nameservers': {
'addresses': [
  '192.168.1.254'
]
  }
}
  }

  However it seems the conversion from the json into netplan yaml
  malforms the yaml slightly causing the netplan apply to fail. Full
  Error below from cloud-init.log

  2020-05-22 12:47:12,682 - netplan.py[DEBUG]: V2 to V2 passthrough
  2020-05-22 12:47:12,683 - util.py[DEBUG]: Writing to 
/etc/netplan/50-cloud-init.yaml - wb: [644] 1047 bytes
  2020-05-22 12:47:12,684 - util.py[DEBUG]: Running command ['netplan', 
'generate'] with allowed return codes [0] (shell=False, capture=True)
  2020-05-22 12:47:12,764 - util.py[WARNING]: failed stage init-local
  2020-05-22 12:47:12,764 - util.py[DEBUG]: failed stage init-local
  Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 653, in 
status_wrapper
  ret = functor(name, args)
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in 
main_init
  init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 710, in 
apply_network_config
  return self.distro.apply_network_config(netcfg, bring_up=bring_up)
File "/usr/lib/python3/dist-

[Yahoo-eng-team] [Bug 1879103] Re: Unattended installation user data conflict

[Ryan Harper]

** Also affects: cloud-init
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1879103

Title:
  Unattended installation user data conflict

Status in cloud-init:
  New
Status in subiquity:
  New

Bug description:
  When passing user-data to the 20.04 unattended installation through an
  iso described at
  https://wiki.ubuntu.com/FoundationsTeam/AutomatedServerInstalls/QuickStart,
  user creation doesn't happen. This is because /dev/sr1 takes priority
  over /var/lib/cloud/seed/nocloud-net/.

  The simple workaround for anyone experiencing this issue is to add the
  following to their user-data script:

late-commands:
  - "eject /dev/sr1"

  I experienced this using Hyper-V.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1879103/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1639955] Re: bad test for snappy systems

[Ryan Harper]

This issue was fixed a while back in:

https://github.com/canonical/cloud-init/commit/4bcc94730

cloud-init retains the legacy bits, but prefers the use of os-release
file or kernel commandline to  for checking for UbuntuCore.


** Changed in: cloud-init (Ubuntu)
   Status: Confirmed => Fix Released

** Changed in: cloud-init
   Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1639955

Title:
  bad test for snappy systems

Status in cloud-init:
  Fix Released
Status in cloud-init package in Ubuntu:
  Fix Released

Bug description:
  Reviewing the latest SRU for cloud-init, I noticed the following:

  def system_is_snappy():
  # channel.ini is configparser loadable.
  # snappy will move to using /etc/system-image/config.d/*.ini
  # this is certainly not a perfect test, but good enough for now.
  content = load_file("/etc/system-image/channel.ini", quiet=True)
  if 'ubuntu-core' in content.lower():
  return True
  if os.path.isdir("/etc/system-image/config.d/"):
  return True
  return False

  This isn't a good test for whether a system is an ubuntu-core system.
  'system-image' is historical baggage, and not likely to be present at
  all in future versions.

  I'm afraid I don't know a good alternative test offhand, but wanted to
  log the bug so someone could look into it rather than being caught by
  surprise when ubuntu-core image contents later change.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1639955/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1869430] Re: cloud-init persists in running state on Kali in AWS

[Ryan Harper]

Excellent news!  Thanks for following up with Kali upstream.  I'm
closing this issue as invalid for cloud-init.  If you find out after the
Kali changes there are still issues, either re-open this bug or file a
new one if the issue/behavior is different.

Thanks!

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1869430

Title:
  cloud-init persists in running state on Kali in AWS

Status in cloud-init:
  Invalid

Bug description:
  Hello,

  We're trying to customize published Kali AMIs using packer & cloud-
  init. The entire process works with Ubuntu, CentOS, and Amazon Linux 2
  targets, but seemingly breaks with Kali. We've tried it with both the
  2020.01 and 2019.03.

  We're also experiencing a long timeout for ec2 data source:

  root@kali:~# cloud-init status --long
  status: running
  time: Fri, 27 Mar 2020 20:06:54 +
  detail:
  DataSourceEc2Local

  root@kali:~# cloud-init analyze blame
  -- Boot Record 01 --
   51.20500s (init-local/search-Ec2Local)
   00.91700s (init-network/config-users-groups)
   00.67200s (init-network/config-growpart)
   00.27400s (init-network/config-resizefs)
   00.24800s (init-network/config-ssh)
   00.00600s (init-network/consume-user-data)
   00.00300s (init-network/check-cache)

  Attached is the log tarball produced by cloud-init. We'd appreciate
  any hints as to what may be happening. It's worth noting that these
  targets are starting in a VPC without direct connection to the outside
  world, but there's a squid proxy available for web traffic. We have
  relevant parts set up to use that proxy.

  
  Thanks!

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1869430/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1860164] Re: cloud-init generates a traceback if a default route already exists during ephemeral network setup

[Ryan Harper]

** Changed in: cloud-init
   Status: Expired => Incomplete

** Changed in: cloud-init
   Importance: Undecided => Medium

** Changed in: cloud-init
   Status: Incomplete => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1860164

Title:
  cloud-init generates a traceback if a default route already exists
  during ephemeral network setup

Status in cloud-init:
  Triaged

Bug description:
  If a route already exists when the ephemeral network exists cloud-init
  will generate the following traceback:

  2020-01-16 21:14:22,584 - util.py[DEBUG]: Getting data from  failed
  Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 
760, in find_source
  if s.update_metadata([EventType.BOOT_NEW_INSTANCE]):
File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 
649, in update_metadata
  result = self.get_data()
File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 
273, in get_data
  return_value = self._get_data()
File 
"/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceOracle.py", line 
195, in _get_data
  with dhcp.EphemeralDHCPv4(net.find_fallback_nic()):
File "/usr/lib/python2.7/site-packages/cloudinit/net/dhcp.py", line 57, in 
__enter__
  return self.obtain_lease()
File "/usr/lib/python2.7/site-packages/cloudinit/net/dhcp.py", line 109, in 
obtain_lease
  ephipv4.__enter__()
File "/usr/lib/python2.7/site-packages/cloudinit/net/__init__.py", line 
920, in __enter__
  self._bringup_static_routes()
File "/usr/lib/python2.7/site-packages/cloudinit/net/__init__.py", line 
974, in _bringup_static_routes
  ['dev', self.interface], capture=True)
File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 2083, in 
subp
  cmd=args)
  ProcessExecutionError: Unexpected error while running command.

  This is a regression from 19.1 on SUSE where exiting routes were
  simply skipped.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1860164/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1867029] Re: Package ifupdown breaks network configuration by cloud-init

[Ryan Harper]

** Also affects: ifupdown (Ubuntu)
   Importance: Undecided
   Status: New

** Tags added: uec-images

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1867029

Title:
  Package ifupdown breaks network configuration by cloud-init

Status in cloud-init:
  Confirmed
Status in ifupdown package in Ubuntu:
  New

Bug description:
  It appears the `ifupdown` package breaks networking configuration that
  cloud-init would normally handle (?) on both providers. The result, if
  you image an instance with the `ifupdown` package installed, is that
  the image is not bootable by Azure/AWS. I'm not familiar with how
  Azure/AWS/etc use cloud-init for network start up however. I'm filing
  here for now, in case `cloud-init` needs to be more defensive against
  `ifupdown`.

  To reproduce with Packer (I confirmed this method):

  * Add necessary Azure subscription env vars from `ifupdown-bug.json`
  * `packer build ifupdown-bug.json`
  * Try to boot an instance with resulting image
  * Instance will never boot -- more specifically, networking never comes up. 
See `boot.log` for an example boot

  To reproduce manually (only a guess, I did not confirm):

  * Boot 18.04 ubuntu instance
  * Install ifupdown
  * Image the instance
  * Try to boot instance using new image
  * Instance won't boot

  I hit this with Packer, creating new images for booting instances in
  scaling groups -- `ifupdown` is brought in by `salt`. If this is
  reproducible via manual installation, there could be some backup
  images/snapshots that are unable to boot though.

  This appears to be because cloud-init network start up operations are
  broken by whatever `ifupdown` sysv/systemd scripts perform on boot.
  With `ifupdown` installed on Azure, `eth0` does not get an IP address,
  the instance never fully boots according to Azure, and it will enter
  an error state. On AWS, the instance boots, but network operations
  like SSH fail.

  The `boot.log` is from Azure. Eventually `cloud-init` gives an
  exception (see `exception.log`), but this seems like a secondary issue
  related to networking being down.

  I did try to upgrade cloud-init using nightly PPAs (version 20.1 I
  believe), but no luck. This is on Ubuntu 18.04 instances from Azure
  marketplace and AWS Canonical AMI, cloud-init version
  `19.4-33-gbb4131a2-0ubuntu1~18.04.1`.

  Related: https://github.com/Azure/WALinuxAgent/issues/1612

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1867029/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1863773] Re: cloud-init network fails if ipv6 is disabled

[Ryan Harper]

Thanks for reporting the bug and attaching the logs.  Looking at the
logs and your description it appears to be an issue with your Debian
image itself.

The files,  /etc/network/if-pre-up.d/cloud_inet6 , and
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg   are not part of
cloud-init itself.

The latter file is configuring cloud-init to not touch networking and
that the image will bring it up.  The former file looks like it needs
some error handling in the case of disabling ipv6.

Looking at the Debian AWS/EC2 FAQ[1] it suggests that you can[2]:

"""
Please report bugs to the cloud.debian.org pseudo-package, when they are 
related to the choices made when building the image (which packages to include, 
which customisation was made, etc.). Advanced users can add the usertags to 
triage the report more precisely.
"""

As this appears to be an issue with the AMI, I'm going to mark this bug Invalid 
for cloud-init.  If you feel that there is an issue with cloud-init, please 
mark this bug New and add some addition
information to help explain how cloud-init is involved.


1. https://wiki.debian.org/Amazon/EC2/FAQ
2. 
https://wiki.debian.org/Amazon/EC2/FAQ#Q:_Where_do_I_report_issues_with_the_images_.3F

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1863773

Title:
  cloud-init network fails if ipv6 is disabled

Status in cloud-init:
  Invalid

Bug description:
  I'm using cloud-init on a Debian machien in AWS.

  On these machiesn, ipv6 is disabled by setting
  /proc/sys/net/ipv6/conf/all/disable_ipv6 to 1.  This causes the
  network to fail:

  bas@machine:/etc/network$ sudo ifup ens5
  Internet Systems Consortium DHCP Client 4.4.1
  Copyright 2004-2018 Internet Systems Consortium.
  All rights reserved.
  For info, please visit https://www.isc.org/software/dhcp/

  Listening on LPF/ens5/02:45:ca:0d:93:76
  Sending on   LPF/ens5/02:45:ca:0d:93:76
  Sending on   Socket/fallback
  DHCPREQUEST for 10.0.16.20 on ens5 to 255.255.255.255 port 67
  DHCPACK of 10.0.16.20 from 10.0.16.1
  RTNETLINK answers: File exists
  bound to 10.0.16.20 -- renewal in 1796 seconds.

  
  Could not get a link-local address
  run-parts: /etc/network/if-pre-up.d/cloud_inet6 exited with return code 1
  ifup: failed to bring up ens5

  
  Even worse, on boot the machines do get a proper ipv4 address, but 
(apparently because setting up the network fails), the dhcp (v4) daemon is 
stopped, and the machines "mysteriously" drop off the net after the lease ends.

  
  1. AWS with official Debian buster images 
(https://wiki.debian.org/Cloud/AmazonEC2Image/Buster)
  2. 

  bas@machine:~$ cat /etc/network/interfaces
  # Include files from /etc/network/interfaces.d:
  source-directory /etc/network/interfaces.d

  # Cloud images dynamically generate config fragments for newly
  # attached interfaces. See /etc/udev/rules.d/75-cloud-ifupdown.rules
  # and /etc/network/cloud-ifupdown-helper. Dynamically generated
  # configuration fragments are stored in /run:
  source-directory /run/network/interfaces.d

  bas@machine:~$ cat /run/network/interfaces.d/ens5
  auto ens5
  allow-hotplug ens5

  iface ens5 inet dhcp

  iface ens5 inet6 manual
try_dhcp 1

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1863773/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1863943] [NEW] do not log imdsv2 token headers

[Ryan Harper]

Public bug reported:

Cloud-init currently logs all headers when processing URLs.  On Ec2,
this includes the IMDSv2 token negotiation and all IMDS interactions.
The value of seeing the headers in the log is quite useful, especially
for confirming whether cloud-init is using IMDSv2 or not; however the
actual value of the token does not aide in this effort.

Reported by: https://github.com/ishug86

** Affects: cloud-init
 Importance: Medium
 Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1863943

Title:
  do not log imdsv2 token headers

Status in cloud-init:
  In Progress

Bug description:
  Cloud-init currently logs all headers when processing URLs.  On Ec2,
  this includes the IMDSv2 token negotiation and all IMDS interactions.
  The value of seeing the headers in the log is quite useful, especially
  for confirming whether cloud-init is using IMDSv2 or not; however the
  actual value of the token does not aide in this effort.

  Reported by: https://github.com/ishug86

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1863943/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1862417] Re: cloud-init: Attempt to mkswap on a partition fails: invalid block count argument: ''

[Ryan Harper]

 The machine I'm working on uses cloud-init to update itself, it might 
only have the fix after the updates.
 ah, interesting
 cat /etc/cloud/build.info  
 that'll give us a point in time for which version you have 
 and I suspect you're right, the top of your cloud-init.log will the 
original version 
 build_name: serverserial: 20190514
 Definitely way older than the PR.
 yep
 I suspect the fix in our case is to use the latest image of Ubuntu 
from end Jan 2020.
 yep

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1862417

Title:
  cloud-init: Attempt to mkswap on a partition fails: invalid block
  count argument: ''

Status in cloud-init:
  Invalid

Bug description:
  If an attempt is made to configure a swap partition on an Ubuntu
  Bionic machine as follows (not a swap file, a swap partition), the
  attempt to mkswap fails.

  The expected behaviour is that mkswap and swapon are executed
  correctly, and /dev/xvdg becomes a valid swap disk. In addition, when
  filename points at a partition, size and maxsize should be ignored.

  fs_setup:
- label: vidi
  device: /dev/xvde
  filesystem: ext4
- label: swap
  device: /dev/xvdg
  filesystem: swap
  mounts:
  - [ /dev/xvde, /var/lib/vidispine, ext4, defaults, 0, 0 ]
  - [ /dev/xvdg, none, swap, sw, 0, 0 ]
  swap:
  filename: /dev/xvdg
  size: auto
  maxsize: 17179869184
  mount_default_fields: [ None, None, "auto", "defaults", "0", "2" ]

  When the machine starts up for the first time, the following error is
  logged after the swap size parameter is passed as the empty string:

  2020-02-07 20:21:55,242 - cc_disk_setup.py[WARNING]: Force flag for swap is 
unknown.
  2020-02-07 20:21:55,255 - util.py[WARNING]: Failed during filesystem operation
  Failed to exec of '['/sbin/mkswap', '/dev/xvdg', '-L', 'swap', '']':
  Unexpected error while running command.
  Command: ['/sbin/mkswap', '/dev/xvdg', '-L', 'swap', '']
  Exit code: 1
  Reason: -
  Stdout: 
  Stderr: mkswap: invalid block count argument: ''
  2020-02-07 20:21:55,530 - cc_mounts.py[WARNING]: Activate mounts: FAIL:swapon 
-a
  2020-02-07 20:21:55,530 - util.py[WARNING]: Activate mounts: FAIL:swapon -a

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1862417/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1861460] [NEW] cloud-init should parse initramfs rendered netplan if present

[Ryan Harper]

Public bug reported:

initramfs-tools used to only execute klibc based networking with some
resolvconf hooks.

In recent releases, it has been greatly improved to use
isc-dhcp-client instead of klibc, support vlan= key (like in
dracut-network), bring up Z devices using chzdev, and generate netplan
yaml from all of the above.

Above improvements were driven in part by Oracle Cloud and in part by
Subiquity netbooting on Z.

Thus these days, instead of trying to reparse klibc files in
/run/net-*, cloud-init should simply import /run/netplan/$device.yaml
files as the ip=* provided networking information on the command line.
I do not currently see cloud-init doing that in e.g.
/cloudinit/net/cmdline.py

** Affects: cloud-init
 Importance: Wishlist
 Status: Triaged

** Changed in: cloud-init
   Importance: Undecided => Wishlist

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1861460

Title:
  cloud-init should parse initramfs rendered netplan if present

Status in cloud-init:
  Triaged

Bug description:
  initramfs-tools used to only execute klibc based networking with some
  resolvconf hooks.

  In recent releases, it has been greatly improved to use
  isc-dhcp-client instead of klibc, support vlan= key (like in
  dracut-network), bring up Z devices using chzdev, and generate netplan
  yaml from all of the above.

  Above improvements were driven in part by Oracle Cloud and in part by
  Subiquity netbooting on Z.

  Thus these days, instead of trying to reparse klibc files in
  /run/net-*, cloud-init should simply import /run/netplan/$device.yaml
  files as the ip=* provided networking information on the command line.
  I do not currently see cloud-init doing that in e.g.
  /cloudinit/net/cmdline.py

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1861460/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1857397] Re: CloudStack data source cannot determine Virtual Router address on RHEL/CentOS 8

[Ryan Harper]

*** This bug is a duplicate of bug 1843334 ***
https://bugs.launchpad.net/bugs/1843334

** This bug has been marked a duplicate of bug 1843334
   Change location of DHCP leases in CloudStack provider as it doesn't work for 
RHEL8

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1857397

Title:
  CloudStack data source cannot determine Virtual Router address on
  RHEL/CentOS 8

Status in cloud-init:
  Fix Released

Bug description:
  The problem: cloud-init is unable to determine CloudStack Virtual
  Router address on RHEL/CentOS 8 machines.

  Why: because RHEL/CentOS 8 does not support systemd-networkd and
  NetworkManager lease files format cannot be parsed by current cloud-
  init CloudStack data source implementation.

  Please see more details about this in merge request
  https://github.com/canonical/cloud-init/pull/141: changes from this
  request were successfully tested.

  What else: most likely other data sources also have the same problem.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1857397/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1858615] Re: Fail to boot when NoCloud datasource is included

[Ryan Harper]

Thanks for filing a bug.  I've added a dmidecode task to track the issue
with the tool.  It may also affect the kernel package, and possibly
firmware (though that's not something that Ubuntu provides).  Cloud-init
and any other tool may invoke this package and it should not reboot the
system; but there's no issue with cloud-init or other callers of the
tool   As such, I'm marking the cloud-init task as invalid.

** Also affects: dmidecode (Ubuntu)
   Importance: Undecided
   Status: New

** Summary changed:

- Fail to boot when NoCloud datasource is included
+ dmidecode triggers system reboot on Inforce 6640

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1858615

Title:
  dmidecode triggers system reboot on Inforce 6640

Status in cloud-init:
  Invalid
Status in dmidecode package in Ubuntu:
  New

Bug description:
  Device: Inforce 6640
  
https://www.inforcecomputing.com/products/single-board-computers-sbc/qualcomm-snapdragon-820-inforce-6640-sbc
  SoC: Snapdragon 820

  sysname='Linux',
  nodename='ubuntu',
  release='4.15.0-1069-snapdragon', 
  version='#76-Ubuntu SMP Tue Nov 26 16:10:14 UTC 2019', 
  machine='aarch64'

  The issue is caused by following commit.
  Inforce 6640 doesn't have functional demidecode.
  System will reboot when executing dmidecode.

  commit 3416e2ee7f65defdb15aab861a85767d13e8c34c
  Author: Robert Schweikert 
  Date: Sat Oct 29 09:29:53 2016 -0400
  dmidecode: Allow dmidecode to be used on aarch64
  aarch64 systems have functional dmidecode, so allow that to be used.
  - aarch64 has support for dmidecode as well

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1858615/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1851552] Re: since ubuntu 18 bionic release and latest, the ubuntu18 cloud image is unable to boot up on openstack instance

[Ryan Harper]

I'm marking the cloud-init task invalid as at this time the logs point
to a nested virtualization/openstack issue with devices not being
present; not related to cloud-init.  If further investigation points to
an issue with cloud-init you can move the cloud-init task back to New.

** Changed in: cloud-init (Ubuntu)
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1851552

Title:
  since ubuntu 18 bionic release and latest, the ubuntu18 cloud image is
  unable to boot up on openstack instance

Status in cloud-init:
  New
Status in networking-calico:
  New
Status in OpenStack Compute (nova):
  New
Status in OpenStack Community Project:
  New
Status in qemu-kvm:
  New
Status in cloud-init package in Ubuntu:
  Invalid
Status in qemu package in Ubuntu:
  New

Bug description:
  Openstack Queens release which is running on ubuntu 18 LTS Controller and 
Compute.
  Tried to boot up the instance via horizon dashboard without success.
  Nova flow works perfect.
  When access to console I discovered that the boot process stuck in the middle.
  [[0;1;31m TIME [0m] Timed out waiting for device dev-vdb.device.
  [[0;1;33mDEPEND[0m] Dependency failed for /mnt.
  [[0;1;33mDEPEND[0m] Dependency failed for File System Check on /dev/vdb.
  It receives IP but looks like not get configured at time.
  since ubuntu 18 there is netplan feature managing the network interfaces
  please advise.

  more details as follow:
  https://bugs.launchpad.net/networking-calico/+bug/1851548

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1851552/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1853376] [NEW] fix debian packaging warnings/errors

[Ryan Harper]

Public bug reported:

E: cloud-init source: untranslatable-debconf-templates cloud-init.templates: 6
W: cloud-init source: missing-file-from-potfiles-in grub.templates
W: cloud-init source: build-depends-on-obsolete-package build-depends: 
dh-systemd => use debhelper (>= 9.20160709)
W: cloud-init source: timewarp-standards-version (2011-12-16 < 2014-09-17)
W: cloud-init source: ancient-standards-version 3.9.6 (released 2014-09-17) 
(current is 4.4.1)
W: cloud-init source: binary-nmu-debian-revision-in-source 
19.3-244-gbee7e918-1~bddeb~20.04.1
W: cloud-init: binary-without-manpage usr/bin/cloud-id
W: cloud-init: binary-without-manpage usr/bin/cloud-init
W: cloud-init: binary-without-manpage usr/bin/cloud-init-per
W: cloud-init: command-with-path-in-maintainer-script postinst:141 
/usr/sbin/grub-install
W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-config.service cloud-init.target
W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-final.service cloud-init.target
W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init-local.service cloud-init.target
W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init.service cloud-init.target
W: cloud-init: systemd-service-file-shutdown-problems 
lib/systemd/system/cloud-init.service
N: 1 tag overridden (1 error)

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1853376

Title:
  fix debian packaging warnings/errors

Status in cloud-init:
  New

Bug description:
  E: cloud-init source: untranslatable-debconf-templates cloud-init.templates: 6
  W: cloud-init source: missing-file-from-potfiles-in grub.templates
  W: cloud-init source: build-depends-on-obsolete-package build-depends: 
dh-systemd => use debhelper (>= 9.20160709)
  W: cloud-init source: timewarp-standards-version (2011-12-16 < 2014-09-17)
  W: cloud-init source: ancient-standards-version 3.9.6 (released 2014-09-17) 
(current is 4.4.1)
  W: cloud-init source: binary-nmu-debian-revision-in-source 
19.3-244-gbee7e918-1~bddeb~20.04.1
  W: cloud-init: binary-without-manpage usr/bin/cloud-id
  W: cloud-init: binary-without-manpage usr/bin/cloud-init
  W: cloud-init: binary-without-manpage usr/bin/cloud-init-per
  W: cloud-init: command-with-path-in-maintainer-script postinst:141 
/usr/sbin/grub-install
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-config.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-final.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init-local.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init.service cloud-init.target
  W: cloud-init: systemd-service-file-shutdown-problems 
lib/systemd/system/cloud-init.service
  N: 1 tag overridden (1 error)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1853376/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1852456] [NEW] doc: list of modules is no longer present

[Ryan Harper]

Public bug reported:

the list of modules has disappeared from the documentation sidebar.

- version 19.3: https://cloudinit.readthedocs.io/en/19.3/topics/modules.html
- version 19.2: https://cloudinit.readthedocs.io/en/19.2/topics/modules.html

In 19.2, the sidebar has an entry for each module, and it's a lot easier
to find and navigate to appropriate module.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1852456

Title:
  doc: list of modules is no longer present

Status in cloud-init:
  New

Bug description:
  the list of modules has disappeared from the documentation sidebar.

  - version 19.3: https://cloudinit.readthedocs.io/en/19.3/topics/modules.html
  - version 19.2: https://cloudinit.readthedocs.io/en/19.2/topics/modules.html

  In 19.2, the sidebar has an entry for each module, and it's a lot
  easier to find and navigate to appropriate module.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1852456/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1851438] Re: cloud-init disk_setup fails to partition disk for Ubuntu18

[Ryan Harper]

On trusty; this complains but works.

ubuntu@ubuntu:~$ dpkg --list | grep util-linux
ii  util-linux  2.20.1-5.1ubuntu20.9
   amd64Miscellaneous system utilities
ubuntu@ubuntu:~$ cat /proc/partitions 
major minor  #blocks  name

 2530   10485760 vda
 2531   10484736 vda1
 253   16   10485760 vdb
 253   32366 vdc
  1101048575 sr0
ubuntu@ubuntu:~$ sudo bash
sudo: unable to resolve host ubuntu
root@ubuntu:~# echo "0," | sfdisk --Linux --unit=S --force /dev/vdb 
Checking that no-one is using this disk right now ...
OK

Disk /dev/vdb: 20805 cylinders, 16 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/vdb: unrecognized partition table type
Old situation:
No partitions found
Warning: bad partition start (earliest 1)
New situation:
Units = sectors of 512 bytes, counting from 0

   Device BootStart   End   #sectors  Id  System
/dev/vdb1 1  20971519   20971519  83  Linux
/dev/vdb2 0 -  0   0  Empty
/dev/vdb3 0 -  0   0  Empty
/dev/vdb4 0 -  0   0  Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
root@ubuntu:~# cat /proc/partitions 
major minor  #blocks  name

 2530   10485760 vda
 2531   10484736 vda1
 253   16   10485760 vdb
 253   17   10485759 vdb1
 253   32366 vdc
  1101048575 sr0


** Also affects: util-linux (Ubuntu)
   Importance: Undecided
   Status: New

** Also affects: util-linux (Ubuntu Eoan)
   Importance: Undecided
   Status: New

** Also affects: util-linux (Ubuntu Xenial)
   Importance: Undecided
   Status: New

** Also affects: util-linux (Ubuntu Focal)
   Importance: Undecided
   Status: New

** Also affects: util-linux (Ubuntu Bionic)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1851438

Title:
  cloud-init disk_setup fails to partition disk for Ubuntu18

Status in cloud-init:
  Triaged
Status in util-linux package in Ubuntu:
  New
Status in util-linux source package in Xenial:
  New
Status in util-linux source package in Bionic:
  New
Status in util-linux source package in Eoan:
  New
Status in util-linux source package in Focal:
  New

Bug description:
  Pasting disk_setup for cloud-config:

  disk_setup:
/dev/xvde:
   layout: True
   overwrite: False
   type: mbr
  fs_setup:
-device: /dev/xvde
 filesystem: ext4
 label: data
 overwrite: false
 partition: auto

  I want to attach and mount a /data disk on the VM using cloud-init so
  I just want to single partition 100% of the disk.

  Error while running the sfdisk command for partitioning the disk (please see 
attached file).
  OS: Ubuntu18

  How I repro-ed it outside cloud-init environment:
  1. Open XenCenter, add a new disk, say /dev/xvdc
  2. Run `/sbin/sfdisk --Linux --unit=S --force /dev/xvdc` and specify start 
sector as 0. Because from the cloud-init logs and source code, I figured that 
it was picking start sector as 0. Save it and see the error.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1851438/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1850642] Re: Cloud init unable to find the metadata service but can CURL it

[Ryan Harper]

*** This bug is a duplicate of bug 1821102 ***
https://bugs.launchpad.net/bugs/1821102

The logs do contain that bug, but I'm not sure that's the failure here.

Looking at the cloud-init logs, we can see the Ephemeral DHCP start and
obtain an lease, but the end point does not respond:

2019-10-30 13:28:41,516 - dhcp.py[DEBUG]: Received dhcp lease on eth0 for 
136.156.90.74/255.255.254.0
2019-10-30 13:28:41,516 - __init__.py[DEBUG]: Attempting setup of ephemeral 
network on eth0 with 136.156.90.74/23 brd 136.156.91.255
2019-10-30 13:28:41,516 - util.py[DEBUG]: Running command ['ip', '-family', 
'inet', 'addr', 'add', '136.156.90.74/23', 'broadcast', '136.156.91.255', 
'dev', 'eth0'] with allowed return codes [0] (shell=False, capture=True)
2019-10-30 13:28:41,519 - util.py[DEBUG]: Running command ['ip', '-family', 
'inet', 'link', 'set', 'dev', 'eth0', 'up'] with allowed return codes [0] 
(shell=False, capture=True)
2019-10-30 13:28:41,522 - util.py[DEBUG]: Running command ['ip', 'route', 
'show', '0.0.0.0/0'] with allowed return codes [0] (shell=False, capture=True)
2019-10-30 13:28:41,525 - util.py[DEBUG]: Running command ['ip', '-4', 'route', 
'add', '136.156.91.254', 'dev', 'eth0', 'src', '136.156.90.74'] with allowed 
return codes [0] (shell=False, capture=True)
2019-10-30 13:28:41,527 - util.py[DEBUG]: Running command ['ip', '-4', 'route', 
'add', 'default', 'via', '136.156.91.254', 'dev', 'eth0'] with allowed return 
codes [0] (shell=False, capture=True)
2019-10-30 13:29:21,573 - util.py[DEBUG]: Resolving URL: http://169.254.169.254 
took 40.044 seconds
2019-10-30 13:29:21,574 - url_helper.py[DEBUG]: [0/1] open 
'http://169.254.169.254/openstack' with {'url': 
'http://169.254.169.254/openstack', 'headers': {'User-Agent': 
'Cloud-Init/18.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 10.0} 
configuration
2019-10-30 13:29:31,608 - url_helper.py[DEBUG]: Calling 
'http://169.254.169.254/openstack' failed [10/-1s]: request error 
[HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with 
url: /openstack (Caused by 
ConnectTimeoutError(, 'Connection to 169.254.169.254 timed out. (connect 
timeout=10.0)'))]
2019-10-30 13:29:31,608 - DataSourceOpenStack.py[DEBUG]: Giving up on OpenStack 
md from ['http://169.254.169.254/openstack'] after 10 seconds
2019-10-30 13:29:31,609 - util.py[DEBUG]: Crawl of metadata service took 50.079 
seconds

Cloud-init did not find the OpenStack datasource at local time:

2019-10-30 13:29:31,689 - main.py[DEBUG]: No local datasource found

So we write out a fallback network config (dhcp on eth0).

2019-10-30 13:29:31,722 - stages.py[INFO]: Applying network configuration from 
fallback bringup=False: {'version': 1, 'config': [{'subnets': [{'type': 
'dhcp'}], 'type': 'physical', 'name': 'eth0', 'mac_address': 
'fa:16:3e:f4:b5:1d'}]}
2019-10-30 13:29:31,723 - __init__.py[DEBUG]: Selected renderer 'sysconfig' 
from priority list: None
2019-10-30 13:29:31,725 - util.py[DEBUG]: Writing to 
/etc/sysconfig/network-scripts/ifcfg-eth0 - wb: [644] 159 bytes
2019-10-30 13:29:31,726 - util.py[DEBUG]: Restoring selinux mode for 
/etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False)
2019-10-30 13:29:31,726 - util.py[DEBUG]: Restoring selinux mode for 
/etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False)

And then exit cloud-init-local.service:

2019-10-30 13:29:31,730 - util.py[DEBUG]: cloud-init mode 'init' took
52.389 seconds (52.39)

Now, the OS networking layer comes up:

Oct 30 13:29:31.623535 localhost.localdomain cloud-init[732]: 2019-10-30 
13:29:31,619 - util.py[WARNING]: No active metadata service found
Oct 30 13:29:31.751295 localhost.localdomain systemd[1]: Started Initial 
cloud-init job (pre-networking).
Oct 30 13:29:31.754472 localhost.localdomain systemd[1]: Reached target Network 
(Pre).
Oct 30 13:29:31.756812 localhost.localdomain systemd[1]: Starting LSB: Bring 
up/down networking...
Oct 30 13:29:31.961050 localhost.localdomain network[866]: Bringing up loopback 
interface:  [  OK  ]
Oct 30 13:29:31.990878 localhost.localdomain network[866]: Bringing up 
interface eth0:
Oct 30 13:29:32.034411 localhost.localdomain dhclient[995]: DHCPDISCOVER on 
eth0 to 255.255.255.255 port 67 interval 8 (xid=0xeceb89e)
Oct 30 13:29:32.055157 localhost.localdomain dhclient[995]: DHCPREQUEST on eth0 
to 255.255.255.255 port 67 (xid=0xeceb89e)
Oct 30 13:29:32.055188 localhost.localdomain dhclient[995]: DHCPOFFER from 
136.156.90.11
Oct 30 13:29:32.055796 localhost.localdomain dhclient[995]: DHCPACK from 
136.156.90.11 (xid=0xeceb89e)
Oct 30 13:29:34.132189 localhost.localdomain NET[1054]: 
/usr/sbin/dhclient-script : updated /etc/resolv.conf
Oct 30 13:29:34.166284 host-136-156-90-74 dhclient[995]: bound to 136.156.90.74 
-- renewal in 34571 seconds.
Oct 30 13:29:34.167466 host-136-156-90-74 network[866]: Determining IP 
information for eth0... done.
Oct 30 13:29:34.224637 host-136-156-90-74 network[866]: [  OK  ]
Oct 30 13:29:34.245397 host-136-

[Yahoo-eng-team] [Bug 1848471] Re: No infiniband support in netplan renderer

[Ryan Harper]

** Also affects: netplan.io (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Importance: Undecided => Wishlist

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1848471

Title:
  No infiniband support in netplan renderer

Status in cloud-init:
  Triaged
Status in netplan.io package in Ubuntu:
  New

Bug description:
  Infiniband support has been added to both the sysconfig and eni
  renderers (at time of writing, eni support is unmerged from
  https://code.launchpad.net/~darren-birkett/cloud-init/+git/cloud-
  init/+merge/373759).

  We need to add support to the netplan renderer too, but at this time
  netplan itself does not support the long form HWADDR that IB devices
  have. Once that is added, we should add support for infiniband devices
  in netplan too.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1848471/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1848243] Re: network-scripts(eni) is deprecated in CentOS/RHEL 8

[Ryan Harper]

cloud-init already has support for writing sysconfig formatted files on
systems which only include NetworkManager, specifically NetworkManager
includes a plugin module, 'ifcfg-rh' which reads the sysconfig files for
NetworkManager and cloud-init will enable this plugin as needed.

Please re-open this bug with more details as to what is not working.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1848243

Title:
  network-scripts(eni) is deprecated in CentOS/RHEL 8

Status in cloud-init:
  Invalid

Bug description:
  The network-scripts package has been removed from the standard
  CentOS/RHEL 8 install and is considered deprecated[1]. Currently it is
  possible to install the network-scripts package but it may not be
  available in the future. It is possible to continue to use eni via
  NetworkManager[2].

  [1] 
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_and_managing_networking/index#legacy_network_scripts_support
  [2] 
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_and_managing_networking/index#Using-NetworkManager-with-sysconfig-files_getting-started-with-networking-devices-with-NetworkManager

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1848243/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1846511] Re: cloud-init does not run after upgrade due to bad 90_dpkg.cfg

[Ryan Harper]

This was an release-branch only issue; it did not affect upstream cloud-
init.

** Also affects: cloud-init (Ubuntu)
   Importance: Undecided
   Status: New

** Also affects: cloud-init (Ubuntu Eoan)
   Importance: Undecided
   Status: New

** Changed in: cloud-init (Ubuntu Eoan)
   Status: New => Fix Released

** Changed in: cloud-init (Ubuntu Eoan)
   Importance: Undecided => Critical

** Changed in: cloud-init
   Status: Fix Committed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1846511

Title:
  cloud-init does not run after upgrade due to bad 90_dpkg.cfg

Status in cloud-init:
  Invalid
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Eoan:
  Fix Released

Bug description:
  Integration tests on Eoan are failing on the lxd and nocloud kvm
  platforms. In both cases the test system is not reachable via SSH.

  LXD log: https://paste.ubuntu.com/p/r2hwqZQfm2/

  KVM log: https://paste.ubuntu.com/p/W3N22yBxKN/

  This failure mode began with the test run of September 26. On the same
  system the Disco test run succeeds.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1846511/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1846513] Re: debian 10 cloud-init stuck in systemd-networkd after 1. reboot

[Ryan Harper]

I'm marking this invalid for cloud-init as the logs do not indicate
there is any cloud-init specific failure.  As you debug if you find
something that cloud-init should have done, please move this bug state
back to New.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1846513

Title:
  debian 10 cloud-init stuck in systemd-networkd after 1. reboot

Status in cloud-init:
  Invalid

Bug description:
  i have create a virtual machine using the following official debian
  image: https://cdimage.debian.org/cdimage/openstack/current-10/

  they have version 18.3-6 installed.

  after the initial boot of the vm cloud-init works as it should. but
  when a reboot occurred it seems to be hanging in networkd-systemd
  bringing up interfaces. see attached log file.

  i have also manually upgraded to cloud-init 19.2-1 and that had the
  same issue.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1846513/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1837106] Re: "socket.getaddrinfo" of "metadata.google.internal" fails on GCE

[Ryan Harper]

** Changed in: cloud-init
   Importance: Undecided => Medium

** Changed in: cloud-init
   Status: Expired => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1837106

Title:
  "socket.getaddrinfo" of "metadata.google.internal" fails on GCE

Status in cloud-init:
  Triaged

Bug description:
  When booting an Ubuntu 18.04 based image on GCE, we see the following
  messages in the log:

  2019-05-30 00:05:27,818 - util.py[DEBUG]: Resolving URL: 
http://metadata.google.internal/computeMetadata/v1/ took 0.001 seconds
  2019-05-30 00:05:27,818 - DataSourceGCE.py[DEBUG]: 
http://metadata.google.internal/computeMetadata/v1/ is not resolvable
  2019-05-30 00:05:27,818 - util.py[DEBUG]: Crawl of GCE metadata service took 
0.013 seconds
  2019-05-30 00:05:27,818 - DataSourceGCE.py[WARNING]: address 
"http://metadata.google.internal/computeMetadata/v1/"; is not resolvable 

  Further, the contents of "/run/cloud-init/instance-data.json" doesn't
  have any meaningful data.

  What I've found is that, read_md() in DataSourceGCE.py will call
  util.is_resolvable_url() on the address
  "http://metadata.google.internal/computeMetadata/v1/";, which results
  is calling socket.getaddrinfo() for "metadata.google.internal", and
  it's this socket.getaddrinfo() call that fails.

  This failure appears to be due to the fact that "cloud-init.service"
  does not ensure it waits for DNS (i.e. "systemd-resolved.service") to
  be working before it runs. I say this because:

  1. If I add "After=systemd-resolved.service" to the "cloud-
  init.service" definition, this failures goes away.

  2. If I run "cloud-init init" after the system has booted up (i.e.
  after enough time has passed, such that DNS is working), the failure
  doesn't occur.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1837106/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1832645] Re: fs_setup/disk_setup: option to wait for the device to exist before continuing

[Ryan Harper]

** Changed in: cloud-init
   Status: Expired => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1832645

Title:
  fs_setup/disk_setup: option to wait for the device to exist before
  continuing

Status in cloud-init:
  Triaged

Bug description:
  When using the AWS::EC2::Volume and AWS::EC2::VolumeAttachment options
  to add a volume to an AWS::EC2::Instance on AWS EC2, the volume is not
  immediately available on the instance.

  This causes fs_setup and disk_setup to fail.

  What would prevent this failure is a "wait" option on both fs_setup
  and disk_setup, which if true, will cause cloud-init to wait until the
  device exists (caused by AWS catching up and attaching the device)
  before continuing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1832645/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1844191] [NEW] azure advanced networking sometimes triggers duplicate mac detection

[Ryan Harper]

Public bug reported:

Hi, we're still being affected by this on Azure with 19.2-24-ge7881d5c-
0ubuntu1~18.04.1 - using PACKER to build from image: BuildSource :
Marketplace/Canonical/UbuntuServer/18.04-DAILY-LTS

Here is the packer config:

"provisioners": [
{
  "type": "shell",
  "inline": [
"while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 
'Waiting for cloud-init...'; sleep 1; done"
  ]
},
{
"type": "ansible",
"playbook_file": "{{user `ansible_playbook`}}",
"user": "packer",
"extra_arguments": [ "--extra-vars", "codeVersion={{user 
`code_version`}} managed_image_name={{user `managed_image_name`}}" ]
},
{
"type": "shell",
"execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh 
'{{ .Path }}'",
"inline_shebang": "/bin/sh -x",
"inline": [ "/usr/sbin/waagent -force -deprovision+user && export 
HISTSIZE=0 && sync" ]
}]


Here is the playbook:

---
- hosts: all
  remote_user: ubuntu
  become: yes
  become_method: sudo
  become_user: root

  environment:
DEBIAN_FRONTEND: noninteractive


Note: we are applying `enableAcceleratedNetworking: true` to the NIC,
anecdotally we think this is related.

Usually our playbook has more in it (obviously) but Azure kept pointing
fingers at us that our image was causing the problem, so I ran this test
simply deploying a blank deprovisioned image via our same process.

And here's what happens on the serial console log:


[   20.337603] sh[910]: + [ -e /var/lib/cloud/instance/obj.pkl ]
[   20.343177] sh[910]: + echo cleaning persistent cloud-init object
[   20.349027] [  OK  ] Started Network Time Synchronization.
[  OK  ] Reached target System Time Synchronized.
sh[910]: cleaning persistent cloud-init object
[   20.361066] sh[910]: + rm /var/lib/cloud/instance/obj.pkl
[   20.412333] sh[910]: + exit 0
[   34.282291] cloud-init[938]: Cloud-init v. 
19.2-24-ge7881d5c-0ubuntu1~18.04.1 running 'init-local' at Mon, 16 Sep 2019 
18:02:23 +. Up 32.02 seconds.
[   34.288809] cloud-init[938]: 2019-09-16 18:02:25,262 - util.py[WARNING]: 
failed stage init-local
[   34.423057] cloud-init[938]: failed run of stage init-local
[   34.437716] cloud-init[938]: 

[   34.441088] cloud-init[938]: Traceback (most recent call last):
[   34.443719] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 653, in 
status_wrapper
[   34.448072] cloud-init[938]: ret = functor(name, args)
[   34.450532] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in main_init
[   34.454849] cloud-init[938]: 
init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
[   34.458725] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 697, in 
apply_network_config
[   34.463421] cloud-init[938]: net.wait_for_physdevs(netcfg)
[   34.466051] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 344, in 
wait_for_physdevs
[   34.470673] cloud-init[938]: present_macs = 
get_interfaces_by_mac().keys()
[   34.473964] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 633, in 
get_interfaces_by_mac
[   34.479325] cloud-init[938]: (name, ret[mac], mac))
[   34.481838] cloud-init[938]: RuntimeError: duplicate mac found! both 'eth0' 
and 'enP1s1' have mac '00:0d:3a:7c:f7:3f'
[   34.486614] cloud-init[938]: 

[FAILED] Failed to start Initial cloud-init job (pre-networking).
See 'systemctl status cloud-init-local.service' for details.
[  OK  ] Reached target Network (Pre).
 Starting Network Service...
[  OK  ] Started Network Service.
 Starting Wait for Network to be Configured...
 Starting Network Name Resolution...
[  OK  ] Started Wait for Network to be Configured.
 Starting Initial cloud-init job (metadata service crawler)...
[  OK  ] Started Network Name Resolution.
[  OK  ] Reached target Host and Network Name Lookups.
[  OK  ] Reached target Network.


When this happens, the machine never boots, and we get an
OSProvisioningTimedOut error after about 30 minutes, and the machine
never reaches healthy state.

** Affects: cloud-init
 Importance: High
 Status: Triaged

** Changed in: cloud-init
   Importance: Undecided => High

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1844191

Title:
  azure advanced networking sometimes triggers duplicate mac detection

Status in cloud-init:
  Triaged

Bug description:
  Hi, we're still being affected by this on Azure with 1

[Yahoo-eng-team] [Bug 1843634] Re: cloud-init misconfigure the network on SLES

[Ryan Harper]

OK, thanks for the logs.  Could you re-attach those running via sudo (or
as root)?  The default user on SLES does not have permissions to read
the journal.

What I see so far looks like networking did not come up after cloud-
init-local.service completes and writes out a network config.

2019-09-11 18:00:15,242 - stages.py[INFO]: Applying network
configuration from ds bringup=False: {'ethernets': {'eth0': {'set-name':
'eth0', 'match': {'macaddress': u'00:0d:3a:6e:6f:8f'}, 'dhcp4': True}},
'version': 2}

This results in the following files being written:

% cat test_azure_sles/etc/sysconfig/network/ifcfg-eth0 
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=dhcp
DEVICE=eth0
HWADDR=00:0d:3a:6e:6f:8f
NM_CONTROLLED=no
ONBOOT=yes
STARTMODE=auto
TYPE=Ethernet
USERCTL=no

Upstream cloud-init on SLES does not generate/update /etc/resolv.conf
but in the logs the cloud-init in does:

2019-09-11 18:00:15,246 - util.py[DEBUG]: Writing to 
/etc/sysconfig/network/ifcfg-eth0 - wb: [644] 191 bytes
2019-09-11 18:00:15,247 - util.py[DEBUG]: Reading from /etc/resolv.conf 
(quiet=False)
2019-09-11 18:00:15,247 - util.py[DEBUG]: Read 795 bytes from /etc/resolv.conf
2019-09-11 18:00:15,247 - util.py[DEBUG]: Writing to /etc/resolv.conf - wb: 
[644] 866 bytes

At first, I thought maybe it was missing this commit:

% git show b74ebca563a21332b29482c8029e7908f60225a4
commit b74ebca563a21332b29482c8029e7908f60225a4
Author: Robert Schweikert 
Date:   Wed Jan 23 22:35:32 2019 +

net/sysconfig: do not write a resolv.conf file with only the header.

Writing the file with no dns information may prevent distro tools
from writing a resolv.conf file with dns information obtained from
a dhcp server.

diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index ae41f7b..fd8e501 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -557,6 +557,8 @@ class Renderer(renderer.Renderer):
 content.add_nameserver(nameserver)
 for searchdomain in network_state.dns_searchdomains:
 content.add_search_domain(searchdomain)
+if not str(content):
+return None
 header = _make_header(';')
 content_str = str(content)
 if not content_str.startswith(header):
@@ -666,7 +668,8 @@ class Renderer(renderer.Renderer):
 dns_path = util.target_path(target, self.dns_path)
 resolv_content = self._render_dns(network_state,
   existing_dns_path=dns_path)
-util.write_file(dns_path, resolv_content, file_mode)
+if resolv_content:
+util.write_file(dns_path, resolv_content, file_mode)
 if self.networkmanager_conf_path:
 nm_conf_path = util.target_path(target,
 self.networkmanager_conf_path)
diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
index d679e92..5313d2d 100644
--- a/tests/unittests/test_net.py
+++ b/tests/unittests/test_net.py
@@ -2098,6 +2098,10 @@ TYPE=Ethernet
 USERCTL=no
 """
 self.assertEqual(expected, found[nspath + 'ifcfg-interface0'])
+# The configuration has no nameserver information make sure we
+# do not write the resolv.conf file
+respath = '/etc/resolv.conf'
+self.assertNotIn(respath, found.keys())
 
 def test_config_with_explicit_loopback(self):
 ns = network_state.parse_net_config_data(CONFIG_V1_EXPLICIT_LOOPBACK)
@@ -2456,6 +2460,10 @@ TYPE=Ethernet
 USERCTL=no
 """
 self.assertEqual(expected, found[nspath + 'ifcfg-interface0'])
+# The configuration has no nameserver information make sure we
+# do not write the resolv.conf file
+respath = '/etc/resolv.conf'
+self.assertNotIn(respath, found.keys())
 
 def test_config_with_explicit_loopback(self):
 ns = network_state.parse_net_config_data(CONFIG_V1_EXPLICIT_LOOPBACK)

But, I believe that is in 19.1 (or likely patched in the distro version).
Later in the boot, we can see that networking didn't actually come up as Azure 
datasource can't find a lease file and then goes into some sort of fallback 
mode which tries to bring up networking (it does) but not with dhcp which is 
why you're missing DNS (it's provided via option to the DHCP response.


2019-09-11 18:00:15,946 - azure.py[DEBUG]: Unable to find endpoint in dhclient 
logs.  Falling back to check lease files
2019-09-11 18:00:15,946 - azure.py[DEBUG]: Looking for endpoint in lease file 
/var/lib/dhcp/dhclient.eth0.leases
2019-09-11 18:00:15,946 - handlers.py[DEBUG]: start: 
azure-ds/_get_value_from_leases_file: _get_value_from_leases_file
2019-09-11 18:00:15,946 - util.py[DEBUG]: Reading from 
/var/lib/dhcp/dhclient.eth0.leases (quiet=False)
2019-09-11 18:00:15,947 - azure.py[ERROR]: Failed to read 
/var/lib/dhcp/dhclient.eth0.leases: [Errno 2] No such file or directory: 
'/var/lib/dhcp/dhclient.e

[Yahoo-eng-team] [Bug 1843502] Re: Network config is incorrectly parsed when nameservers are specified

[Ryan Harper]

Issue is related to local changes, marking invalid.

** Changed in: cloud-init
   Status: Incomplete => Invalid

** Changed in: cloud-init (Suse)
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1843502

Title:
  Network config is incorrectly parsed when  nameservers are specified

Status in cloud-init:
  Invalid
Status in cloud-init package in Suse:
  Invalid

Bug description:
  The issue was reproduced on Azure with cloud-init 19.1 on a SLES12 SP4 
machine. Looking at the code, the same behavior could be reproduced on any 
other configuration where the cloud provider specifies nameservers in the 
network configuration.
  The specified nameservers in network configuration are ignored and cloud-init 
raises an error.
  In network_state.py the function _v2_common builds a name_cmd dictionary 
which is then passed to the function handle_nameserver. The handle_nameserver 
has a decorator that enforces that passed in dictionary to have the key 
"address". But the _v2_common build a dictionary that has the key "addresses" 
instead. That results in raising an error.
  Here's a snapshot of the cloud-init.log

  2019-09-09 16:21:29,479 - network_state.py[DEBUG]: v2(nameserver) -> 
v1(nameserver):
  {'search': 'xkf00b0rtzgejkug4xc2pcinre.xx.internal.cloudapp.net', 'type': 
'nameserver', 'addresses': '168.63.129.16'}
  2019-09-09 16:21:29,479 - network_state.py[WARNING]: Skipping invalid 
command: {'nameservers': {'search': 
'xkf00b0rtzgejkug4xc2pcinre.xx.internal.cloudapp.net', 'addresses': 
'168.63.129.16'}, 'eth0': {'set-name': 'eth0', 'match': {'macaddress': 
u'00:0d:3a:6d:ca:25'}, 'dhcp4': True}}
  Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 321, in parse_config_v2
  self._v2_common(command)
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 697, in _v2_common
  self.handle_nameserver(name_cmd)
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 118, in decorator
  required_keys))
  InvalidCommand: Command missing set(['address']) of required keys ['address']

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1843502/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1843502] Re: Network config is incorrectly parsed when nameservers are specified

[Ryan Harper]

Thanks for the bug and the logs.  Looking at the network-config that was
generated:

>>> print(yaml.dump(nc, default_flow_style=False, indent=4))
ethernets:
eth0:
dhcp4: true
match:
macaddress: 00:0d:3a:6d:ca:25
set-name: eth0
nameservers:
addresses: 168.63.129.16
search: xkf00b0rtzgejk

The bug is that nameservers needs to be indented *under* eth0.

However, cloud-init upstream does not parse or process nameservers[1]
from Azure metadata, so I can't understand why you have this bug unless
the cloud-init 19.1 on SLES has some downstream patches.

1. https://git.launchpad.net/cloud-
init/tree/cloudinit/sources/DataSourceAzure.py#n1305


** Changed in: cloud-init
   Status: New => Incomplete

** Also affects: cloud-init (Suse)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1843502

Title:
  Network config is incorrectly parsed when  nameservers are specified

Status in cloud-init:
  Incomplete
Status in cloud-init package in Suse:
  New

Bug description:
  The issue was reproduced on Azure with cloud-init 19.1 on a SLES12 SP4 
machine. Looking at the code, the same behavior could be reproduced on any 
other configuration where the cloud provider specifies nameservers in the 
network configuration.
  The specified nameservers in network configuration are ignored and cloud-init 
raises an error.
  In network_state.py the function _v2_common builds a name_cmd dictionary 
which is then passed to the function handle_nameserver. The handle_nameserver 
has a decorator that enforces that passed in dictionary to have the key 
"address". But the _v2_common build a dictionary that has the key "addresses" 
instead. That results in raising an error.
  Here's a snapshot of the cloud-init.log

  2019-09-09 16:21:29,479 - network_state.py[DEBUG]: v2(nameserver) -> 
v1(nameserver):
  {'search': 'xkf00b0rtzgejkug4xc2pcinre.xx.internal.cloudapp.net', 'type': 
'nameserver', 'addresses': '168.63.129.16'}
  2019-09-09 16:21:29,479 - network_state.py[WARNING]: Skipping invalid 
command: {'nameservers': {'search': 
'xkf00b0rtzgejkug4xc2pcinre.xx.internal.cloudapp.net', 'addresses': 
'168.63.129.16'}, 'eth0': {'set-name': 'eth0', 'match': {'macaddress': 
u'00:0d:3a:6d:ca:25'}, 'dhcp4': True}}
  Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 321, in parse_config_v2
  self._v2_common(command)
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 697, in _v2_common
  self.handle_nameserver(name_cmd)
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 118, in decorator
  required_keys))
  InvalidCommand: Command missing set(['address']) of required keys ['address']

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1843502/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1843602] [NEW] cloud-init collect logs needs to defer to distro for debug data

[Ryan Harper]

Public bug reported:

While triaging a bug in sles'based distro using cloud-init, the collect-
logs output had a few issues:

1) the user did not have systemd journal privs

Unexpected error while running command.
Command: ['journalctl', '--boot=0', '-o', 'short-precise']
Exit code: 1
Reason: -
Stdout: 
Stderr: Hint: You are currently not seeing messages from other users and the 
system.
  Users in the 'systemd-journal' group can see all messages. Pass 
-q to
  turn off this notice.
No journal files were opened due to insufficient permissions.

2) we tried to collect dpkg-version on a non debian distro

Unexpected error while running command.
Command: ['dpkg-query', '--show', '-f=${Version}\n', 'cloud-init']
Exit code: -
Reason: [Errno 2] No such file or directory
Stdout: -
Stderr: -

3) version file is empty

4) dmesg as non-root user fails

Unexpected error while running command.
Command: ['dmesg']
Exit code: 1
Reason: -
Stdout: 
Stderr: dmesg: read kernel buffer failed: Operation not permitted

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1843602

Title:
  cloud-init collect logs needs to defer to distro for debug data

Status in cloud-init:
  New

Bug description:
  While triaging a bug in sles'based distro using cloud-init, the
  collect-logs output had a few issues:

  1) the user did not have systemd journal privs

  Unexpected error while running command.
  Command: ['journalctl', '--boot=0', '-o', 'short-precise']
  Exit code: 1
  Reason: -
  Stdout: 
  Stderr: Hint: You are currently not seeing messages from other users and the 
system.
Users in the 'systemd-journal' group can see all messages. Pass 
-q to
turn off this notice.
  No journal files were opened due to insufficient permissions.

  2) we tried to collect dpkg-version on a non debian distro

  Unexpected error while running command.
  Command: ['dpkg-query', '--show', '-f=${Version}\n', 'cloud-init']
  Exit code: -
  Reason: [Errno 2] No such file or directory
  Stdout: -
  Stderr: -

  3) version file is empty

  4) dmesg as non-root user fails

  Unexpected error while running command.
  Command: ['dmesg']
  Exit code: 1
  Reason: -
  Stdout: 
  Stderr: dmesg: read kernel buffer failed: Operation not permitted

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1843602/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1839860] Re: cloud-init error while MAAS commissioning (PXE phase) P9 Witherspoon

[Ryan Harper]

Looks like there's nothing for cloud-init here.  Please reopen the
cloud-init task if that changes.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1839860

Title:
  cloud-init error while MAAS commissioning (PXE phase) P9 Witherspoon

Status in cloud-init:
  Invalid
Status in MAAS:
  Incomplete
Status in The Ubuntu-power-systems project:
  New

Bug description:
  While trying to commissioning bobone (P9 withersppon machine with
  OpenBMC in Server team's MAAS) the PXE phase ended with the following
  cloud-init error (shown at sol):

   Starting Wait until snapd is fully seeded...

  Ubuntu 18.04.3 LTS ubuntu hvc0

  ubuntu login: [  131.162174] cloud-init[5497]: Can not apply stage config, no 
datasource found! Likely bad things to come!
  [  131.162320] cloud-init[5497]: 

  [  131.162414] cloud-init[5497]: Traceback (most recent call last):
  [  131.162512] cloud-init[5497]:   File 
"/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 485, in 
main_modules
  [  131.162614] cloud-init[5497]: init.fetch(existing="trust")
  [  131.162678] cloud-init[5497]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 351, in fetch
  [  131.162776] cloud-init[5497]: return 
self._get_data_source(existing=existing)
  [  131.162851] cloud-init[5497]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 261, in 
_get_data_source
  [  131.162934] cloud-init[5497]: pkg_list, self.reporter)
  [  131.163005] cloud-init[5497]:   File 
"/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 741, in 
find_source
  [  131.163104] cloud-init[5497]: raise DataSourceNotFoundException(msg)
  [  131.163177] cloud-init[5497]: 
cloudinit.sources.DataSourceNotFoundException: Did not find any data source, 
searched classes: ()
  [  131.163269] cloud-init[5497]: 

  [  131.53] cloud-init[5551]: Can not apply stage final, no datasource 
found! Likely bad things to come!
  [  131.566820] cloud-init[5551]: 

  [  131.566922] cloud-init[5551]: Traceback (most recent call last):
  [  131.567004] cloud-init[5551]:   File 
"/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 485, in 
main_modules
  [  131.567116] cloud-init[5551]: init.fetch(existing="trust")
  [  131.567193] cloud-init[5551]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 351, in fetch
  [  131.567274] cloud-init[5551]: return 
self._get_data_source(existing=existing)
  [  131.567348] cloud-init[5551]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 261, in 
_get_data_source
  [  131.567438] cloud-init[5551]: pkg_list, self.reporter)
  [  131.567508] cloud-init[5551]:   File 
"/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 741, in 
find_source
  [  131.567598] cloud-init[5551]: raise DataSourceNotFoundException(msg)
  [  131.567679] cloud-init[5551]: 
cloudinit.sources.DataSourceNotFoundException: Did not find any data source, 
searched classes: ()
  [  131.567779] cloud-init[5551]: 


  Ubuntu 18.04.3 LTS ubuntu hvc0

  ubuntu login:


  MAAS log (from UI):

  TIMEEVENT
    Mon, 12 Aug. 2019 11:10:45 PXE Request - commissioning
    Mon, 12 Aug. 2019 11:08:43 Node powered on
    Mon, 12 Aug. 2019 11:08:14 Powering node on
    Mon, 12 Aug. 2019 11:08:14 Node - Started commissioning on 'bobone'.
    Mon, 12 Aug. 2019 11:08:14 Node changed status - From 'New' to 
'Commissioning'
    Mon, 12 Aug. 2019 11:08:14 User starting node commissioning - (jfh)
    Mon, 12 Aug. 2019 11:07:04 Node powered off


  With that system cannot complete the Commissioning phase.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1839860/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1839491] Re: Manually performed partitioning changes get reverted on reboot

[Ryan Harper]

Moving cloud-init task to invalid, no bug/work for cloud-init.

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1839491

Title:
  Manually performed partitioning changes get reverted on reboot

Status in cloud-init:
  Invalid
Status in MAAS:
  Triaged

Bug description:
  Hello,

  I am facing an issue where I need to make changes to the initially
  deployed partition layout, but upon making those changes and
  rebooting, the partition layout gets reverted.

  My env:
  MAAS version: 2.6.0 (7802-g59416a869-0ubuntu1~18.04.1)
  System vendor: HP
  System product: ProLiant DL360 Gen9 (780021-S01)
  System version: Unknown
  Mainboard product: ProLiant DL360 Gen9
  Mainboard firmware version: P89
  Mainboard firmware date: 12/27/2015
  CPU model: Intel(R) Xeon(R) CPU E5-2690 v3
  Deployed (16.04 LTS "Xenial Xerus")
  Kernel: xenial (ga-16.04)
  Power type: ipmi
  Power driver: LAN_2_0 [IPMI 2.0]
  Power boot type: EFI boot
  Architecture amd64/generic
  Minimum Kernel: no minimum kernel
  Interfaces: eno1, eno2, noe3, eno4, eno49, eno50. Only eno49 is used.
  Storage: sda Physical 1TB, sdb Physical 1TB.

  
  Steps to reproduce:

  1. Deploy MAAS with the following partition configuration:
  sda-part1 536.9 MB Partition fat32 formatted filesystem mounted at /boot/efi
  sda-part2 100.0 GB Partition ext4 formatted filesystem mounted at /

  2. Check the partitions on the node:

  $ lsblk

  NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
  sda  8:00 931.5G  0 disk 
  |-sda1   8:10   512M  0 part /boot/efi
  `-sda2   8:20   931G  0 part /
  sdb  8:16   0 931.5G  0 disk 

  
  Here we notice the initial partitioning scheme is not respected. This could 
be related to the main issue of partitioning changes being reverted, but could 
also be a separate issue.

  3. Boot an ubuntu ISO and go into rescue mode. I used ubuntu-16.04.6
  -server-amd64.iso

  4. Choose "Do not use a root filesystem" and "Execute a shell in the
  installer environment".

  4. Run the following commands:

  $ e2fsck -f /dev/sda2

  $ resize2fs /dev/sda2 150G

  $ e2fsck -f /dev/sda2

  $ sudo parted /dev/sda

  (parted) unit GiB print

  (parted) resizepart

  Partition number? 2

  End? 200GiB

  (parted) print

  You should see partition 2 resized.

  (parted) quit

  $ e2fsck -f /dev/sda2

  5. Confirm

  $ fdisk -l

  6. Sync writes

  $ sync

  7. Reboot the node. Remove ISO image.

  8. Let system boot, check partitions again:

  $ lsblk

  NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
  sda  8:00 931.5G  0 disk 
  |-sda1   8:10   512M  0 part /boot/efi
  `-sda2   8:20   931G  0 part /
  sdb  8:16   0 931.5G  0 disk 

  We can see see that the changes were reverted.

  If I remove cloud-init, I can successfully re-partition and reboot,
  without the changes being reverted.

  Attached logs before and after repartition.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1839491/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1757054] Re: OpenStack datasource does not deal with 404 Not Found correctly

[Ryan Harper]

*** This bug is a duplicate of bug 1702160 ***
https://bugs.launchpad.net/bugs/1702160

** This bug has been marked a duplicate of bug 1702160
   OpenStack datasource should not retry user-data on 404

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1757054

Title:
  OpenStack datasource does not deal with 404 Not Found correctly

Status in cloud-init:
  New

Bug description:
  OpenStack datasource does not deal with 404 Not Found correctly.If we
  boot a vm without userdata in OpenStack environment, CloudInit retry
  to get usedata when 404 Not Found returned. The GuestOS start up times
  will increase.I think it's not needed and a waste of time.

  Cloud-Init LOG:

  2018-03-19 11:09:16,391 - url_helper.py[DEBUG]: [0/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:16,414 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:17,415 - url_helper.py[DEBUG]: [1/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:17,445 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:18,446 - url_helper.py[DEBUG]: [2/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:18,468 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:19,470 - url_helper.py[DEBUG]: [3/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:19,535 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:20,536 - url_helper.py[DEBUG]: [4/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:20,556 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:21,557 - url_helper.py[DEBUG]: [5/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:21,584 - openstack.py[DEBUG]: Failed reading optional path 
http://169.254.169.254:80/openstack/2015-10-15/user_data due to: 404 Client 
Error: Not Found

  Go through the code, I found the following logic which i think it's not 
correct.
  cloudinit/sources/helpers/openstack.py
  def should_retry_cb(_request_args, cause):
  try:
  code = int(cause.code)
  if code >= 400:
  return False
  except (TypeError, ValueError):
  # Older versions of requests didn't have a code.
  pass
  return True

  response = url_helper.readurl(path,
     retries=self.retries,
     ssl_details=self.ssl_details,
     timeout=self.timeout,
      exception_cb=should_retry_cb)

  1.When 404 returned, do not need to retry.
  2.exception_cb=should_retry_cb need to be changed to 
exception_cb=should_skip_retry_cb.  May be a Bug here:)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1757054/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1758339] Re: cloud-init init-local tries to get data over network

[Ryan Harper]

Please reopen (mark as new) if you feel there's something that cloud-
init is not doing correctly.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1758339

Title:
  cloud-init init-local tries to get data over network

Status in cloud-init:
  Invalid

Bug description:
  init-local tries to gather data from metadata service before the
  network is ready which causes a WARNING message sent to syslog. I
  believe everything ends up being successful and it's just some noise
  in syslog during instance launch.

  Mar 23 04:18:27 task-302 systemd[1]: Starting Initial cloud-init job 
(pre-networking)...
  Mar 23 04:18:33 task-302 cloud-init[399]: Cloud-init v. 17.2 running 
'init-local' at Fri, 23 Mar 2018 04:18:28 +. Up 4.98 seconds.
  Mar 23 04:18:33 task-302 cloud-init[399]: 2018-03-23 04:18:33,395 - 
util.py[WARNING]: Failed fetching dynamic/instance-identity from url 
http://169.254.169.254/2009-04-04/dynamic/instance-identity
  Mar 23 04:18:33 task-302 systemd[1]: Started Initial cloud-init job 
(pre-networking).
  Mar 23 04:18:34 task-302 systemd[1]: Starting Initial cloud-init job 
(metadata service crawler)...
  Mar 23 04:18:34 task-302 cloud-init[989]: Cloud-init v. 17.2 running 'init' 
at Fri, 23 Mar 2018 04:18:34 +. Up 11.03 seconds.
  Mar 23 04:18:34 task-302 cloud-init[989]: ci-info: 
++Net device 
info++
  Mar 23 04:18:34 task-302 cloud-init[989]: ci-info: 
++--+-+---+---+---+
  Mar 23 04:18:34 task-302 cloud-init[989]: ci-info: | Device |  Up  |  
 Address   |  Mask | Scope | Hw-Address|
  Mar 23 04:18:34 task-302 cloud-init[989]: ci-info: 
++--+-+---+---+---+

  cloud-init version
  cloud-init -v
  /usr/bin/cloud-init 17.2
  ii  cloud-init   17.2-35-gf576b2a2-0ubuntu1~16.04.2   
   all  Init scripts for cloud instances

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1758339/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1758409] Re: integration tests: restructure ssh timeout

[Ryan Harper]

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1758409

Title:
  integration tests: restructure ssh timeout

Status in cloud-init:
  Fix Released

Bug description:
  # Summary
  During the integration tests, currently if SSH to instance times out it holds 
up testing for over an hour in an attempt to SSH to an instance; note the 
timestamp jump on: https://paste.ubuntu.com/p/NBQKwm9wdG/

  The _ssh_connect function was originally written for the nocloud_kvm
  platform and used as a method for determining if an instance was up
  and accessible. As such, the function is doing double duty and not
  correctly focused on SSH'ing to an up and running instance and has a
  bug in it as it is waiting far too long.

  # Action plan

  1. For the nocloud_kvm platform when when starting and before
  _wait_for_system, there should be a check if an instance is accessible
  during the is_running check. This could be done again by SSH with a
  number of retries, but should be taken care of inside the nocloud_kvm
  platform itself and not in the SSH connect function.

  2. Update the _ssh_connect to timeout quickly, reduce wait on banner,
  and only retry up to 3 times.

  # Noted Files
  tests/cloud_tests/platforms/platforms.py:_ssh_connect()
  tests/cloud_tests/platforms/nocloudkvm/instance.py:start()

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1758409/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1768468] Re: error in ca_certs module

[Ryan Harper]

I'm marking the cloud-init task invalid; I don't believe cloud-init did
anything wrong; but please set the task back to New if you have new
information showing that cloud-init didn't do something quite right.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1768468

Title:
  error in ca_certs module

Status in cloud-init:
  Invalid
Status in ca-certificates package in Ubuntu:
  New

Bug description:
  Hello,
  I'm using cloud-init in order to customize Nutanix AHV images.
  I'm using the ca_certs module to upload our private ca chain (root and 
intermediate) but in the log I found a log that states that
  -
  Cloud-init v. 18.2 running 'init-local' at Wed, 02 May 2018 08:54:58 +. 
Up 6.25 seconds.
  Updating certificates in /etc/ssl/certs...
  rehash: skipping cloud-init-ca-certs.pem,it does not contain exactly one 
certificate or CRL
  1 added, 0 removed; done.
  Running hooks in /etc/ca-certificates/update.d...
  done.
  -
  And the certificates are not trusted.
  Ubuntu version is 18.04.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1768468/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1767002] Re: When stop(deallocate) and then start a RHEL 7.5 VM, cloud-init cannot mount /dev/sdb1 to /mnt.

[Ryan Harper]

I'm closing this bug as Fixed Released.

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1767002

Title:
  When stop(deallocate) and then start a RHEL 7.5 VM, cloud-init cannot
  mount /dev/sdb1 to /mnt.

Status in cloud-init:
  Fix Released

Bug description:
  When stop(deallocate) and then start a RHEL VM, cloud-init cannot
  mount /dev/sdb1 to /mnt.

  You see this error:

  /var/log/cloud-init.log:
  2018-03-23 06:30:52,018 - util.py[DEBUG]: Running command ['mount', '-o', 
'ro,sync', '-t', 'auto', '/dev/disk/cloud/azure_resource-part1', 
'/tmp/tmp9iIt2n'] with allowed return codes [0] (shell=False, capture=True)
  2018-03-23 06:30:52,038 - util.py[DEBUG]: Failed mount of 
'/dev/disk/cloud/azure_resource-part1' as 'auto': Unexpected error while 
running command.
  Command: ['mount', '-o', 'ro,sync', '-t', 'auto', 
'/dev/disk/cloud/azure_resource-part1', '/tmp/tmp9iIt2n']
  Exit code: 32
  Reason: -
  Stdout: -
  Stderr: mount: unknown filesystem type 'ntfs'
  2018-03-23 06:30:52,038 - util.py[DEBUG]: Recursively deleting /tmp/tmp9iIt2n
  2018-03-23 06:30:52,038 - DataSourceAzure.py[DEBUG]: reformattable=False: 
partition 1 (/dev/disk/cloud/azure_resource-part1 -> /dev/sdb1) on device 
/dev/disk/cloud/azure_resource was ntfs formatted but mount of 
/dev/disk/cloud/azure_resource-part1 failed: Failed mounting 
/dev/disk/cloud/azure_resource-part1 to /tmp/tmp9iIt2n due to: Unexpected error 
while running command.
  Command: ['mount', '-o', 'ro,sync', '-t', 'auto', 
'/dev/disk/cloud/azure_resource-part1', '/tmp/tmp9iIt2n']
  Exit code: 32
  Reason: -
  Stdout: -
  Stderr: mount: unknown filesystem type 'ntfs'

  This has been patched here: https://code.launchpad.net/~paul-meyer
  /cloud-init/+git/cloud-init/+ref/fix-ntfs-mount.

  Thanks,

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1767002/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1768468] Re: error in ca_certs module

[Ryan Harper]

Hi,  I don't believe this is a cloud-init bug.  Cloud-init wrote the two
certificate files requested.

2018-05-02 08:55:00,913 - stages.py[DEBUG]: Running module ca-certs () with 
frequency once-per-instance
2018-05-02 08:55:00,914 - handlers.py[DEBUG]: start: 
init-network/config-ca-certs: running config-ca-certs with frequency 
once-per-instance
2018-05-02 08:55:00,914 - util.py[DEBUG]: Writing to 
/var/lib/cloud/instances/44574988-144d-485a-9386-2ec4f4f545c4/sem/config_ca_certs
 - wb: [644] 24 bytes
2018-05-02 08:55:00,914 - helpers.py[DEBUG]: Running config-ca-certs using lock 
()
2018-05-02 08:55:00,914 - cc_ca_certs.py[DEBUG]: Adding 2 certificates
2018-05-02 08:55:00,916 - util.py[DEBUG]: Writing to 
/usr/share/ca-certificates/cloud-init-ca-certs.crt - wb: [644] 4545 bytes
2018-05-02 08:55:00,917 - util.py[DEBUG]: Reading from 
/etc/ca-certificates.conf (quiet=False)
2018-05-02 08:55:00,917 - util.py[DEBUG]: Read 5898 bytes from 
/etc/ca-certificates.conf
2018-05-02 08:55:00,917 - util.py[DEBUG]: Writing to /etc/ca-certificates.conf 
- wb: [644] 5922 bytes
2018-05-02 08:55:00,918 - cc_ca_certs.py[DEBUG]: Updating certificates
2018-05-02 08:55:00,918 - util.py[DEBUG]: Running command 
['update-ca-certificates'] with allowed return codes [0] (shell=False, 
capture=False)
2018-05-02 08:55:01,845 - handlers.py[DEBUG]: finish: 
init-network/config-ca-certs: SUCCESS: config-ca-certs ran successfully

The error you show mentions a 'cloud-init-ca-certs.pem' It may be
related the ca-certificates package?  I'm adding the package as a task.

** Also affects: ca-certificates (Ubuntu)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1768468

Title:
  error in ca_certs module

Status in cloud-init:
  Invalid
Status in ca-certificates package in Ubuntu:
  New

Bug description:
  Hello,
  I'm using cloud-init in order to customize Nutanix AHV images.
  I'm using the ca_certs module to upload our private ca chain (root and 
intermediate) but in the log I found a log that states that
  -
  Cloud-init v. 18.2 running 'init-local' at Wed, 02 May 2018 08:54:58 +. 
Up 6.25 seconds.
  Updating certificates in /etc/ssl/certs...
  rehash: skipping cloud-init-ca-certs.pem,it does not contain exactly one 
certificate or CRL
  1 added, 0 removed; done.
  Running hooks in /etc/ca-certificates/update.d...
  done.
  -
  And the certificates are not trusted.
  Ubuntu version is 18.04.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1768468/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1788960] Re: rhevm related cloud-init integration test fail for cloud-init v18.3

[Ryan Harper]

Thanks for filing the bug.  I believe this issue has been resolved in
this commit:

commit a8dcad9ac62bb1d2a4f7489960395bad6cac9382
Author: Scott Moser 
Date:   Wed Sep 5 16:02:25 2018 +

tests: Disallow use of util.subp except for where needed.

In many cases, cloud-init uses 'util.subp' to run a subprocess.
This is not really desirable in our unit tests as it makes the tests
dependent upon existance of those utilities.

The change here is to modify the base test case class (CiTestCase) to
raise exception any time subp is called.  Then, fix all callers.
For cases where subp is necessary or actually desired, we can use it
via
  a.) context hander CiTestCase.allow_subp(value)
  b.) class level self.allowed_subp = value

Both cases the value is a list of acceptable executable names that
will be called (essentially argv[0]).

Some cleanups in AltCloud were done as the code was being updated.


** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1788960

Title:
  rhevm related cloud-init integration test fail for cloud-init v18.3

Status in cloud-init:
  Fix Released

Bug description:
  There are 3 of the rhevm related integration test failure:
  test_mount_cb_fails, test_no_udevadm_cmd test_no_udevadm_cmd, seems
  all related to "user_data_rhevm()"

  
  OS:  Ubuntu 16.04 Xenial
  cloud-init: v18.3 dab59087155d3963849a36b3f63ee662047f708b

  
  log snip:
  peter@U1604CloudinitTools:~/cloud-init$  tox -e citest -- run --verbose \
  > --preserve-data --data-dir ./debug \
  > 
  GLOB sdist-make: /home/peter/cloud-init/setup.py
  citest create: /home/peter/cloud-init/.tox/citest
  citest installdeps: -r/home/peter/cloud-init/integration-requirements.txt
  citest inst: /home/peter/cloud-init/.tox/dist/cloud-init-18.3.zip
  citest installed: 
asn1crypto==0.24.0,bcrypt==3.1.4,boto3==1.5.9,botocore==1.8.50,certifi==2018.8.24,cffi==1.11.5,chardet==3.0.4,cloud-init==18.3,configobj==5.0.6,cryptography==2.3.1,docutils==0.14,idna==2.7,Jinja2==2.10,jmespath==0.9.3,jsonpatch==1.23,jsonpointer==2.0,jsonschema==2.6.0,MarkupSafe==1.0,oauthlib==2.1.0,paramiko==2.4.1,pbr==4.2.0,pyasn1==0.4.4,pycparser==2.18,pylxd==2.2.7,PyNaCl==1.2.1,python-dateutil==2.7.3,python-simplestreams==0.1.0,PyYAML==3.13,requests==2.19.1,requests-toolbelt==0.8.0,requests-unixsocket==0.1.5,s3transfer==0.1.13,six==1.11.0,urllib3==1.23,ws4py==0.5.1
  citest runtests: PYTHONHASHSEED='2994029421'
  citest runtests: commands[0] | /home/peter/cloud-init/.tox/citest/bin/python 
-m tests.cloud_tests run --verbose --preserve-data --data-dir debug

  
  ==
  ERROR: Test user_data_rhevm() where mount_cb fails.
  --
  Traceback (most recent call last):
File 
"/home/peter/cloud-init/tests/unittests/test_datasource/test_altcloud.py", line 
296, in test_mount_cb_fails
  self.assertEqual(False, dsrc.user_data_rhevm())
File "/home/peter/cloud-init/cloudinit/sources/DataSourceAltCloud.py", line 
198, in user_data_rhevm
  (cmd_out, _err) = util.udevadm_settle(exists=floppy_dev, timeout=5)
  TypeError: 'NoneType' object is not iterable
   >> begin captured logging << 
  cloudinit.util: DEBUG: Running command ['true'] with allowed return codes [0] 
(shell=False, capture=True)
  cloudinit.sources.DataSourceAltCloud: DEBUG: Command: true
  Output
  - >> end captured logging << -

  ==
  ERROR: Test user_data_rhevm() with no udevadm command.
  --
  Traceback (most recent call last):
File 
"/home/peter/cloud-init/tests/unittests/test_datasource/test_altcloud.py", line 
324, in test_no_udevadm_cmd
  self.assertEqual(False, dsrc.user_data_rhevm())
File "/home/peter/cloud-init/cloudinit/sources/DataSourceAltCloud.py", line 
198, in user_data_rhevm
  (cmd_out, _err) = util.udevadm_settle(exists=floppy_dev, timeout=5)
  TypeError: 'NoneType' object is not iterable
   >> begin captured logging << 
  cloudinit.util: DEBUG: Running command ['modprobe', 'floppy'] with allowed 
return codes [0] (shell=False, capture=True)
  cloudinit.sources.DataSourceAltCloud: DEBUG: Command: modprobe floppy
  Output
  - >> end captured logging << -

  ==
  ERROR: Test user_data_rhevm() where udevadm fails.
  --
  Traceback (most recent call last):
File 
"/home/peter/cloud-in

[Yahoo-eng-team] [Bug 1806338] Re: pip2 install zbar

[Ryan Harper]

Hi,

I don't think this is related to the cloud-init project so I'm marking
the cloud-init task as invalid.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1806338

Title:
  pip2 install zbar

Status in cloud-init:
  Invalid

Bug description:
  2 . 1@linux-e9a2:~/Downloads/HPLIP_3181136-Latest> su -c "pip2 install zbar"
  Password:
  /usr/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py:83: 
RequestsDependencyWarning: Old version of cryptography ([1, 3, 1]) may cause 
slowdown.
warnings.warn(warning, RequestsDependencyWarning)
  Collecting zbar
Using cached 
https://files.pythonhosted.org/packages/33/54/cc5819efc9ee7e34b60b41e1d2d4753b6dd0c26a41c9a552611f66aa106e/zbar-0.10.tar.bz2
  Installing collected packages: zbar
Running setup.py install for zbar ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, 
tokenize;__file__='/tmp/pip-install-GIARuQ/zbar/setup.py';f=getattr(tokenize, 
'open', open)(__file__);code=f.read().replace('\r\n', 
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record 
/tmp/pip-record-qH93Ju/install-record.txt --single-version-externally-managed 
--compile:
  running install
  running build
  running build_ext
  building 'zbar' extension
  creating build
  creating build/temp.linux-x86_64-2.7
  gcc -pthread -fno-strict-aliasing -fmessage-length=0 
-grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector 
-funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -g 
-DNDEBUG -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 
-fstack-protector -funwind-tables -fasynchronous-unwind-tables 
-fstack-clash-protection -g -DOPENSSL_LOAD_CONF -fwrapv -fPIC 
-I/usr/include/python2.7 -c zbarmodule.c -o 
build/temp.linux-x86_64-2.7/zbarmodule.o
  gcc: error: unrecognized command line option ‘-fstack-clash-protection’
  gcc: error: unrecognized command line option ‘-fstack-clash-protection’
  error: command 'gcc' failed with exit status 1
  
  
  Command "/usr/bin/python -u -c "import setuptools, 
tokenize;__file__='/tmp/pip-install-GIARuQ/zbar/setup.py';f=getattr(tokenize, 
'open', open)(__file__);code=f.read().replace('\r\n', 
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record 
/tmp/pip-record-qH93Ju/install-record.txt --single-version-externally-managed 
--compile" failed with error code 1 in /tmp/pip-install-GIARuQ/zbar/

  
  su -c "pip2 install tesserocr"
  Password:
  /usr/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py:83: 
RequestsDependencyWarning: Old version of cryptography ([1, 3, 1]) may cause 
slowdown.
warnings.warn(warning, RequestsDependencyWarning)
  Collecting tesserocr
Using cached 
https://files.pythonhosted.org/packages/f8/6d/4e81e041f33a4419e59edcb1dbdf3c56e9393f60f5ef531381bd67a1339b/tesserocr-2.3.1.tar.gz
  Complete output from command python setup.py egg_info:
  Supporting tesseract v3.04.00
  Configs from pkg-config: {'libraries': ['lept', 'tesseract'], 
'cython_compile_time_env': {'TESSERACT_VERSION': 197632}, 'include_dirs': 
['/usr/include']}
  Unable to find pgen, not compiling formal grammar.
  /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown 
distribution option: 'python_requires'
warnings.warn(msg)
  Compiling /tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Plex/Scanners.py 
because it changed.
  Compiling /tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Plex/Actions.py 
because it changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/Scanning.py because it 
changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/Visitor.py because it 
changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/FlowControl.py because 
it changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Runtime/refnanny.pyx because it 
changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/FusedNode.py because it 
changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Tempita/_tempita.py because it 
changed.
  [1/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/FlowControl.py
  [2/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/FusedNode.py
  [3/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/Scanning.py
  [4/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/Visitor.py
  [5/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Plex/Actions.py
  [6/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Plex/Scanners.py
  [7/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.2

[Yahoo-eng-team] [Bug 1645824] Re: NoCloud source doesn't work on FreeBSD

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1645824

Title:
  NoCloud source doesn't work on FreeBSD

Status in cloud-init:
  Fix Released

Bug description:
  Hey guys,

  I'm trying to use cloud-init on FreeBSD using CD to seed metadata, the
  thing is that it had some issues:

  - Mount option 'sync' is not allowed for cd9660 filesystem.
  - I optimized the list of filesystems that needed to be scanned for metadata 
by having three lists (vfat, iso9660, and label list) and then checking against 
them to see which filesystem option needs to be passed to mount command.

  Additionally I'm going to push some changes to FreeBSD cloud-init
  package so it can build last version. I will open another ticket for
  fixing networking in FreeBSD as it doesn't support sysfs
  (/sys/class/net/) by default.

  Thanks!

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1645824/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1669875] Re: identify openstack vmware platform

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1669875

Title:
  identify openstack vmware platform

Status in cloud-init:
  Fix Released
Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  We need a way to identify locally that we are running on openstack
  when inside a guest of VmWare.

  Related bugs:
https://bugs.launchpad.net/cloud-init/+bugs?field.tag=dsid-nova

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1669875/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1821102] Re: Cloud-init should not setup ephemeral ipv4 if apply_network_config is False for OpenStack

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1821102

Title:
  Cloud-init should not setup ephemeral ipv4 if apply_network_config is
  False for OpenStack

Status in cloud-init:
  Fix Released

Bug description:
  As fixed in bug #1749717, cloud-init will attempt to configure an
  ephemeral ipv4 address on the first interface to fetch OpenStack (and
  probably others) networking config via a metadata URL.

  There's a couple of issues with this implementation that affect our
  OpenStack cloud.

  Access to our metadata server on 169.254.169.254 is delivered by an
  additional route delivered by DHCP, which is not configured via cloud-
  init's dhcp.py (that is probably another bug).

  Also, we needed to bump up the timeouts for accessing our metadata, as
  we're a largeish cloud and the defaults were way too low. We actually
  copied the timeout/retry values from the Ec2 Datasource.

  So the result is that users are left waiting for cloud-init-local
  stage to timeout, as the additional route to the metadata server isn't
  configured, which was 2 mins in our config.

  I believe a simple fix for this situation would be to skip the
  ephemeral ipv4 setup if the datastore config has apply_network_config:
  False

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1821102/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1806701] Re: cloud-init may hang OS boot process due to grep for the entire ISO file when it is attached

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1806701

Title:
  cloud-init may hang OS boot process due to grep for the entire ISO
  file when it is attached

Status in cloud-init:
  Fix Released

Bug description:
  We have found in our test for SLES15 with cloud-init installed, if we
  attach a ISO file with the VM before VM is boot, it often takes more
  than 10 minutes to start the SLES OS. Sometimes it failed to start the
  SLES OS at all.

  We've root caused it is due to the  "is_cdrom_ovf()" func of "tools
  /ds-identify".

  In this function, there is the following logic to detect if an ISO contains 
certain string:
  >local idstr="http://schemas.dmtf.org/ovf/environment/1";
  >grep --quiet --ignore-case "$idstr" "${PATH_ROOT}$dev"
  ref: https://git.launchpad.net/cloud-init/tree/tools/ds-identify

  It is trying to grep the who ISO file for a certain string, which causes 
intense IO pressure for the system.
  What is worse is that sometimes the ISO file is large (e.g. >5GB for 
installer DVD) and it is mounted over NFS. The "grep" process often consume 99% 
CPU and seems hang. Then the systemd starts more and more "grep" process which 
smoke the CPU and consumes all the IO bandwidth for the ISO file. Then the 
system may hang for a long time and sometimes failed to start.

  To fix this issue, I suggest that we should not grep for the entire
  ISO file. Rather then we should just check if the file/dir exists with
  os.path.exists().

  
  -debug log snip
  pek2-gosv-16-dhcp180:~ # ps -ef
  UIDPID  PPID  C STIME TTY  TIME CMD
  root 1 0  0 13:32 ?00:00:04 /usr/lib/systemd/systemd 
--switched-root --system --deserialize 24
  …
  root   474 1  0 13:34 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root   482   474  2 13:34 ?00:00:15 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  root  1020 1  0 13:35 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root  1039  1020  1 13:35 ?00:00:07 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  polkitd   1049 1  0 13:37 ?00:00:00 /usr/lib/polkit-1/polkitd 
--no-debug
  root  1051 1  0 13:37 ?00:00:00 /usr/sbin/wickedd --systemd 
--foreground
  root  1052 1  0 13:37 ?00:00:00 
/usr/lib/systemd/systemd-logind
  root  1054 1  0 13:37 ?00:00:00 /usr/sbin/wickedd-nanny 
--systemd --foreground
  root  1073 1  0 13:37 ?00:00:00 /usr/bin/vmtoolsd
  root  1097 1  0 13:37 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root  1110  1097  1 13:37 ?00:00:04 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  root  1304 1  0 13:38 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root  1312  1304  1 13:38 ?00:00:03 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  root  1537 1  0 13:40 ?00:00:00 /usr/bin/plymouth --wait
  root  1613 1  0 13:40 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root  1645  1613  0 13:40 ?00:00:02 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  …

  
  Grep use nearly 100% cpu, system very slow.

  top - 13:46:37 up 26 min,  2 users,  load average: 14.14, 15.03, 10.57
  Tasks: 225 total,   6 running, 219 sleeping,   0 stopped,   0 zombie
  %Cpu(s): 40.1 us, 49.3 sy,  0.0 ni,  0.0 id,  1.4 wa,  0.0 hi,  9.1 si,  0.0 
st
  KiB Mem :  1000916 total,64600 free,   355880 used,   580436 buff/cache
  KiB Swap:  1288168 total,  1285600 free, 2568 used.   492688 avail Mem

PID USER  PR  NIVIRTRESSHR S  %CPU  %MEM TIME+ COMMAND
  4427 root  20   0   40100   3940   3084 R 99.90 0.394   0:27.41 top
  1016 root  20   0  197796   4852   3400 R 99.90 0.485   1:26.44 vmtoolsd
  1723 root  20   07256   1860   1556 D 99.90 0.186   0:28.44 grep
484 root  20   07256   1684   1396 D 99.90 0.168   1:51.22 grep
  1278 root  20   07256   1856   1556 D 99.90 0.185   0:38.44 grep
  1398 root  20   07256   1860   1556 R 99.90 0.186   0:28.53 grep
  1061 root  20   07256   1856   1556 D 99.90 0.185   0:56.62 grep
  -debug log snip

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1806701/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to  

[Yahoo-eng-team] [Bug 1827238] Re: Machines fail to deploy because cloud-init needs to accept both netplan spellings for grat arp

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1827238

Title:
  Machines fail to deploy because cloud-init needs to accept both
  netplan spellings for grat arp

Status in cloud-init:
  Fix Released
Status in curtin:
  Invalid
Status in MAAS:
  Fix Released

Bug description:
  Many nodes failed to boot after installation.

  Here is one example, beartic.

  beartic.
  finishes install: 2019-05-01T10:36:17+00:00 beartic systemd[1]: Stopping 
Unattended Upgrades Shutdown...

  dhcp's after reboot:
  
10.244.40.32/var/log/maas/rsyslog/leafeon/2019-05-01/messages:2019-05-01T10:38:10+00:00
 leafeon dhcpd[21312]: DHCPDISCOVER from 14:02:ec:41:c7:dc via broam
  
10.244.40.32/var/log/maas/rsyslog/leafeon/2019-05-01/messages:2019-05-01T10:38:10+00:00
 leafeon dhcpd[21312]: DHCPOFFER on 10.244.41.7 to 14:02:ec:41:c7:dc via broam
  
10.244.40.32/var/log/maas/rsyslog/leafeon/2019-05-01/messages:2019-05-01T10:38:13+00:00
 leafeon dhcpd[21312]: DHCPREQUEST for 10.244.41.7 (10.244.40.30) from 
14:02:ec:41:c7:dc via broam
  
10.244.40.32/var/log/maas/rsyslog/leafeon/2019-05-01/messages:2019-05-01T10:38:13+00:00
 leafeon dhcpd[21312]: DHCPACK on 10.244.41.7 to 14:02:ec:41:c7:dc via broam

  grub and grub.cfg:
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:13 
provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 
10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:13 
provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 
10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:14 
provisioningserver.rackdservices.tftp: [info] grubx64.efi requested by 
10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/command.lst 
requested by 10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/fs.lst requested 
by 10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/crypto.lst 
requested by 10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/terminal.lst 
requested by 10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg requested by 
10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-14:02:ec:41:c7:dc 
requested by 10.244.41.7

  but we never got any rsyslog message or api calls after that.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1827238/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1812857] Re: RuntimeError: duplicate mac found! both 'ens4' and 'bond0' have mac '9c:XX:XX:46:5d:91'

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1812857

Title:
  RuntimeError: duplicate mac found! both 'ens4' and 'bond0' have mac
  '9c:XX:XX:46:5d:91'

Status in cloud-init:
  Fix Released

Bug description:
  2019-01-22 13:58:22,667 - util.py[DEBUG]: failed stage init   
   
  Traceback (most recent call last):
   
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 658, in 
status_wrapper   
  ret = functor(name, args) 
   
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in 
main_init
  init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
   
File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 648, in 
apply_network_config   
  netcfg, src = self._find_networking_config()  
   
File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 635, in 
_find_networking_config
  if self.datasource and hasattr(self.datasource, 'network_config'):
   
File 
"/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceConfigDrive.py", 
line 155, in network_config  
  self.network_json, known_macs=self.known_macs)
   
File 
"/usr/lib/python3/dist-packages/cloudinit/sources/helpers/openstack.py", line 
655, in convert_net_json
  known_macs = net.get_interfaces_by_mac()  
   
File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 595, 
in get_interfaces_by_mac
  (name, ret[mac], mac))
   
  RuntimeError: duplicate mac found! both 'ens4' and 'bond0' have mac 
'9c:XX:XX:46:5d:91'  

  
  Net config:
  2019-01-22 13:56:20,055 - stages.py[DEBUG]: applying net config names for 
{'version': 1, 'config': [{'mtu': 1500, 'type': 'bond', 'subnets': [{'type': 
'dhcp4'}], 'params': {'mac_address': '9c:XX:XX:46:5d:91', 'bond_up-delay': 
25, 'bond_mimon': 100, 'bond_xmit_hash_policy': 'layer2+3', 'bond_mode': 
'802.3ad'}, 'name': 'bond0', 'bond_interfaces': ['ens4', 'e
  ns4d1']}, {'type': 'physical', 'mtu': 1500, 'subnets': [{'type': 'dhcp4'}], 
'mac_address': 'f4:XX:XX:44:6f:f0', 'name': 'eno1'}, {'type': 'physical', 
'subnets': [], 'mac_address': '9c:XX:XX:46:5d:91', 'name': 'ens4'}, {'type': 
'physical', 'subnets': [], 'mac_address': '9c:XX:XX:46:5d:92', 'name': 
'ens4d1'}, {'type': 'nameserver', 'address': '8.8.8.8'}]}  

  OS: Ubuntu 18.04.1 LTS
  cloud-init: 18.4-0ubuntu1~18.04.1
  cloud-provider: OpenStack
  Datasource: ConfigDrive

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1812857/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1833192] Re: VMware: post custom script isn't run correctly

[Ryan Harper]

** Also affects: cloud-init
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1833192

Title:
  VMware: post custom script isn't run correctly

Status in cloud-init:
  Fix Released
Status in cloud-init package in Ubuntu:
  New

Bug description:
  In DataSouceOVF.py, it could launch the user defined script before the
  customization and after the customization. In current design, the post
  customization script is added as the rc.local service and launched
  after the VM reboot. But actually the cloud-init doesn't need to
  reboot the VM, so the post customization script is not launched as we
  expected. We need to leverage the cc_scripts_per_instance module and
  let it to trigger the post customization script.

  ProblemType: Bug
  DistroRelease: Ubuntu 16.04
  Package: evince (not installed)
  ProcVersionSignature: Ubuntu 4.4.0-21.37-generic 4.4.6
  Uname: Linux 4.4.0-21-generic x86_64
  ApportVersion: 2.20.1-0ubuntu2.18
  Architecture: amd64
  Date: Tue Jun 18 08:58:04 2019
  InstallationDate: Installed on 2018-08-03 (319 days ago)
  InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Release amd64 
(20160420.3)
  ProcEnviron:
   TERM=linux
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  SourcePackage: evince
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1833192/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1836598] Re: cc_ntp generates wrong config for ntpd in debian (buster)

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1836598

Title:
  cc_ntp generates wrong config for ntpd in debian (buster)

Status in cloud-init:
  Fix Released

Bug description:
  when i use the following configuration:

  ntp:
enabled: true
ntp_client: ntp
pools: [0.int.pool.ntp.org, 1.int.pool.ntp.org, ntp.myorg.org]
servers:
  - ntp.server.local
  - ntp.ubuntu.com
  - 192.168.23.2

  wrong configuration pools received,
  cat /etc/ntpd.conf

  ...skip...
  # pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
  # pick a different set every time it starts up.  Please consider joining the
  # pool: 
  # poolspool 0.int.pool.ntp.org iburst
  pool 1.int.pool.ntp.org iburst
  pool ntp.myorg.org iburst
  # servers
  server ntp.server.local iburst
  server ntp.ubuntu.com iburst
  server 192.168.23.2 iburst

  # Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html 
for
  ...skip...

  wrong line >> # poolspool 0.int.pool.ntp.org iburst

  
  system>
  Debian GNU/Linux 10 (buster)
  cloud-init 18.3-6

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1836598/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1833264] Re: cloud-init-generator hardcodes path to ds-identify

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1833264

Title:
  cloud-init-generator hardcodes path to ds-identify

Status in cloud-init:
  Fix Released

Bug description:
  On systemd systems, the cloud-init-generator uses the ds-identify
  program and hardcodes the path to the program.  The value is correct
  for Ubuntu/Debian and other systems, however on centos-based systems,
  ds-identify is installed down libexec path.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1833264/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1836921] Re: Release 19.2

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1836921

Title:
  Release 19.2

Status in cloud-init:
  Fix Released

Bug description:
  This bug tracks cloud-init upstream release of version 19.2.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1836921/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1836921] [NEW] Release 19.2

[Ryan Harper]

Public bug reported:

This bug tracks cloud-init upstream release of version 19.2.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1836921

Title:
  Release 19.2

Status in cloud-init:
  New

Bug description:
  This bug tracks cloud-init upstream release of version 19.2.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1836921/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1835008] Re: network_v2 does not set mtu on eni config

[Ryan Harper]

for non-physical interfaces, common attributes like mtu were not copied
into the internal state when merging v2 config.

** Changed in: curtin
   Status: New => Invalid

** Changed in: cloud-init
   Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1835008

Title:
  network_v2 does not set mtu on eni config

Status in cloud-init:
  Confirmed
Status in curtin:
  Invalid
Status in MAAS:
  Invalid

Bug description:
  The MTU is not set on the bond, nor vlans on the bond, nor the bridges
  attached to the vlans on a network_v2 configuration. It's only set on
  the underlying interfaces (eth0, eth1)

  This does not happen if given a network_v1 configuration.

  (I'm deploying ubuntu xenial via MAAS 2.6.0, setting
  force_v1_network_yaml=true is my current workaround.)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1835008/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1834190] Re: Modules steps not taken into account since 19.1

[Ryan Harper]

I've added snapd so they can look into what's going on with
snapd.seeded.service.

** Also affects: snapd
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: Incomplete => Invalid

** Summary changed:

- Modules steps not taken into account since 19.1
+ snapd.seeded.service blocks for a long time preventing cloud-init from 
completing

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1834190

Title:
  snapd.seeded.service blocks for a long time preventing cloud-init from
  completing

Status in cloud-init:
  Invalid
Status in snapd:
  New

Bug description:
  We use qcow2 images to install baremetal servers, our datasource is to have a 
little partition as ConfigDrive with meta_data.json and vendor_data.json is 
necessary.
  The vendor_data.json provides informations about default user password 
settings.

  This actually works well for every linux distrib we use unless cloud-
  init is 19.1.

  The modules steps are no longer working:

  cat /run/cloud-init/status.json 
  {
   "v1": {
"datasource": "DataSourceConfigDrive [net,ver=2][source=/dev/sda4]",
"init": {
 "errors": [],
 "finished": 1561465042.7556307,
 "start": 1561465042.1868145
},
"init-local": {
 "errors": [],
 "finished": 1561465035.4210315,
 "start": 1561465034.724825
},
"modules-config": {
 "errors": [],
 "finished": null,
 "start": null
},
"modules-final": {
 "errors": [],
 "finished": null,
 "start": null
},
"modules-init": {
 "errors": [],
 "finished": null,
 "start": null
},
"stage": null
   }
  }

  
  and we can see in the /var/lib/cloud/instance/sem directory that the modules 
doesn't have the sem file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1834190/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1833731] [NEW] cloud-init analyze output not formatted cleanly on Azure

[Ryan Harper]

Public bug reported:

Azure added additional reporting trace points, the result is that
analyze/blame output doesn't look well formatted.

-- Boot Record 01 --
The total time elapsed since completing an event is printed after the "@" 
character.
The time the event takes is printed after the "+" character.

Starting stage: init-local
|`->no cache found @00.00500s +00.0s
Starting stage: init-local/search-Azure
Starting stage: azure-ds/_get_data
|`->found azure asset tag @00.05000s +00.01100s
Starting stage: azure-ds/crawl_metadata
|`->list_possible_azure_ds_devs @00.06200s +00.25000s
|`->load_azure_ds_dir @00.31300s +00.0s
Starting stage: azure-ds/load_azure_ds_dir
Starting stage: azure-ds/read_azure_ovf
|`->load_azure_ovf_pubkeys @00.31700s +00.0s
|`->_extract_preprovisioned_vm_setting @00.31700s +00.00100s
Finished stage: (azure-ds/read_azure_ovf) 00.00400 seconds 

Finished stage: (azure-ds/load_azure_ds_dir) 00.00500 seconds

Starting stage: azure-ds/get_metadata_from_imds
|`->_get_metadata_from_imds @00.49100s +00.01000s
Finished stage: (azure-ds/get_metadata_from_imds) 00.19000 seconds 

|`->_get_random_seed @00.52100s +00.00100s
Finished stage: (azure-ds/crawl_metadata) 00.47100 seconds 

|`->maybe_remove_ubuntu_network_config_scripts @00.53200s +00.00100s
|`->write_files @00.53300s +00.00300s
Finished stage: (azure-ds/_get_data) 00.48600 seconds 

Finished stage: (init-local/search-Azure) 00.48900 seconds

|`->network config from imds @00.59600s +00.0s
Finished stage: (init-local) 00.73800 seconds 

Starting stage: init-network
|`->restored from cache with run check: DataSourceAzure [seed=/var/lib/waagent] 
@03.12900s +00.03300s
|`->network config from imds @03.21700s +00.00100s
Starting stage: init-network/setup-datasource
Starting stage: azure-ds/setup
Starting stage: azure-ds/_negotiate
Starting stage: azure-ds/bounce_network_with_azure_hostname
|`->temporary_hostname @03.23000s +00.0s
Finished stage: (azure-ds/bounce_network_with_azure_hostname) 00.00300 seconds 

Starting stage: azure-ds/get_metadata_from_fabric
Starting stage: azure-ds/register_with_azure_and_fetch_data
|`->generate_certificate @03.23500s +00.10500s
Starting stage: azure-ds/find_endpoint
|`->_networkd_get_value_from_leases @03.34000s +00.00200s
Finished stage: (azure-ds/find_endpoint) 00.00200 seconds 

Starting stage: azure-ds/parse_certificates
|`->_decrypt_certs_from_xml @03.36100s +00.01000s
Starting stage: azure-ds/_get_ssh_key_from_cert
|`->_run_x509_action @03.37200s +00.00600s
Finished stage: (azure-ds/_get_ssh_key_from_cert) 00.04900 seconds 

Starting stage: azure-ds/_get_fingerprint_from_cert
|`->_run_x509_action @03.42100s +00.00600s
Finished stage: (azure-ds/_get_fingerprint_from_cert) 00.00600 seconds 

Finished stage: (azure-ds/parse_certificates) 00.06700 seconds

|`->_report_ready @03.42800s +00.00700s
Finished stage: (azure-ds/register_with_azure_and_fetch_data) 00.20200 seconds 

Finished stage: (azure-ds/get_metadata_from_fabric) 00.20200 seconds

Finished stage: (azure-ds/_negotiate) 00.20800 seconds

Finished stage: (azure-ds/setup) 00.20800 seconds

Finished stage: (init-network/setup-datasource) 00.20800 seconds

|`->reading and applying user-data @03.44500s +00.00300s
|`->reading and applying vendor-data @03.44900s +00.0s
Starting stage: init-network/activate-datasource
Starting stage: azure-ds/activate
Starting stage: azure-ds/address_ephemeral_resize
|`->wait for ephemeral disk @03.48600s +00.00100s
|`->can_dev_be_reformatted @03.48700s +00.00100s
Finished stage: (azure-ds/address_ephemeral_resize) 00.00200 seconds 

Finished stage: (azure-ds/activate) 00.00200 seconds

Finished stage: (init-network/activate-datasource) 00.00400 seconds

|`->config-migrator ran successfully @03.54000s +00.00400s
|`->config-seed_random ran successfully @03.54600s +00.00200s
|`->config-bootcmd ran successfully @03.54900s +00.0s
|`->config-write-files ran successfully @03.55000s +00.0s
|`->config-growpart ran successfully @03.55100s +00.10400s
|`->config-resizefs ran successfully @03.65600s +00.04900s
|`->config-disk_setup ran successfully @03.70600s +03.42300s
|`->config-mounts ran successfully @07.13000s +00.22300s
|`->config-set_hostname ran successfully @07.35400s +00.00200s
|`->config-update_hostname ran successfully @07.35700s +00.00200s
|`->config-update_etc_hosts ran successfully @07.35900s +00.00100s
|`->config-ca-certs ran successfully @07.36000s +00.00100s
|`->config-rsyslog ran successfully @07.36200s +00.00100s
|`->config-users-groups ran successfully @07.36400s +00.32700s
|`->config-ssh ran successfully @07.69200s +00.26200s
Finished stage: (init-network) 04.83900 seconds

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1833731

Title:
  cloud-init analyze output not formatted cleanly on Azure

S

[Yahoo-eng-team] [Bug 1833460] [NEW] cloud_tests on lxd snap error exporting image

[Ryan Harper]

Public bug reported:

Attempting a cloud_test run on lxd platform had this issue.

(diglett) cloud-init % tox -r -e citest -- run --verbose --platform=lxd 
--os-name centos70 --rpm ~/cloud-init-19.1+44.g12efb6d3-1.el7.noarch.rpm 
--result /tmp/centos7-result.yaml
GLOB sdist-make: /srv/tmp/rharper/cloud-init/setup.py
citest create: /srv/tmp/rharper/cloud-init/.tox/citest
citest installdeps: -r/srv/tmp/rharper/cloud-init/integration-requirements.txt
citest inst: /srv/tmp/rharper/cloud-init/.tox/.tmp/package/1/cloud-init-19.1.zip
citest installed: 
asn1crypto==0.24.0,attrs==19.1.0,bcrypt==3.1.6,boto3==1.5.9,botocore==1.8.50,certifi==2019.6.16,cffi==1.12.3,chardet==3.0.4,cloud-init==19.1,configobj==5.0.6,cryptography==2.7,docutils==0.14,idna==2.8,Jinja2==2.10.1,jmespath==0.9.4,jsonpatch==1.23,jsonpointer==2.0,jsonschema==3.0.1,linecache2==1.0.0,MarkupSafe==1.1.1,oauthlib==3.0.1,paramiko==2.4.1,pbr==5.3.0,pkg-resources==0.0.0,pyasn1==0.4.5,pycparser==2.19,pylxd==2.2.7,PyNaCl==1.3.0,pyrsistent==0.15.2,python-dateutil==2.8.0,python-simplestreams==0.1.0,PyYAML==5.1.1,requests==2.22.0,requests-toolbelt==0.9.1,requests-unixsocket==0.1.5,s3transfer==0.1.13,six==1.12.0,traceback2==1.4.0,unittest2==1.1.0,urllib3==1.25.3,ws4py==0.5.1
citest run-test-pre: PYTHONHASHSEED='2218838469'
citest runtests: commands[0] | 
/srv/tmp/rharper/cloud-init/.tox/citest/bin/python -m tests.cloud_tests run 
--verbose --platform=lxd --os-name centos70 --rpm 
/home/rharper/cloud-init-19.1+44.g12efb6d3-1.el7.noarch.rpm --result 
/tmp/centos7-result.yaml
2019-06-19 19:18:44,550 - tests.cloud_tests - DEBUG - running with args: 
Namespace(data_dir=None, deb=None, feature_override={}, os_name=['centos70'], 
platform=['lxd'], ppa=None, preserve_data=False, preserve_instance=False, 
quiet=False, repo=None, result='/tmp/centos7-result.yaml', 
rpm='/home/rharper/cloud-init-19.1+44.g12efb6d3-1.el7.noarch.rpm', script=None, 
subcmd='run', 
test_config=['/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/bugs/lp1511485.yaml',
 '/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/bugs/lp1611074.yaml', 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/bugs/lp1628337.yaml', 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/add_apt_repositories.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/alter_completion_message.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/configure_instance_trusted_ca_certificates.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/configure_instances_ssh_keys.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/including_user_groups.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/install_arbitrary_packages.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/install_run_chef_recipes.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/run_apt_upgrade.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/run_commands.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/run_commands_first_boot.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/setup_run_puppet.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/writing_out_arbitrary_files.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/main/command_output_simple.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_conf.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_disable_suites.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_primary.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_proxy.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_security.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_sources_key.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_sources_list.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_sources_ppa.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_pipelining_disable.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_pipelining_os.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/bootcmd.yaml', 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/byobu.yaml', 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/ca_certs.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/debug_disable.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/debug_enable.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/clo

[Yahoo-eng-team] [Bug 1833264] [NEW] cloud-init-generator hardcodes path to ds-identify

[Ryan Harper]

Public bug reported:

On systemd systems, the cloud-init-generator uses the ds-identify
program and hardcodes the path to the program.  The value is correct for
Ubuntu/Debian and other systems, however on centos-based systems, ds-
identify is installed down libexec path.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1833264

Title:
  cloud-init-generator hardcodes path to ds-identify

Status in cloud-init:
  New

Bug description:
  On systemd systems, the cloud-init-generator uses the ds-identify
  program and hardcodes the path to the program.  The value is correct
  for Ubuntu/Debian and other systems, however on centos-based systems,
  ds-identify is installed down libexec path.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1833264/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1800848] Re: does not recognize cloned KVM VM as new instance

[Ryan Harper]

OK, good to know.

w.r.t SLES 15 and cloud-init; if you've attached a NoCloud datasource
correctly (your logs indicate that you are) then I would say yes this is
a bug.

Now, upstream since 18.4 there have been quite a few fixes applied which
may not have made it into SLES15, not 100% sure about that but:

% git log 18.4..HEAD --oneline | grep -i suse
bb0b6f1 net/sysconfig: write out SUSE-compatible IPv6 config
dfe50e3 tox: Update testenv for openSUSE Leap to 15.0
3f12012 sysconfig: On SUSE, use STARTMODE instead of ONBOOT
744c423 Add cloud-id binary to packages for SUSE
e0084a5 systemd: On SUSE ensure cloud-init.service runs before wicked
4ea64f1 update detection of openSUSE variants


** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1800848

Title:
  does not recognize cloned KVM VM as new instance

Status in cloud-init:
  Invalid
Status in cloud-init package in Suse:
  Fix Released

Bug description:
  On a SLES 15 KVM VM on Proxmox VE  cloud-init 18.2-3.13 from
  module/repo Public-Cloud-Module_15-0  cloud-init fails to recognize a
  newly cloned VM from a template where cloud-init has initially been
  run for test purposes as a new instance. VM is using NoCloud resource.

  This leads to network configuration not applied and thus duplicate IP
  addresses when VM is started:

  % tail -f /var/log/cloud-init.log | grep -v util.py
  2018-10-31 13:55:27,370 - __init__.py[INFO]: 
/var/lib/cloud/data/previous-hostname differs from /etc/hostname, assuming user 
maintained hostname.
  2018-10-31 13:57:37,431 - main.py[DEBUG]: No kernel command line url found.
  2018-10-31 13:57:37,431 - main.py[DEBUG]: Closing stdin.
  2018-10-31 13:57:37,454 - main.py[DEBUG]: Checking to see if files that we 
need already exist from a previous run that would allow us to stop early.
  2018-10-31 13:57:37,455 - main.py[DEBUG]: Execution continuing, no previous 
run detected that would allow us to stop early.
  2018-10-31 13:57:37,455 - handlers.py[DEBUG]: start: 
init-network/check-cache: attempting to read from cache [trust]
  2018-10-31 13:57:37,463 - stages.py[DEBUG]: restored from cache with run 
check: DataSourceNoCloudNet [seed=/dev/sr0] [dsmode=net]
  2018-10-31 13:57:37,464 - handlers.py[DEBUG]: finish: 
init-network/check-cache: SUCCESS: restored from cache with run check: 
DataSourceNoCloudNet [seed=/dev/sr0][dsmode=net]
  2018-10-31 13:57:37,488 - stages.py[DEBUG]: previous iid found to be 
414842fe12da6f1078eca77443e6ab84592299ba
  2018-10-31 13:57:37,493 - main.py[DEBUG]: [net] init will now be targeting 
instance id: 414842fe12da6f1078eca77443e6ab84592299ba. new=False
  2018-10-31 13:57:37,514 - stages.py[DEBUG]: applying net config names for 
{'version': 1, 'config': [{'type': 'physical', 'name': 'eth0', 'mac_address': 
'76:61:cf:d5:65:b2', 'subnets': [{'type': 'static', 'address': '10.0.88.32', 
'netmask': '255.255.0.0', 'gateway': '10.0.0.4'}, {'type': 'static', 'address': 
'auto'}]}, {'type': 'nameserver', 'address': ['10.0.0.4'], 'search': 
['qs.de']}]}
  2018-10-31 13:57:37,515 - stages.py[DEBUG]: Using distro class 
  2018-10-31 13:57:37,529 - __init__.py[DEBUG]: no work necessary for renaming 
of [['76:61:cf:d5:65:b2', 'eth0', 'virtio_net', '0x0001']]
  2018-10-31 13:57:37,530 - stages.py[DEBUG]: not a new instance. network 
config is not applied.

  Network configuration obviously was changed however.

  Commands used for cloning (using a self-made shell script)

  qm shutdown 1032 ; qm clone 1032 2200 --name slesmaster ; qm template 2200
  qm clone 2200 2201 --name sles1 ; qm set 2201 --ipconfig0 
ip=10.0.88.201/8,gw=10.0.0.4 ; qm start 2201
  qm clone 2200 2202 --name sles2 ; qm set 2202 --ipconfig0 
ip=10.0.88.202/8,gw=10.0.0.4 ; qm start 2202

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1800848/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1822615] [NEW] cloud-tests tree_run/bddeb fail to build deb

[Ryan Harper]

Public bug reported:

While looking to verify an apt_pipelining test-case fix I ran this:

% tox -e citest -- tree_run --verbose --os-name bionic --test
modules/apt_pipelining_disable --test modules/apt_pipelining_os --result
/tmp/result.yaml

During which, it would fail like this:

2019-03-31 16:13:11,743 - tests.cloud_tests - ERROR - stage part: build deb on 
system encountered error: Unexpected error while running command.
Command: /root/packages/bddeb -d
Exit code: 1
Reason: -



 ==   
ERROR: 
cloudinit.cmd.tests.test_query.TestQuery.test_handle_args_dumps_all_instance_data
--   
Traceback (most recent call last):   
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/tests/test_query.py",
 line 153, in test_handle_args_dumps_all_instance_data
self.assertEqual(0, query.handle_args('anyname', args))  
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/query.py",
 line 124, in handle_args
instance_data['userdata'] = util.load_file(user_data_fn) 
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/util.py",
 line 1359, in load_file
with open(fname, 'rb') as ifh:   
FileNotFoundError: [Errno 2] No such file or directory: 'ud' 
 
==   
ERROR: 
cloudinit.cmd.tests.test_query.TestQuery.test_handle_args_list_keys_errors_when_varname_is_not_a_dict
--   
Traceback (most recent call last):   
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/tests/test_query.py",
 line 255, in test_handle_args_list_keys_errors_when_varname_is_not_a_dict
self.assertEqual(1, query.handle_args('anyname', args))  
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/query.py",
 line 124, in handle_args
instance_data['userdata'] = util.load_file(user_data_fn) 
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/util.py",
 line 1359, in load_file
with open(fname, 'rb') as ifh:   
FileNotFoundError: [Errno 2] No such file or directory: 'ud' 
 
==   
ERROR: 
cloudinit.cmd.tests.test_query.TestQuery.test_handle_args_list_keys_sorts_nested_keys_when_varname
--   
Traceback (most recent call last):   
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/tests/test_query.py",
 line 239, in test_handle_args_list_keys_sorts_nested_keys_when_varname
self.assertEqual(0, query.handle_args('anyname', args))  
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/query.py",
 line 124, in handle_args
instance_data['userdata'] = util.load_file(user_data_fn) 
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/util.py",
 line 1359, in load_file
with open(fname, 'rb') as ifh:   
FileNotFoundError: [Errno 2] No such file or directory: 'ud' 
 
==   
ERROR: 
cloudinit.cmd.tests.test_query.TestQuery.test_handle_args_list_keys_sorts_top_level_keys_when_no_varname
--   
Traceback (most recent call last):   
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/tests/test_query.py",
 line 224, in test_handle_args_list_keys_sorts_top_level_keys_when_no_varname
self.assertEqual(0, query.handle_args('anyname', args))  
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/query.py",
 line 124, in handle_args
instance_data['userdata'] = util.load_file(user_data_fn) 
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/util.py",
 line 1359, in load_file
with open(fname, 'rb') as ifh:   
FileNotFoundError: [Errno 2] No such file or directory: 'ud' 

==

[Yahoo-eng-team] [Bug 1819994] Re: An error occurs when MAAS Deploy 18.04 on ThinkSystem SR590

[Ryan Harper]

2019-03-14 17:32:34,606 - __init__.py[DEBUG]: Selected renderer
'sysconfig' from priority list: None

This is a cloud-init bug.  The sysconfig renderer has NetworkManager
support, this triggered cloud-init to render sysconfig instead of
netplan.


** Also affects: cloud-init
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Importance: Undecided => High

** Changed in: cloud-init
   Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1819994

Title:
  An error occurs when MAAS Deploy 18.04 on ThinkSystem SR590

Status in cloud-init:
  Confirmed
Status in MAAS:
  Incomplete
Status in Provider for Plainbox - Canonical Certification Server:
  Confirmed

Bug description:
  Configuration:
  UEFI/BIOS:TEE136S
  IMM/BMC:  CDI333V
  CPU:  Intel(R) Xeon(R) Platinum 8253 CPU @ 2.20GHz
  Memory:   16G DIMM * 12
  Raid card:ThinkSystem RAID 530-8i 
  NIC Card: Intel X722 LOM

  Reproduce Steps:
  1.Config "network" as first boot
  2.Power on machine
  3.Visit TC through web browser and Commission machine
  4.When commission complete, deploy ubuntu 18.04 LTS on SUT
  5.The Error appeared during OS deploy.

  Deploy errors like the following(you can view the attachment for
  details):

  cloud-init[] Date_and_time - handlers.py[WARNING]: failed posting
  event: start: modules-final/config-: running config-

  cloud-init[] Date_and_time - handlers.py[WARNING]: failed posting
  event: fainish: modules-final: SUCCESS: running modules for final

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1819994/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1819913] [NEW] cloud-init on xenial may generate network config on every boot

[Ryan Harper]

Public bug reported:

On at least EC2 with cloud-init xenial release, the Ec2 datasource allows the 
EventType.BOOT
event to update metadata and will regenerate network configuration on each 
boot. Cloud-init releases newer than Xenial are not affected since cloud-init 
will detect
which datasource to use and does not perform searching.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1819913

Title:
  cloud-init on xenial may generate network config on every boot

Status in cloud-init:
  New

Bug description:
  On at least EC2 with cloud-init xenial release, the Ec2 datasource allows the 
EventType.BOOT
  event to update metadata and will regenerate network configuration on each 
boot. Cloud-init releases newer than Xenial are not affected since cloud-init 
will detect
  which datasource to use and does not perform searching.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1819913/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1815051] Re: Bionic netplan render invalid yaml duplicate anchor declaration for nameserver entries

[Ryan Harper]

** Also affects: cloud-init (Ubuntu Cosmic)
   Importance: Undecided
   Status: New

** Also affects: cloud-init (Ubuntu Xenial)
   Importance: Undecided
   Status: New

** Also affects: cloud-init (Ubuntu Bionic)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1815051

Title:
  Bionic netplan render invalid yaml duplicate anchor declaration for
  nameserver entries

Status in cloud-init:
  In Progress
Status in cloud-init package in Ubuntu:
  In Progress
Status in cloud-init source package in Xenial:
  New
Status in cloud-init source package in Bionic:
  New
Status in cloud-init source package in Cosmic:
  New

Bug description:
  The netplan configuration redeclares the nameservers anchor for every
  single section (vlans, bonds), and use the same id for similar entries
  (id001).

  In this specific case the network configuration in maas have a bond0
  with two vlans, bond0.3502 and bond0.3503, and an untagged bond1
  without vlans. The rendered 50-cloud-init.yaml looks like this:

  network:
  version: 2
  ethernets:
  ...
  bonds:
  ...
  bond1:
  ...
  nameservers: &id001 <- anchor declaration here
  addresses:
  - 255.255.255.1
  - 255.255.255.2
  - 255.255.255.3
  - 255.255.255.5
  search:
  - customer.domain
  - maas
  ...
  bondM:
  ...
  nameservers: *id001

 vlans:
  bond0.3502:
  ...
  nameservers: &id001 <- anchor redeclaration here
  addresses:
  - 255.255.255.1
  - 255.255.255.2
  - 255.255.255.3
  - 255.255.255.5
  search:
  - customer.domain
  - maas
  bond0.3503:
  ...
  nameservers: *id001

  As the cloudinit renders an invalid yaml file, the netplan apply
  produces the following error: (due to the anchor redeclaration in the
  vlans section):

 Invalid YAML at /etc/netplan/50-cloud-init.yaml line 118 column 25:
  second occurence

  This render bug prevents us using the untagged bond and the bond with
  the vlans in the same configuration.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1815051/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1815051] Re: Bionic netplan render invalid yaml duplicate anchor declaration for nameserver entries

[Ryan Harper]

** Also affects: cloud-init (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: cloud-init (Ubuntu)
   Importance: Undecided => High

** Changed in: cloud-init (Ubuntu)
   Status: New => In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1815051

Title:
  Bionic netplan render invalid yaml duplicate anchor declaration for
  nameserver entries

Status in cloud-init:
  In Progress
Status in cloud-init package in Ubuntu:
  In Progress
Status in cloud-init source package in Xenial:
  New
Status in cloud-init source package in Bionic:
  New
Status in cloud-init source package in Cosmic:
  New

Bug description:
  The netplan configuration redeclares the nameservers anchor for every
  single section (vlans, bonds), and use the same id for similar entries
  (id001).

  In this specific case the network configuration in maas have a bond0
  with two vlans, bond0.3502 and bond0.3503, and an untagged bond1
  without vlans. The rendered 50-cloud-init.yaml looks like this:

  network:
  version: 2
  ethernets:
  ...
  bonds:
  ...
  bond1:
  ...
  nameservers: &id001 <- anchor declaration here
  addresses:
  - 255.255.255.1
  - 255.255.255.2
  - 255.255.255.3
  - 255.255.255.5
  search:
  - customer.domain
  - maas
  ...
  bondM:
  ...
  nameservers: *id001

 vlans:
  bond0.3502:
  ...
  nameservers: &id001 <- anchor redeclaration here
  addresses:
  - 255.255.255.1
  - 255.255.255.2
  - 255.255.255.3
  - 255.255.255.5
  search:
  - customer.domain
  - maas
  bond0.3503:
  ...
  nameservers: *id001

  As the cloudinit renders an invalid yaml file, the netplan apply
  produces the following error: (due to the anchor redeclaration in the
  vlans section):

 Invalid YAML at /etc/netplan/50-cloud-init.yaml line 118 column 25:
  second occurence

  This render bug prevents us using the untagged bond and the bond with
  the vlans in the same configuration.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1815051/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1811446] [NEW] chpasswd: is mangling certain password hashes

[Ryan Harper]

Public bug reported:

#cloud-config

# from 1 files
# part-001

---
chpasswd:
expire: false
list: 'root:$2y$10$8BQjxjVByHA/Ee.O1bCXtO8S7Y5WojbXWqnqYpUW.BrPx/Dlew1Va

'


>From #cloud-init

 Hey there, I'm not sure whether I'm running into a bug or not
 I'm trying to set the password hash for the root user on a system 
using the chpasswd module
 It should match new hash at this line in the module but it doens't 
seem to match
 
https://github.com/cloud-init/cloud-init/blame/master/cloudinit/config/cc_set_passwords.py#L163
 I can confirm this when running it through 
https://regex101.com/r/Nj7VTZ/1
 Then I was thinking, isn't [] for lists of characters rather than 
lists of strings
 Changing it to \$(1|2a|2y|5|6)(\$.+){2} does work
 At least in regex101
 smoser, you any idea, I saw you commited the change: 
https://github.com/cloud-init/cloud-init/commit/21632972df034c200578e1fbc121a07f20bb8774
 marlinc_: i'd think yes. that is a bug for the '2a' and '2y'

** Affects: cloud-init
 Importance: Undecided
 Status: New

** Merge proposal linked:
   https://code.launchpad.net/~marlinc/cloud-init/+git/cloud-init/+merge/361683

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1811446

Title:
  chpasswd: is mangling certain password hashes

Status in cloud-init:
  New

Bug description:
  #cloud-config

  # from 1 files
  # part-001

  ---
  chpasswd:
  expire: false
  list: 'root:$2y$10$8BQjxjVByHA/Ee.O1bCXtO8S7Y5WojbXWqnqYpUW.BrPx/Dlew1Va

  '

  
  From #cloud-init

   Hey there, I'm not sure whether I'm running into a bug or not
   I'm trying to set the password hash for the root user on a system 
using the chpasswd module
   It should match new hash at this line in the module but it doens't 
seem to match
   
https://github.com/cloud-init/cloud-init/blame/master/cloudinit/config/cc_set_passwords.py#L163
   I can confirm this when running it through 
https://regex101.com/r/Nj7VTZ/1
   Then I was thinking, isn't [] for lists of characters rather than 
lists of strings
   Changing it to \$(1|2a|2y|5|6)(\$.+){2} does work
   At least in regex101
   smoser, you any idea, I saw you commited the change: 
https://github.com/cloud-init/cloud-init/commit/21632972df034c200578e1fbc121a07f20bb8774
   marlinc_: i'd think yes. that is a bug for the '2a' and '2y'

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1811446/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1809277] [NEW] nocloud-net doesn't read included network-config file if using seedfrom=URL

[Ryan Harper]

Public bug reported:

Currently nocloud-net does not process network-config file if included
in the seed provided via a URL.  This is by-design, however, now that
several datasources have this model of using an ephemeral DHCP process
to fetch user/metadata, which includes network-configuration and then
apply that; nocloud-net could transition to this model.

** Affects: cloud-init
 Importance: Low
 Status: Triaged

** Changed in: cloud-init
   Importance: Undecided => Low

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1809277

Title:
  nocloud-net doesn't read included network-config file if using
  seedfrom=URL

Status in cloud-init:
  Triaged

Bug description:
  Currently nocloud-net does not process network-config file if included
  in the seed provided via a URL.  This is by-design, however, now that
  several datasources have this model of using an ephemeral DHCP process
  to fetch user/metadata, which includes network-configuration and then
  apply that; nocloud-net could transition to this model.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1809277/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 74747] Re: Default sources.list file has source packages enabled by default

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/74747

Title:
  Default sources.list file has source packages enabled by default

Status in cloud-init:
  Fix Released
Status in apt-setup package in Ubuntu:
  Fix Released
Status in cloud-init package in Ubuntu:
  Fix Released
Status in apt-setup source package in Xenial:
  Fix Released
Status in cloud-init source package in Xenial:
  Confirmed
Status in apt-setup source package in Bionic:
  Fix Released
Status in cloud-init source package in Bionic:
  Confirmed
Status in apt-setup source package in Cosmic:
  Fix Released
Status in cloud-init source package in Cosmic:
  Fix Released

Bug description:
  The default sources.list file has source packages enabled by default,
  this is bad for the average user (especially those on modems) because
  they are very unlikely to use source packages, however they will still
  have the download overhead of the packages list.

  For most people the deb-src lines could simply be commented out by
  default.

  (Bug reported at the behest of Robert Collins)

  Implementing this would probably invalidate bug 301602. See also bug
  987264.

  Mailing list discussion:
  
https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2013-May/thread.html#14503
  
https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2013-July/thread.html#14617

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/74747/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1797231] Re: ec2 integration test failure on changed instance-data.json

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1797231

Title:
  ec2 integration test failure on changed instance-data.json

Status in cloud-init:
  Fix Released

Bug description:
  Integration tests for ec2 expected instance-data.json to contain the
  following keys:

  
  {'ds': {'meta_data': {'network' }}} but meta_data changed to meta-data to 
better represent the raw data received directly from ec2's metadata service.

  
  Update integration tests to use meta_data instead of meta-data.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1797231/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1805854] Re: [feature-request] Add non-x86 Ubuntu EC2 mirrors in to default cloud-init configuration

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1805854

Title:
  [feature-request] Add non-x86 Ubuntu EC2 mirrors in to default cloud-
  init configuration

Status in cloud-init:
  Fix Released

Bug description:
  With the announcement of EC2s A1 instances we'd (CPC) like to be able
  to rely on cloud-init to configure archive mirrors for non-x86
  architectures.

  Currently the AMIs have the mirrors configured using
  /etc/cloud/cloud.cfg.d/95_mirrors.cfg which points to the in region
  archive mirrors for non-x86 architectures.

  ```
  # Add archive mirrors for non-x86 architectures
  system_info:
     package_mirrors:
   - arches: [i386, amd64]
     failsafe:
   primary: http://archive.ubuntu.com/ubuntu
   security: http://security.ubuntu.com/ubuntu
     search:
   primary:
     - http://%(ec2_region)s.ec2.archive.ubuntu.com/ubuntu/
     - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/
     - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/
   security: []
   - arches: [armhf, armel, default]
     failsafe:
   primary: http://ports.ubuntu.com/ubuntu-ports
   security: http://ports.ubuntu.com/ubuntu-ports
     search:
   primary:
     - http://%(ec2_region)s.ec2.ports.ubuntu.com/ubuntu-ports/
     - 
http://%(availability_zone)s.clouds.ports.ubuntu.com/ubuntu-ports/
     - http://%(region)s.clouds.ports.ubuntu.com/ubuntu-ports/
   security: []
  ```

  See http://paste.ubuntu.com/p/QGvGKpKYKb/ for current AMIs that can be
  used for testing.

  So the feature request is that coud-init configure the above mirrors
  automatically without us having to write
  /etc/cloud/cloud.cfg.d/95_mirrors.cfg on the image.

  Please let me know if you would like me to spin up instances for
  testing instead or if you need any further info.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1805854/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1803598] Re: Do not retry polling IMDS for reprovisiondata during timeout

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1803598

Title:
  Do not retry polling IMDS for reprovisiondata during timeout

Status in cloud-init:
  Fix Released

Bug description:
  Azure retries polling IMDS for metadata and reprovisiondata when a
  timeout occurs. However this should not happen when polling for
  reprovisiondata as a timeout could indicate that the IP address has
  changed and that we should do a dhcp instead of polling, which is
  going to result in timeout during every poll.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1803598/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1799338] Re: cloud-init won't reformat NTFS ephemeral drive on SLES 15

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1799338

Title:
  cloud-init won't reformat NTFS ephemeral drive on SLES 15

Status in cloud-init:
  Fix Released

Bug description:
  Commit aa4eeb808 (Paul Meyer 2018-05-23 15:45:39 -0400  710) detects that the 
platform doesn't support NTFS volumes by looking for the appropriate error 
message in the exception it catches. The exact syntax of that error message 
differs between RHEL (the target distro for Paul's merge) and SUSE:
RHEL mount: unknown filesystem type 'ntfs'
SUSE mount: /dev/sdb1: unknown filesystem type 'ntfs'

  As a result, cloud-init on SUSE VMs in Azure doesn't properly detect
  that the distro doesn't support NTFS and thus will not reformat the
  ephemeral volume on Azure.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1799338/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1805201] Re: collect-logs traceback on non-root user

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1805201

Title:
  collect-logs traceback on non-root user

Status in cloud-init:
  Fix Released

Bug description:
  Permissions errors are seen when running collect-logs as non-root user
  cloud-init 18.4.24

  $ cloud-init collect-logs
  Traceback (most recent call last):
File "/usr/bin/cloud-init", line 11, in 
  load_entry_point('cloud-init==18.4', 'console_scripts', 'cloud-init')()
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 904, in 
main
  get_uptime=True, func=functor, args=(name, args))
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2514, in 
log_time
  ret = func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/cloudinit/cmd/devel/logs.py", line 
125, in handle_collect_logs_args
  collect_logs(args.tarfile, args.userdata, args.verbosity)
File "/usr/lib/python3/dist-packages/cloudinit/cmd/devel/logs.py", line 
113, in collect_logs
  os.path.join(run_dir, 'cloud-init'))
File "/usr/lib/python3.6/shutil.py", line 359, in copytree
  raise Error(errors)
  shutil.Error: [('/run/cloud-init/instance-data-sensitive.json', 
'/tmp/tmphc62cg6h/cloud-init-logs-2018-11-26/run/cloud-init/instance-data-sensitive.json',
 "[Errno 13] Permission denied: 
'/run/cloud-init/instance-data-sensitive.json'")]

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1805201/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1797199] Re: kvm integration test failures due to invalid config-disk path

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1797199

Title:
  kvm integration test failures due to invalid config-disk path

Status in cloud-init:
  Fix Released

Bug description:
  Cosmic and bionic integration tests are failing on nocloud kvm
  platforms due to invalid/strict assertions.

  FAIL: test_instance_data_json_kvm 
(tests.cloud_tests.testcases.modules.set_password_expire.TestPasswordExpire)
  Validate instance-data.json content by nocloud-kvm platform.
  --
  Traceback (most recent call last):
File 
"/var/lib/jenkins/slaves/torkoal/workspace/cloud-init-integration-nocloud-kvm-c/cloud-init/tests/cloud_tests/testcases/base.py",
 line 265, in test_instance_data_json_kvm
  self.assertEqual('config-disk (/dev/vda)', v1_data['subplatform'])
  AssertionError: 'config-disk (/dev/vda)' != 'config-disk (/dev/vdb)'
  - config-disk (/dev/vda)
  ? ^
  + config-disk (/dev/vdb)
  ?

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1797199/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1798117] Re: juju sends "network" top level key to user.network-config in lxd containers

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1798117

Title:
  juju sends "network" top level key to user.network-config in lxd
  containers

Status in cloud-init:
  Fix Released
Status in juju:
  Fix Committed
Status in cloud-init package in Ubuntu:
  Confirmed

Bug description:
  == Short summary ==
  In lxd containers launched by juju, 
  /var/lib/cloud/seed/nocloud-net/network-config has:
   has:
   network:
 config: disabled

  That is invalid content.  Cloud-init assumes content in 'network-config'
  is already namespaced to 'network'.  The correct content would be:

config: disabled

  == Easy recreate ==
  $ lxc launch ubuntu-daily:bionic \
 "--config=user.network-config={'network': {'config': {'disabled'}}}"

  == Longer Info ==
  When looking at bug 1651497, I see containers that run cloud-init
  have errors in a container's cloud-init log 
  (http://paste.ubuntu.com/p/5mKXC8pMwH/) like:
AttributeError: 'NoneType' object has no attribute 'iter_interfaces'
  and
Failed to rename devices: Failed to apply network config names. Found bad 
network config version: None

  After some looking guessing I realized that juju must be attempting to
  disable cloud-init's network configuration via sending the following
  into the nocloud seed (/var/lib/cloud/seed/nocloud-net/network-config)
  via 'user.network-config'.

  cloud-init can clearly handle this better, but juju should not be
  sending invalid configuration.

  
  Related bugs:
   * bug 1651497: iscsid.service fails to start in container, results in failed 
dist-upgrade later on

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: cloud-init 18.3-9-g2e62cb8a-0ubuntu1~18.04.2
  ProcVersionSignature: Ubuntu 4.18.0-8.9-generic 4.18.7
  Uname: Linux 4.18.0-8-generic x86_64
  ApportVersion: 2.20.9-0ubuntu7.4
  Architecture: amd64
  CloudName: NoCloud
  Date: Tue Oct 16 14:33:12 2018
  PackageArchitecture: all
  ProcEnviron:
   TERM=xterm-256color
   PATH=(custom, no user)
   LANG=C.UTF-8
  SourcePackage: cloud-init
  UpgradeStatus: No upgrade log present (probably fresh install)
  cloud-init-log-warnings:
   2018-10-16 14:32:01,706 - stages.py[WARNING]: Failed to rename devices: 
Failed to apply network config names. Found bad network config version: None
   2018-10-16 14:32:01,707 - util.py[WARNING]: failed stage init-local
   AttributeError: 'NoneType' object has no attribute 'version'
   2018-10-16 14:32:02,366 - stages.py[WARNING]: Failed to rename devices: 
Failed to apply network config names. Found bad network config version: None
  user_data.txt:
   #cloud-config
   {}

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1798117/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1799594] Re: Azure - Report ready during preprovisioning as soon as we get the ReprovisionData

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1799594

Title:
  Azure - Report ready during preprovisioning as soon as we get the
  ReprovisionData

Status in cloud-init:
  Fix Released

Bug description:
  When reusing a preprovisioned VM, report ready to Azure fabric as soon
  as we get the reprovision data and the goal state so that we are not
  delayed by the cloud-init stage switch, saving 2-3 seconds. Also
  reduce logging when polling IMDS for reprovision data.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1799594/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1798189] Re: cloud-init query: /run/cloud/instance-data-sensitive.json not generated on upgrade

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1798189

Title:
  cloud-init query: /run/cloud/instance-data-sensitive.json not
  generated on upgrade

Status in cloud-init:
  Fix Released

Bug description:
  /run/cloud-init/instance-data.json & instance-data-sensitive.json not
  regenerated on upgrade.

  Between cloud-init from 18.3-9 -> 18.4.0 cloud-init transitioned from
  a single sensitive  /run/cloud-init/instance-data.json  that was read-
  only root to two separate files:  /run/cloud-init/instance-data-
  sensitive.json (root readable) and  /run/cloud-init/instance-data.json
  (world readable).

  cloud-init query subcommand attempts to read the instance-data.json
  when getuid is non-root, and instance-data-sensitive.json when getuid
  is root.

  Since /run/cloud-init/instance-data*json is only regenerated on
  reboot, "cloud-init query" after an upgrade emits the following errors

  # as non-root
  ubuntu@mybox $ cloud-init query --all
  ERROR: Missing instance-data.json file: /run/cloud-init/instance-data.json

  # as root user
  ubuntu@mybox $ sudo cloud-init query --all
  ERROR: Missing instance-data.json file: 
/run/cloud-init/instance-data-sensitive.json

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1798189/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1799709] Re: service order is incorrect for SUSE distros

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1799709

Title:
  service order is incorrect for SUSE distros

Status in cloud-init:
  Fix Released

Bug description:
  With db50bc0d9 the sysconfig renderer was enabled for openSUSE and
  SUSE Linux Enterprise. This requires that cloud-init.service starts
  before wicked, but at present the service is started after wicked.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1799709/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1798424] Re: Xenial Azure: Make generation of network config from IMDS hotplug scripts configurable opt-in

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1798424

Title:
  Xenial Azure: Make generation of network config from IMDS  hotplug
  scripts configurable opt-in

Status in cloud-init:
  Fix Released
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Xenial:
  Fix Released

Bug description:
  === Begin SRU Template ===
  [Impact]
  By default, Xenial needs to rely on existing cloud image hotplug scripts and 
only generate fallback network config (dhcp on eth0) by default. If consumers 
want to generate dynamic network from Azure's IMDS service, thus removing cloud 
image hotplug scripts, then a datasource configuration option is surfaced.

  
  [Test Case]
  1. Deploy stock Xenial cloud image
  2. upgrade cloud-init -proposed
  3. Run cloud-init clean --reboot --logs
  4. Confirm that network is not sourced from IMDS content and hotplug scripts 
still exist
  5. Add datasource configuration setting Azure: apply_network_config: true
  6. Run cloud-init clean --reboot --logs
  7.  Confirm that network is sourced from IMDS and hotplug scripts are removed.

  
  [Regression Potential]

  [Other Info]
  Upstream commit at
https://git.launchpad.net/cloud-init/commit/?id=15a75ea1

  === End SRU Template ===

  
  === Original Description ===

  
  cloud-init v. 18.4-0ubuntu1~16.04.1 in -proposed automatically renders 
network configuration from Azure's IMDS by default instead of fallback config 
of dhcp on eth0. This represents a difference in behavior from current Xenial.

  On Xenial Azure, Ubuntu cloud images have udev scripts to handle
  network hotplug. Azure datasource has the ability to read full network
  config from their IMDS service and render hotplugged devices as well
  as remove the cloud-image default scripts.

  Make the cloud-init hotplug behavior configurable and default it to
  off in Xenial.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1798424/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1808380] Re: Release 18.5

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1808380

Title:
  Release 18.5

Status in cloud-init:
  Fix Released

Bug description:
  This bug tracks cloud-init upstream release of version 18.5.

  == Release Notes ==
  Hello All,

  Cloud-init release 18.5 is now available

  The 18.5 release:
* spanned just over two months in length.
* had 13 contributors from 8 domains.
* fixed 18 launchpad.net issues.

  Highlights:
   - Azure now supports waking from preprovision state via netlink messages.
   - New cli command 'cloud-id' to display what cloud on which an instance
 is running.
   - write_files config module now supports appending to a file
   - instance-data.json standardized platform and subplatform values
   - select ubuntu archive mirror for armel, armhf, and arm64

  
  == ChangeLog ==
   - tests: add Disco release [Joshua Powers]
   - net: render 'metric' values in per-subnet routes (LP: #1805871)
   - write_files: add support for appending to files. [James Baxter]
   - config: On ubuntu select cloud archive mirrors for armel, armhf, arm64.
 (LP: #1805854)
   - dhclient-hook: cleanups, tests and fix a bug on 'down' event.
   - NoCloud: Allow top level 'network' key in network-config. (LP: #1798117)
   - ovf: Fix ovf network config generation gateway/routes (LP: #1806103)
   - azure: detect vnet migration via netlink media change event
 [Tamilmani Manoharan]
   - Azure: fix copy/paste error in error handling when reading azure ovf.
 [Adam DePue]
   - tests: fix incorrect order of mocks in test_handle_zfs_root.
   - doc: Change dns_nameserver property to dns_nameservers. [Tomer Cohen]
   - OVF: identify label iso9660 filesystems with label 'OVF ENV'.
   - logs: collect-logs ignore instance-data-sensitive.json on non-root user
 (LP: #1805201)
   - net: Ephemeral*Network: add connectivity check via URL
   - azure: _poll_imds only retry on 404. Fail on Timeout (LP: #1803598)
   - resizefs: Prefix discovered devpath with '/dev/' when path does not
 exist [Igor Galić]
   - azure: retry imds polling on requests.Timeout (LP: LP:1800223)
   - azure: Accept variation in error msg from mount for ntfs volumes
 [Jason Zions] (LP: #1799338)
   - azure: fix regression introduced when persisting ephemeral dhcp lease
 [asakkurr]
   - azure: add udev rules to create cloud-init Gen2 disk name symlinks
 (LP: #1797480)
   - tests: ec2 mock missing httpretty user-data and instance-identity routes
   - azure: remove /etc/netplan/90-hotplug-azure.yaml when net from IMDS
   - azure: report ready to fabric after reprovision and reduce logging
 [asakkurr] (LP: #1799594)
   - query: better error when missing read permission on instance-data
   - instance-data: fallback to instance-data.json if sensitive is absent.
 (LP: #1798189)
   - docs: remove colon from network v1 config example. [Tomer Cohen]
   - Add cloud-id binary to packages for SUSE [Jason Zions]
   - systemd: On SUSE ensure cloud-init.service runs before wicked
 [Robert Schweikert] (LP: #1799709)
   - update detection of openSUSE variants [Robert Schweikert]
   - azure: Add apply_network_config option to disable network from IMDS
 (LP: #1798424)
   - Correct spelling in an error message (udevadm). [Katie McLaughlin]
   - tests: meta_data key changed to meta-data in ec2 instance-data.json
 (LP: #1797231)
   - tests: fix kvm integration test to assert flexible config-disk path
 (LP: #1797199)
   - tools: Add cloud-id command line utility
   - instance-data: Add standard keys platform and subplatform. Refactor ec2.
   - net: ignore nics that have "zero" mac address. (LP: #1796917)
   - tests: fix apt_configure_primary to be more flexible
   - Ubuntu: update sources.list to comment out deb-src entries. (LP: #74747)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1808380/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1808380] [NEW] Release 18.5

[Ryan Harper]

Public bug reported:

This bug tracks cloud-init upstream release of version 18.5.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1808380

Title:
  Release 18.5

Status in cloud-init:
  New

Bug description:
  This bug tracks cloud-init upstream release of version 18.5.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1808380/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1735950] Re: ValueError: Old and New apt format defined with unequal values True vs False @ apt_preserve_sources_list

[Ryan Harper]

This bug is believed to be fixed in curtin in version 18.2. If this is
still a problem for you, please make a comment and set the state back to
New

Thank you.

** Changed in: curtin
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1735950

Title:
  ValueError: Old and New apt format defined with unequal values True vs
  False @ apt_preserve_sources_list

Status in cloud-init:
  Invalid
Status in curtin:
  Fix Released
Status in MAAS:
  Fix Released
Status in MAAS 2.3 series:
  Won't Fix

Bug description:
  All nodes have these same failed events:

  Node post-installation failure - 'cloudinit' running modules for
  config

  Node post-installation failure - 'cloudinit' running config-apt-
  configure with frequency once-per-instance

  
  Experiencing odd issues with the squid proxy not being reachable.

  From a deployed node that had the event errors.

  $ sudo cat /var/log/cloud-init.log | http://paste.ubuntu.com/26098787/
  $ sudo cat /var/log/cloud-init-output.log | http://paste.ubuntu.com/26098802/

  ubuntu@os-util-00:~$ sudo apt install htop
  sudo: unable to resolve host os-util-00
  Reading package lists... Done
  Building dependency tree   
  Reading state information... Done
  The following NEW packages will be installed:
htop
  0 upgraded, 1 newly installed, 0 to remove and 14 not upgraded.
  Need to get 76.4 kB of archives.
  After this operation, 215 kB of additional disk space will be used.
  Err:1 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 htop 
amd64 2.0.1-1ubuntu1
Could not connect to 10.10.0.110:8000 (10.10.0.110). - connect (113: No 
route to host)
  E: Failed to fetch 
http://archive.ubuntu.com/ubuntu/pool/universe/h/htop/htop_2.0.1-1ubuntu1_amd64.deb
  Could not connect to 10.10.0.110:8000 (10.10.0.110). - connect (113: No route 
to host)

  E: Unable to fetch some archives, maybe run apt-get update or try with
  --fix-missing?


  
  Not sure if these things are related (the proxy not being reachable, and the 
node event errors)  something is not right.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1735950/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1807030] Re: Deployment fails due to missing EFI directory on system with no EFI support

[Ryan Harper]

Please reopen if you feel there's an issue with how curtin is handling
the late_commands.

** Changed in: curtin
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1807030

Title:
  Deployment fails due to missing EFI directory on system with no EFI
  support

Status in cloud-init:
  Invalid
Status in curtin:
  Invalid
Status in MAAS:
  Invalid

Bug description:
  Attempting to deploy various versions of Ubuntu via MAAS 2.4.2
  (7034-g2f5deb8b8-0ubuntu1). I've tried 16.04.5 and 18.04.1 and end up
  with messages in the logs like such:


  Looking in the system settings, this system does not use EFI at all.
  It is purely BIOS mode, yet the installer is complaining about a
  missing efi directory when it fails.

  The machine is successfully commissioned, and commissioning does not
  detect EFI and thus does not create a /boot/efi partition as it is not
  necessary.  Watching the node boot via console, it clearly is doing a
  BIOS mode PXE boot from the NICs, it is not loading an EFI environment
  first.

  
  A search and skimming of the manuals for this model (ProLiant SL230s) shows 
that it has no EFI options available as well: 
  https://support.hpe.com/hpsc/doc/public/display?docId=c03239129
  https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c03239183

  Installation finished. No error reported.


  Running command ['udevadm', 'settle'] with allowed return codes [0]
  (capture=False)

  Running command ['umount', '/tmp/tmpmg3cwxp7/target/sys'] with allowed
  return codes [0] (capture=False)

  Running command ['umount', '/tmp/tmpmg3cwxp7/target/proc'] with
  allowed return codes [0] (capture=False)

  Running command ['umount', '/tmp/tmpmg3cwxp7/target/dev'] with allowed
  return codes [0] (capture=False)

  finish: cmd-install/stage-curthooks/builtin/cmd-curthooks: SUCCESS:
  curtin command curthooks

  start: cmd-install/stage-hook/builtin/cmd-hook: curtin command hook

  Finalizing /tmp/tmpmg3cwxp7/target

  finish: cmd-install/stage-hook/builtin/cmd-hook: SUCCESS: curtin
  command hook

  curtin: Installation failed with exception: Unexpected error while
  running command.

  Command: ['grep', 'efi', '/proc/mounts']

  Exit code: 1

  Reason: -

  Stdout: ''

  Stderr: ''

  A full paste of the install lot from the MAAS web ui is here:
  https://pastebin.canonical.com/p/6SCncBtHGd/

  And the node's config data from MAAS can be found here:
  https://pastebin.canonical.com/p/dbV7PTVnYw/

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1807030/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1795508] [NEW] cloud-init clean from within /var/lib/cloud-init/instance

[Ryan Harper]

Public bug reported:

Attempted to cloud-init clean from a directory clean will remove:

/var/lib/cloud/instance# cloud-init clean --logs --reboot 
Traceback (most recent call last):
  File "/usr/bin/cloud-init", line 9, in 
load_entry_point('cloud-init==18.3', 'console_scripts', 'cloud-init')()
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 904, in main
get_uptime=True, func=functor, args=(name, args))
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2514, in 
log_time
ret = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/clean.py", line 81, in 
handle_clean_args
exit_code = remove_artifacts(args.remove_logs, args.remove_seed)
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/clean.py", line 75, in 
remove_artifacts
return 1
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
next(self.gen)
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 832, in chdir
os.chdir(curr)
FileNotFoundError: [Errno 2] No such file or directory: 
'/var/lib/cloud/instances/ce3aca12-4e37-4ef9-bc51-170db3d25881'

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1795508

Title:
  cloud-init clean from within /var/lib/cloud-init/instance

Status in cloud-init:
  New

Bug description:
  Attempted to cloud-init clean from a directory clean will remove:

  /var/lib/cloud/instance# cloud-init clean --logs --reboot 
  Traceback (most recent call last):
File "/usr/bin/cloud-init", line 9, in 
  load_entry_point('cloud-init==18.3', 'console_scripts', 'cloud-init')()
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 904, in 
main
  get_uptime=True, func=functor, args=(name, args))
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2514, in 
log_time
  ret = func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/cloudinit/cmd/clean.py", line 81, in 
handle_clean_args
  exit_code = remove_artifacts(args.remove_logs, args.remove_seed)
File "/usr/lib/python3/dist-packages/cloudinit/cmd/clean.py", line 75, in 
remove_artifacts
  return 1
File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
  next(self.gen)
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 832, in chdir
  os.chdir(curr)
  FileNotFoundError: [Errno 2] No such file or directory: 
'/var/lib/cloud/instances/ce3aca12-4e37-4ef9-bc51-170db3d25881'

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1795508/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1759616] Re: [2.x] CentOS networking adds weird route

[Ryan Harper]

This is an expected route for RHEL/Centos defaults.  Please re-open if
you think cloud-init needs to do something here.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1759616

Title:
  [2.x] CentOS networking adds weird route

Status in cloud-init:
  Invalid
Status in MAAS:
  Incomplete

Bug description:
  MAAS CentOS deployed machine has a weird route:

  169.254.0.0/16 dev ens3 scope link metric 1002

  [centos@precise ~]$ ip route sh
  default via 192.168.122.1 dev ens3
  169.254.0.0/16 dev ens3 scope link metric 1002
  192.168.122.0/24 dev ens3 proto kernel scope link src 192.168.122.6
  192.168.133.0/24 via 192.168.122.1 dev ens3

  Deploying Ubuntu doesn't yield the same config.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1759616/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1749722] Re: NTP: take into account systemd-timesyncd where present

[Ryan Harper]

** Also affects: systemd (Ubuntu)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1749722

Title:
  NTP: take into account systemd-timesyncd where present

Status in cloud-init:
  In Progress
Status in systemd package in Ubuntu:
  New

Bug description:
  Currently, the NTP module configures ntpd during cloud-init install by
  installing and configuring ntpd.

  ntpd competes with systemd-timesyncd on systemd distros like Ubuntu
  Xenial.

  Ideally the NTP module should configure systemd-timesyncd where
  present, falling back to ntpd where not present.

  This stops two separate daemons (ntpd and systemd-timesyncd) competing
  with each other to set the time, where systemd-timesyncd (on Ubuntu at
  least) has an internal hardcoded compiled in timeserver to fall back
  on if no timeserver is configured (which is bad, but what can you do).

  The competing timeserver behaviour is invisible when the machine can
  see the net, but logs this error constantly when the machine cannot
  see the net:

  systemd-timesyncd[527]: Timed out waiting for reply from
  91.189.94.4:123 (ntp.ubuntu.com).

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1749722/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1721157] [NEW] netplan render drops bridge_stp setting

[Ryan Harper]

Public bug reported:

cloud-init tip fails to render a netplan bridge configuration with a stp
value.

% cat network_configs/bridge-simple.yaml 
showtrace: true
network:
version: 1
config:
# Physical interfaces.
- type: physical
  name: eth0
  mac_address: "52:54:00:12:34:00"
  subnets:
  - type: dhcp4
- type: physical
  name: eth1
  mac_address: "52:54:00:12:34:02"
# Bridge
- type: bridge
  name: br0
  bridge_interfaces:
- eth1
  params:
  bridge_fd: 150
  bridge_stp: 'off'


Currently renders a netplan config of:

network:
version: 2
ethernets:
eth0:
dhcp4: true
match:
macaddress: '52:54:00:12:34:00'
set-name: eth0
eth1:
match:
macaddress: '52:54:00:12:34:02'
set-name: eth1
bridges:
br0:
interfaces:
- eth1
parameters:
forward-delay: 150


netplan in artful with bridge stp support enables STP by default (unless 
disabled)
This prevents setting some values of forward-delay.  In any case, we should 
translate
the bridge_stp value to the netplan boolean.

** Affects: cloud-init
 Importance: Undecided
 Status: New


** Tags: curtin

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1721157

Title:
  netplan render drops bridge_stp setting

Status in cloud-init:
  New

Bug description:
  cloud-init tip fails to render a netplan bridge configuration with a
  stp value.

  % cat network_configs/bridge-simple.yaml 
  showtrace: true
  network:
  version: 1
  config:
  # Physical interfaces.
  - type: physical
name: eth0
mac_address: "52:54:00:12:34:00"
subnets:
- type: dhcp4
  - type: physical
name: eth1
mac_address: "52:54:00:12:34:02"
  # Bridge
  - type: bridge
name: br0
bridge_interfaces:
  - eth1
params:
bridge_fd: 150
bridge_stp: 'off'

  
  Currently renders a netplan config of:

  network:
  version: 2
  ethernets:
  eth0:
  dhcp4: true
  match:
  macaddress: '52:54:00:12:34:00'
  set-name: eth0
  eth1:
  match:
  macaddress: '52:54:00:12:34:02'
  set-name: eth1
  bridges:
  br0:
  interfaces:
  - eth1
  parameters:
  forward-delay: 150

  
  netplan in artful with bridge stp support enables STP by default (unless 
disabled)
  This prevents setting some values of forward-delay.  In any case, we should 
translate
  the bridge_stp value to the netplan boolean.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1721157/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1720426] [NEW] network-config should allow dhcp client option values

[Ryan Harper]

Public bug reported:

DHCP clients can send specific values in the dhcp request such as
hostname, lease time, client_id, vendor_id, etc.

The dhcp configuration option in v1, and v2 (and others) only has dhcp
as a boolean.


v1:
network:
  version: 1
  config:
- type: physical
  name: eth0
  mac_address: 00:11:22:33:44:55
  subnets:
- type: dhcp
  hostname: myhostname
  client:  client_id_1
  lease: 3600
  
Would render in eni:

auto iface eth0 dhcp
  hostname myhostname
  client: client_id_1
  lease 3600

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1720426

Title:
  network-config should allow dhcp client option values

Status in cloud-init:
  New

Bug description:
  DHCP clients can send specific values in the dhcp request such as
  hostname, lease time, client_id, vendor_id, etc.

  The dhcp configuration option in v1, and v2 (and others) only has dhcp
  as a boolean.

  
  v1:
  network:
version: 1
config:
  - type: physical
name: eth0
mac_address: 00:11:22:33:44:55
subnets:
  - type: dhcp
hostname: myhostname
client:  client_id_1
lease: 3600

  Would render in eni:

  auto iface eth0 dhcp
hostname myhostname
client: client_id_1
lease 3600

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1720426/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1716773] [NEW] cloud-init doesn't cache network_config property in cache

[Ryan Harper]

Public bug reported:

When init net stage runs, it will restore the cloud object cache in
/var/lib/cloud/instance/obj.pkl

This cache dump is created before cloud-init processes the datasource,
which means some of the data cloud-init has fetched is not cached.

In particular, we've observed that the 'network_config' attribute is
populated and set in init-local mode, but when we run init-net; it will
re-run due to the underlying object value still be set to None (the
default) as we failed to cache the object after modifying it.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1716773

Title:
  cloud-init doesn't cache network_config property in cache

Status in cloud-init:
  New

Bug description:
  When init net stage runs, it will restore the cloud object cache in
  /var/lib/cloud/instance/obj.pkl

  This cache dump is created before cloud-init processes the datasource,
  which means some of the data cloud-init has fetched is not cached.

  In particular, we've observed that the 'network_config' attribute is
  populated and set in init-local mode, but when we run init-net; it
  will re-run due to the underlying object value still be set to None
  (the default) as we failed to cache the object after modifying it.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1716773/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1712695] Re: Curtin network-passthrough configures /e/n/i.d/50-cloud-init.cfg instead of /e/n/i

[Ryan Harper]

They cannot just parse /etc/network/interfaces;  curtin has always
written out an eni that includes a source
/etc/network/interfaces.d/*.cfg (meaning juju has a bug if they don't
read all eni configs).

Anyone parsing eni needs to read in all of the configs.   In which case,
they would pickup any configs that cloud-init renders (including
/etc/network/interfaces.d/50-cloud-init.cfg

outside of maas, juju has to parse cloud-init rendered network config
(on clouds) anyhow which is always written at the location specified in
this bug.

** Changed in: curtin
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1712695

Title:
  Curtin network-passthrough configures /e/n/i.d/50-cloud-init.cfg
  instead of /e/n/i

Status in cloud-init:
  New
Status in curtin:
  Invalid
Status in MAAS:
  New

Bug description:
  When using curtin's network passthrough, it writes config in
  /etc/network/interfaces.d/50-cloud-init.cfg instead of
  /etc/network/interfaces.

  This is likely to break juju provided that (i believe) it parses
  /e/n/i to do their bridge configuration.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1712695/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1710932] Re: Doesn't configure landscape client

[Ryan Harper]

You've an error in your cloud-config file

ubuntu@withkvm:/var/log$ cat /etc/cloud/cloud.cfg.d/99_landscape_client

cloud-config files need a .cfg extension to be read.

If I use your landscape config this way, I see landscape-client
installed and file written.

% cat landscape.yaml 
#cloud-config
landscape:
  client:
url: "https://192.168.122.1/message-system";
ping_url: "http://192.168.122.1/ping";
data_path: "/var/lib/landscape/client"
#http_proxy: "http://my.proxy.com/foobar";
tags: "maas"
computer_title: withkvm
#https_proxy: fooproxy
registration_key: test
account_name: standalone

% lxc launch ubuntu-daily:artful --config=user.user-data="$(cat
landscape.yaml)"

# wait a bit for install to complete

% lxc exec measured-hyena -- dpkg --list | grep landscape
ii  landscape-client   16.03-0ubuntu3   
 amd64The Landscape administration system client
ii  landscape-common   16.03-0ubuntu3   
 amd64The Landscape administration system client - Common files

it appears to be working fine.  If you feel there's still a bug, please
re-open.


** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1710932

Title:
  Doesn't configure landscape client

Status in cloud-init:
  Invalid

Bug description:
  ubuntu@withkvm:/var/log$ cat cloud-init.log | grep landscape
  2017-08-15 17:17:28,660 - stages.py[DEBUG]: Running module landscape () with 
frequency once-per-instance
  2017-08-15 17:17:28,660 - handlers.py[DEBUG]: start: 
modules-final/config-landscape: running config-landscape with frequency 
once-per-instance
  2017-08-15 17:17:28,664 - util.py[DEBUG]: Writing to 
/var/lib/cloud/instances/yar6cr/sem/config_landscape - wb: [420] 25 bytes
  2017-08-15 17:17:28,665 - helpers.py[DEBUG]: Running config-landscape using 
lock ()
  2017-08-15 17:17:28,665 - handlers.py[DEBUG]: finish: 
modules-final/config-landscape: SUCCESS: config-landscape ran successfully

  ubuntu@withkvm:/var/log$ cat /etc/cloud/cloud.cfg.d/99_landscape_client 
  landscape:
client:
  url: "https://192.168.122.1/message-system";
  ping_url: "http://192.168.122.1/ping";
  data_path: "/var/lib/landscape/client"
  #http_proxy: "http://my.proxy.com/foobar";
  tags: "maas"
  computer_title: withkvm
  #https_proxy: fooproxy
  registration_key: test
  account_name: standalone

  ubuntu@withkvm:/var/log$ dpkg -l | grep landscape-client
  ubuntu@withkvm:/var/log$ cat /etc/landscape/client.conf
  cat: /etc/landscape/client.conf: No such file or directory

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1710932/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1658734] Re: DNS queries of does-not-exist.example.com and example.invalid

[Ryan Harper]

** Bug watch added: Red Hat Bugzilla #1468192
   https://bugzilla.redhat.com/show_bug.cgi?id=1468192

** Also affects: cloud-init (CentOS) via
   https://bugzilla.redhat.com/show_bug.cgi?id=1468192
   Importance: Unknown
   Status: Unknown

** Merge proposal linked:
   https://code.launchpad.net/~rmccabe/cloud-init/+git/cloud-init/+merge/328877

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1658734

Title:
  DNS queries of does-not-exist.example.com and example.invalid

Status in cloud-init:
  Confirmed
Status in cloud-init package in CentOS:
  Unknown

Bug description:
  cloud-init makes several DNS queries for does-not-exist.example.com
  and example.invalid (and also some random names).
  https://git.launchpad.net/cloud-init/tree/cloudinit/util.py#n1100

  We understand that it does this to detect the kind of DNS redirection
  that's done an many universities, some ISPs, and services like OpenDNS
  (when used for filtering or typo correction).

  However, it can be problematic in an environment where an intrusion
  detection system might flag these queries as potentially malicious,
  and in a system where DNS redirection is not used it unnecessarily
  increases boot time.

  It looks like the feature was written to make it possible to disable
  it or provide specific redirection IPs, but that it never gained a
  config option to control it.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1658734/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1709715] [NEW] cloud-init apply_net_config_names doesn't grok v2 configs

[Ryan Harper]

Public bug reported:

when supplying cloud-init with a network-configuration in version:2
format, the rename code doesn't find any of the set-name parameters as
the function expects a v1 config format.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1709715

Title:
  cloud-init apply_net_config_names doesn't grok v2 configs

Status in cloud-init:
  New

Bug description:
  when supplying cloud-init with a network-configuration in version:2
  format, the rename code doesn't find any of the set-name parameters as
  the function expects a v1 config format.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1709715/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp