[Yahoo-eng-team] [Bug 2012144] [NEW] [OVN] adding/removing floating IPs neutron server errors about binding port

[Ryan M Belgrave]

Public bug reported:

Using Zed and Ubuntu and OVN as the ml2 driver.

Neutron Server Version 21.0.0
OVN Version 22.09.0

When adding/removing floating IPs the neutron server errors with the
following

2023-02-23 03:30:05.842 25044 INFO
neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-
accc7595-320b-47e4-93e4-b07f6b205295 - - - - - -] Refusing to bind port
5266e0cd-1064-4baa-9679-8c5f2eb13d29 on host sora due to the OVN chassis
bridge mapping physical networks [] not supporting physical network:
provider

2023-02-23 03:30:05.843 25044 ERROR neutron.plugins.ml2.managers [None
req-accc7595-320b-47e4-93e4-b07f6b205295 - - - - - -] Failed to bind
port 5266e0cd-1064-4baa-9679-8c5f2eb13d29 on host sora for vnic_type
normal using segments [{'id': '5621a693-771d-4a57-beb4-d7a6e8dfc1b9',
'network_type': 'flat', 'physical_network': 'provider',
'segmentation_id': None, 'network_id':
'71cbb38e-dc91-4db4-9a3a-7e499cd3fd69'}]

The floating IPs work as expected though so I am unsure why this error
is given.

The host has been setup with the following bridge and mapping

ovs-vsctl --may-exist add-br br-provider -- set bridge br-provider 
protocols=OpenFlow13
ovs-vsctl set open . external-ids:ovn-bridge-mappings=provider:br-provider
ovs-vsctl --may-exist add-port br-provider veth1-provider


Looking at the ovn driver code I can see it gets the ovn bridge mappings here 
https://github.com/openstack/neutron/blob/stable/zed/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py#L850

and if I get the mappings by hand I can see them set as expected bellow:

# ovn-sbctl list Chassis
_uuid   : ac259942-1da8-425e-aaad-40f861771353
encaps  : [17ba4674-5955-4dd7-847d-37ffc94dbc38, 
3bf53123-6d0b-4422-8afc-7b52530ea782]
external_ids: {}
hostname: sora
name: "b8edbb46-b62a-4ca2-be87-872a83eb03d5"
nb_cfg  : 0
other_config: {ct-no-masked-label="true", datapath-type=system, 
iface-types="bareudp,erspan,geneve,gre,gtpu,internal,ip6erspan,ip6gre,lisp,patch,stt,system,tap,vxlan",
 is-interconn="false", mac-binding-timestamp="true", 
ovn-bridge-mappings="provider:br-provider", ovn-chassis-mac-mappings="", 
ovn-cms-options=enable-chassis-as-gw, ovn-enable-lflow-cache="true", 
ovn-limit-lflow-cache="", ovn-memlimit-lflow-cache-kb="", 
ovn-monitor-all="false", ovn-trim-limit-lflow-cache="", ovn-trim-timeout-ms="", 
ovn-trim-wmark-perc-lflow-cache="", port-up-notif="true"}
transport_zones : []
vtep_logical_switches: []

I believe this error is being generated here
https://github.com/openstack/neutron/blob/stable/zed/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py#L1004

but I am unsure why since everything still seems to work?

** Affects: neutron
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2012144

Title:
  [OVN] adding/removing floating IPs neutron server errors about binding
  port

Status in neutron:
  New

Bug description:
  Using Zed and Ubuntu and OVN as the ml2 driver.

  Neutron Server Version 21.0.0
  OVN Version 22.09.0

  When adding/removing floating IPs the neutron server errors with the
  following

  2023-02-23 03:30:05.842 25044 INFO
  neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-
  accc7595-320b-47e4-93e4-b07f6b205295 - - - - - -] Refusing to bind
  port 5266e0cd-1064-4baa-9679-8c5f2eb13d29 on host sora due to the OVN
  chassis bridge mapping physical networks [] not supporting physical
  network: provider

  2023-02-23 03:30:05.843 25044 ERROR neutron.plugins.ml2.managers [None
  req-accc7595-320b-47e4-93e4-b07f6b205295 - - - - - -] Failed to bind
  port 5266e0cd-1064-4baa-9679-8c5f2eb13d29 on host sora for vnic_type
  normal using segments [{'id': '5621a693-771d-4a57-beb4-d7a6e8dfc1b9',
  'network_type': 'flat', 'physical_network': 'provider',
  'segmentation_id': None, 'network_id':
  '71cbb38e-dc91-4db4-9a3a-7e499cd3fd69'}]

  The floating IPs work as expected though so I am unsure why this error
  is given.

  The host has been setup with the following bridge and mapping

  ovs-vsctl --may-exist add-br br-provider -- set bridge br-provider 
protocols=OpenFlow13
  ovs-vsctl set open . external-ids:ovn-bridge-mappings=provider:br-provider
  ovs-vsctl --may-exist add-port br-provider veth1-provider

  
  Looking at the ovn driver code I can see it gets the ovn bridge mappings here 
https://github.com/openstack/neutron/blob/stable/zed/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py#L850

  and if I get the mappings by hand I can see them set as expected
  bellow:

  # ovn-sbctl list Chassis
  _uuid   : ac259942-1da8-425e-aaad-40f861771353
  encaps  : [17ba4674-5955-4dd7-847d-37ffc94dbc38, 
3bf53123-6d0b-4422-8afc-7b52530ea782]
  external_ids: {}
  hostname: sora
  

[Yahoo-eng-team] [Bug 1940535] [NEW] CPU topologies documentation possible bug

[Matt Ryan]

Public bug reported:


This bug tracker is for errors with the documentation, use the following
as a template and remove or add fields as you see fit. Convert [ ] into
[x] to check boxes:

- [ ] This doc is inaccurate in this way: __
- [ ] This is a doc addition request.
- [x] I have a fix to the document that I can paste below including example: 
input and output. 

The line

"This will ensure the instance gets scheduled to a host with SMT by
requesting hosts that do not report the HW_CPU_HYPERTHREADING trait."

should be

"This will ensure the instance gets scheduled to a host without SMT by
requesting hosts that do not report the HW_CPU_HYPERTHREADING trait."

note the change from "with" to "without".


If you have a troubleshooting or support issue, use the following  resources:

 - The mailing list: https://lists.openstack.org
 - IRC: 'openstack' channel on OFTC

---
Release: 23.1.0.dev370 on 2021-02-22 12:52:17
SHA: 54b48b7dd42b68270189fe1b49bc67db5b94795d
Source: 
https://opendev.org/openstack/nova/src/doc/source/admin/cpu-topologies.rst
URL: https://docs.openstack.org/nova/latest/admin/cpu-topologies.html

** Affects: nova
 Importance: Undecided
 Status: New


** Tags: doc

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1940535

Title:
  CPU topologies documentation possible bug

Status in OpenStack Compute (nova):
  New

Bug description:

  This bug tracker is for errors with the documentation, use the
  following as a template and remove or add fields as you see fit.
  Convert [ ] into [x] to check boxes:

  - [ ] This doc is inaccurate in this way: __
  - [ ] This is a doc addition request.
  - [x] I have a fix to the document that I can paste below including example: 
input and output. 

  The line

  "This will ensure the instance gets scheduled to a host with SMT by
  requesting hosts that do not report the HW_CPU_HYPERTHREADING trait."

  should be

  "This will ensure the instance gets scheduled to a host without SMT by
  requesting hosts that do not report the HW_CPU_HYPERTHREADING trait."

  note the change from "with" to "without".

  
  If you have a troubleshooting or support issue, use the following  resources:

   - The mailing list: https://lists.openstack.org
   - IRC: 'openstack' channel on OFTC

  ---
  Release: 23.1.0.dev370 on 2021-02-22 12:52:17
  SHA: 54b48b7dd42b68270189fe1b49bc67db5b94795d
  Source: 
https://opendev.org/openstack/nova/src/doc/source/admin/cpu-topologies.rst
  URL: https://docs.openstack.org/nova/latest/admin/cpu-topologies.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1940535/+subscriptions


-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1927020] Re: cloudconfig not writing maas data source

[Ryan Harper]

I'm marking the cloud-init task invalid.  After getting config and logs
we confirmed that the cloud-init package in the Debian image does not
contain a recent handle_maas_preseed() that Ubuntu uses and curtin/MAAS
rely upon to install the MAAS datasource correctly.

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1927020

Title:
  cloudconfig not writing maas data source

Status in cloud-init:
  Invalid
Status in curtin:
  Invalid

Bug description:
  further background https://discourse.maas.io/t/debian10-fails-on-
  final-reboot/4486

  I'm deploying a debian 10 buster image using MAAS. No errors are
  reported in MAAS or curtin install but on the final boot cloud-init
  reports `Failed to load metadata and userdata`. checking the config
  for the machine in maas shows all the cloudconfig: to connect to maas
  but the files with oauth credentials etc dont seem to have been copied
  to the target.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1927020/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1927020] Re: cloudconfig not writing maas data source

[Ryan Harper]

I'm marking the curtin task invalid; after getting config and logs we
confirmed that the cloud-init package in the Debian image does not
contain a recent handle_maas_preseed() that Ubuntu uses and curtin/MAAS
rely upon to install the MAAS datasource correctly.

** Changed in: curtin
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1927020

Title:
  cloudconfig not writing maas data source

Status in cloud-init:
  Invalid
Status in curtin:
  Invalid

Bug description:
  further background https://discourse.maas.io/t/debian10-fails-on-
  final-reboot/4486

  I'm deploying a debian 10 buster image using MAAS. No errors are
  reported in MAAS or curtin install but on the final boot cloud-init
  reports `Failed to load metadata and userdata`. checking the config
  for the machine in maas shows all the cloudconfig: to connect to maas
  but the files with oauth credentials etc dont seem to have been copied
  to the target.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1927020/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1925454] Re: Update Ubuntu 20.04 default network-config by cloud-config

[Ryan Harper]

I'm marking this bug as Invalid.  cloud-init is working as expected for
NoCloud datasource.  Changes to the network-config during execution
can't affect cloud-inits behavior during boot with the NoCloud
datasource.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1925454

Title:
  Update Ubuntu 20.04 default network-config  by cloud-config

Status in cloud-init:
  Invalid

Bug description:
  This is a follow-up on the case https://bugs.launchpad.net/cloud-
  init/+bug/1924922

  I cannot update default Ubuntu network-config by writing user-data
  file in cloud-config.

  Attached are logs. Below is a snippet from the cloud-config

  #cloud-config
  write_files:
- path: /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
  permissions: '0644'
  content: |
network: 
  config: disabled
- path: /etc/netplan/50-cloud-init.yaml
  permissions: '0644'
  content: |
network:
  version: 2
  ethernets:
eth0:
  dhcp4: true
  addresses:
- 192.168.53.3/26
  gateway4: 192.168.53.62
  nameservers:
addresses:
  - 1.1.1.1
  - 8.8.8.8
  runcmd:
- [sudo, netplan, try]
- [sudo, netplan, generate]
- [sudo, netplan, apply]
  power_state:
mode: 'reboot'
message: 'reboot triggered by cloud-init'

  -

  Result is such that the hardcoded network-config ('optional': True, not 
waiting for DHCP server) is attached/generated with IP as, for an example - 
192.168.53.4, not the IP I really want 192.168.53.3. This leads to couple of 
problems, one as specified in bug/1924922, that ssh_import_id is not working 
(for the time is not sync up)
  Is there a way to overwrite this default with user-data, and not to go to the 
config partition and manually change it?

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1925454/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1924922] Re: ssh-import-id gives bad handshake

[Ryan Harper]

I'm marking this bug invalid, cloud-init is doing what it was told to do
and it appears looks like the user-data config relies on networking but
the network-config was marked optional.

Please move this bug state back to New if you have further questions or
issues.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1924922

Title:
  ssh-import-id gives bad handshake

Status in cloud-init:
  Invalid

Bug description:
  Ubuntu 20.04 server runs with user data as below,  yet it raises the
  error from launchpad.net

  2020-04-01 17:24:18,815 ERROR HTTPSConnectionPool(host='launchpad.net', 
port=443): Max retries exceeded with url: /~aniecki/+sshkeys (C
  aused by SSLError(SSLError("bad handshake: Error([('SSL routines', 
'tls_process_server_certificate', 'certificate verify failed')])")))
  Cloud-init v. 20.4.1-0ubuntu1~20.04.1 running 'modules:config' at Wed, 01 Apr 
2020 17:24:17 +. Up 42.20 seconds.

  Apparently this is the error because the host time is not yet synch
  up, but the ntp is also set there,

  
  #cloud-config

  ntp:
enabled: true
ntp_client: chrony

  users:
- name: kevo
  primary-group: users
  shell: /bin/bash
  sudo: ALL=(ALL) NOPASSWD:ALL
  groups: users
  ssh_import_id: aniecki
  lock_passwd: true
  ssh_pwauth: false

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1924922/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1893770] Re: Cloudinit resets network scripts to default configuration DHCP once Config Drive is removed after sometime

[Ryan Harper]

** Changed in: cloud-init
   Status: Expired => In Progress

** Changed in: cloud-init
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1893770

Title:
  Cloudinit resets network scripts to default configuration DHCP once
  Config Drive is removed after sometime

Status in cloud-init:
  In Progress

Bug description:
  Cloudinit version: 19.1
  Platform: OpenStack based.
  OS: RHEL, SUSE

  We use config drive(/dev/sr0) as a data source to configure network
  interfaces in the guest but configdrive is not always available and
  may be removed after couple of hours from the hypervisor.

  On a first boot cloudinit uses data provided in config drive and
  updates system level network scripts /etc/sysconfig/ifcfg-* (Static
  configuration of networks) files and also configures the interface in
  the guest.

  As long as the configdrive is available, reboots will relay on system
  scripts for the configuring network but once configdrive is removed,
  datasource becomes None meaning it neither system script nor config
  drive

  which makes cloud init to configure default network which is DHCP

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1893770/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1899487] Re: cloud-init hard codes MTU configuration at initial deploy time

[Ryan Harper]

" # curl http://169.254.169.254/openstack/2018-08-27/network_data.json
{"links": [{"id": "tapa035fb68-01", "vif_id": 
"a035fb68-010c-42e3-8da7-ea3c36a0d607", "type": "ovs", "mtu": 8942, 
"ethernet_mac_address": "fa:16:3e:31:26:f7"}], "networks": [{"id": "network0", 
"type": "ipv4_dhcp", "link": "tapa035fb68-01", "network_id": 
"b4ef84c0-1235-48a8-aaf7-03fab7ef5367"}], "services": []}"

How is cloud-init to know from this network-config.json that DHCP will
provide an MTU value?  How does it know that it should ignore the
provided MTU?  If DHCP is providing MTU, should network-config.json then
not provide the MTU value?


** Also affects: netplan.io (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: New => Incomplete

** Changed in: netplan.io (Ubuntu)
   Status: New => Incomplete

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1899487

Title:
  cloud-init hard codes MTU configuration at initial deploy time

Status in cloud-init:
  Incomplete
Status in netplan.io package in Ubuntu:
  Incomplete

Bug description:
  When using OpenStack cloud provider `cloud-init` will write out
  /etc/netplan/50-cloud-init.yaml at initial instance boot and not
  update it on subsequent boots of the instance.

  The OpenStack metadata service provides information about MTU for the
  network [0] and cloud-init takes this value and writes it into the
  netplan configuration [1].

  A side effect of configuring the MTU through netplan is that the
  `systemd-networkd` [Link] section [2] gets the MTUBytes value filled
  and this in turn makes `systemd-networkd` ignore the MTU value
  provided by DHCP [3][4].

  During the lifetime of a cloud events occur that will force a operator
  to reduce the MTU available to instances attached to its overlay
  networks. This may happen because of software imposed change of tunnel
  type (GRE -> VXLAN, VXLAN -> GENEVE) or change of topology or
  encapsulation in the physical network equipment.

  To maximize performance these clouds have configured their instances
  to use the maximum available MTU without leaving any headroom to
  account for such changes and the only way to move forward is to reduce
  the available MTU on the instances. We are facing a concrete challenge
  with this now where we have users wanting to migrate from VXLAN
  tunnels to GENEVE tunnels with 38 byte header size.

  0: # curl http://169.254.169.254/openstack/2018-08-27/network_data.json
  {"links": [{"id": "tapa035fb68-01", "vif_id": 
"a035fb68-010c-42e3-8da7-ea3c36a0d607", "type": "ovs", "mtu": 8942, 
"ethernet_mac_address": "fa:16:3e:31:26:f7"}], "networks": [{"id": "network0", 
"type": "ipv4_dhcp", "link": "tapa035fb68-01", "network_id": 
"b4ef84c0-1235-48a8-aaf7-03fab7ef5367"}], "services": []}

  1: # cat /etc/netplan/50-cloud-init.yaml 
  # This file is generated from information provided by the datasource.  Changes
  # to it will not persist across an instance reboot.  To disable cloud-init's
  # network configuration capabilities, write a file
  # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
  # network: {config: disabled}
  network:
  version: 2
  ethernets:
  ens2:
  dhcp4: true
  match:
  macaddress: fa:16:3e:31:26:f7
  mtu: 8950
  set-name: ens2

  2: # cat /run/systemd/network/10-netplan-ens2.link 
  [Match]
  MACAddress=fa:16:3e:31:26:f7

  [Link]
  Name=ens2
  WakeOnLan=off
  MTUBytes=8950

  3: # cat /run/systemd/network/10-netplan-ens2.network 
  [Match]
  MACAddress=fa:16:3e:31:26:f7
  Name=ens2

  [Link]
  MTUBytes=8950

  [Network]
  DHCP=ipv4
  LinkLocalAddressing=ipv6

  [DHCP]
  RouteMetric=100
  UseMTU=true

  4: Oct 12 13:30:18 canary-3 systemd-networkd[24084]:
  /run/systemd/network/10-netplan-ens2.network: MTUBytes= in [Link]
  section and UseMTU= in [DHCP] section are set. Disabling UseMTU=.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1899487/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1890803] Re: Groovy amd64 / arm64 / PowerPC deployment seems not working

[Ryan Harper]

@Sean  you're right...  the cloud-init fix was to handle the case there is *no* 
/etc/fstab ... 
so any other place that expects /etc/fstab to exist must also handle missing 
file.

Looking at overlayroot, it expects to be able to update/modify the
baked-in fstab, but it's not there.

https://git.launchpad.net/cloud-initramfs-tools/tree/overlayroot/scripts
/init-bottom/overlayroot#n313


** Also affects: cloud-initramfs-tools
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1890803

Title:
  Groovy amd64 / arm64 / PowerPC deployment seems not working

Status in cloud-init:
  Invalid
Status in cloud-initramfs-tools:
  New
Status in MAAS:
  Triaged
Status in ubuntu-kernel-tests:
  New

Bug description:
  All bare-metal deployment tasks have failed on our bare-metal maas
  server and the PowerMAAS / HyperMAAS (image synced on maas server).

  Deployment failed with:
    Installation was aborted.

  Need to further investigate this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1890803/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1890803] Re: Groovy amd64 / arm64 / PowerPC deployment seems not working

[Ryan Harper]

I'm marking cloud-init task invalid; by the time cloud-init runs, rootfs
should be rw using overlayfs backed by tmpfs.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1890803

Title:
  Groovy amd64 / arm64 / PowerPC deployment seems not working

Status in cloud-init:
  Invalid
Status in MAAS:
  Incomplete
Status in ubuntu-kernel-tests:
  New

Bug description:
  All bare-metal deployment tasks have failed on our bare-metal maas
  server and the PowerMAAS / HyperMAAS (image synced on maas server).

  Deployment failed with:
    Installation was aborted.

  Need to further investigate this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1890803/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1890717] [NEW] Keystone deploy enters ERROR state with nothing running on port 35337

[Ryan Farrell]

Public bug reported:

A fresh deployment of Keystone has failed; the charm entered and error
state then become blocked with a message indicating it was waiting for a
certain number of peers.

2020-08-06 17:15:33 DEBUG identity-service-relation-changed
RuntimeError: The call within manager.py failed with the error: 'Unable
to establish connection to http://localhost:35337/v3/auth/tokens:
HTTPConnectionPool(host='localhost', port=35337): Max retries exceeded
with url: /v3/auth/tokens (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused',))'. The call was: path=['resolve_role_id'],
args=('member',), kwargs={}, api_version=None

Logs indicated that there was nothing listing on port 35337, which was
confirmed with nc:

# keystone/0  - working unit
ubuntu@juju-2bccdc-9-lxd-1:~$ sudo netstat -tupln | grep 35337  

   
tcp6   0  0 :::35337:::*LISTEN  
30815/apache2

#keystone/1  - non working unit
ubuntu@juju-2bccdc-10-lxd-1:~$ sudo netstat -tupln | grep 35337


Additionally the directory: /etc/apache2/ssl/keystone was completely empty - we 
expected there to be symlinks to certs / keyst there.


This deployment was using cs:~openstack-charmers-next/keystone-500 - 
redeploying the unit was successful.

** Affects: keystone
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1890717

Title:
  Keystone deploy enters ERROR state with nothing running on port  35337

Status in OpenStack Identity (keystone):
  New

Bug description:
  A fresh deployment of Keystone has failed; the charm entered and error
  state then become blocked with a message indicating it was waiting for
  a certain number of peers.

  2020-08-06 17:15:33 DEBUG identity-service-relation-changed
  RuntimeError: The call within manager.py failed with the error:
  'Unable to establish connection to
  http://localhost:35337/v3/auth/tokens:
  HTTPConnectionPool(host='localhost', port=35337): Max retries exceeded
  with url: /v3/auth/tokens (Caused by
  NewConnectionError(': Failed to establish a new connection: [Errno 111]
  Connection refused',))'. The call was: path=['resolve_role_id'],
  args=('member',), kwargs={}, api_version=None

  Logs indicated that there was nothing listing on port 35337, which was
  confirmed with nc:

  # keystone/0  - working unit
  ubuntu@juju-2bccdc-9-lxd-1:~$ sudo netstat -tupln | grep 35337

 
  tcp6   0  0 :::35337:::*LISTEN
  30815/apache2

  #keystone/1  - non working unit
  ubuntu@juju-2bccdc-10-lxd-1:~$ sudo netstat -tupln | grep 35337

  
  Additionally the directory: /etc/apache2/ssl/keystone was completely empty - 
we expected there to be symlinks to certs / keyst there.

  
  This deployment was using cs:~openstack-charmers-next/keystone-500 - 
redeploying the unit was successful.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1890717/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1888298] Re: cc_timezone fails on Ubuntu Bionic and Xenial minimal

[Ryan Harper]

https://cloud-images.ubuntu.com/daily/server/minimal/daily/focal/current
/focal-minimal-cloudimg-amd64.manifest

Has tzdata ...

This looks like a cloudimg issue, not cloud-init.

https://cloud-
images.ubuntu.com/daily/server/minimal/daily/bionic/current/bionic-
minimal-cloudimg-amd64.manifest


** Also affects: cloud-images
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1888298

Title:
  cc_timezone fails on Ubuntu Bionic and Xenial minimal

Status in cloud-images:
  New
Status in cloud-init:
  Incomplete

Bug description:
  Summary
  ===
  On Ubuntu Bionic and Xenial minimal images, there is no tzdata package. As a 
result, when cloud-init tries to set the timezone it will fail and produce a 
stack trace.

  Expected Result
  ===
  No trace and no failure of the cloud-config.service :)

  Actual result
  ===
  2020-07-20 18:13:22,515 - util.py[DEBUG]: Running module timezone () failed
    File "/usr/lib/python3/dist-packages/cloudinit/config/cc_timezone.py", line 
47, in handle
  cloud.distro.set_timezone(timezone)
    File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 
165, in set_timezone
  distros.set_etc_timezone(tz=tz, tz_file=self._find_tz_file(tz))
  OSError: Invalid timezone America/Vancouver, no file found at 
/usr/share/zoneinfo/America/Vancouver

  Steps to reproduce
  ===
  $ wget 
https://cloud-images.ubuntu.com/daily/server/minimal/releases/bionic/release/ubuntu-18.04-minimal-cloudimg-amd64.img
  $ multipass launch file:///$(pwd)/ubuntu-18.04-minimal-cloudimg-amd64.img 
--name=bionic-minimal
  $ multipass exec bionic-minimal -- sudo systemctl list-units --failed 
--no-legend
  # note that cloud-config.service fails
  $ multipass exec bionic-minimal -- sudo cat /var/log/cloud-init.log | grep 
timezone

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/1888298/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1887380] [NEW] Attaching virtual GPU devices to guests in nova

[ryan]

Public bug reported:


This bug tracker is for errors with the documentation, use the following
as a template and remove or add fields as you see fit. Convert [ ] into
[x] to check boxes:

- [X] This is a doc addition request.

Hi, a problem came up when we are using nova(Queens) configured with the
vGPU feature to create several instances. It seems multiple instances
preempt the same vGPU resource, in our case, on the exact same instance
which has acquired a vGPU already. Here is the error reported in the
log:

"libvirt.libvirtError: Requested operation is not valid: mediated device
/sys/bus/mdev/devices/xxx is in use by driver QEMU, domain xxx"

Apparently, nova is trying to allocate the vGPU resource that is already
being used by another instance. Also, we ruled out a situation that
there is not enough vGPU resources on the host. In our case, 25% of
instances fell into error-creating state while we are only creating
instances which only need 50% of all vGPU resources. From our
perspective, the problem is with the nova-scheduler. Any idea how to
work this out?

Thanks

Ruien Zhang
zhangru...@bytedance.com

---
Release: 21.1.0.dev214 on 2020-04-28 20:09:00
SHA: d19f1ac47b0a5fe1dd80b7187087e5810501f16c
Source: https://opendev.org/openstack/nova/src/doc/source/admin/virtual-gpu.rst
URL: https://docs.openstack.org/nova/latest/admin/virtual-gpu.html

** Affects: nova
 Importance: Undecided
 Status: New


** Tags: doc

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1887380

Title:
  Attaching virtual GPU devices to guests in nova

Status in OpenStack Compute (nova):
  New

Bug description:

  This bug tracker is for errors with the documentation, use the
  following as a template and remove or add fields as you see fit.
  Convert [ ] into [x] to check boxes:

  - [X] This is a doc addition request.

  Hi, a problem came up when we are using nova(Queens) configured with
  the vGPU feature to create several instances. It seems multiple
  instances preempt the same vGPU resource, in our case, on the exact
  same instance which has acquired a vGPU already. Here is the error
  reported in the log:

  "libvirt.libvirtError: Requested operation is not valid: mediated
  device /sys/bus/mdev/devices/xxx is in use by driver QEMU, domain xxx"

  Apparently, nova is trying to allocate the vGPU resource that is
  already being used by another instance. Also, we ruled out a situation
  that there is not enough vGPU resources on the host. In our case, 25%
  of instances fell into error-creating state while we are only creating
  instances which only need 50% of all vGPU resources. From our
  perspective, the problem is with the nova-scheduler. Any idea how to
  work this out?

  Thanks

  Ruien Zhang
  zhangru...@bytedance.com

  ---
  Release: 21.1.0.dev214 on 2020-04-28 20:09:00
  SHA: d19f1ac47b0a5fe1dd80b7187087e5810501f16c
  Source: 
https://opendev.org/openstack/nova/src/doc/source/admin/virtual-gpu.rst
  URL: https://docs.openstack.org/nova/latest/admin/virtual-gpu.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1887380/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1883907] Re: EC2 data source does not properly return the instance ID when cached data exists

[Ryan Harper]

I'm marking this invalid; if you find more information that would
indicate that cloud-init is not booting like a new instance after
capturing on Ec2, please reopen this bug and include updated
information.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1883907

Title:
  EC2 data source does not properly return the instance ID when cached
  data exists

Status in cloud-init:
  Invalid

Bug description:
  When creating an AMI from a running instance without cleaning the
  cloud-init cache an instance from the new AMI is no properly
  identified as a new instance and none of the PER_INSTANCE tasks will
  be executed.

  The problem is that the Ec2 data source will return the cached
  instance-id rather than the new instance ID.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1883907/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1881814] Re: OpenStack datasource detection fails during init on Hyper-V compute nodes

[Ryan Harper]

I'm marking the cloud-init task invalid; if you believe there is still a
bug in cloud-init, please re-open this task along with new information
explaining the bug.


** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1881814

Title:
  OpenStack datasource detection fails during init on Hyper-V compute
  nodes

Status in cloud-init:
  Invalid
Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  The OpenStack datasource detection fails on an Hyper-V compute (Nova)
  node (running on-premises, no cloud provider). It does not try to
  attempt to use the metdata endpoint and the instance fails to
  initialize properly unless another datasource is used (like
  ConfigDrive or EC2, the latter which gives a warning).

  The issue is in cloudinit/sources/DataSourceOpenStack.py, in the
  detect_openstack function. The DMI data does not match anything
  expected.

  In Hyper-V the following info is returned from DMI:

  grep "" /sys/class/dmi/id/*

  /sys/class/dmi/id/bios_date:05/23/2012
  /sys/class/dmi/id/bios_vendor:American Megatrends Inc.
  /sys/class/dmi/id/bios_version:090006
  /sys/class/dmi/id/board_name:Virtual Machine
  /sys/class/dmi/id/board_serial:8096-7783-9998-*snip*
  /sys/class/dmi/id/board_vendor:Microsoft Corporation
  /sys/class/dmi/id/board_version:7.0
  /sys/class/dmi/id/chassis_asset_tag:8096-7783-9998-*snip*
  /sys/class/dmi/id/chassis_serial:8096-7783-9998-*snip*
  /sys/class/dmi/id/chassis_type:3
  /sys/class/dmi/id/chassis_vendor:Microsoft Corporation
  /sys/class/dmi/id/chassis_version:7.0
  
/sys/class/dmi/id/modalias:dmi:bvnAmericanMegatrendsInc.:bvr090006:bd05/23/2012:svnMicrosoftCorporation:pnVirtualMachine:pvr7.0:rvnMicrosoftCorporation:rnVirtualMachine:rvr7.0:cvnMicrosoftCorporation:ct3:cvr7.0:
  grep: /sys/class/dmi/id/power: Is a directory
  /sys/class/dmi/id/product_name:Virtual Machine
  /sys/class/dmi/id/product_serial:8096-7783-9998-*snip*
  /sys/class/dmi/id/product_uuid:f41d9e0e-c208-*snip*
  /sys/class/dmi/id/product_version:7.0
  grep: /sys/class/dmi/id/subsystem: Is a directory
  /sys/class/dmi/id/sys_vendor:Microsoft Corporation
  
/sys/class/dmi/id/uevent:MODALIAS=dmi:bvnAmericanMegatrendsInc.:bvr090006:bd05/23/2012:svnMicrosoftCorporation:pnVirtualMachine:pvr7.0:rvnMicrosoftCorporation:rnVirtualMachine:rvr7.0:cvnMicrosoftCorporation:ct3:cvr7.0:

  cat /proc/1/environ

  HOME=/ init=/sbin/init NETWORK_SKIP_ENSLAVED= TERM=linux
  BOOT_IMAGE=/boot/vmlinuz-5.4.0-33-generic drop_caps=
  PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  PWD=/ rootmnt=/root

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1881814/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1881806] Re: cloud-init fails to initialize with virtual router

[Ryan Harper]

This issue has already been fixed in upstream cloud-init, see:

https://github.com/canonical/cloud-
init/commit/5352dd99eb2937b4eaaaf596b40ad7ca69d87f64

And

 https://bugs.launchpad.net/cloud-init/+bug/1639263

For downstream, you'll need RedHat/Centos to cherry pick this commit.

If possible, you can test with our daily rpm build for Centos8 using
this COPR repo:

https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/


** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1881806

Title:
  cloud-init fails to initialize with virtual router

Status in cloud-init:
  Invalid
Status in CentOS:
  Unknown

Bug description:
  Cloud-init fails to initialize with RHEL8.1 and CentOS8.1 (RPM cloud-
  init-18.5-7.el8_1.1.noarch) when routing to the VMs are provider by
  virtual router (vrouter). In this scenario, we are spawning a VM on
  OpenStack using Contrail as the network provider. With cloud-init-18.5
  version, we see the following exception during the VM boot phase
  causing init and final stages of cloud-init to fail.

  Impact: cloud-init fails to inject ssh-keys, grow root partition and
  other critical functions.

  Version cloud-init-18.2-6.el8.noarch of cloud-init did not have this
  issue.

  Reproducibility: always
   

  2020-06-02 19:17:15,641 - util.py[WARNING]: failed stage init
  failed run of stage init
  
  Traceback (most recent call last):
    File "/usr/lib/python3.6/site-packages/cloudinit/cmd/main.py", line 652, in 
status_wrapper
  ret = functor(name, args)
    File "/usr/lib/python3.6/site-packages/cloudinit/cmd/main.py", line 362, in 
main_init
  init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
    File "/usr/lib/python3.6/site-packages/cloudinit/stages.py", line 649, in 
apply_network_config
  netcfg, src = self._find_networking_config()
    File "/usr/lib/python3.6/site-packages/cloudinit/stages.py", line 636, in 
_find_networking_config
  if self.datasource and hasattr(self.datasource, 'network_config'):
    File 
"/usr/lib/python3.6/site-packages/cloudinit/sources/DataSourceOpenStack.py", 
line 115, in network_config
  self.network_json, known_macs=None)
    File 
"/usr/lib/python3.6/site-packages/cloudinit/sources/helpers/openstack.py", line 
645, in convert_net_json
  'Unknown network_data link type: %s' % link['type'])
  ValueError: Unknown network_data link type: vrouter

  The error seems to be coming from
  cloudinit/sources/helpers/openstack.py file which only expects 'link'
  type to be only of one of the following:

  valid_keys = {
  'physical': [
  'name',
  'type',
  'mac_address',
  'subnets',
  'params',
  'mtu',
  ],

  However, with Contrail, the network_config object that gets passed to
  the method convert_net_json looks something like the following object:

  {
    "services": [],
    "networks": [
  {
    "network_id": "f42c8806-c128-4ef8-af9e-d4a3856dde23",
    "type": "ipv4",
    "netmask": "255.255.255.192",
    "link": "tap722bcc7f-5a",
    "routes": [
  {
    "netmask": "0.0.0.0",
    "network": "0.0.0.0",
    "gateway": "10.140.145.126"
  }
    ],
    "ip_address": "10.140.145.110",
    "id": "network0"
  }
    ],
    "links": [
  {
    "type": "vrouter",
    "vif_id": "722bcc7f-5ae5-4564-ad43-ff9a5357dd36",
    "ethernet_mac_address": "02:72:2b:cc:7f:5a",
    "id": "tap722bcc7f-5a",
    "mtu": null
  }
    ]
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1881806/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1880177] Re: cloud init netplan fails to apply (v2 18.04 cloud image)

[Ryan Harper]

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1880177

Title:
  cloud init netplan fails to apply (v2 18.04 cloud image)

Status in cloud-init:
  Invalid

Bug description:
  Hi,
  I have been running up against this for a while and haven't found a work 
around no matter how much I change the format of the source yaml. 

  Environment:
   - vSphere deploying using Terraform and the VMWare Guest Info datasource 
(https://github.com/vmware/cloud-init-vmware-guestinfo)
   - Ubuntu 18.04 Cloud image
   - Cloud-init v. 19.4-33-gbb4131a2-0ubuntu1~18.04.1
   
  Issue:

  When cloud init tries to apply the network configuration (using v2
  syntax) it results in the following error from netplan:

  Stderr: /etc/netplan/50-cloud-init.yaml:10:13: Error in network definition: 
malformed address '10.1.400.140/24', must be X.X.X.X/NN or X:X:X:X:X:X:X:X/NN
  - 10.1.400.140/24

  It is obvious why the error is happening because when you check the
  resulting yaml file it is wrongly indented:

  # This file is generated from information provided by the datasource.  Changes
  # to it will not persist across an instance reboot.  To disable cloud-init's
  # network configuration capabilities, write a file
  # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
  # network: {config: disabled}
  network:
  ethernets:
  ens192:
  addresses:
  - 10.1.1.50/24
  dhcp4: false
  gateway4: null
  nameservers:
  addresses: []
  set-name: TEST
  ens224:
  addresses:
  - 10.2.2.14/24
  dhcp4: false
  gateway4: 10.2.2.254
  nameservers:
  addresses:
  - 10.2.2.254
  set-name: MGMT
  ens256:
  addresses:
  - 192.168.1.14/24
  dhcp4: false
  gateway4: 192.168.1.254
  nameservers:
  addresses:
  - 192.168.1.254
  set-name: TEST2
  version: 2

  However the input yaml from the VMware datasource looks correct:
  #vmware-rpctool "info-get guestinfo.metadata" | base64 -d
  network:
version: 2
ethernets:
  ens192:
dhcp4: false
set-name: TEST
addresses: [10.1.1.50/24]
gateway4: 
nameservers:
  addresses: []
  ens224:
dhcp4: false
set-name: MGMT
addresses: [10.2.2.14/24]
gateway4: 10.2.2.254
nameservers:
  addresses: [10.2.2.254]
  ens256:
dhcp4: false
set-name: TEST2
addresses: [192.168.1.14/24]
gateway4: 192.168.1.254
nameservers:
  addresses: [192.168.1.254]
  local-hostname: server.mgmt
  instance-id: server.mgmt

  And cloud init seems to interpret it correctly to json (from cloud-
  init.log):

  {
'ens192': {
  'dhcp4': False,
  'set-name': 'TEST',
  'addresses': [
'10.1.1.50/24'
  ],
  'gateway4': None,
  'nameservers': {
'addresses': [
  
]
  }
},
'ens224': {
  'dhcp4': False,
  'set-name': 'MGMT',
  'addresses': [
'10.2.2.14/24'
  ],
  'gateway4': '10.2.2.254',
  'nameservers': {
'addresses': [
  '10.2.2.254'
]
  }
},
'ens256': {
  'dhcp4': False,
  'set-name': 'TEST2',
  'addresses': [
'192.168.1.14/24'
  ],
  'gateway4': '192.168.1.254',
  'nameservers': {
'addresses': [
  '192.168.1.254'
]
  }
}
  }

  However it seems the conversion from the json into netplan yaml
  malforms the yaml slightly causing the netplan apply to fail. Full
  Error below from cloud-init.log

  2020-05-22 12:47:12,682 - netplan.py[DEBUG]: V2 to V2 passthrough
  2020-05-22 12:47:12,683 - util.py[DEBUG]: Writing to 
/etc/netplan/50-cloud-init.yaml - wb: [644] 1047 bytes
  2020-05-22 12:47:12,684 - util.py[DEBUG]: Running command ['netplan', 
'generate'] with allowed return codes [0] (shell=False, capture=True)
  2020-05-22 12:47:12,764 - util.py[WARNING]: failed stage init-local
  2020-05-22 12:47:12,764 - util.py[DEBUG]: failed stage init-local
  Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 653, in 
status_wrapper
  ret = functor(name, args)
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in 
main_init
  init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 710, in 
apply_network_config
  return self.distro.apply_network_config(netcfg, bring_up=bring_up)
File 

[Yahoo-eng-team] [Bug 1879103] Re: Unattended installation user data conflict

[Ryan Harper]

** Also affects: cloud-init
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1879103

Title:
  Unattended installation user data conflict

Status in cloud-init:
  New
Status in subiquity:
  New

Bug description:
  When passing user-data to the 20.04 unattended installation through an
  iso described at
  https://wiki.ubuntu.com/FoundationsTeam/AutomatedServerInstalls/QuickStart,
  user creation doesn't happen. This is because /dev/sr1 takes priority
  over /var/lib/cloud/seed/nocloud-net/.

  The simple workaround for anyone experiencing this issue is to add the
  following to their user-data script:

late-commands:
  - "eject /dev/sr1"

  I experienced this using Hyper-V.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1879103/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1639955] Re: bad test for snappy systems

[Ryan Harper]

This issue was fixed a while back in:

https://github.com/canonical/cloud-init/commit/4bcc94730

cloud-init retains the legacy bits, but prefers the use of os-release
file or kernel commandline to  for checking for UbuntuCore.


** Changed in: cloud-init (Ubuntu)
   Status: Confirmed => Fix Released

** Changed in: cloud-init
   Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1639955

Title:
  bad test for snappy systems

Status in cloud-init:
  Fix Released
Status in cloud-init package in Ubuntu:
  Fix Released

Bug description:
  Reviewing the latest SRU for cloud-init, I noticed the following:

  def system_is_snappy():
  # channel.ini is configparser loadable.
  # snappy will move to using /etc/system-image/config.d/*.ini
  # this is certainly not a perfect test, but good enough for now.
  content = load_file("/etc/system-image/channel.ini", quiet=True)
  if 'ubuntu-core' in content.lower():
  return True
  if os.path.isdir("/etc/system-image/config.d/"):
  return True
  return False

  This isn't a good test for whether a system is an ubuntu-core system.
  'system-image' is historical baggage, and not likely to be present at
  all in future versions.

  I'm afraid I don't know a good alternative test offhand, but wanted to
  log the bug so someone could look into it rather than being caught by
  surprise when ubuntu-core image contents later change.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1639955/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1869430] Re: cloud-init persists in running state on Kali in AWS

[Ryan Harper]

Excellent news!  Thanks for following up with Kali upstream.  I'm
closing this issue as invalid for cloud-init.  If you find out after the
Kali changes there are still issues, either re-open this bug or file a
new one if the issue/behavior is different.

Thanks!

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1869430

Title:
  cloud-init persists in running state on Kali in AWS

Status in cloud-init:
  Invalid

Bug description:
  Hello,

  We're trying to customize published Kali AMIs using packer & cloud-
  init. The entire process works with Ubuntu, CentOS, and Amazon Linux 2
  targets, but seemingly breaks with Kali. We've tried it with both the
  2020.01 and 2019.03.

  We're also experiencing a long timeout for ec2 data source:

  root@kali:~# cloud-init status --long
  status: running
  time: Fri, 27 Mar 2020 20:06:54 +
  detail:
  DataSourceEc2Local

  root@kali:~# cloud-init analyze blame
  -- Boot Record 01 --
   51.20500s (init-local/search-Ec2Local)
   00.91700s (init-network/config-users-groups)
   00.67200s (init-network/config-growpart)
   00.27400s (init-network/config-resizefs)
   00.24800s (init-network/config-ssh)
   00.00600s (init-network/consume-user-data)
   00.00300s (init-network/check-cache)

  Attached is the log tarball produced by cloud-init. We'd appreciate
  any hints as to what may be happening. It's worth noting that these
  targets are starting in a VPC without direct connection to the outside
  world, but there's a squid proxy available for web traffic. We have
  relevant parts set up to use that proxy.

  
  Thanks!

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1869430/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1860164] Re: cloud-init generates a traceback if a default route already exists during ephemeral network setup

[Ryan Harper]

** Changed in: cloud-init
   Status: Expired => Incomplete

** Changed in: cloud-init
   Importance: Undecided => Medium

** Changed in: cloud-init
   Status: Incomplete => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1860164

Title:
  cloud-init generates a traceback if a default route already exists
  during ephemeral network setup

Status in cloud-init:
  Triaged

Bug description:
  If a route already exists when the ephemeral network exists cloud-init
  will generate the following traceback:

  2020-01-16 21:14:22,584 - util.py[DEBUG]: Getting data from  failed
  Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 
760, in find_source
  if s.update_metadata([EventType.BOOT_NEW_INSTANCE]):
File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 
649, in update_metadata
  result = self.get_data()
File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 
273, in get_data
  return_value = self._get_data()
File 
"/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceOracle.py", line 
195, in _get_data
  with dhcp.EphemeralDHCPv4(net.find_fallback_nic()):
File "/usr/lib/python2.7/site-packages/cloudinit/net/dhcp.py", line 57, in 
__enter__
  return self.obtain_lease()
File "/usr/lib/python2.7/site-packages/cloudinit/net/dhcp.py", line 109, in 
obtain_lease
  ephipv4.__enter__()
File "/usr/lib/python2.7/site-packages/cloudinit/net/__init__.py", line 
920, in __enter__
  self._bringup_static_routes()
File "/usr/lib/python2.7/site-packages/cloudinit/net/__init__.py", line 
974, in _bringup_static_routes
  ['dev', self.interface], capture=True)
File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 2083, in 
subp
  cmd=args)
  ProcessExecutionError: Unexpected error while running command.

  This is a regression from 19.1 on SUSE where exiting routes were
  simply skipped.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1860164/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1867029] Re: Package ifupdown breaks network configuration by cloud-init

[Ryan Harper]

** Also affects: ifupdown (Ubuntu)
   Importance: Undecided
   Status: New

** Tags added: uec-images

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1867029

Title:
  Package ifupdown breaks network configuration by cloud-init

Status in cloud-init:
  Confirmed
Status in ifupdown package in Ubuntu:
  New

Bug description:
  It appears the `ifupdown` package breaks networking configuration that
  cloud-init would normally handle (?) on both providers. The result, if
  you image an instance with the `ifupdown` package installed, is that
  the image is not bootable by Azure/AWS. I'm not familiar with how
  Azure/AWS/etc use cloud-init for network start up however. I'm filing
  here for now, in case `cloud-init` needs to be more defensive against
  `ifupdown`.

  To reproduce with Packer (I confirmed this method):

  * Add necessary Azure subscription env vars from `ifupdown-bug.json`
  * `packer build ifupdown-bug.json`
  * Try to boot an instance with resulting image
  * Instance will never boot -- more specifically, networking never comes up. 
See `boot.log` for an example boot

  To reproduce manually (only a guess, I did not confirm):

  * Boot 18.04 ubuntu instance
  * Install ifupdown
  * Image the instance
  * Try to boot instance using new image
  * Instance won't boot

  I hit this with Packer, creating new images for booting instances in
  scaling groups -- `ifupdown` is brought in by `salt`. If this is
  reproducible via manual installation, there could be some backup
  images/snapshots that are unable to boot though.

  This appears to be because cloud-init network start up operations are
  broken by whatever `ifupdown` sysv/systemd scripts perform on boot.
  With `ifupdown` installed on Azure, `eth0` does not get an IP address,
  the instance never fully boots according to Azure, and it will enter
  an error state. On AWS, the instance boots, but network operations
  like SSH fail.

  The `boot.log` is from Azure. Eventually `cloud-init` gives an
  exception (see `exception.log`), but this seems like a secondary issue
  related to networking being down.

  I did try to upgrade cloud-init using nightly PPAs (version 20.1 I
  believe), but no luck. This is on Ubuntu 18.04 instances from Azure
  marketplace and AWS Canonical AMI, cloud-init version
  `19.4-33-gbb4131a2-0ubuntu1~18.04.1`.

  Related: https://github.com/Azure/WALinuxAgent/issues/1612

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1867029/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1863773] Re: cloud-init network fails if ipv6 is disabled

[Ryan Harper]

Thanks for reporting the bug and attaching the logs.  Looking at the
logs and your description it appears to be an issue with your Debian
image itself.

The files,  /etc/network/if-pre-up.d/cloud_inet6 , and
/etc/cloud/cloud.cfg.d/99-disable-network-config.cfg   are not part of
cloud-init itself.

The latter file is configuring cloud-init to not touch networking and
that the image will bring it up.  The former file looks like it needs
some error handling in the case of disabling ipv6.

Looking at the Debian AWS/EC2 FAQ[1] it suggests that you can[2]:

"""
Please report bugs to the cloud.debian.org pseudo-package, when they are 
related to the choices made when building the image (which packages to include, 
which customisation was made, etc.). Advanced users can add the usertags to 
triage the report more precisely.
"""

As this appears to be an issue with the AMI, I'm going to mark this bug Invalid 
for cloud-init.  If you feel that there is an issue with cloud-init, please 
mark this bug New and add some addition
information to help explain how cloud-init is involved.


1. https://wiki.debian.org/Amazon/EC2/FAQ
2. 
https://wiki.debian.org/Amazon/EC2/FAQ#Q:_Where_do_I_report_issues_with_the_images_.3F

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1863773

Title:
  cloud-init network fails if ipv6 is disabled

Status in cloud-init:
  Invalid

Bug description:
  I'm using cloud-init on a Debian machien in AWS.

  On these machiesn, ipv6 is disabled by setting
  /proc/sys/net/ipv6/conf/all/disable_ipv6 to 1.  This causes the
  network to fail:

  bas@machine:/etc/network$ sudo ifup ens5
  Internet Systems Consortium DHCP Client 4.4.1
  Copyright 2004-2018 Internet Systems Consortium.
  All rights reserved.
  For info, please visit https://www.isc.org/software/dhcp/

  Listening on LPF/ens5/02:45:ca:0d:93:76
  Sending on   LPF/ens5/02:45:ca:0d:93:76
  Sending on   Socket/fallback
  DHCPREQUEST for 10.0.16.20 on ens5 to 255.255.255.255 port 67
  DHCPACK of 10.0.16.20 from 10.0.16.1
  RTNETLINK answers: File exists
  bound to 10.0.16.20 -- renewal in 1796 seconds.

  
  Could not get a link-local address
  run-parts: /etc/network/if-pre-up.d/cloud_inet6 exited with return code 1
  ifup: failed to bring up ens5

  
  Even worse, on boot the machines do get a proper ipv4 address, but 
(apparently because setting up the network fails), the dhcp (v4) daemon is 
stopped, and the machines "mysteriously" drop off the net after the lease ends.

  
  1. AWS with official Debian buster images 
(https://wiki.debian.org/Cloud/AmazonEC2Image/Buster)
  2. 

  bas@machine:~$ cat /etc/network/interfaces
  # Include files from /etc/network/interfaces.d:
  source-directory /etc/network/interfaces.d

  # Cloud images dynamically generate config fragments for newly
  # attached interfaces. See /etc/udev/rules.d/75-cloud-ifupdown.rules
  # and /etc/network/cloud-ifupdown-helper. Dynamically generated
  # configuration fragments are stored in /run:
  source-directory /run/network/interfaces.d

  bas@machine:~$ cat /run/network/interfaces.d/ens5
  auto ens5
  allow-hotplug ens5

  iface ens5 inet dhcp

  iface ens5 inet6 manual
try_dhcp 1

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1863773/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1863943] [NEW] do not log imdsv2 token headers

[Ryan Harper]

Public bug reported:

Cloud-init currently logs all headers when processing URLs.  On Ec2,
this includes the IMDSv2 token negotiation and all IMDS interactions.
The value of seeing the headers in the log is quite useful, especially
for confirming whether cloud-init is using IMDSv2 or not; however the
actual value of the token does not aide in this effort.

Reported by: https://github.com/ishug86

** Affects: cloud-init
 Importance: Medium
 Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1863943

Title:
  do not log imdsv2 token headers

Status in cloud-init:
  In Progress

Bug description:
  Cloud-init currently logs all headers when processing URLs.  On Ec2,
  this includes the IMDSv2 token negotiation and all IMDS interactions.
  The value of seeing the headers in the log is quite useful, especially
  for confirming whether cloud-init is using IMDSv2 or not; however the
  actual value of the token does not aide in this effort.

  Reported by: https://github.com/ishug86

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1863943/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1862417] Re: cloud-init: Attempt to mkswap on a partition fails: invalid block count argument: ''

[Ryan Harper]

 The machine I'm working on uses cloud-init to update itself, it might 
only have the fix after the updates.
 ah, interesting
 cat /etc/cloud/build.info  
 that'll give us a point in time for which version you have 
 and I suspect you're right, the top of your cloud-init.log will the 
original version 
 build_name: serverserial: 20190514
 Definitely way older than the PR.
 yep
 I suspect the fix in our case is to use the latest image of Ubuntu 
from end Jan 2020.
 yep

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1862417

Title:
  cloud-init: Attempt to mkswap on a partition fails: invalid block
  count argument: ''

Status in cloud-init:
  Invalid

Bug description:
  If an attempt is made to configure a swap partition on an Ubuntu
  Bionic machine as follows (not a swap file, a swap partition), the
  attempt to mkswap fails.

  The expected behaviour is that mkswap and swapon are executed
  correctly, and /dev/xvdg becomes a valid swap disk. In addition, when
  filename points at a partition, size and maxsize should be ignored.

  fs_setup:
- label: vidi
  device: /dev/xvde
  filesystem: ext4
- label: swap
  device: /dev/xvdg
  filesystem: swap
  mounts:
  - [ /dev/xvde, /var/lib/vidispine, ext4, defaults, 0, 0 ]
  - [ /dev/xvdg, none, swap, sw, 0, 0 ]
  swap:
  filename: /dev/xvdg
  size: auto
  maxsize: 17179869184
  mount_default_fields: [ None, None, "auto", "defaults", "0", "2" ]

  When the machine starts up for the first time, the following error is
  logged after the swap size parameter is passed as the empty string:

  2020-02-07 20:21:55,242 - cc_disk_setup.py[WARNING]: Force flag for swap is 
unknown.
  2020-02-07 20:21:55,255 - util.py[WARNING]: Failed during filesystem operation
  Failed to exec of '['/sbin/mkswap', '/dev/xvdg', '-L', 'swap', '']':
  Unexpected error while running command.
  Command: ['/sbin/mkswap', '/dev/xvdg', '-L', 'swap', '']
  Exit code: 1
  Reason: -
  Stdout: 
  Stderr: mkswap: invalid block count argument: ''
  2020-02-07 20:21:55,530 - cc_mounts.py[WARNING]: Activate mounts: FAIL:swapon 
-a
  2020-02-07 20:21:55,530 - util.py[WARNING]: Activate mounts: FAIL:swapon -a

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1862417/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1861460] [NEW] cloud-init should parse initramfs rendered netplan if present

[Ryan Harper]

Public bug reported:

initramfs-tools used to only execute klibc based networking with some
resolvconf hooks.

In recent releases, it has been greatly improved to use
isc-dhcp-client instead of klibc, support vlan= key (like in
dracut-network), bring up Z devices using chzdev, and generate netplan
yaml from all of the above.

Above improvements were driven in part by Oracle Cloud and in part by
Subiquity netbooting on Z.

Thus these days, instead of trying to reparse klibc files in
/run/net-*, cloud-init should simply import /run/netplan/$device.yaml
files as the ip=* provided networking information on the command line.
I do not currently see cloud-init doing that in e.g.
/cloudinit/net/cmdline.py

** Affects: cloud-init
 Importance: Wishlist
 Status: Triaged

** Changed in: cloud-init
   Importance: Undecided => Wishlist

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1861460

Title:
  cloud-init should parse initramfs rendered netplan if present

Status in cloud-init:
  Triaged

Bug description:
  initramfs-tools used to only execute klibc based networking with some
  resolvconf hooks.

  In recent releases, it has been greatly improved to use
  isc-dhcp-client instead of klibc, support vlan= key (like in
  dracut-network), bring up Z devices using chzdev, and generate netplan
  yaml from all of the above.

  Above improvements were driven in part by Oracle Cloud and in part by
  Subiquity netbooting on Z.

  Thus these days, instead of trying to reparse klibc files in
  /run/net-*, cloud-init should simply import /run/netplan/$device.yaml
  files as the ip=* provided networking information on the command line.
  I do not currently see cloud-init doing that in e.g.
  /cloudinit/net/cmdline.py

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1861460/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1857397] Re: CloudStack data source cannot determine Virtual Router address on RHEL/CentOS 8

[Ryan Harper]

*** This bug is a duplicate of bug 1843334 ***
https://bugs.launchpad.net/bugs/1843334

** This bug has been marked a duplicate of bug 1843334
   Change location of DHCP leases in CloudStack provider as it doesn't work for 
RHEL8

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1857397

Title:
  CloudStack data source cannot determine Virtual Router address on
  RHEL/CentOS 8

Status in cloud-init:
  Fix Released

Bug description:
  The problem: cloud-init is unable to determine CloudStack Virtual
  Router address on RHEL/CentOS 8 machines.

  Why: because RHEL/CentOS 8 does not support systemd-networkd and
  NetworkManager lease files format cannot be parsed by current cloud-
  init CloudStack data source implementation.

  Please see more details about this in merge request
  https://github.com/canonical/cloud-init/pull/141: changes from this
  request were successfully tested.

  What else: most likely other data sources also have the same problem.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1857397/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1858615] Re: Fail to boot when NoCloud datasource is included

[Ryan Harper]

Thanks for filing a bug.  I've added a dmidecode task to track the issue
with the tool.  It may also affect the kernel package, and possibly
firmware (though that's not something that Ubuntu provides).  Cloud-init
and any other tool may invoke this package and it should not reboot the
system; but there's no issue with cloud-init or other callers of the
tool   As such, I'm marking the cloud-init task as invalid.

** Also affects: dmidecode (Ubuntu)
   Importance: Undecided
   Status: New

** Summary changed:

- Fail to boot when NoCloud datasource is included
+ dmidecode triggers system reboot on Inforce 6640

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1858615

Title:
  dmidecode triggers system reboot on Inforce 6640

Status in cloud-init:
  Invalid
Status in dmidecode package in Ubuntu:
  New

Bug description:
  Device: Inforce 6640
  
https://www.inforcecomputing.com/products/single-board-computers-sbc/qualcomm-snapdragon-820-inforce-6640-sbc
  SoC: Snapdragon 820

  sysname='Linux',
  nodename='ubuntu',
  release='4.15.0-1069-snapdragon', 
  version='#76-Ubuntu SMP Tue Nov 26 16:10:14 UTC 2019', 
  machine='aarch64'

  The issue is caused by following commit.
  Inforce 6640 doesn't have functional demidecode.
  System will reboot when executing dmidecode.

  commit 3416e2ee7f65defdb15aab861a85767d13e8c34c
  Author: Robert Schweikert 
  Date: Sat Oct 29 09:29:53 2016 -0400
  dmidecode: Allow dmidecode to be used on aarch64
  aarch64 systems have functional dmidecode, so allow that to be used.
  - aarch64 has support for dmidecode as well

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1858615/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1851552] Re: since ubuntu 18 bionic release and latest, the ubuntu18 cloud image is unable to boot up on openstack instance

[Ryan Harper]

I'm marking the cloud-init task invalid as at this time the logs point
to a nested virtualization/openstack issue with devices not being
present; not related to cloud-init.  If further investigation points to
an issue with cloud-init you can move the cloud-init task back to New.

** Changed in: cloud-init (Ubuntu)
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1851552

Title:
  since ubuntu 18 bionic release and latest, the ubuntu18 cloud image is
  unable to boot up on openstack instance

Status in cloud-init:
  New
Status in networking-calico:
  New
Status in OpenStack Compute (nova):
  New
Status in OpenStack Community Project:
  New
Status in qemu-kvm:
  New
Status in cloud-init package in Ubuntu:
  Invalid
Status in qemu package in Ubuntu:
  New

Bug description:
  Openstack Queens release which is running on ubuntu 18 LTS Controller and 
Compute.
  Tried to boot up the instance via horizon dashboard without success.
  Nova flow works perfect.
  When access to console I discovered that the boot process stuck in the middle.
  [[0;1;31m TIME [0m] Timed out waiting for device dev-vdb.device.
  [[0;1;33mDEPEND[0m] Dependency failed for /mnt.
  [[0;1;33mDEPEND[0m] Dependency failed for File System Check on /dev/vdb.
  It receives IP but looks like not get configured at time.
  since ubuntu 18 there is netplan feature managing the network interfaces
  please advise.

  more details as follow:
  https://bugs.launchpad.net/networking-calico/+bug/1851548

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1851552/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1853376] [NEW] fix debian packaging warnings/errors

[Ryan Harper]

Public bug reported:

E: cloud-init source: untranslatable-debconf-templates cloud-init.templates: 6
W: cloud-init source: missing-file-from-potfiles-in grub.templates
W: cloud-init source: build-depends-on-obsolete-package build-depends: 
dh-systemd => use debhelper (>= 9.20160709)
W: cloud-init source: timewarp-standards-version (2011-12-16 < 2014-09-17)
W: cloud-init source: ancient-standards-version 3.9.6 (released 2014-09-17) 
(current is 4.4.1)
W: cloud-init source: binary-nmu-debian-revision-in-source 
19.3-244-gbee7e918-1~bddeb~20.04.1
W: cloud-init: binary-without-manpage usr/bin/cloud-id
W: cloud-init: binary-without-manpage usr/bin/cloud-init
W: cloud-init: binary-without-manpage usr/bin/cloud-init-per
W: cloud-init: command-with-path-in-maintainer-script postinst:141 
/usr/sbin/grub-install
W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-config.service cloud-init.target
W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-final.service cloud-init.target
W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init-local.service cloud-init.target
W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init.service cloud-init.target
W: cloud-init: systemd-service-file-shutdown-problems 
lib/systemd/system/cloud-init.service
N: 1 tag overridden (1 error)

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1853376

Title:
  fix debian packaging warnings/errors

Status in cloud-init:
  New

Bug description:
  E: cloud-init source: untranslatable-debconf-templates cloud-init.templates: 6
  W: cloud-init source: missing-file-from-potfiles-in grub.templates
  W: cloud-init source: build-depends-on-obsolete-package build-depends: 
dh-systemd => use debhelper (>= 9.20160709)
  W: cloud-init source: timewarp-standards-version (2011-12-16 < 2014-09-17)
  W: cloud-init source: ancient-standards-version 3.9.6 (released 2014-09-17) 
(current is 4.4.1)
  W: cloud-init source: binary-nmu-debian-revision-in-source 
19.3-244-gbee7e918-1~bddeb~20.04.1
  W: cloud-init: binary-without-manpage usr/bin/cloud-id
  W: cloud-init: binary-without-manpage usr/bin/cloud-init
  W: cloud-init: binary-without-manpage usr/bin/cloud-init-per
  W: cloud-init: command-with-path-in-maintainer-script postinst:141 
/usr/sbin/grub-install
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-config.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-final.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init-local.service cloud-init.target
  W: cloud-init: systemd-service-file-refers-to-unusual-wantedby-target 
lib/systemd/system/cloud-init.service cloud-init.target
  W: cloud-init: systemd-service-file-shutdown-problems 
lib/systemd/system/cloud-init.service
  N: 1 tag overridden (1 error)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1853376/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1852456] [NEW] doc: list of modules is no longer present

[Ryan Harper]

Public bug reported:

the list of modules has disappeared from the documentation sidebar.

- version 19.3: https://cloudinit.readthedocs.io/en/19.3/topics/modules.html
- version 19.2: https://cloudinit.readthedocs.io/en/19.2/topics/modules.html

In 19.2, the sidebar has an entry for each module, and it's a lot easier
to find and navigate to appropriate module.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1852456

Title:
  doc: list of modules is no longer present

Status in cloud-init:
  New

Bug description:
  the list of modules has disappeared from the documentation sidebar.

  - version 19.3: https://cloudinit.readthedocs.io/en/19.3/topics/modules.html
  - version 19.2: https://cloudinit.readthedocs.io/en/19.2/topics/modules.html

  In 19.2, the sidebar has an entry for each module, and it's a lot
  easier to find and navigate to appropriate module.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1852456/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1835037] Re: Upgrade from bionic-rocky to bionic-stein failed migrations.

[Ryan Beisner]

The fix merged in master and is in the current stable charms as of
19.10.

** Changed in: charm-nova-cloud-controller
   Status: In Progress => Fix Committed

** Changed in: charm-nova-cloud-controller
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1835037

Title:
  Upgrade from bionic-rocky to bionic-stein failed migrations.

Status in OpenStack nova-cloud-controller charm:
  Fix Released
Status in OpenStack Compute (nova):
  New

Bug description:
  We were trying to upgrade from rocky to stein using the charm
  procedure described here:

  https://docs.openstack.org/project-deploy-guide/charm-deployment-
  guide/latest/app-upgrade-openstack.html

  and we got into this problem,

  
  2019-07-02 09:56:44 ERROR juju-log online_data_migrations failed
  b'Running batches of 50 until complete\nError attempting to run \n9 rows matched query 
populate_user_id, 0 
migrated\n+-+--+---+\n|
  Migration  | Total Needed | Completed 
|\n+-+--+---+\n|
 create_incomplete_consumers |  0   | 0 |\n| 
delete_build_requests_with_no_instance_uuid |  0   | 0 |\n| 
fill_virtual_interface_list |  0   | 0 |\n| 
migrate_empty_ratio |  0   | 0 |\n|  
migrate_keypairs_to_api_db |  0   | 0 |\n|   
migrate_quota_classes_to_api_db   |  0   | 0 |\n|
migrate_quota_limits_to_api_db   |  0   | 0 |\n|  
migration_migrate_to_uuid  |  0   | 0 |\n| 
populate_missing_availability_zones |  0   | 0 |\n| 
 populate_queued_for_delete |  0   | 0 |\n| 
  populate_user_id  |  9   | 0 |\n|
populate_uuids   |  0   | 0 |\n| 
service_uuids_online_data_migration |  0   | 0 
|\n+-+--+---+\nSome
 migrations failed unexpectedly. Check log for details.\n'

  What should we do to get this fixed?

  Regards,

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-nova-cloud-controller/+bug/1835037/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1851438] Re: cloud-init disk_setup fails to partition disk for Ubuntu18

[Ryan Harper]

On trusty; this complains but works.

ubuntu@ubuntu:~$ dpkg --list | grep util-linux
ii  util-linux  2.20.1-5.1ubuntu20.9
   amd64Miscellaneous system utilities
ubuntu@ubuntu:~$ cat /proc/partitions 
major minor  #blocks  name

 2530   10485760 vda
 2531   10484736 vda1
 253   16   10485760 vdb
 253   32366 vdc
  1101048575 sr0
ubuntu@ubuntu:~$ sudo bash
sudo: unable to resolve host ubuntu
root@ubuntu:~# echo "0," | sfdisk --Linux --unit=S --force /dev/vdb 
Checking that no-one is using this disk right now ...
OK

Disk /dev/vdb: 20805 cylinders, 16 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/vdb: unrecognized partition table type
Old situation:
No partitions found
Warning: bad partition start (earliest 1)
New situation:
Units = sectors of 512 bytes, counting from 0

   Device BootStart   End   #sectors  Id  System
/dev/vdb1 1  20971519   20971519  83  Linux
/dev/vdb2 0 -  0   0  Empty
/dev/vdb3 0 -  0   0  Empty
/dev/vdb4 0 -  0   0  Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
root@ubuntu:~# cat /proc/partitions 
major minor  #blocks  name

 2530   10485760 vda
 2531   10484736 vda1
 253   16   10485760 vdb
 253   17   10485759 vdb1
 253   32366 vdc
  1101048575 sr0


** Also affects: util-linux (Ubuntu)
   Importance: Undecided
   Status: New

** Also affects: util-linux (Ubuntu Eoan)
   Importance: Undecided
   Status: New

** Also affects: util-linux (Ubuntu Xenial)
   Importance: Undecided
   Status: New

** Also affects: util-linux (Ubuntu Focal)
   Importance: Undecided
   Status: New

** Also affects: util-linux (Ubuntu Bionic)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1851438

Title:
  cloud-init disk_setup fails to partition disk for Ubuntu18

Status in cloud-init:
  Triaged
Status in util-linux package in Ubuntu:
  New
Status in util-linux source package in Xenial:
  New
Status in util-linux source package in Bionic:
  New
Status in util-linux source package in Eoan:
  New
Status in util-linux source package in Focal:
  New

Bug description:
  Pasting disk_setup for cloud-config:

  disk_setup:
/dev/xvde:
   layout: True
   overwrite: False
   type: mbr
  fs_setup:
-device: /dev/xvde
 filesystem: ext4
 label: data
 overwrite: false
 partition: auto

  I want to attach and mount a /data disk on the VM using cloud-init so
  I just want to single partition 100% of the disk.

  Error while running the sfdisk command for partitioning the disk (please see 
attached file).
  OS: Ubuntu18

  How I repro-ed it outside cloud-init environment:
  1. Open XenCenter, add a new disk, say /dev/xvdc
  2. Run `/sbin/sfdisk --Linux --unit=S --force /dev/xvdc` and specify start 
sector as 0. Because from the cloud-init logs and source code, I figured that 
it was picking start sector as 0. Save it and see the error.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1851438/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1850642] Re: Cloud init unable to find the metadata service but can CURL it

[Ryan Harper]

*** This bug is a duplicate of bug 1821102 ***
https://bugs.launchpad.net/bugs/1821102

The logs do contain that bug, but I'm not sure that's the failure here.

Looking at the cloud-init logs, we can see the Ephemeral DHCP start and
obtain an lease, but the end point does not respond:

2019-10-30 13:28:41,516 - dhcp.py[DEBUG]: Received dhcp lease on eth0 for 
136.156.90.74/255.255.254.0
2019-10-30 13:28:41,516 - __init__.py[DEBUG]: Attempting setup of ephemeral 
network on eth0 with 136.156.90.74/23 brd 136.156.91.255
2019-10-30 13:28:41,516 - util.py[DEBUG]: Running command ['ip', '-family', 
'inet', 'addr', 'add', '136.156.90.74/23', 'broadcast', '136.156.91.255', 
'dev', 'eth0'] with allowed return codes [0] (shell=False, capture=True)
2019-10-30 13:28:41,519 - util.py[DEBUG]: Running command ['ip', '-family', 
'inet', 'link', 'set', 'dev', 'eth0', 'up'] with allowed return codes [0] 
(shell=False, capture=True)
2019-10-30 13:28:41,522 - util.py[DEBUG]: Running command ['ip', 'route', 
'show', '0.0.0.0/0'] with allowed return codes [0] (shell=False, capture=True)
2019-10-30 13:28:41,525 - util.py[DEBUG]: Running command ['ip', '-4', 'route', 
'add', '136.156.91.254', 'dev', 'eth0', 'src', '136.156.90.74'] with allowed 
return codes [0] (shell=False, capture=True)
2019-10-30 13:28:41,527 - util.py[DEBUG]: Running command ['ip', '-4', 'route', 
'add', 'default', 'via', '136.156.91.254', 'dev', 'eth0'] with allowed return 
codes [0] (shell=False, capture=True)
2019-10-30 13:29:21,573 - util.py[DEBUG]: Resolving URL: http://169.254.169.254 
took 40.044 seconds
2019-10-30 13:29:21,574 - url_helper.py[DEBUG]: [0/1] open 
'http://169.254.169.254/openstack' with {'url': 
'http://169.254.169.254/openstack', 'headers': {'User-Agent': 
'Cloud-Init/18.5'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 10.0} 
configuration
2019-10-30 13:29:31,608 - url_helper.py[DEBUG]: Calling 
'http://169.254.169.254/openstack' failed [10/-1s]: request error 
[HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with 
url: /openstack (Caused by 
ConnectTimeoutError(, 'Connection to 169.254.169.254 timed out. (connect 
timeout=10.0)'))]
2019-10-30 13:29:31,608 - DataSourceOpenStack.py[DEBUG]: Giving up on OpenStack 
md from ['http://169.254.169.254/openstack'] after 10 seconds
2019-10-30 13:29:31,609 - util.py[DEBUG]: Crawl of metadata service took 50.079 
seconds

Cloud-init did not find the OpenStack datasource at local time:

2019-10-30 13:29:31,689 - main.py[DEBUG]: No local datasource found

So we write out a fallback network config (dhcp on eth0).

2019-10-30 13:29:31,722 - stages.py[INFO]: Applying network configuration from 
fallback bringup=False: {'version': 1, 'config': [{'subnets': [{'type': 
'dhcp'}], 'type': 'physical', 'name': 'eth0', 'mac_address': 
'fa:16:3e:f4:b5:1d'}]}
2019-10-30 13:29:31,723 - __init__.py[DEBUG]: Selected renderer 'sysconfig' 
from priority list: None
2019-10-30 13:29:31,725 - util.py[DEBUG]: Writing to 
/etc/sysconfig/network-scripts/ifcfg-eth0 - wb: [644] 159 bytes
2019-10-30 13:29:31,726 - util.py[DEBUG]: Restoring selinux mode for 
/etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False)
2019-10-30 13:29:31,726 - util.py[DEBUG]: Restoring selinux mode for 
/etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False)

And then exit cloud-init-local.service:

2019-10-30 13:29:31,730 - util.py[DEBUG]: cloud-init mode 'init' took
52.389 seconds (52.39)

Now, the OS networking layer comes up:

Oct 30 13:29:31.623535 localhost.localdomain cloud-init[732]: 2019-10-30 
13:29:31,619 - util.py[WARNING]: No active metadata service found
Oct 30 13:29:31.751295 localhost.localdomain systemd[1]: Started Initial 
cloud-init job (pre-networking).
Oct 30 13:29:31.754472 localhost.localdomain systemd[1]: Reached target Network 
(Pre).
Oct 30 13:29:31.756812 localhost.localdomain systemd[1]: Starting LSB: Bring 
up/down networking...
Oct 30 13:29:31.961050 localhost.localdomain network[866]: Bringing up loopback 
interface:  [  OK  ]
Oct 30 13:29:31.990878 localhost.localdomain network[866]: Bringing up 
interface eth0:
Oct 30 13:29:32.034411 localhost.localdomain dhclient[995]: DHCPDISCOVER on 
eth0 to 255.255.255.255 port 67 interval 8 (xid=0xeceb89e)
Oct 30 13:29:32.055157 localhost.localdomain dhclient[995]: DHCPREQUEST on eth0 
to 255.255.255.255 port 67 (xid=0xeceb89e)
Oct 30 13:29:32.055188 localhost.localdomain dhclient[995]: DHCPOFFER from 
136.156.90.11
Oct 30 13:29:32.055796 localhost.localdomain dhclient[995]: DHCPACK from 
136.156.90.11 (xid=0xeceb89e)
Oct 30 13:29:34.132189 localhost.localdomain NET[1054]: 
/usr/sbin/dhclient-script : updated /etc/resolv.conf
Oct 30 13:29:34.166284 host-136-156-90-74 dhclient[995]: bound to 136.156.90.74 
-- renewal in 34571 seconds.
Oct 30 13:29:34.167466 host-136-156-90-74 network[866]: Determining IP 
information for eth0... done.
Oct 30 13:29:34.224637 host-136-156-90-74 network[866]: [  OK  ]
Oct 30 13:29:34.245397 

[Yahoo-eng-team] [Bug 1848471] Re: No infiniband support in netplan renderer

[Ryan Harper]

** Also affects: netplan.io (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Importance: Undecided => Wishlist

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1848471

Title:
  No infiniband support in netplan renderer

Status in cloud-init:
  Triaged
Status in netplan.io package in Ubuntu:
  New

Bug description:
  Infiniband support has been added to both the sysconfig and eni
  renderers (at time of writing, eni support is unmerged from
  https://code.launchpad.net/~darren-birkett/cloud-init/+git/cloud-
  init/+merge/373759).

  We need to add support to the netplan renderer too, but at this time
  netplan itself does not support the long form HWADDR that IB devices
  have. Once that is added, we should add support for infiniband devices
  in netplan too.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1848471/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1849154] [NEW] Live Migrations complete but occasionally fail to update the Openstack Database

[Ryan Farrell]

Public bug reported:

[Description]
Occasionally when evacuating vms off of nova compute hosts for host reboots, a 
vms migration will be reported as complete in the migration list, but queries 
to the openstack api, such as 'openstack show uuid' will report the host & 
hypervisor-hostname unchanged. The only indication that something is wrong is 
that power_state will be NOSTATE.  We can see that the instance is in fact 
migrated and running on the new host with 'sudo virsh list --all | grep 
$instance_name'. 

In order to resolve this issue we perform a direct database edit such
as:

'update instances
set host="$newhost", node="$newhost.domain", progress="0"
where uuid="" and deleted="0";' 

* In one instance, the 'progress' value was stuck at 99 and I needed to
set that to 0 in the database as well.

[Expected]
Its expected that the live migration completes and that the instance in the 
openstack database correctly reflects the name of the new host, and its power 
state.

[Impact]
Instances that are found to be in power state NOSTATE are blocked from 
performing certain actions; instances in this state do not self recover.


[Environment]
Openstack Queens; Nova 17.0.10
libvirtd/virsh: 4.0.0
ceph: 12.2.8
neutron-openvswitch: 12.0.5

[Logs]
In this particular set of logs (sosreports from the live migration source and 
destination hosts); the instance that was in error had uuid 
67f328d0-cb5e-416a-9af4-c6e47e68a1e0.

** Affects: nova
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1849154

Title:
  Live Migrations complete but occasionally fail to update the Openstack
  Database

Status in OpenStack Compute (nova):
  New

Bug description:
  [Description]
  Occasionally when evacuating vms off of nova compute hosts for host reboots, 
a vms migration will be reported as complete in the migration list, but queries 
to the openstack api, such as 'openstack show uuid' will report the host & 
hypervisor-hostname unchanged. The only indication that something is wrong is 
that power_state will be NOSTATE.  We can see that the instance is in fact 
migrated and running on the new host with 'sudo virsh list --all | grep 
$instance_name'. 

  In order to resolve this issue we perform a direct database edit such
  as:

  'update instances
  set host="$newhost", node="$newhost.domain", progress="0"
  where uuid="" and deleted="0";' 

  * In one instance, the 'progress' value was stuck at 99 and I needed
  to set that to 0 in the database as well.

  [Expected]
  Its expected that the live migration completes and that the instance in the 
openstack database correctly reflects the name of the new host, and its power 
state.

  [Impact]
  Instances that are found to be in power state NOSTATE are blocked from 
performing certain actions; instances in this state do not self recover.

  
  [Environment]
  Openstack Queens; Nova 17.0.10
  libvirtd/virsh: 4.0.0
  ceph: 12.2.8
  neutron-openvswitch: 12.0.5

  [Logs]
  In this particular set of logs (sosreports from the live migration source and 
destination hosts); the instance that was in error had uuid 
67f328d0-cb5e-416a-9af4-c6e47e68a1e0.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1849154/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1848243] Re: network-scripts(eni) is deprecated in CentOS/RHEL 8

[Ryan Harper]

cloud-init already has support for writing sysconfig formatted files on
systems which only include NetworkManager, specifically NetworkManager
includes a plugin module, 'ifcfg-rh' which reads the sysconfig files for
NetworkManager and cloud-init will enable this plugin as needed.

Please re-open this bug with more details as to what is not working.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1848243

Title:
  network-scripts(eni) is deprecated in CentOS/RHEL 8

Status in cloud-init:
  Invalid

Bug description:
  The network-scripts package has been removed from the standard
  CentOS/RHEL 8 install and is considered deprecated[1]. Currently it is
  possible to install the network-scripts package but it may not be
  available in the future. It is possible to continue to use eni via
  NetworkManager[2].

  [1] 
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_and_managing_networking/index#legacy_network_scripts_support
  [2] 
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_and_managing_networking/index#Using-NetworkManager-with-sysconfig-files_getting-started-with-networking-devices-with-NetworkManager

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1848243/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1846511] Re: cloud-init does not run after upgrade due to bad 90_dpkg.cfg

[Ryan Harper]

This was an release-branch only issue; it did not affect upstream cloud-
init.

** Also affects: cloud-init (Ubuntu)
   Importance: Undecided
   Status: New

** Also affects: cloud-init (Ubuntu Eoan)
   Importance: Undecided
   Status: New

** Changed in: cloud-init (Ubuntu Eoan)
   Status: New => Fix Released

** Changed in: cloud-init (Ubuntu Eoan)
   Importance: Undecided => Critical

** Changed in: cloud-init
   Status: Fix Committed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1846511

Title:
  cloud-init does not run after upgrade due to bad 90_dpkg.cfg

Status in cloud-init:
  Invalid
Status in cloud-init package in Ubuntu:
  Fix Released
Status in cloud-init source package in Eoan:
  Fix Released

Bug description:
  Integration tests on Eoan are failing on the lxd and nocloud kvm
  platforms. In both cases the test system is not reachable via SSH.

  LXD log: https://paste.ubuntu.com/p/r2hwqZQfm2/

  KVM log: https://paste.ubuntu.com/p/W3N22yBxKN/

  This failure mode began with the test run of September 26. On the same
  system the Disco test run succeeds.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1846511/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1846513] Re: debian 10 cloud-init stuck in systemd-networkd after 1. reboot

[Ryan Harper]

I'm marking this invalid for cloud-init as the logs do not indicate
there is any cloud-init specific failure.  As you debug if you find
something that cloud-init should have done, please move this bug state
back to New.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1846513

Title:
  debian 10 cloud-init stuck in systemd-networkd after 1. reboot

Status in cloud-init:
  Invalid

Bug description:
  i have create a virtual machine using the following official debian
  image: https://cdimage.debian.org/cdimage/openstack/current-10/

  they have version 18.3-6 installed.

  after the initial boot of the vm cloud-init works as it should. but
  when a reboot occurred it seems to be hanging in networkd-systemd
  bringing up interfaces. see attached log file.

  i have also manually upgraded to cloud-init 19.2-1 and that had the
  same issue.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1846513/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1837106] Re: "socket.getaddrinfo" of "metadata.google.internal" fails on GCE

[Ryan Harper]

** Changed in: cloud-init
   Importance: Undecided => Medium

** Changed in: cloud-init
   Status: Expired => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1837106

Title:
  "socket.getaddrinfo" of "metadata.google.internal" fails on GCE

Status in cloud-init:
  Triaged

Bug description:
  When booting an Ubuntu 18.04 based image on GCE, we see the following
  messages in the log:

  2019-05-30 00:05:27,818 - util.py[DEBUG]: Resolving URL: 
http://metadata.google.internal/computeMetadata/v1/ took 0.001 seconds
  2019-05-30 00:05:27,818 - DataSourceGCE.py[DEBUG]: 
http://metadata.google.internal/computeMetadata/v1/ is not resolvable
  2019-05-30 00:05:27,818 - util.py[DEBUG]: Crawl of GCE metadata service took 
0.013 seconds
  2019-05-30 00:05:27,818 - DataSourceGCE.py[WARNING]: address 
"http://metadata.google.internal/computeMetadata/v1/; is not resolvable 

  Further, the contents of "/run/cloud-init/instance-data.json" doesn't
  have any meaningful data.

  What I've found is that, read_md() in DataSourceGCE.py will call
  util.is_resolvable_url() on the address
  "http://metadata.google.internal/computeMetadata/v1/;, which results
  is calling socket.getaddrinfo() for "metadata.google.internal", and
  it's this socket.getaddrinfo() call that fails.

  This failure appears to be due to the fact that "cloud-init.service"
  does not ensure it waits for DNS (i.e. "systemd-resolved.service") to
  be working before it runs. I say this because:

  1. If I add "After=systemd-resolved.service" to the "cloud-
  init.service" definition, this failures goes away.

  2. If I run "cloud-init init" after the system has booted up (i.e.
  after enough time has passed, such that DNS is working), the failure
  doesn't occur.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1837106/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1832645] Re: fs_setup/disk_setup: option to wait for the device to exist before continuing

[Ryan Harper]

** Changed in: cloud-init
   Status: Expired => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1832645

Title:
  fs_setup/disk_setup: option to wait for the device to exist before
  continuing

Status in cloud-init:
  Triaged

Bug description:
  When using the AWS::EC2::Volume and AWS::EC2::VolumeAttachment options
  to add a volume to an AWS::EC2::Instance on AWS EC2, the volume is not
  immediately available on the instance.

  This causes fs_setup and disk_setup to fail.

  What would prevent this failure is a "wait" option on both fs_setup
  and disk_setup, which if true, will cause cloud-init to wait until the
  device exists (caused by AWS catching up and attaching the device)
  before continuing.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1832645/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1844191] [NEW] azure advanced networking sometimes triggers duplicate mac detection

[Ryan Harper]

Public bug reported:

Hi, we're still being affected by this on Azure with 19.2-24-ge7881d5c-
0ubuntu1~18.04.1 - using PACKER to build from image: BuildSource :
Marketplace/Canonical/UbuntuServer/18.04-DAILY-LTS

Here is the packer config:

"provisioners": [
{
  "type": "shell",
  "inline": [
"while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 
'Waiting for cloud-init...'; sleep 1; done"
  ]
},
{
"type": "ansible",
"playbook_file": "{{user `ansible_playbook`}}",
"user": "packer",
"extra_arguments": [ "--extra-vars", "codeVersion={{user 
`code_version`}} managed_image_name={{user `managed_image_name`}}" ]
},
{
"type": "shell",
"execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh 
'{{ .Path }}'",
"inline_shebang": "/bin/sh -x",
"inline": [ "/usr/sbin/waagent -force -deprovision+user && export 
HISTSIZE=0 && sync" ]
}]


Here is the playbook:

---
- hosts: all
  remote_user: ubuntu
  become: yes
  become_method: sudo
  become_user: root

  environment:
DEBIAN_FRONTEND: noninteractive


Note: we are applying `enableAcceleratedNetworking: true` to the NIC,
anecdotally we think this is related.

Usually our playbook has more in it (obviously) but Azure kept pointing
fingers at us that our image was causing the problem, so I ran this test
simply deploying a blank deprovisioned image via our same process.

And here's what happens on the serial console log:


[   20.337603] sh[910]: + [ -e /var/lib/cloud/instance/obj.pkl ]
[   20.343177] sh[910]: + echo cleaning persistent cloud-init object
[   20.349027] [  OK  ] Started Network Time Synchronization.
[  OK  ] Reached target System Time Synchronized.
sh[910]: cleaning persistent cloud-init object
[   20.361066] sh[910]: + rm /var/lib/cloud/instance/obj.pkl
[   20.412333] sh[910]: + exit 0
[   34.282291] cloud-init[938]: Cloud-init v. 
19.2-24-ge7881d5c-0ubuntu1~18.04.1 running 'init-local' at Mon, 16 Sep 2019 
18:02:23 +. Up 32.02 seconds.
[   34.288809] cloud-init[938]: 2019-09-16 18:02:25,262 - util.py[WARNING]: 
failed stage init-local
[   34.423057] cloud-init[938]: failed run of stage init-local
[   34.437716] cloud-init[938]: 

[   34.441088] cloud-init[938]: Traceback (most recent call last):
[   34.443719] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 653, in 
status_wrapper
[   34.448072] cloud-init[938]: ret = functor(name, args)
[   34.450532] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in main_init
[   34.454849] cloud-init[938]: 
init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
[   34.458725] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 697, in 
apply_network_config
[   34.463421] cloud-init[938]: net.wait_for_physdevs(netcfg)
[   34.466051] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 344, in 
wait_for_physdevs
[   34.470673] cloud-init[938]: present_macs = 
get_interfaces_by_mac().keys()
[   34.473964] cloud-init[938]:   File 
"/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 633, in 
get_interfaces_by_mac
[   34.479325] cloud-init[938]: (name, ret[mac], mac))
[   34.481838] cloud-init[938]: RuntimeError: duplicate mac found! both 'eth0' 
and 'enP1s1' have mac '00:0d:3a:7c:f7:3f'
[   34.486614] cloud-init[938]: 

[FAILED] Failed to start Initial cloud-init job (pre-networking).
See 'systemctl status cloud-init-local.service' for details.
[  OK  ] Reached target Network (Pre).
 Starting Network Service...
[  OK  ] Started Network Service.
 Starting Wait for Network to be Configured...
 Starting Network Name Resolution...
[  OK  ] Started Wait for Network to be Configured.
 Starting Initial cloud-init job (metadata service crawler)...
[  OK  ] Started Network Name Resolution.
[  OK  ] Reached target Host and Network Name Lookups.
[  OK  ] Reached target Network.


When this happens, the machine never boots, and we get an
OSProvisioningTimedOut error after about 30 minutes, and the machine
never reaches healthy state.

** Affects: cloud-init
 Importance: High
 Status: Triaged

** Changed in: cloud-init
   Importance: Undecided => High

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1844191

Title:
  azure advanced networking sometimes triggers duplicate mac detection

Status in cloud-init:
  Triaged

Bug description:
  Hi, we're still being affected by this on Azure with 

[Yahoo-eng-team] [Bug 1843634] Re: cloud-init misconfigure the network on SLES

[Ryan Harper]

OK, thanks for the logs.  Could you re-attach those running via sudo (or
as root)?  The default user on SLES does not have permissions to read
the journal.

What I see so far looks like networking did not come up after cloud-
init-local.service completes and writes out a network config.

2019-09-11 18:00:15,242 - stages.py[INFO]: Applying network
configuration from ds bringup=False: {'ethernets': {'eth0': {'set-name':
'eth0', 'match': {'macaddress': u'00:0d:3a:6e:6f:8f'}, 'dhcp4': True}},
'version': 2}

This results in the following files being written:

% cat test_azure_sles/etc/sysconfig/network/ifcfg-eth0 
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=dhcp
DEVICE=eth0
HWADDR=00:0d:3a:6e:6f:8f
NM_CONTROLLED=no
ONBOOT=yes
STARTMODE=auto
TYPE=Ethernet
USERCTL=no

Upstream cloud-init on SLES does not generate/update /etc/resolv.conf
but in the logs the cloud-init in does:

2019-09-11 18:00:15,246 - util.py[DEBUG]: Writing to 
/etc/sysconfig/network/ifcfg-eth0 - wb: [644] 191 bytes
2019-09-11 18:00:15,247 - util.py[DEBUG]: Reading from /etc/resolv.conf 
(quiet=False)
2019-09-11 18:00:15,247 - util.py[DEBUG]: Read 795 bytes from /etc/resolv.conf
2019-09-11 18:00:15,247 - util.py[DEBUG]: Writing to /etc/resolv.conf - wb: 
[644] 866 bytes

At first, I thought maybe it was missing this commit:

% git show b74ebca563a21332b29482c8029e7908f60225a4
commit b74ebca563a21332b29482c8029e7908f60225a4
Author: Robert Schweikert 
Date:   Wed Jan 23 22:35:32 2019 +

net/sysconfig: do not write a resolv.conf file with only the header.

Writing the file with no dns information may prevent distro tools
from writing a resolv.conf file with dns information obtained from
a dhcp server.

diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index ae41f7b..fd8e501 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -557,6 +557,8 @@ class Renderer(renderer.Renderer):
 content.add_nameserver(nameserver)
 for searchdomain in network_state.dns_searchdomains:
 content.add_search_domain(searchdomain)
+if not str(content):
+return None
 header = _make_header(';')
 content_str = str(content)
 if not content_str.startswith(header):
@@ -666,7 +668,8 @@ class Renderer(renderer.Renderer):
 dns_path = util.target_path(target, self.dns_path)
 resolv_content = self._render_dns(network_state,
   existing_dns_path=dns_path)
-util.write_file(dns_path, resolv_content, file_mode)
+if resolv_content:
+util.write_file(dns_path, resolv_content, file_mode)
 if self.networkmanager_conf_path:
 nm_conf_path = util.target_path(target,
 self.networkmanager_conf_path)
diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
index d679e92..5313d2d 100644
--- a/tests/unittests/test_net.py
+++ b/tests/unittests/test_net.py
@@ -2098,6 +2098,10 @@ TYPE=Ethernet
 USERCTL=no
 """
 self.assertEqual(expected, found[nspath + 'ifcfg-interface0'])
+# The configuration has no nameserver information make sure we
+# do not write the resolv.conf file
+respath = '/etc/resolv.conf'
+self.assertNotIn(respath, found.keys())
 
 def test_config_with_explicit_loopback(self):
 ns = network_state.parse_net_config_data(CONFIG_V1_EXPLICIT_LOOPBACK)
@@ -2456,6 +2460,10 @@ TYPE=Ethernet
 USERCTL=no
 """
 self.assertEqual(expected, found[nspath + 'ifcfg-interface0'])
+# The configuration has no nameserver information make sure we
+# do not write the resolv.conf file
+respath = '/etc/resolv.conf'
+self.assertNotIn(respath, found.keys())
 
 def test_config_with_explicit_loopback(self):
 ns = network_state.parse_net_config_data(CONFIG_V1_EXPLICIT_LOOPBACK)

But, I believe that is in 19.1 (or likely patched in the distro version).
Later in the boot, we can see that networking didn't actually come up as Azure 
datasource can't find a lease file and then goes into some sort of fallback 
mode which tries to bring up networking (it does) but not with dhcp which is 
why you're missing DNS (it's provided via option to the DHCP response.


2019-09-11 18:00:15,946 - azure.py[DEBUG]: Unable to find endpoint in dhclient 
logs.  Falling back to check lease files
2019-09-11 18:00:15,946 - azure.py[DEBUG]: Looking for endpoint in lease file 
/var/lib/dhcp/dhclient.eth0.leases
2019-09-11 18:00:15,946 - handlers.py[DEBUG]: start: 
azure-ds/_get_value_from_leases_file: _get_value_from_leases_file
2019-09-11 18:00:15,946 - util.py[DEBUG]: Reading from 
/var/lib/dhcp/dhclient.eth0.leases (quiet=False)
2019-09-11 18:00:15,947 - azure.py[ERROR]: Failed to read 
/var/lib/dhcp/dhclient.eth0.leases: [Errno 2] No such file or directory: 

[Yahoo-eng-team] [Bug 1843502] Re: Network config is incorrectly parsed when nameservers are specified

[Ryan Harper]

Issue is related to local changes, marking invalid.

** Changed in: cloud-init
   Status: Incomplete => Invalid

** Changed in: cloud-init (Suse)
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1843502

Title:
  Network config is incorrectly parsed when  nameservers are specified

Status in cloud-init:
  Invalid
Status in cloud-init package in Suse:
  Invalid

Bug description:
  The issue was reproduced on Azure with cloud-init 19.1 on a SLES12 SP4 
machine. Looking at the code, the same behavior could be reproduced on any 
other configuration where the cloud provider specifies nameservers in the 
network configuration.
  The specified nameservers in network configuration are ignored and cloud-init 
raises an error.
  In network_state.py the function _v2_common builds a name_cmd dictionary 
which is then passed to the function handle_nameserver. The handle_nameserver 
has a decorator that enforces that passed in dictionary to have the key 
"address". But the _v2_common build a dictionary that has the key "addresses" 
instead. That results in raising an error.
  Here's a snapshot of the cloud-init.log

  2019-09-09 16:21:29,479 - network_state.py[DEBUG]: v2(nameserver) -> 
v1(nameserver):
  {'search': 'xkf00b0rtzgejkug4xc2pcinre.xx.internal.cloudapp.net', 'type': 
'nameserver', 'addresses': '168.63.129.16'}
  2019-09-09 16:21:29,479 - network_state.py[WARNING]: Skipping invalid 
command: {'nameservers': {'search': 
'xkf00b0rtzgejkug4xc2pcinre.xx.internal.cloudapp.net', 'addresses': 
'168.63.129.16'}, 'eth0': {'set-name': 'eth0', 'match': {'macaddress': 
u'00:0d:3a:6d:ca:25'}, 'dhcp4': True}}
  Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 321, in parse_config_v2
  self._v2_common(command)
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 697, in _v2_common
  self.handle_nameserver(name_cmd)
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 118, in decorator
  required_keys))
  InvalidCommand: Command missing set(['address']) of required keys ['address']

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1843502/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1843502] Re: Network config is incorrectly parsed when nameservers are specified

[Ryan Harper]

Thanks for the bug and the logs.  Looking at the network-config that was
generated:

>>> print(yaml.dump(nc, default_flow_style=False, indent=4))
ethernets:
eth0:
dhcp4: true
match:
macaddress: 00:0d:3a:6d:ca:25
set-name: eth0
nameservers:
addresses: 168.63.129.16
search: xkf00b0rtzgejk

The bug is that nameservers needs to be indented *under* eth0.

However, cloud-init upstream does not parse or process nameservers[1]
from Azure metadata, so I can't understand why you have this bug unless
the cloud-init 19.1 on SLES has some downstream patches.

1. https://git.launchpad.net/cloud-
init/tree/cloudinit/sources/DataSourceAzure.py#n1305


** Changed in: cloud-init
   Status: New => Incomplete

** Also affects: cloud-init (Suse)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1843502

Title:
  Network config is incorrectly parsed when  nameservers are specified

Status in cloud-init:
  Incomplete
Status in cloud-init package in Suse:
  New

Bug description:
  The issue was reproduced on Azure with cloud-init 19.1 on a SLES12 SP4 
machine. Looking at the code, the same behavior could be reproduced on any 
other configuration where the cloud provider specifies nameservers in the 
network configuration.
  The specified nameservers in network configuration are ignored and cloud-init 
raises an error.
  In network_state.py the function _v2_common builds a name_cmd dictionary 
which is then passed to the function handle_nameserver. The handle_nameserver 
has a decorator that enforces that passed in dictionary to have the key 
"address". But the _v2_common build a dictionary that has the key "addresses" 
instead. That results in raising an error.
  Here's a snapshot of the cloud-init.log

  2019-09-09 16:21:29,479 - network_state.py[DEBUG]: v2(nameserver) -> 
v1(nameserver):
  {'search': 'xkf00b0rtzgejkug4xc2pcinre.xx.internal.cloudapp.net', 'type': 
'nameserver', 'addresses': '168.63.129.16'}
  2019-09-09 16:21:29,479 - network_state.py[WARNING]: Skipping invalid 
command: {'nameservers': {'search': 
'xkf00b0rtzgejkug4xc2pcinre.xx.internal.cloudapp.net', 'addresses': 
'168.63.129.16'}, 'eth0': {'set-name': 'eth0', 'match': {'macaddress': 
u'00:0d:3a:6d:ca:25'}, 'dhcp4': True}}
  Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 321, in parse_config_v2
  self._v2_common(command)
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 697, in _v2_common
  self.handle_nameserver(name_cmd)
File "/usr/lib/python2.7/site-packages/cloudinit/net/network_state.py", 
line 118, in decorator
  required_keys))
  InvalidCommand: Command missing set(['address']) of required keys ['address']

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1843502/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1843602] [NEW] cloud-init collect logs needs to defer to distro for debug data

[Ryan Harper]

Public bug reported:

While triaging a bug in sles'based distro using cloud-init, the collect-
logs output had a few issues:

1) the user did not have systemd journal privs

Unexpected error while running command.
Command: ['journalctl', '--boot=0', '-o', 'short-precise']
Exit code: 1
Reason: -
Stdout: 
Stderr: Hint: You are currently not seeing messages from other users and the 
system.
  Users in the 'systemd-journal' group can see all messages. Pass 
-q to
  turn off this notice.
No journal files were opened due to insufficient permissions.

2) we tried to collect dpkg-version on a non debian distro

Unexpected error while running command.
Command: ['dpkg-query', '--show', '-f=${Version}\n', 'cloud-init']
Exit code: -
Reason: [Errno 2] No such file or directory
Stdout: -
Stderr: -

3) version file is empty

4) dmesg as non-root user fails

Unexpected error while running command.
Command: ['dmesg']
Exit code: 1
Reason: -
Stdout: 
Stderr: dmesg: read kernel buffer failed: Operation not permitted

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1843602

Title:
  cloud-init collect logs needs to defer to distro for debug data

Status in cloud-init:
  New

Bug description:
  While triaging a bug in sles'based distro using cloud-init, the
  collect-logs output had a few issues:

  1) the user did not have systemd journal privs

  Unexpected error while running command.
  Command: ['journalctl', '--boot=0', '-o', 'short-precise']
  Exit code: 1
  Reason: -
  Stdout: 
  Stderr: Hint: You are currently not seeing messages from other users and the 
system.
Users in the 'systemd-journal' group can see all messages. Pass 
-q to
turn off this notice.
  No journal files were opened due to insufficient permissions.

  2) we tried to collect dpkg-version on a non debian distro

  Unexpected error while running command.
  Command: ['dpkg-query', '--show', '-f=${Version}\n', 'cloud-init']
  Exit code: -
  Reason: [Errno 2] No such file or directory
  Stdout: -
  Stderr: -

  3) version file is empty

  4) dmesg as non-root user fails

  Unexpected error while running command.
  Command: ['dmesg']
  Exit code: 1
  Reason: -
  Stdout: 
  Stderr: dmesg: read kernel buffer failed: Operation not permitted

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1843602/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1839860] Re: cloud-init error while MAAS commissioning (PXE phase) P9 Witherspoon

[Ryan Harper]

Looks like there's nothing for cloud-init here.  Please reopen the
cloud-init task if that changes.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1839860

Title:
  cloud-init error while MAAS commissioning (PXE phase) P9 Witherspoon

Status in cloud-init:
  Invalid
Status in MAAS:
  Incomplete
Status in The Ubuntu-power-systems project:
  New

Bug description:
  While trying to commissioning bobone (P9 withersppon machine with
  OpenBMC in Server team's MAAS) the PXE phase ended with the following
  cloud-init error (shown at sol):

   Starting Wait until snapd is fully seeded...

  Ubuntu 18.04.3 LTS ubuntu hvc0

  ubuntu login: [  131.162174] cloud-init[5497]: Can not apply stage config, no 
datasource found! Likely bad things to come!
  [  131.162320] cloud-init[5497]: 

  [  131.162414] cloud-init[5497]: Traceback (most recent call last):
  [  131.162512] cloud-init[5497]:   File 
"/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 485, in 
main_modules
  [  131.162614] cloud-init[5497]: init.fetch(existing="trust")
  [  131.162678] cloud-init[5497]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 351, in fetch
  [  131.162776] cloud-init[5497]: return 
self._get_data_source(existing=existing)
  [  131.162851] cloud-init[5497]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 261, in 
_get_data_source
  [  131.162934] cloud-init[5497]: pkg_list, self.reporter)
  [  131.163005] cloud-init[5497]:   File 
"/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 741, in 
find_source
  [  131.163104] cloud-init[5497]: raise DataSourceNotFoundException(msg)
  [  131.163177] cloud-init[5497]: 
cloudinit.sources.DataSourceNotFoundException: Did not find any data source, 
searched classes: ()
  [  131.163269] cloud-init[5497]: 

  [  131.53] cloud-init[5551]: Can not apply stage final, no datasource 
found! Likely bad things to come!
  [  131.566820] cloud-init[5551]: 

  [  131.566922] cloud-init[5551]: Traceback (most recent call last):
  [  131.567004] cloud-init[5551]:   File 
"/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 485, in 
main_modules
  [  131.567116] cloud-init[5551]: init.fetch(existing="trust")
  [  131.567193] cloud-init[5551]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 351, in fetch
  [  131.567274] cloud-init[5551]: return 
self._get_data_source(existing=existing)
  [  131.567348] cloud-init[5551]:   File 
"/usr/lib/python3/dist-packages/cloudinit/stages.py", line 261, in 
_get_data_source
  [  131.567438] cloud-init[5551]: pkg_list, self.reporter)
  [  131.567508] cloud-init[5551]:   File 
"/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 741, in 
find_source
  [  131.567598] cloud-init[5551]: raise DataSourceNotFoundException(msg)
  [  131.567679] cloud-init[5551]: 
cloudinit.sources.DataSourceNotFoundException: Did not find any data source, 
searched classes: ()
  [  131.567779] cloud-init[5551]: 


  Ubuntu 18.04.3 LTS ubuntu hvc0

  ubuntu login:


  MAAS log (from UI):

  TIMEEVENT
    Mon, 12 Aug. 2019 11:10:45 PXE Request - commissioning
    Mon, 12 Aug. 2019 11:08:43 Node powered on
    Mon, 12 Aug. 2019 11:08:14 Powering node on
    Mon, 12 Aug. 2019 11:08:14 Node - Started commissioning on 'bobone'.
    Mon, 12 Aug. 2019 11:08:14 Node changed status - From 'New' to 
'Commissioning'
    Mon, 12 Aug. 2019 11:08:14 User starting node commissioning - (jfh)
    Mon, 12 Aug. 2019 11:07:04 Node powered off


  With that system cannot complete the Commissioning phase.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1839860/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1839491] Re: Manually performed partitioning changes get reverted on reboot

[Ryan Harper]

Moving cloud-init task to invalid, no bug/work for cloud-init.

** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1839491

Title:
  Manually performed partitioning changes get reverted on reboot

Status in cloud-init:
  Invalid
Status in MAAS:
  Triaged

Bug description:
  Hello,

  I am facing an issue where I need to make changes to the initially
  deployed partition layout, but upon making those changes and
  rebooting, the partition layout gets reverted.

  My env:
  MAAS version: 2.6.0 (7802-g59416a869-0ubuntu1~18.04.1)
  System vendor: HP
  System product: ProLiant DL360 Gen9 (780021-S01)
  System version: Unknown
  Mainboard product: ProLiant DL360 Gen9
  Mainboard firmware version: P89
  Mainboard firmware date: 12/27/2015
  CPU model: Intel(R) Xeon(R) CPU E5-2690 v3
  Deployed (16.04 LTS "Xenial Xerus")
  Kernel: xenial (ga-16.04)
  Power type: ipmi
  Power driver: LAN_2_0 [IPMI 2.0]
  Power boot type: EFI boot
  Architecture amd64/generic
  Minimum Kernel: no minimum kernel
  Interfaces: eno1, eno2, noe3, eno4, eno49, eno50. Only eno49 is used.
  Storage: sda Physical 1TB, sdb Physical 1TB.

  
  Steps to reproduce:

  1. Deploy MAAS with the following partition configuration:
  sda-part1 536.9 MB Partition fat32 formatted filesystem mounted at /boot/efi
  sda-part2 100.0 GB Partition ext4 formatted filesystem mounted at /

  2. Check the partitions on the node:

  $ lsblk

  NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
  sda  8:00 931.5G  0 disk 
  |-sda1   8:10   512M  0 part /boot/efi
  `-sda2   8:20   931G  0 part /
  sdb  8:16   0 931.5G  0 disk 

  
  Here we notice the initial partitioning scheme is not respected. This could 
be related to the main issue of partitioning changes being reverted, but could 
also be a separate issue.

  3. Boot an ubuntu ISO and go into rescue mode. I used ubuntu-16.04.6
  -server-amd64.iso

  4. Choose "Do not use a root filesystem" and "Execute a shell in the
  installer environment".

  4. Run the following commands:

  $ e2fsck -f /dev/sda2

  $ resize2fs /dev/sda2 150G

  $ e2fsck -f /dev/sda2

  $ sudo parted /dev/sda

  (parted) unit GiB print

  (parted) resizepart

  Partition number? 2

  End? 200GiB

  (parted) print

  You should see partition 2 resized.

  (parted) quit

  $ e2fsck -f /dev/sda2

  5. Confirm

  $ fdisk -l

  6. Sync writes

  $ sync

  7. Reboot the node. Remove ISO image.

  8. Let system boot, check partitions again:

  $ lsblk

  NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
  sda  8:00 931.5G  0 disk 
  |-sda1   8:10   512M  0 part /boot/efi
  `-sda2   8:20   931G  0 part /
  sdb  8:16   0 931.5G  0 disk 

  We can see see that the changes were reverted.

  If I remove cloud-init, I can successfully re-partition and reboot,
  without the changes being reverted.

  Attached logs before and after repartition.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1839491/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1757054] Re: OpenStack datasource does not deal with 404 Not Found correctly

[Ryan Harper]

*** This bug is a duplicate of bug 1702160 ***
https://bugs.launchpad.net/bugs/1702160

** This bug has been marked a duplicate of bug 1702160
   OpenStack datasource should not retry user-data on 404

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1757054

Title:
  OpenStack datasource does not deal with 404 Not Found correctly

Status in cloud-init:
  New

Bug description:
  OpenStack datasource does not deal with 404 Not Found correctly.If we
  boot a vm without userdata in OpenStack environment, CloudInit retry
  to get usedata when 404 Not Found returned. The GuestOS start up times
  will increase.I think it's not needed and a waste of time.

  Cloud-Init LOG:

  2018-03-19 11:09:16,391 - url_helper.py[DEBUG]: [0/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:16,414 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:17,415 - url_helper.py[DEBUG]: [1/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:17,445 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:18,446 - url_helper.py[DEBUG]: [2/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:18,468 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:19,470 - url_helper.py[DEBUG]: [3/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:19,535 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:20,536 - url_helper.py[DEBUG]: [4/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:20,556 - url_helper.py[DEBUG]: Please wait 1 seconds while 
we wait to try again
  2018-03-19 11:09:21,557 - url_helper.py[DEBUG]: [5/6] open 
'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 
'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': 
{'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 
'timeout': 50.0} configuration
  2018-03-19 11:09:21,584 - openstack.py[DEBUG]: Failed reading optional path 
http://169.254.169.254:80/openstack/2015-10-15/user_data due to: 404 Client 
Error: Not Found

  Go through the code, I found the following logic which i think it's not 
correct.
  cloudinit/sources/helpers/openstack.py
  def should_retry_cb(_request_args, cause):
  try:
  code = int(cause.code)
  if code >= 400:
  return False
  except (TypeError, ValueError):
  # Older versions of requests didn't have a code.
  pass
  return True

  response = url_helper.readurl(path,
     retries=self.retries,
     ssl_details=self.ssl_details,
     timeout=self.timeout,
      exception_cb=should_retry_cb)

  1.When 404 returned, do not need to retry.
  2.exception_cb=should_retry_cb need to be changed to 
exception_cb=should_skip_retry_cb.  May be a Bug here:)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1757054/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1758339] Re: cloud-init init-local tries to get data over network

[Ryan Harper]

Please reopen (mark as new) if you feel there's something that cloud-
init is not doing correctly.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1758339

Title:
  cloud-init init-local tries to get data over network

Status in cloud-init:
  Invalid

Bug description:
  init-local tries to gather data from metadata service before the
  network is ready which causes a WARNING message sent to syslog. I
  believe everything ends up being successful and it's just some noise
  in syslog during instance launch.

  Mar 23 04:18:27 task-302 systemd[1]: Starting Initial cloud-init job 
(pre-networking)...
  Mar 23 04:18:33 task-302 cloud-init[399]: Cloud-init v. 17.2 running 
'init-local' at Fri, 23 Mar 2018 04:18:28 +. Up 4.98 seconds.
  Mar 23 04:18:33 task-302 cloud-init[399]: 2018-03-23 04:18:33,395 - 
util.py[WARNING]: Failed fetching dynamic/instance-identity from url 
http://169.254.169.254/2009-04-04/dynamic/instance-identity
  Mar 23 04:18:33 task-302 systemd[1]: Started Initial cloud-init job 
(pre-networking).
  Mar 23 04:18:34 task-302 systemd[1]: Starting Initial cloud-init job 
(metadata service crawler)...
  Mar 23 04:18:34 task-302 cloud-init[989]: Cloud-init v. 17.2 running 'init' 
at Fri, 23 Mar 2018 04:18:34 +. Up 11.03 seconds.
  Mar 23 04:18:34 task-302 cloud-init[989]: ci-info: 
++Net device 
info++
  Mar 23 04:18:34 task-302 cloud-init[989]: ci-info: 
++--+-+---+---+---+
  Mar 23 04:18:34 task-302 cloud-init[989]: ci-info: | Device |  Up  |  
 Address   |  Mask | Scope | Hw-Address|
  Mar 23 04:18:34 task-302 cloud-init[989]: ci-info: 
++--+-+---+---+---+

  cloud-init version
  cloud-init -v
  /usr/bin/cloud-init 17.2
  ii  cloud-init   17.2-35-gf576b2a2-0ubuntu1~16.04.2   
   all  Init scripts for cloud instances

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1758339/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1758409] Re: integration tests: restructure ssh timeout

[Ryan Harper]

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1758409

Title:
  integration tests: restructure ssh timeout

Status in cloud-init:
  Fix Released

Bug description:
  # Summary
  During the integration tests, currently if SSH to instance times out it holds 
up testing for over an hour in an attempt to SSH to an instance; note the 
timestamp jump on: https://paste.ubuntu.com/p/NBQKwm9wdG/

  The _ssh_connect function was originally written for the nocloud_kvm
  platform and used as a method for determining if an instance was up
  and accessible. As such, the function is doing double duty and not
  correctly focused on SSH'ing to an up and running instance and has a
  bug in it as it is waiting far too long.

  # Action plan

  1. For the nocloud_kvm platform when when starting and before
  _wait_for_system, there should be a check if an instance is accessible
  during the is_running check. This could be done again by SSH with a
  number of retries, but should be taken care of inside the nocloud_kvm
  platform itself and not in the SSH connect function.

  2. Update the _ssh_connect to timeout quickly, reduce wait on banner,
  and only retry up to 3 times.

  # Noted Files
  tests/cloud_tests/platforms/platforms.py:_ssh_connect()
  tests/cloud_tests/platforms/nocloudkvm/instance.py:start()

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1758409/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1768468] Re: error in ca_certs module

[Ryan Harper]

I'm marking the cloud-init task invalid; I don't believe cloud-init did
anything wrong; but please set the task back to New if you have new
information showing that cloud-init didn't do something quite right.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1768468

Title:
  error in ca_certs module

Status in cloud-init:
  Invalid
Status in ca-certificates package in Ubuntu:
  New

Bug description:
  Hello,
  I'm using cloud-init in order to customize Nutanix AHV images.
  I'm using the ca_certs module to upload our private ca chain (root and 
intermediate) but in the log I found a log that states that
  -
  Cloud-init v. 18.2 running 'init-local' at Wed, 02 May 2018 08:54:58 +. 
Up 6.25 seconds.
  Updating certificates in /etc/ssl/certs...
  rehash: skipping cloud-init-ca-certs.pem,it does not contain exactly one 
certificate or CRL
  1 added, 0 removed; done.
  Running hooks in /etc/ca-certificates/update.d...
  done.
  -
  And the certificates are not trusted.
  Ubuntu version is 18.04.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1768468/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1767002] Re: When stop(deallocate) and then start a RHEL 7.5 VM, cloud-init cannot mount /dev/sdb1 to /mnt.

[Ryan Harper]

I'm closing this bug as Fixed Released.

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1767002

Title:
  When stop(deallocate) and then start a RHEL 7.5 VM, cloud-init cannot
  mount /dev/sdb1 to /mnt.

Status in cloud-init:
  Fix Released

Bug description:
  When stop(deallocate) and then start a RHEL VM, cloud-init cannot
  mount /dev/sdb1 to /mnt.

  You see this error:

  /var/log/cloud-init.log:
  2018-03-23 06:30:52,018 - util.py[DEBUG]: Running command ['mount', '-o', 
'ro,sync', '-t', 'auto', '/dev/disk/cloud/azure_resource-part1', 
'/tmp/tmp9iIt2n'] with allowed return codes [0] (shell=False, capture=True)
  2018-03-23 06:30:52,038 - util.py[DEBUG]: Failed mount of 
'/dev/disk/cloud/azure_resource-part1' as 'auto': Unexpected error while 
running command.
  Command: ['mount', '-o', 'ro,sync', '-t', 'auto', 
'/dev/disk/cloud/azure_resource-part1', '/tmp/tmp9iIt2n']
  Exit code: 32
  Reason: -
  Stdout: -
  Stderr: mount: unknown filesystem type 'ntfs'
  2018-03-23 06:30:52,038 - util.py[DEBUG]: Recursively deleting /tmp/tmp9iIt2n
  2018-03-23 06:30:52,038 - DataSourceAzure.py[DEBUG]: reformattable=False: 
partition 1 (/dev/disk/cloud/azure_resource-part1 -> /dev/sdb1) on device 
/dev/disk/cloud/azure_resource was ntfs formatted but mount of 
/dev/disk/cloud/azure_resource-part1 failed: Failed mounting 
/dev/disk/cloud/azure_resource-part1 to /tmp/tmp9iIt2n due to: Unexpected error 
while running command.
  Command: ['mount', '-o', 'ro,sync', '-t', 'auto', 
'/dev/disk/cloud/azure_resource-part1', '/tmp/tmp9iIt2n']
  Exit code: 32
  Reason: -
  Stdout: -
  Stderr: mount: unknown filesystem type 'ntfs'

  This has been patched here: https://code.launchpad.net/~paul-meyer
  /cloud-init/+git/cloud-init/+ref/fix-ntfs-mount.

  Thanks,

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1767002/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1768468] Re: error in ca_certs module

[Ryan Harper]

Hi,  I don't believe this is a cloud-init bug.  Cloud-init wrote the two
certificate files requested.

2018-05-02 08:55:00,913 - stages.py[DEBUG]: Running module ca-certs () with 
frequency once-per-instance
2018-05-02 08:55:00,914 - handlers.py[DEBUG]: start: 
init-network/config-ca-certs: running config-ca-certs with frequency 
once-per-instance
2018-05-02 08:55:00,914 - util.py[DEBUG]: Writing to 
/var/lib/cloud/instances/44574988-144d-485a-9386-2ec4f4f545c4/sem/config_ca_certs
 - wb: [644] 24 bytes
2018-05-02 08:55:00,914 - helpers.py[DEBUG]: Running config-ca-certs using lock 
()
2018-05-02 08:55:00,914 - cc_ca_certs.py[DEBUG]: Adding 2 certificates
2018-05-02 08:55:00,916 - util.py[DEBUG]: Writing to 
/usr/share/ca-certificates/cloud-init-ca-certs.crt - wb: [644] 4545 bytes
2018-05-02 08:55:00,917 - util.py[DEBUG]: Reading from 
/etc/ca-certificates.conf (quiet=False)
2018-05-02 08:55:00,917 - util.py[DEBUG]: Read 5898 bytes from 
/etc/ca-certificates.conf
2018-05-02 08:55:00,917 - util.py[DEBUG]: Writing to /etc/ca-certificates.conf 
- wb: [644] 5922 bytes
2018-05-02 08:55:00,918 - cc_ca_certs.py[DEBUG]: Updating certificates
2018-05-02 08:55:00,918 - util.py[DEBUG]: Running command 
['update-ca-certificates'] with allowed return codes [0] (shell=False, 
capture=False)
2018-05-02 08:55:01,845 - handlers.py[DEBUG]: finish: 
init-network/config-ca-certs: SUCCESS: config-ca-certs ran successfully

The error you show mentions a 'cloud-init-ca-certs.pem' It may be
related the ca-certificates package?  I'm adding the package as a task.

** Also affects: ca-certificates (Ubuntu)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1768468

Title:
  error in ca_certs module

Status in cloud-init:
  Invalid
Status in ca-certificates package in Ubuntu:
  New

Bug description:
  Hello,
  I'm using cloud-init in order to customize Nutanix AHV images.
  I'm using the ca_certs module to upload our private ca chain (root and 
intermediate) but in the log I found a log that states that
  -
  Cloud-init v. 18.2 running 'init-local' at Wed, 02 May 2018 08:54:58 +. 
Up 6.25 seconds.
  Updating certificates in /etc/ssl/certs...
  rehash: skipping cloud-init-ca-certs.pem,it does not contain exactly one 
certificate or CRL
  1 added, 0 removed; done.
  Running hooks in /etc/ca-certificates/update.d...
  done.
  -
  And the certificates are not trusted.
  Ubuntu version is 18.04.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1768468/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1788960] Re: rhevm related cloud-init integration test fail for cloud-init v18.3

[Ryan Harper]

Thanks for filing the bug.  I believe this issue has been resolved in
this commit:

commit a8dcad9ac62bb1d2a4f7489960395bad6cac9382
Author: Scott Moser 
Date:   Wed Sep 5 16:02:25 2018 +

tests: Disallow use of util.subp except for where needed.

In many cases, cloud-init uses 'util.subp' to run a subprocess.
This is not really desirable in our unit tests as it makes the tests
dependent upon existance of those utilities.

The change here is to modify the base test case class (CiTestCase) to
raise exception any time subp is called.  Then, fix all callers.
For cases where subp is necessary or actually desired, we can use it
via
  a.) context hander CiTestCase.allow_subp(value)
  b.) class level self.allowed_subp = value

Both cases the value is a list of acceptable executable names that
will be called (essentially argv[0]).

Some cleanups in AltCloud were done as the code was being updated.


** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1788960

Title:
  rhevm related cloud-init integration test fail for cloud-init v18.3

Status in cloud-init:
  Fix Released

Bug description:
  There are 3 of the rhevm related integration test failure:
  test_mount_cb_fails, test_no_udevadm_cmd test_no_udevadm_cmd, seems
  all related to "user_data_rhevm()"

  
  OS:  Ubuntu 16.04 Xenial
  cloud-init: v18.3 dab59087155d3963849a36b3f63ee662047f708b

  
  log snip:
  peter@U1604CloudinitTools:~/cloud-init$  tox -e citest -- run --verbose \
  > --preserve-data --data-dir ./debug \
  > 
  GLOB sdist-make: /home/peter/cloud-init/setup.py
  citest create: /home/peter/cloud-init/.tox/citest
  citest installdeps: -r/home/peter/cloud-init/integration-requirements.txt
  citest inst: /home/peter/cloud-init/.tox/dist/cloud-init-18.3.zip
  citest installed: 
asn1crypto==0.24.0,bcrypt==3.1.4,boto3==1.5.9,botocore==1.8.50,certifi==2018.8.24,cffi==1.11.5,chardet==3.0.4,cloud-init==18.3,configobj==5.0.6,cryptography==2.3.1,docutils==0.14,idna==2.7,Jinja2==2.10,jmespath==0.9.3,jsonpatch==1.23,jsonpointer==2.0,jsonschema==2.6.0,MarkupSafe==1.0,oauthlib==2.1.0,paramiko==2.4.1,pbr==4.2.0,pyasn1==0.4.4,pycparser==2.18,pylxd==2.2.7,PyNaCl==1.2.1,python-dateutil==2.7.3,python-simplestreams==0.1.0,PyYAML==3.13,requests==2.19.1,requests-toolbelt==0.8.0,requests-unixsocket==0.1.5,s3transfer==0.1.13,six==1.11.0,urllib3==1.23,ws4py==0.5.1
  citest runtests: PYTHONHASHSEED='2994029421'
  citest runtests: commands[0] | /home/peter/cloud-init/.tox/citest/bin/python 
-m tests.cloud_tests run --verbose --preserve-data --data-dir debug

  
  ==
  ERROR: Test user_data_rhevm() where mount_cb fails.
  --
  Traceback (most recent call last):
File 
"/home/peter/cloud-init/tests/unittests/test_datasource/test_altcloud.py", line 
296, in test_mount_cb_fails
  self.assertEqual(False, dsrc.user_data_rhevm())
File "/home/peter/cloud-init/cloudinit/sources/DataSourceAltCloud.py", line 
198, in user_data_rhevm
  (cmd_out, _err) = util.udevadm_settle(exists=floppy_dev, timeout=5)
  TypeError: 'NoneType' object is not iterable
   >> begin captured logging << 
  cloudinit.util: DEBUG: Running command ['true'] with allowed return codes [0] 
(shell=False, capture=True)
  cloudinit.sources.DataSourceAltCloud: DEBUG: Command: true
  Output
  - >> end captured logging << -

  ==
  ERROR: Test user_data_rhevm() with no udevadm command.
  --
  Traceback (most recent call last):
File 
"/home/peter/cloud-init/tests/unittests/test_datasource/test_altcloud.py", line 
324, in test_no_udevadm_cmd
  self.assertEqual(False, dsrc.user_data_rhevm())
File "/home/peter/cloud-init/cloudinit/sources/DataSourceAltCloud.py", line 
198, in user_data_rhevm
  (cmd_out, _err) = util.udevadm_settle(exists=floppy_dev, timeout=5)
  TypeError: 'NoneType' object is not iterable
   >> begin captured logging << 
  cloudinit.util: DEBUG: Running command ['modprobe', 'floppy'] with allowed 
return codes [0] (shell=False, capture=True)
  cloudinit.sources.DataSourceAltCloud: DEBUG: Command: modprobe floppy
  Output
  - >> end captured logging << -

  ==
  ERROR: Test user_data_rhevm() where udevadm fails.
  --
  Traceback (most recent call last):
File 

[Yahoo-eng-team] [Bug 1806338] Re: pip2 install zbar

[Ryan Harper]

Hi,

I don't think this is related to the cloud-init project so I'm marking
the cloud-init task as invalid.

** Changed in: cloud-init
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1806338

Title:
  pip2 install zbar

Status in cloud-init:
  Invalid

Bug description:
  2 . 1@linux-e9a2:~/Downloads/HPLIP_3181136-Latest> su -c "pip2 install zbar"
  Password:
  /usr/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py:83: 
RequestsDependencyWarning: Old version of cryptography ([1, 3, 1]) may cause 
slowdown.
warnings.warn(warning, RequestsDependencyWarning)
  Collecting zbar
Using cached 
https://files.pythonhosted.org/packages/33/54/cc5819efc9ee7e34b60b41e1d2d4753b6dd0c26a41c9a552611f66aa106e/zbar-0.10.tar.bz2
  Installing collected packages: zbar
Running setup.py install for zbar ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, 
tokenize;__file__='/tmp/pip-install-GIARuQ/zbar/setup.py';f=getattr(tokenize, 
'open', open)(__file__);code=f.read().replace('\r\n', 
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record 
/tmp/pip-record-qH93Ju/install-record.txt --single-version-externally-managed 
--compile:
  running install
  running build
  running build_ext
  building 'zbar' extension
  creating build
  creating build/temp.linux-x86_64-2.7
  gcc -pthread -fno-strict-aliasing -fmessage-length=0 
-grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector 
-funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -g 
-DNDEBUG -fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 
-fstack-protector -funwind-tables -fasynchronous-unwind-tables 
-fstack-clash-protection -g -DOPENSSL_LOAD_CONF -fwrapv -fPIC 
-I/usr/include/python2.7 -c zbarmodule.c -o 
build/temp.linux-x86_64-2.7/zbarmodule.o
  gcc: error: unrecognized command line option ‘-fstack-clash-protection’
  gcc: error: unrecognized command line option ‘-fstack-clash-protection’
  error: command 'gcc' failed with exit status 1
  
  
  Command "/usr/bin/python -u -c "import setuptools, 
tokenize;__file__='/tmp/pip-install-GIARuQ/zbar/setup.py';f=getattr(tokenize, 
'open', open)(__file__);code=f.read().replace('\r\n', 
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record 
/tmp/pip-record-qH93Ju/install-record.txt --single-version-externally-managed 
--compile" failed with error code 1 in /tmp/pip-install-GIARuQ/zbar/

  
  su -c "pip2 install tesserocr"
  Password:
  /usr/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py:83: 
RequestsDependencyWarning: Old version of cryptography ([1, 3, 1]) may cause 
slowdown.
warnings.warn(warning, RequestsDependencyWarning)
  Collecting tesserocr
Using cached 
https://files.pythonhosted.org/packages/f8/6d/4e81e041f33a4419e59edcb1dbdf3c56e9393f60f5ef531381bd67a1339b/tesserocr-2.3.1.tar.gz
  Complete output from command python setup.py egg_info:
  Supporting tesseract v3.04.00
  Configs from pkg-config: {'libraries': ['lept', 'tesseract'], 
'cython_compile_time_env': {'TESSERACT_VERSION': 197632}, 'include_dirs': 
['/usr/include']}
  Unable to find pgen, not compiling formal grammar.
  /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown 
distribution option: 'python_requires'
warnings.warn(msg)
  Compiling /tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Plex/Scanners.py 
because it changed.
  Compiling /tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Plex/Actions.py 
because it changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/Scanning.py because it 
changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/Visitor.py because it 
changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/FlowControl.py because 
it changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Runtime/refnanny.pyx because it 
changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/FusedNode.py because it 
changed.
  Compiling 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Tempita/_tempita.py because it 
changed.
  [1/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/FlowControl.py
  [2/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/FusedNode.py
  [3/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/Scanning.py
  [4/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Compiler/Visitor.py
  [5/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Plex/Actions.py
  [6/8] Cythonizing 
/tmp/easy_install-YGWrKs/Cython-0.29.1/Cython/Plex/Scanners.py
  [7/8] Cythonizing 

[Yahoo-eng-team] [Bug 1645824] Re: NoCloud source doesn't work on FreeBSD

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1645824

Title:
  NoCloud source doesn't work on FreeBSD

Status in cloud-init:
  Fix Released

Bug description:
  Hey guys,

  I'm trying to use cloud-init on FreeBSD using CD to seed metadata, the
  thing is that it had some issues:

  - Mount option 'sync' is not allowed for cd9660 filesystem.
  - I optimized the list of filesystems that needed to be scanned for metadata 
by having three lists (vfat, iso9660, and label list) and then checking against 
them to see which filesystem option needs to be passed to mount command.

  Additionally I'm going to push some changes to FreeBSD cloud-init
  package so it can build last version. I will open another ticket for
  fixing networking in FreeBSD as it doesn't support sysfs
  (/sys/class/net/) by default.

  Thanks!

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1645824/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1669875] Re: identify openstack vmware platform

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1669875

Title:
  identify openstack vmware platform

Status in cloud-init:
  Fix Released
Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  We need a way to identify locally that we are running on openstack
  when inside a guest of VmWare.

  Related bugs:
https://bugs.launchpad.net/cloud-init/+bugs?field.tag=dsid-nova

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1669875/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1821102] Re: Cloud-init should not setup ephemeral ipv4 if apply_network_config is False for OpenStack

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1821102

Title:
  Cloud-init should not setup ephemeral ipv4 if apply_network_config is
  False for OpenStack

Status in cloud-init:
  Fix Released

Bug description:
  As fixed in bug #1749717, cloud-init will attempt to configure an
  ephemeral ipv4 address on the first interface to fetch OpenStack (and
  probably others) networking config via a metadata URL.

  There's a couple of issues with this implementation that affect our
  OpenStack cloud.

  Access to our metadata server on 169.254.169.254 is delivered by an
  additional route delivered by DHCP, which is not configured via cloud-
  init's dhcp.py (that is probably another bug).

  Also, we needed to bump up the timeouts for accessing our metadata, as
  we're a largeish cloud and the defaults were way too low. We actually
  copied the timeout/retry values from the Ec2 Datasource.

  So the result is that users are left waiting for cloud-init-local
  stage to timeout, as the additional route to the metadata server isn't
  configured, which was 2 mins in our config.

  I believe a simple fix for this situation would be to skip the
  ephemeral ipv4 setup if the datastore config has apply_network_config:
  False

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1821102/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1806701] Re: cloud-init may hang OS boot process due to grep for the entire ISO file when it is attached

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1806701

Title:
  cloud-init may hang OS boot process due to grep for the entire ISO
  file when it is attached

Status in cloud-init:
  Fix Released

Bug description:
  We have found in our test for SLES15 with cloud-init installed, if we
  attach a ISO file with the VM before VM is boot, it often takes more
  than 10 minutes to start the SLES OS. Sometimes it failed to start the
  SLES OS at all.

  We've root caused it is due to the  "is_cdrom_ovf()" func of "tools
  /ds-identify".

  In this function, there is the following logic to detect if an ISO contains 
certain string:
  >local idstr="http://schemas.dmtf.org/ovf/environment/1;
  >grep --quiet --ignore-case "$idstr" "${PATH_ROOT}$dev"
  ref: https://git.launchpad.net/cloud-init/tree/tools/ds-identify

  It is trying to grep the who ISO file for a certain string, which causes 
intense IO pressure for the system.
  What is worse is that sometimes the ISO file is large (e.g. >5GB for 
installer DVD) and it is mounted over NFS. The "grep" process often consume 99% 
CPU and seems hang. Then the systemd starts more and more "grep" process which 
smoke the CPU and consumes all the IO bandwidth for the ISO file. Then the 
system may hang for a long time and sometimes failed to start.

  To fix this issue, I suggest that we should not grep for the entire
  ISO file. Rather then we should just check if the file/dir exists with
  os.path.exists().

  
  -debug log snip
  pek2-gosv-16-dhcp180:~ # ps -ef
  UIDPID  PPID  C STIME TTY  TIME CMD
  root 1 0  0 13:32 ?00:00:04 /usr/lib/systemd/systemd 
--switched-root --system --deserialize 24
  …
  root   474 1  0 13:34 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root   482   474  2 13:34 ?00:00:15 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  root  1020 1  0 13:35 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root  1039  1020  1 13:35 ?00:00:07 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  polkitd   1049 1  0 13:37 ?00:00:00 /usr/lib/polkit-1/polkitd 
--no-debug
  root  1051 1  0 13:37 ?00:00:00 /usr/sbin/wickedd --systemd 
--foreground
  root  1052 1  0 13:37 ?00:00:00 
/usr/lib/systemd/systemd-logind
  root  1054 1  0 13:37 ?00:00:00 /usr/sbin/wickedd-nanny 
--systemd --foreground
  root  1073 1  0 13:37 ?00:00:00 /usr/bin/vmtoolsd
  root  1097 1  0 13:37 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root  1110  1097  1 13:37 ?00:00:04 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  root  1304 1  0 13:38 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root  1312  1304  1 13:38 ?00:00:03 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  root  1537 1  0 13:40 ?00:00:00 /usr/bin/plymouth --wait
  root  1613 1  0 13:40 ?00:00:00 /bin/sh 
/usr/lib/cloud-init/ds-identify
  root  1645  1613  0 13:40 ?00:00:02 grep --quiet --ignore-case 
http://schemas.dmtf.org/ovf/environment/1 /dev/sr1
  …

  
  Grep use nearly 100% cpu, system very slow.

  top - 13:46:37 up 26 min,  2 users,  load average: 14.14, 15.03, 10.57
  Tasks: 225 total,   6 running, 219 sleeping,   0 stopped,   0 zombie
  %Cpu(s): 40.1 us, 49.3 sy,  0.0 ni,  0.0 id,  1.4 wa,  0.0 hi,  9.1 si,  0.0 
st
  KiB Mem :  1000916 total,64600 free,   355880 used,   580436 buff/cache
  KiB Swap:  1288168 total,  1285600 free, 2568 used.   492688 avail Mem

PID USER  PR  NIVIRTRESSHR S  %CPU  %MEM TIME+ COMMAND
  4427 root  20   0   40100   3940   3084 R 99.90 0.394   0:27.41 top
  1016 root  20   0  197796   4852   3400 R 99.90 0.485   1:26.44 vmtoolsd
  1723 root  20   07256   1860   1556 D 99.90 0.186   0:28.44 grep
484 root  20   07256   1684   1396 D 99.90 0.168   1:51.22 grep
  1278 root  20   07256   1856   1556 D 99.90 0.185   0:38.44 grep
  1398 root  20   07256   1860   1556 R 99.90 0.186   0:28.53 grep
  1061 root  20   07256   1856   1556 D 99.90 0.185   0:56.62 grep
  -debug log snip

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1806701/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to   

[Yahoo-eng-team] [Bug 1827238] Re: Machines fail to deploy because cloud-init needs to accept both netplan spellings for grat arp

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1827238

Title:
  Machines fail to deploy because cloud-init needs to accept both
  netplan spellings for grat arp

Status in cloud-init:
  Fix Released
Status in curtin:
  Invalid
Status in MAAS:
  Fix Released

Bug description:
  Many nodes failed to boot after installation.

  Here is one example, beartic.

  beartic.
  finishes install: 2019-05-01T10:36:17+00:00 beartic systemd[1]: Stopping 
Unattended Upgrades Shutdown...

  dhcp's after reboot:
  
10.244.40.32/var/log/maas/rsyslog/leafeon/2019-05-01/messages:2019-05-01T10:38:10+00:00
 leafeon dhcpd[21312]: DHCPDISCOVER from 14:02:ec:41:c7:dc via broam
  
10.244.40.32/var/log/maas/rsyslog/leafeon/2019-05-01/messages:2019-05-01T10:38:10+00:00
 leafeon dhcpd[21312]: DHCPOFFER on 10.244.41.7 to 14:02:ec:41:c7:dc via broam
  
10.244.40.32/var/log/maas/rsyslog/leafeon/2019-05-01/messages:2019-05-01T10:38:13+00:00
 leafeon dhcpd[21312]: DHCPREQUEST for 10.244.41.7 (10.244.40.30) from 
14:02:ec:41:c7:dc via broam
  
10.244.40.32/var/log/maas/rsyslog/leafeon/2019-05-01/messages:2019-05-01T10:38:13+00:00
 leafeon dhcpd[21312]: DHCPACK on 10.244.41.7 to 14:02:ec:41:c7:dc via broam

  grub and grub.cfg:
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:13 
provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 
10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:13 
provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 
10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:14 
provisioningserver.rackdservices.tftp: [info] grubx64.efi requested by 
10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/command.lst 
requested by 10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/fs.lst requested 
by 10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/crypto.lst 
requested by 10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/x86_64-efi/terminal.lst 
requested by 10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg requested by 
10.244.41.7
  10.244.40.30/var/log/maas/rackd.log:2019-05-01 10:38:15 
provisioningserver.rackdservices.tftp: [info] /grub/grub.cfg-14:02:ec:41:c7:dc 
requested by 10.244.41.7

  but we never got any rsyslog message or api calls after that.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1827238/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1812857] Re: RuntimeError: duplicate mac found! both 'ens4' and 'bond0' have mac '9c:XX:XX:46:5d:91'

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1812857

Title:
  RuntimeError: duplicate mac found! both 'ens4' and 'bond0' have mac
  '9c:XX:XX:46:5d:91'

Status in cloud-init:
  Fix Released

Bug description:
  2019-01-22 13:58:22,667 - util.py[DEBUG]: failed stage init   
   
  Traceback (most recent call last):
   
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 658, in 
status_wrapper   
  ret = functor(name, args) 
   
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in 
main_init
  init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
   
File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 648, in 
apply_network_config   
  netcfg, src = self._find_networking_config()  
   
File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 635, in 
_find_networking_config
  if self.datasource and hasattr(self.datasource, 'network_config'):
   
File 
"/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceConfigDrive.py", 
line 155, in network_config  
  self.network_json, known_macs=self.known_macs)
   
File 
"/usr/lib/python3/dist-packages/cloudinit/sources/helpers/openstack.py", line 
655, in convert_net_json
  known_macs = net.get_interfaces_by_mac()  
   
File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 595, 
in get_interfaces_by_mac
  (name, ret[mac], mac))
   
  RuntimeError: duplicate mac found! both 'ens4' and 'bond0' have mac 
'9c:XX:XX:46:5d:91'  

  
  Net config:
  2019-01-22 13:56:20,055 - stages.py[DEBUG]: applying net config names for 
{'version': 1, 'config': [{'mtu': 1500, 'type': 'bond', 'subnets': [{'type': 
'dhcp4'}], 'params': {'mac_address': '9c:XX:XX:46:5d:91', 'bond_up-delay': 
25, 'bond_mimon': 100, 'bond_xmit_hash_policy': 'layer2+3', 'bond_mode': 
'802.3ad'}, 'name': 'bond0', 'bond_interfaces': ['ens4', 'e
  ns4d1']}, {'type': 'physical', 'mtu': 1500, 'subnets': [{'type': 'dhcp4'}], 
'mac_address': 'f4:XX:XX:44:6f:f0', 'name': 'eno1'}, {'type': 'physical', 
'subnets': [], 'mac_address': '9c:XX:XX:46:5d:91', 'name': 'ens4'}, {'type': 
'physical', 'subnets': [], 'mac_address': '9c:XX:XX:46:5d:92', 'name': 
'ens4d1'}, {'type': 'nameserver', 'address': '8.8.8.8'}]}  

  OS: Ubuntu 18.04.1 LTS
  cloud-init: 18.4-0ubuntu1~18.04.1
  cloud-provider: OpenStack
  Datasource: ConfigDrive

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1812857/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1833192] Re: VMware: post custom script isn't run correctly

[Ryan Harper]

** Also affects: cloud-init
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1833192

Title:
  VMware: post custom script isn't run correctly

Status in cloud-init:
  Fix Released
Status in cloud-init package in Ubuntu:
  New

Bug description:
  In DataSouceOVF.py, it could launch the user defined script before the
  customization and after the customization. In current design, the post
  customization script is added as the rc.local service and launched
  after the VM reboot. But actually the cloud-init doesn't need to
  reboot the VM, so the post customization script is not launched as we
  expected. We need to leverage the cc_scripts_per_instance module and
  let it to trigger the post customization script.

  ProblemType: Bug
  DistroRelease: Ubuntu 16.04
  Package: evince (not installed)
  ProcVersionSignature: Ubuntu 4.4.0-21.37-generic 4.4.6
  Uname: Linux 4.4.0-21-generic x86_64
  ApportVersion: 2.20.1-0ubuntu2.18
  Architecture: amd64
  Date: Tue Jun 18 08:58:04 2019
  InstallationDate: Installed on 2018-08-03 (319 days ago)
  InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Release amd64 
(20160420.3)
  ProcEnviron:
   TERM=linux
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  SourcePackage: evince
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1833192/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1836598] Re: cc_ntp generates wrong config for ntpd in debian (buster)

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1836598

Title:
  cc_ntp generates wrong config for ntpd in debian (buster)

Status in cloud-init:
  Fix Released

Bug description:
  when i use the following configuration:

  ntp:
enabled: true
ntp_client: ntp
pools: [0.int.pool.ntp.org, 1.int.pool.ntp.org, ntp.myorg.org]
servers:
  - ntp.server.local
  - ntp.ubuntu.com
  - 192.168.23.2

  wrong configuration pools received,
  cat /etc/ntpd.conf

  ...skip...
  # pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
  # pick a different set every time it starts up.  Please consider joining the
  # pool: 
  # poolspool 0.int.pool.ntp.org iburst
  pool 1.int.pool.ntp.org iburst
  pool ntp.myorg.org iburst
  # servers
  server ntp.server.local iburst
  server ntp.ubuntu.com iburst
  server 192.168.23.2 iburst

  # Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html 
for
  ...skip...

  wrong line >> # poolspool 0.int.pool.ntp.org iburst

  
  system>
  Debian GNU/Linux 10 (buster)
  cloud-init 18.3-6

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1836598/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1833264] Re: cloud-init-generator hardcodes path to ds-identify

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1833264

Title:
  cloud-init-generator hardcodes path to ds-identify

Status in cloud-init:
  Fix Released

Bug description:
  On systemd systems, the cloud-init-generator uses the ds-identify
  program and hardcodes the path to the program.  The value is correct
  for Ubuntu/Debian and other systems, however on centos-based systems,
  ds-identify is installed down libexec path.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1833264/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1836921] Re: Release 19.2

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 19.2. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1836921

Title:
  Release 19.2

Status in cloud-init:
  Fix Released

Bug description:
  This bug tracks cloud-init upstream release of version 19.2.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1836921/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1836921] [NEW] Release 19.2

[Ryan Harper]

Public bug reported:

This bug tracks cloud-init upstream release of version 19.2.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1836921

Title:
  Release 19.2

Status in cloud-init:
  New

Bug description:
  This bug tracks cloud-init upstream release of version 19.2.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1836921/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1835008] Re: network_v2 does not set mtu on eni config

[Ryan Harper]

for non-physical interfaces, common attributes like mtu were not copied
into the internal state when merging v2 config.

** Changed in: curtin
   Status: New => Invalid

** Changed in: cloud-init
   Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1835008

Title:
  network_v2 does not set mtu on eni config

Status in cloud-init:
  Confirmed
Status in curtin:
  Invalid
Status in MAAS:
  Invalid

Bug description:
  The MTU is not set on the bond, nor vlans on the bond, nor the bridges
  attached to the vlans on a network_v2 configuration. It's only set on
  the underlying interfaces (eth0, eth1)

  This does not happen if given a network_v1 configuration.

  (I'm deploying ubuntu xenial via MAAS 2.6.0, setting
  force_v1_network_yaml=true is my current workaround.)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1835008/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1834190] Re: Modules steps not taken into account since 19.1

[Ryan Harper]

I've added snapd so they can look into what's going on with
snapd.seeded.service.

** Also affects: snapd
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Status: Incomplete => Invalid

** Summary changed:

- Modules steps not taken into account since 19.1
+ snapd.seeded.service blocks for a long time preventing cloud-init from 
completing

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1834190

Title:
  snapd.seeded.service blocks for a long time preventing cloud-init from
  completing

Status in cloud-init:
  Invalid
Status in snapd:
  New

Bug description:
  We use qcow2 images to install baremetal servers, our datasource is to have a 
little partition as ConfigDrive with meta_data.json and vendor_data.json is 
necessary.
  The vendor_data.json provides informations about default user password 
settings.

  This actually works well for every linux distrib we use unless cloud-
  init is 19.1.

  The modules steps are no longer working:

  cat /run/cloud-init/status.json 
  {
   "v1": {
"datasource": "DataSourceConfigDrive [net,ver=2][source=/dev/sda4]",
"init": {
 "errors": [],
 "finished": 1561465042.7556307,
 "start": 1561465042.1868145
},
"init-local": {
 "errors": [],
 "finished": 1561465035.4210315,
 "start": 1561465034.724825
},
"modules-config": {
 "errors": [],
 "finished": null,
 "start": null
},
"modules-final": {
 "errors": [],
 "finished": null,
 "start": null
},
"modules-init": {
 "errors": [],
 "finished": null,
 "start": null
},
"stage": null
   }
  }

  
  and we can see in the /var/lib/cloud/instance/sem directory that the modules 
doesn't have the sem file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1834190/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1833731] [NEW] cloud-init analyze output not formatted cleanly on Azure

[Ryan Harper]

Public bug reported:

Azure added additional reporting trace points, the result is that
analyze/blame output doesn't look well formatted.

-- Boot Record 01 --
The total time elapsed since completing an event is printed after the "@" 
character.
The time the event takes is printed after the "+" character.

Starting stage: init-local
|`->no cache found @00.00500s +00.0s
Starting stage: init-local/search-Azure
Starting stage: azure-ds/_get_data
|`->found azure asset tag @00.05000s +00.01100s
Starting stage: azure-ds/crawl_metadata
|`->list_possible_azure_ds_devs @00.06200s +00.25000s
|`->load_azure_ds_dir @00.31300s +00.0s
Starting stage: azure-ds/load_azure_ds_dir
Starting stage: azure-ds/read_azure_ovf
|`->load_azure_ovf_pubkeys @00.31700s +00.0s
|`->_extract_preprovisioned_vm_setting @00.31700s +00.00100s
Finished stage: (azure-ds/read_azure_ovf) 00.00400 seconds 

Finished stage: (azure-ds/load_azure_ds_dir) 00.00500 seconds

Starting stage: azure-ds/get_metadata_from_imds
|`->_get_metadata_from_imds @00.49100s +00.01000s
Finished stage: (azure-ds/get_metadata_from_imds) 00.19000 seconds 

|`->_get_random_seed @00.52100s +00.00100s
Finished stage: (azure-ds/crawl_metadata) 00.47100 seconds 

|`->maybe_remove_ubuntu_network_config_scripts @00.53200s +00.00100s
|`->write_files @00.53300s +00.00300s
Finished stage: (azure-ds/_get_data) 00.48600 seconds 

Finished stage: (init-local/search-Azure) 00.48900 seconds

|`->network config from imds @00.59600s +00.0s
Finished stage: (init-local) 00.73800 seconds 

Starting stage: init-network
|`->restored from cache with run check: DataSourceAzure [seed=/var/lib/waagent] 
@03.12900s +00.03300s
|`->network config from imds @03.21700s +00.00100s
Starting stage: init-network/setup-datasource
Starting stage: azure-ds/setup
Starting stage: azure-ds/_negotiate
Starting stage: azure-ds/bounce_network_with_azure_hostname
|`->temporary_hostname @03.23000s +00.0s
Finished stage: (azure-ds/bounce_network_with_azure_hostname) 00.00300 seconds 

Starting stage: azure-ds/get_metadata_from_fabric
Starting stage: azure-ds/register_with_azure_and_fetch_data
|`->generate_certificate @03.23500s +00.10500s
Starting stage: azure-ds/find_endpoint
|`->_networkd_get_value_from_leases @03.34000s +00.00200s
Finished stage: (azure-ds/find_endpoint) 00.00200 seconds 

Starting stage: azure-ds/parse_certificates
|`->_decrypt_certs_from_xml @03.36100s +00.01000s
Starting stage: azure-ds/_get_ssh_key_from_cert
|`->_run_x509_action @03.37200s +00.00600s
Finished stage: (azure-ds/_get_ssh_key_from_cert) 00.04900 seconds 

Starting stage: azure-ds/_get_fingerprint_from_cert
|`->_run_x509_action @03.42100s +00.00600s
Finished stage: (azure-ds/_get_fingerprint_from_cert) 00.00600 seconds 

Finished stage: (azure-ds/parse_certificates) 00.06700 seconds

|`->_report_ready @03.42800s +00.00700s
Finished stage: (azure-ds/register_with_azure_and_fetch_data) 00.20200 seconds 

Finished stage: (azure-ds/get_metadata_from_fabric) 00.20200 seconds

Finished stage: (azure-ds/_negotiate) 00.20800 seconds

Finished stage: (azure-ds/setup) 00.20800 seconds

Finished stage: (init-network/setup-datasource) 00.20800 seconds

|`->reading and applying user-data @03.44500s +00.00300s
|`->reading and applying vendor-data @03.44900s +00.0s
Starting stage: init-network/activate-datasource
Starting stage: azure-ds/activate
Starting stage: azure-ds/address_ephemeral_resize
|`->wait for ephemeral disk @03.48600s +00.00100s
|`->can_dev_be_reformatted @03.48700s +00.00100s
Finished stage: (azure-ds/address_ephemeral_resize) 00.00200 seconds 

Finished stage: (azure-ds/activate) 00.00200 seconds

Finished stage: (init-network/activate-datasource) 00.00400 seconds

|`->config-migrator ran successfully @03.54000s +00.00400s
|`->config-seed_random ran successfully @03.54600s +00.00200s
|`->config-bootcmd ran successfully @03.54900s +00.0s
|`->config-write-files ran successfully @03.55000s +00.0s
|`->config-growpart ran successfully @03.55100s +00.10400s
|`->config-resizefs ran successfully @03.65600s +00.04900s
|`->config-disk_setup ran successfully @03.70600s +03.42300s
|`->config-mounts ran successfully @07.13000s +00.22300s
|`->config-set_hostname ran successfully @07.35400s +00.00200s
|`->config-update_hostname ran successfully @07.35700s +00.00200s
|`->config-update_etc_hosts ran successfully @07.35900s +00.00100s
|`->config-ca-certs ran successfully @07.36000s +00.00100s
|`->config-rsyslog ran successfully @07.36200s +00.00100s
|`->config-users-groups ran successfully @07.36400s +00.32700s
|`->config-ssh ran successfully @07.69200s +00.26200s
Finished stage: (init-network) 04.83900 seconds

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1833731

Title:
  cloud-init analyze output not formatted cleanly on Azure

[Yahoo-eng-team] [Bug 1833589] [NEW] neutron-dynamic-routing broken by introduction of agent_timestamp to _log_heartbeat()

[Ryan Tidwell]

Public bug reported:

neutron-dynamic-routing is broken due to merging
https://review.opendev.org/#/c/665426/, which adds the argument
'agent_timestamp' to _log_heartbeat(). All neutron-dynamic-routing jobs
are failing with errors like these
http://paste.openstack.org/show/753229/.

** Affects: neutron
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1833589

Title:
  neutron-dynamic-routing broken by introduction of agent_timestamp to
  _log_heartbeat()

Status in neutron:
  New

Bug description:
  neutron-dynamic-routing is broken due to merging
  https://review.opendev.org/#/c/665426/, which adds the argument
  'agent_timestamp' to _log_heartbeat(). All neutron-dynamic-routing
  jobs are failing with errors like these
  http://paste.openstack.org/show/753229/.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1833589/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1833460] [NEW] cloud_tests on lxd snap error exporting image

[Ryan Harper]

Public bug reported:

Attempting a cloud_test run on lxd platform had this issue.

(diglett) cloud-init % tox -r -e citest -- run --verbose --platform=lxd 
--os-name centos70 --rpm ~/cloud-init-19.1+44.g12efb6d3-1.el7.noarch.rpm 
--result /tmp/centos7-result.yaml
GLOB sdist-make: /srv/tmp/rharper/cloud-init/setup.py
citest create: /srv/tmp/rharper/cloud-init/.tox/citest
citest installdeps: -r/srv/tmp/rharper/cloud-init/integration-requirements.txt
citest inst: /srv/tmp/rharper/cloud-init/.tox/.tmp/package/1/cloud-init-19.1.zip
citest installed: 
asn1crypto==0.24.0,attrs==19.1.0,bcrypt==3.1.6,boto3==1.5.9,botocore==1.8.50,certifi==2019.6.16,cffi==1.12.3,chardet==3.0.4,cloud-init==19.1,configobj==5.0.6,cryptography==2.7,docutils==0.14,idna==2.8,Jinja2==2.10.1,jmespath==0.9.4,jsonpatch==1.23,jsonpointer==2.0,jsonschema==3.0.1,linecache2==1.0.0,MarkupSafe==1.1.1,oauthlib==3.0.1,paramiko==2.4.1,pbr==5.3.0,pkg-resources==0.0.0,pyasn1==0.4.5,pycparser==2.19,pylxd==2.2.7,PyNaCl==1.3.0,pyrsistent==0.15.2,python-dateutil==2.8.0,python-simplestreams==0.1.0,PyYAML==5.1.1,requests==2.22.0,requests-toolbelt==0.9.1,requests-unixsocket==0.1.5,s3transfer==0.1.13,six==1.12.0,traceback2==1.4.0,unittest2==1.1.0,urllib3==1.25.3,ws4py==0.5.1
citest run-test-pre: PYTHONHASHSEED='2218838469'
citest runtests: commands[0] | 
/srv/tmp/rharper/cloud-init/.tox/citest/bin/python -m tests.cloud_tests run 
--verbose --platform=lxd --os-name centos70 --rpm 
/home/rharper/cloud-init-19.1+44.g12efb6d3-1.el7.noarch.rpm --result 
/tmp/centos7-result.yaml
2019-06-19 19:18:44,550 - tests.cloud_tests - DEBUG - running with args: 
Namespace(data_dir=None, deb=None, feature_override={}, os_name=['centos70'], 
platform=['lxd'], ppa=None, preserve_data=False, preserve_instance=False, 
quiet=False, repo=None, result='/tmp/centos7-result.yaml', 
rpm='/home/rharper/cloud-init-19.1+44.g12efb6d3-1.el7.noarch.rpm', script=None, 
subcmd='run', 
test_config=['/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/bugs/lp1511485.yaml',
 '/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/bugs/lp1611074.yaml', 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/bugs/lp1628337.yaml', 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/add_apt_repositories.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/alter_completion_message.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/configure_instance_trusted_ca_certificates.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/configure_instances_ssh_keys.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/including_user_groups.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/install_arbitrary_packages.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/install_run_chef_recipes.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/run_apt_upgrade.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/run_commands.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/run_commands_first_boot.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/setup_run_puppet.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/examples/writing_out_arbitrary_files.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/main/command_output_simple.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_conf.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_disable_suites.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_primary.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_proxy.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_security.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_sources_key.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_sources_keyserver.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_sources_list.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_configure_sources_ppa.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_pipelining_disable.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/apt_pipelining_os.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/bootcmd.yaml', 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/byobu.yaml', 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/ca_certs.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/debug_disable.yaml',
 
'/srv/tmp/rharper/cloud-init/tests/cloud_tests/testcases/modules/debug_enable.yaml',
 

[Yahoo-eng-team] [Bug 1833264] [NEW] cloud-init-generator hardcodes path to ds-identify

[Ryan Harper]

Public bug reported:

On systemd systems, the cloud-init-generator uses the ds-identify
program and hardcodes the path to the program.  The value is correct for
Ubuntu/Debian and other systems, however on centos-based systems, ds-
identify is installed down libexec path.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1833264

Title:
  cloud-init-generator hardcodes path to ds-identify

Status in cloud-init:
  New

Bug description:
  On systemd systems, the cloud-init-generator uses the ds-identify
  program and hardcodes the path to the program.  The value is correct
  for Ubuntu/Debian and other systems, however on centos-based systems,
  ds-identify is installed down libexec path.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1833264/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1832636] [NEW] Error creating IPv6 subnet on routed network segment

[Ryan Tidwell]

Public bug reported:

I have a routed provider network with 2 segments. Prior to creating an
IPv6 subnet on a segment, I have created IPv4 subnets and attached a
router to the external network without issue. When I attempt to create
an IPv6 subnet on a segment, the following occurs:

> openstack subnet create --subnet-pool ULA --prefix-length 64
--network-segment 89254a60-1d84-4162-9d89-d678e8103594 --network
c96b20c7-0702-4efa-a980-4e918979300a --ip-version 6 --no-dhcp RACK_1_ULA

BadRequestException: 400: Client Error for url:
http://controller:9696/v2.0/subnets, Invalid input for operation: Failed
to create port on network c96b20c7-0702-4efa-a980-4e918979300a, because
fixed_ips included invalid subnet 1237b25c-a357-40b2-9b5b-f311cd0bb075.

Interestingly enough, the subnet was created despite this error. The
subnet ID referenced in the error message is the UUID of the newly
created subnet:

> openstack subnet show 1237b25c-a357-40b2-9b5b-f311cd0bb075
+---+---+
| Field | Value 

|
+---+---+
| allocation_pools  | fd00::2-fd00::::: 

|
| cidr  | fd00::/64 

|
| created_at| 2019-06-12T19:04:53Z  

|
| description   |   

|
| dns_nameservers   |   

|
| enable_dhcp   | False 

|
| gateway_ip| fd00::1   

|
| host_routes   |   

|
| id| 1237b25c-a357-40b2-9b5b-f311cd0bb075  

|
| ip_version| 6 

|
| ipv6_address_mode | None  

|
| ipv6_ra_mode  | None  

|
| location  | Munch({'cloud': '', 'region_name': '', 'zone': None, 
'project': Munch({'id': 'ea4d84a4ec2449a691eba8e9a664996a', 'name': 'admin', 
'domain_id': 'default', 'domain_name': None})}) |
| name  | RACK_1_ULA

|
| network_id| c96b20c7-0702-4efa-a980-4e918979300a  

|
| prefix_length | None  

|
| project_id| ea4d84a4ec2449a691eba8e9a664996a  
   

[Yahoo-eng-team] [Bug 1800848] Re: does not recognize cloned KVM VM as new instance

[Ryan Harper]

OK, good to know.

w.r.t SLES 15 and cloud-init; if you've attached a NoCloud datasource
correctly (your logs indicate that you are) then I would say yes this is
a bug.

Now, upstream since 18.4 there have been quite a few fixes applied which
may not have made it into SLES15, not 100% sure about that but:

% git log 18.4..HEAD --oneline | grep -i suse
bb0b6f1 net/sysconfig: write out SUSE-compatible IPv6 config
dfe50e3 tox: Update testenv for openSUSE Leap to 15.0
3f12012 sysconfig: On SUSE, use STARTMODE instead of ONBOOT
744c423 Add cloud-id binary to packages for SUSE
e0084a5 systemd: On SUSE ensure cloud-init.service runs before wicked
4ea64f1 update detection of openSUSE variants


** Changed in: cloud-init
   Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1800848

Title:
  does not recognize cloned KVM VM as new instance

Status in cloud-init:
  Invalid
Status in cloud-init package in Suse:
  Fix Released

Bug description:
  On a SLES 15 KVM VM on Proxmox VE  cloud-init 18.2-3.13 from
  module/repo Public-Cloud-Module_15-0  cloud-init fails to recognize a
  newly cloned VM from a template where cloud-init has initially been
  run for test purposes as a new instance. VM is using NoCloud resource.

  This leads to network configuration not applied and thus duplicate IP
  addresses when VM is started:

  % tail -f /var/log/cloud-init.log | grep -v util.py
  2018-10-31 13:55:27,370 - __init__.py[INFO]: 
/var/lib/cloud/data/previous-hostname differs from /etc/hostname, assuming user 
maintained hostname.
  2018-10-31 13:57:37,431 - main.py[DEBUG]: No kernel command line url found.
  2018-10-31 13:57:37,431 - main.py[DEBUG]: Closing stdin.
  2018-10-31 13:57:37,454 - main.py[DEBUG]: Checking to see if files that we 
need already exist from a previous run that would allow us to stop early.
  2018-10-31 13:57:37,455 - main.py[DEBUG]: Execution continuing, no previous 
run detected that would allow us to stop early.
  2018-10-31 13:57:37,455 - handlers.py[DEBUG]: start: 
init-network/check-cache: attempting to read from cache [trust]
  2018-10-31 13:57:37,463 - stages.py[DEBUG]: restored from cache with run 
check: DataSourceNoCloudNet [seed=/dev/sr0] [dsmode=net]
  2018-10-31 13:57:37,464 - handlers.py[DEBUG]: finish: 
init-network/check-cache: SUCCESS: restored from cache with run check: 
DataSourceNoCloudNet [seed=/dev/sr0][dsmode=net]
  2018-10-31 13:57:37,488 - stages.py[DEBUG]: previous iid found to be 
414842fe12da6f1078eca77443e6ab84592299ba
  2018-10-31 13:57:37,493 - main.py[DEBUG]: [net] init will now be targeting 
instance id: 414842fe12da6f1078eca77443e6ab84592299ba. new=False
  2018-10-31 13:57:37,514 - stages.py[DEBUG]: applying net config names for 
{'version': 1, 'config': [{'type': 'physical', 'name': 'eth0', 'mac_address': 
'76:61:cf:d5:65:b2', 'subnets': [{'type': 'static', 'address': '10.0.88.32', 
'netmask': '255.255.0.0', 'gateway': '10.0.0.4'}, {'type': 'static', 'address': 
'auto'}]}, {'type': 'nameserver', 'address': ['10.0.0.4'], 'search': 
['qs.de']}]}
  2018-10-31 13:57:37,515 - stages.py[DEBUG]: Using distro class 
  2018-10-31 13:57:37,529 - __init__.py[DEBUG]: no work necessary for renaming 
of [['76:61:cf:d5:65:b2', 'eth0', 'virtio_net', '0x0001']]
  2018-10-31 13:57:37,530 - stages.py[DEBUG]: not a new instance. network 
config is not applied.

  Network configuration obviously was changed however.

  Commands used for cloning (using a self-made shell script)

  qm shutdown 1032 ; qm clone 1032 2200 --name slesmaster ; qm template 2200
  qm clone 2200 2201 --name sles1 ; qm set 2201 --ipconfig0 
ip=10.0.88.201/8,gw=10.0.0.4 ; qm start 2201
  qm clone 2200 2202 --name sles2 ; qm set 2202 --ipconfig0 
ip=10.0.88.202/8,gw=10.0.0.4 ; qm start 2202

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1800848/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1830240] [NEW] [RFE] Allow subnets from different subnet pools on the same network when using address scopes

[Ryan Tidwell]

Public bug reported:

Currently, neutron enforces that on any given network, all subnets of
the same address family (IPv4/6) must be all allocated from the same
subnet pool. This restriction was put in place before the concept of
address scopes existed, and was meant to help enforce the uniqueness of
subnet CIDR's associated with a network.

When address scopes are used, we really only need to enforce that all
subnets on a given network belong to the same address scope for each
address family. This unlocks use cases for operators that allow them to
build different subnet pools to be used with specific subnet service
types and allocate them from different pools.

For example:

Suppose an operator has different blocks of addresses for floating IP's
and FIP agent gateway ports. This change would allow the operator to
manage the FIP range from one subnet pool and FIP gateway addresses from
another pool, which is something they can't do today because neutron
only allows subnets to be placed on the network if they are allocated
from the same pool.

Proposed Changes:

- When address scopes are not used neutron behaves as it currently does,
enforcing all subnets of the same address family on the same network be
allocated from the same subnet pool

- When address scopes are used the "same subnet pool" constraint is
relaxed, and the check made is that all subnets belong to the same
address scope

- Logic will be added to subnet pool updates to block movement between
address scopes when moving the pool would cause one or more subnets
allocated from the pool to violate the "same address scope" constraint.

- Logic will be added to the subnet "off-board" case to block the
operation when "off-boarding" the requested subnet(s) causes a violation
of the "same address scope" constraint.

As an alternative implementation, introducing address_scope_v4 and
address_scope_v6 fields to the network resource would also give us a way
of associating networks with the address scope. Since these constraints
need to be enforced on the network, perhaps networks need to be aware of
address scopes.

** Affects: neutron
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1830240

Title:
  [RFE] Allow subnets from different subnet pools on the same network
  when using address scopes

Status in neutron:
  New

Bug description:
  Currently, neutron enforces that on any given network, all subnets of
  the same address family (IPv4/6) must be all allocated from the same
  subnet pool. This restriction was put in place before the concept of
  address scopes existed, and was meant to help enforce the uniqueness
  of subnet CIDR's associated with a network.

  When address scopes are used, we really only need to enforce that all
  subnets on a given network belong to the same address scope for each
  address family. This unlocks use cases for operators that allow them
  to build different subnet pools to be used with specific subnet
  service types and allocate them from different pools.

  For example:

  Suppose an operator has different blocks of addresses for floating
  IP's and FIP agent gateway ports. This change would allow the operator
  to manage the FIP range from one subnet pool and FIP gateway addresses
  from another pool, which is something they can't do today because
  neutron only allows subnets to be placed on the network if they are
  allocated from the same pool.

  Proposed Changes:

  - When address scopes are not used neutron behaves as it currently
  does, enforcing all subnets of the same address family on the same
  network be allocated from the same subnet pool

  - When address scopes are used the "same subnet pool" constraint is
  relaxed, and the check made is that all subnets belong to the same
  address scope

  - Logic will be added to subnet pool updates to block movement between
  address scopes when moving the pool would cause one or more subnets
  allocated from the pool to violate the "same address scope"
  constraint.

  - Logic will be added to the subnet "off-board" case to block the
  operation when "off-boarding" the requested subnet(s) causes a
  violation of the "same address scope" constraint.

  As an alternative implementation, introducing address_scope_v4 and
  address_scope_v6 fields to the network resource would also give us a
  way of associating networks with the address scope. Since these
  constraints need to be enforced on the network, perhaps networks need
  to be aware of address scopes.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1830240/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1828607] [NEW] [RFE] DVR Enhancements

[Ryan Tidwell]

Public bug reported:

This involves the following items:

- Support for distributed ingress and egress for IPv6
- Support for running without network nodes. This implies
  * Support for distributed DHCP
  * Support for distributed SNAT
- Ensuring an OpenFlow-based DVR implementation is written in a way that can be 
offloaded to a smart-NIC as hardware support comes online.

Due to the broad nature of these changes, I will propose a spec in
neutron-specs to elaborate on these items.

** Affects: neutron
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1828607

Title:
  [RFE] DVR Enhancements

Status in neutron:
  New

Bug description:
  This involves the following items:

  - Support for distributed ingress and egress for IPv6
  - Support for running without network nodes. This implies
* Support for distributed DHCP
* Support for distributed SNAT
  - Ensuring an OpenFlow-based DVR implementation is written in a way that can 
be offloaded to a smart-NIC as hardware support comes online.

  Due to the broad nature of these changes, I will propose a spec in
  neutron-specs to elaborate on these items.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1828607/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1775250] Re: Implement DVR-aware announcement of fixed IP's in neutron-dynamic-routing

[Ryan Tidwell]

** Changed in: neutron
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1775250

Title:
  Implement DVR-aware announcement of fixed IP's in neutron-dynamic-
  routing

Status in neutron:
  Fix Released

Bug description:
  The current implementation of neutron-dynamic-routing is compatible
  with DVR, but is not optimized for DVR. It currently announces next-
  hops for all tenant subnets through the central router on the network
  node. Announcing next-hops via the FIP gateway on the compute node was
  never implemented due to the pre-requisite of having DVR fast-exit in
  place for packets to be routed through the FIP namespace properly.
  With DVR fast-exit now in place, it's time to consider adding DVR-
  aware /32 and/or /128 announcements for fixed IP's using the FIP
  gateway as the next-hop.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1775250/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1822615] [NEW] cloud-tests tree_run/bddeb fail to build deb

[Ryan Harper]

Public bug reported:

While looking to verify an apt_pipelining test-case fix I ran this:

% tox -e citest -- tree_run --verbose --os-name bionic --test
modules/apt_pipelining_disable --test modules/apt_pipelining_os --result
/tmp/result.yaml

During which, it would fail like this:

2019-03-31 16:13:11,743 - tests.cloud_tests - ERROR - stage part: build deb on 
system encountered error: Unexpected error while running command.
Command: /root/packages/bddeb -d
Exit code: 1
Reason: -



 ==   
ERROR: 
cloudinit.cmd.tests.test_query.TestQuery.test_handle_args_dumps_all_instance_data
--   
Traceback (most recent call last):   
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/tests/test_query.py",
 line 153, in test_handle_args_dumps_all_instance_data
self.assertEqual(0, query.handle_args('anyname', args))  
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/query.py",
 line 124, in handle_args
instance_data['userdata'] = util.load_file(user_data_fn) 
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/util.py",
 line 1359, in load_file
with open(fname, 'rb') as ifh:   
FileNotFoundError: [Errno 2] No such file or directory: 'ud' 
 
==   
ERROR: 
cloudinit.cmd.tests.test_query.TestQuery.test_handle_args_list_keys_errors_when_varname_is_not_a_dict
--   
Traceback (most recent call last):   
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/tests/test_query.py",
 line 255, in test_handle_args_list_keys_errors_when_varname_is_not_a_dict
self.assertEqual(1, query.handle_args('anyname', args))  
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/query.py",
 line 124, in handle_args
instance_data['userdata'] = util.load_file(user_data_fn) 
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/util.py",
 line 1359, in load_file
with open(fname, 'rb') as ifh:   
FileNotFoundError: [Errno 2] No such file or directory: 'ud' 
 
==   
ERROR: 
cloudinit.cmd.tests.test_query.TestQuery.test_handle_args_list_keys_sorts_nested_keys_when_varname
--   
Traceback (most recent call last):   
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/tests/test_query.py",
 line 239, in test_handle_args_list_keys_sorts_nested_keys_when_varname
self.assertEqual(0, query.handle_args('anyname', args))  
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/query.py",
 line 124, in handle_args
instance_data['userdata'] = util.load_file(user_data_fn) 
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/util.py",
 line 1359, in load_file
with open(fname, 'rb') as ifh:   
FileNotFoundError: [Errno 2] No such file or directory: 'ud' 
 
==   
ERROR: 
cloudinit.cmd.tests.test_query.TestQuery.test_handle_args_list_keys_sorts_top_level_keys_when_no_varname
--   
Traceback (most recent call last):   
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/tests/test_query.py",
 line 224, in test_handle_args_list_keys_sorts_top_level_keys_when_no_varname
self.assertEqual(0, query.handle_args('anyname', args))  
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/cmd/query.py",
 line 124, in handle_args
instance_data['userdata'] = util.load_file(user_data_fn) 
  File 
"/run/cloud-init/tmp/tmp2v5cncbz/cloud-init-18.5-57-g6c258412/cloudinit/util.py",
 line 1359, in load_file
with open(fname, 'rb') as ifh:   
FileNotFoundError: [Errno 2] No such file or directory: 'ud' 

[Yahoo-eng-team] [Bug 1820297] [NEW] os-ken error when BGP peer goes down

[Ryan Tidwell]

Public bug reported:

I'm observing backtraces in neutron-dynamic-routing scenario job logs.
The neutron dynamic routing agent is logging the following, seemingly
after a BGP peer goes idle

Feb 26 13:07:55.571869 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: INFO 
neutron_dynamic_routing.services.bgp.agent.driver.os_ken.driver [-] BGP Peer 
192.168.10.129 for remote_as=64522 went DOWN.
Feb 26 13:07:55.572107 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: DEBUG bgpspeaker.peer [-] Peer 192.168.10.129 BGP 
FSM went from Established to Idle {{(pid=21925) bgp_state 
/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/peer.py:237}}
Feb 26 13:07:55.572335 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: ERROR bgpspeaker.base [-] Traceback (most recent 
call last):
Feb 26 13:07:55.572545 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]:   File 
"/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/base.py", 
line 256, in start
Feb 26 13:07:55.572762 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: self._run(*args, **kwargs)
Feb 26 13:07:55.572969 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]:   File 
"/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/speaker.py",
 line 275, in _run
Feb 26 13:07:55.573167 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: self._recv_loop()
Feb 26 13:07:55.573375 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]:   File 
"/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/speaker.py",
 line 571, in _recv_loop
Feb 26 13:07:55.573581 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: self.connection_lost(conn_lost_reason)
Feb 26 13:07:55.573786 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]:   File 
"/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/speaker.py",
 line 596, in connection_lost
Feb 26 13:07:55.573992 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: self._peer.connection_lost(reason)
Feb 26 13:07:55.574191 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]:   File 
"/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/peer.py", 
line 2323, in connection_lost
Feb 26 13:07:55.574391 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: self._protocol.stop()
Feb 26 13:07:55.574590 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]:   File 
"/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/speaker.py",
 line 405, in stop
Feb 26 13:07:55.574797 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: Activity.stop(self)
Feb 26 13:07:55.574996 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]:   File 
"/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/base.py", 
line 314, in stop
Feb 26 13:07:55.575208 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: raise ActivityException(desc='Cannot call stop 
when activity is '
Feb 26 13:07:55.575406 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: ActivityException: 100.1 - Cannot call stop when 
activity is not started or has been stopped already.
Feb 26 13:07:55.575639 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: : ActivityException: 100.1 - Cannot call stop when 
activity is not started or has been stopped already.

I'm not sure of the impact yet. At a minimum, it would be nice to see a
nice log message instead of a backtrace (assuming this is doesn't
indicate a real issue).

** Affects: neutron
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1820297

Title:
  os-ken error when BGP peer goes down

Status in neutron:
  New

Bug description:
  I'm observing backtraces in neutron-dynamic-routing scenario job logs.
  The neutron dynamic routing agent is logging the following, seemingly
  after a BGP peer goes idle

  Feb 26 13:07:55.571869 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: INFO 
neutron_dynamic_routing.services.bgp.agent.driver.os_ken.driver [-] BGP Peer 
192.168.10.129 for remote_as=64522 went DOWN.
  Feb 26 13:07:55.572107 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: DEBUG bgpspeaker.peer [-] Peer 192.168.10.129 BGP 
FSM went from Established to Idle {{(pid=21925) bgp_state 
/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/peer.py:237}}
  Feb 26 13:07:55.572335 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]: ERROR bgpspeaker.base [-] Traceback (most recent 
call last):
  Feb 26 13:07:55.572545 ubuntu-xenial-inap-mtl01-0003035225 
neutron-bgp-dragent[21925]:   File 
"/usr/local/lib/python2.7/dist-packages/os_ken/services/protocols/bgp/base.py", 

[Yahoo-eng-team] [Bug 1819994] Re: An error occurs when MAAS Deploy 18.04 on ThinkSystem SR590

[Ryan Harper]

2019-03-14 17:32:34,606 - __init__.py[DEBUG]: Selected renderer
'sysconfig' from priority list: None

This is a cloud-init bug.  The sysconfig renderer has NetworkManager
support, this triggered cloud-init to render sysconfig instead of
netplan.


** Also affects: cloud-init
   Importance: Undecided
   Status: New

** Changed in: cloud-init
   Importance: Undecided => High

** Changed in: cloud-init
   Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1819994

Title:
  An error occurs when MAAS Deploy 18.04 on ThinkSystem SR590

Status in cloud-init:
  Confirmed
Status in MAAS:
  Incomplete
Status in Provider for Plainbox - Canonical Certification Server:
  Confirmed

Bug description:
  Configuration:
  UEFI/BIOS:TEE136S
  IMM/BMC:  CDI333V
  CPU:  Intel(R) Xeon(R) Platinum 8253 CPU @ 2.20GHz
  Memory:   16G DIMM * 12
  Raid card:ThinkSystem RAID 530-8i 
  NIC Card: Intel X722 LOM

  Reproduce Steps:
  1.Config "network" as first boot
  2.Power on machine
  3.Visit TC through web browser and Commission machine
  4.When commission complete, deploy ubuntu 18.04 LTS on SUT
  5.The Error appeared during OS deploy.

  Deploy errors like the following(you can view the attachment for
  details):

  cloud-init[] Date_and_time - handlers.py[WARNING]: failed posting
  event: start: modules-final/config-: running config-

  cloud-init[] Date_and_time - handlers.py[WARNING]: failed posting
  event: fainish: modules-final: SUCCESS: running modules for final

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1819994/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1820293] [NEW] Error disassociating bgpspeaker from BGP dr-agent

[Ryan Tidwell]

Public bug reported:

I've been noticing the scenario jobs in neutron-dynamic-routing (non-
voting) failing intermittently. Upon further investigation, it appears
these failures are happening when attempting to disassociate a
bgpspeaker from an agent.

2019-02-26 13:08:57.781145 | primary | ==
2019-02-26 13:08:57.781222 | primary | Failed 1 tests - output below:
2019-02-26 13:08:57.781292 | primary | ==
2019-02-26 13:08:57.781317 | primary |
2019-02-26 13:08:57.781574 | primary | 
neutron_dynamic_routing.tests.tempest.scenario.basic.test_basic.BgpSpeakerBasicTest.test_remove_add_speaker_agent[id-aa6c565c-ded3-413b-8dc9-3928b3b0e38f]
2019-02-26 13:08:57.781840 | primary | 
--
2019-02-26 13:08:57.781865 | primary |
2019-02-26 13:08:57.781918 | primary | Captured traceback:
2019-02-26 13:08:57.781971 | primary | ~~~
2019-02-26 13:08:57.782053 | primary | Traceback (most recent call last):
2019-02-26 13:08:57.782322 | primary |   File 
"/opt/stack/new/neutron-dynamic-routing/neutron_dynamic_routing/tests/tempest/scenario/basic/test_basic.py",
 line 103, in test_remove_add_speaker_agent
2019-02-26 13:08:57.782456 | primary | 
self.bgp_client.add_bgp_speaker_to_dragent(agent_id, speaker_id)
2019-02-26 13:08:57.782695 | primary |   File 
"/opt/stack/new/neutron-dynamic-routing/neutron_dynamic_routing/tests/tempest/bgp_client.py",
 line 132, in add_bgp_speaker_to_dragent
2019-02-26 13:08:57.782793 | primary | resp, body = self.post(uri, 
update_body)
2019-02-26 13:08:57.782914 | primary |   File 
"tempest/lib/common/rest_client.py", line 280, in post
2019-02-26 13:08:57.783056 | primary | return self.request('POST', url, 
extra_headers, headers, body, chunked)
2019-02-26 13:08:57.783183 | primary |   File 
"tempest/lib/common/rest_client.py", line 676, in request
2019-02-26 13:08:57.783298 | primary | self._error_checker(resp, 
resp_body)
2019-02-26 13:08:57.783440 | primary |   File 
"tempest/lib/common/rest_client.py", line 797, in _error_checker
2019-02-26 13:08:57.783572 | primary | raise 
exceptions.Conflict(resp_body, resp=resp)
2019-02-26 13:08:57.783714 | primary | tempest.lib.exceptions.Conflict: 
Conflict with state of target resource
2019-02-26 13:08:57.783993 | primary | Details: {u'message': u'BgpDrAgent 
f67420d5-6b3b-4635-8a26-32347b188638 is already associated to a BGP speaker.', 
u'detail': u'', u'type': u'DrAgentAssociationError'}

This should be cleaned up so that these jobs can eventually become
voting jobs, and so that a potential user-facing issue is addressed.

** Affects: neutron
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1820293

Title:
  Error disassociating bgpspeaker from BGP dr-agent

Status in neutron:
  New

Bug description:
  I've been noticing the scenario jobs in neutron-dynamic-routing (non-
  voting) failing intermittently. Upon further investigation, it appears
  these failures are happening when attempting to disassociate a
  bgpspeaker from an agent.

  2019-02-26 13:08:57.781145 | primary | ==
  2019-02-26 13:08:57.781222 | primary | Failed 1 tests - output below:
  2019-02-26 13:08:57.781292 | primary | ==
  2019-02-26 13:08:57.781317 | primary |
  2019-02-26 13:08:57.781574 | primary | 
neutron_dynamic_routing.tests.tempest.scenario.basic.test_basic.BgpSpeakerBasicTest.test_remove_add_speaker_agent[id-aa6c565c-ded3-413b-8dc9-3928b3b0e38f]
  2019-02-26 13:08:57.781840 | primary | 
--
  2019-02-26 13:08:57.781865 | primary |
  2019-02-26 13:08:57.781918 | primary | Captured traceback:
  2019-02-26 13:08:57.781971 | primary | ~~~
  2019-02-26 13:08:57.782053 | primary | Traceback (most recent call last):
  2019-02-26 13:08:57.782322 | primary |   File 
"/opt/stack/new/neutron-dynamic-routing/neutron_dynamic_routing/tests/tempest/scenario/basic/test_basic.py",
 line 103, in test_remove_add_speaker_agent
  2019-02-26 13:08:57.782456 | primary | 
self.bgp_client.add_bgp_speaker_to_dragent(agent_id, speaker_id)
  2019-02-26 13:08:57.782695 | primary |   File 
"/opt/stack/new/neutron-dynamic-routing/neutron_dynamic_routing/tests/tempest/bgp_client.py",
 line 132, in add_bgp_speaker_to_dragent
  2019-02-26 13:08:57.782793 | primary | resp, body = self.post(uri, 
update_body)
  2019-02-26 13:08:57.782914 | primary |   File 
"tempest/lib/common/rest_client.py", line 280, in post
  

[Yahoo-eng-team] [Bug 1819913] [NEW] cloud-init on xenial may generate network config on every boot

[Ryan Harper]

Public bug reported:

On at least EC2 with cloud-init xenial release, the Ec2 datasource allows the 
EventType.BOOT
event to update metadata and will regenerate network configuration on each 
boot. Cloud-init releases newer than Xenial are not affected since cloud-init 
will detect
which datasource to use and does not perform searching.

** Affects: cloud-init
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1819913

Title:
  cloud-init on xenial may generate network config on every boot

Status in cloud-init:
  New

Bug description:
  On at least EC2 with cloud-init xenial release, the Ec2 datasource allows the 
EventType.BOOT
  event to update metadata and will regenerate network configuration on each 
boot. Cloud-init releases newer than Xenial are not affected since cloud-init 
will detect
  which datasource to use and does not perform searching.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1819913/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1819569] [NEW] Swap volumes of vm failed: pivot of disk 'vdb' requires an active copy job

[Ryan Liang]

Public bug reported:

Description
===

I had one volume, vol-1 attached to two VMs, vm-1 and vm-2, then tried
to swap vm-1's volume to another volume vol-2, but failed.

The Cinder backend attached the volume to vm successfully.

Nova reported below error:

Mar 08 01:34:42.984286 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver [None req-1033d1c1-bd3b-4990-a8da-e503d25a041b 
tempest-TestMultiAttachVolumeSwap-890516006 
tempest-TestMultiAttachVolumeSwap-890516006] Failure rebasing volume /dev/sdj 
on vdb.: libvirt.libvirtError: Requested operation is not valid: pivot of disk 
'vdb' requires an active copy job
Mar 08 01:34:42.984521 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver Traceback (most recent call last):
Mar 08 01:34:42.984846 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver   File 
"/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1557, in _swap_volume
Mar 08 01:34:42.985032 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver dev.abort_job(pivot=True)
Mar 08 01:34:42.985214 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver   File 
"/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 754, in abort_job
Mar 08 01:34:42.986537 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver self._guest._domain.blockJobAbort(self._disk, 
flags=flags)
Mar 08 01:34:42.986914 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver   File 
"/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 190, in doit
Mar 08 01:34:42.987246 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver result = proxy_call(self._autowrap, f, *args, 
**kwargs)
Mar 08 01:34:42.987608 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver   File 
"/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 148, in 
proxy_call
Mar 08 01:34:42.987958 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver rv = execute(f, *args, **kwargs)
Mar 08 01:34:42.988276 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver   File 
"/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 129, in execute
Mar 08 01:34:42.988434 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver six.reraise(c, e, tb)
Mar 08 01:34:42.988591 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver   File 
"/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise
Mar 08 01:34:42.988746 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver raise value
Mar 08 01:34:42.988895 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver   File 
"/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 83, in tworker
Mar 08 01:34:42.989045 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver rv = meth(*args, **kwargs)
Mar 08 01:34:42.989194 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver   File 
"/usr/local/lib/python3.5/dist-packages/libvirt.py", line 772, in blockJobAbort
Mar 08 01:34:42.989366 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver if ret == -1: raise libvirtError 
('virDomainBlockJobAbort() failed', dom=self)
Mar 08 01:34:42.989516 UnityiSCSISlave nova-compute[19437]: ERROR 
nova.virt.libvirt.driver libvirt.libvirtError: Requested operation is not 
valid: pivot of disk 'vdb' requires an active copy job


Steps to reproduce
==

1. In OpenStack, attach a volume, vol-1 to two vms, vm-1 and vm-2. Volume vol-1 
is provisioned on external storage system.
2. Swap vol-1 of vm-1 to another volume, vol-2.


Actual results
==

You can see above failure: Failure rebasing volume /dev/sdj (vol-2) on
vdb.


Expected results

vm-1's volume updated to vol-2.


Environment
===

OpenStack: master (Stein) devstack
libvirt-python: 5.1.0
libvirt-bin: 4.0.0-1ubuntu8.5~cloud0


Logs


1. Full n-cpu log locates: 
http://publiclogs.emc.com/51/633451/5/check/EMC_Unity_iSCSI/a2d0e05/EMC_Unity_iSCSI/1218/logs/screen-n-cpu.txt.gz
Search via request id: req-1033d1c1-bd3b-4990-a8da-e503d25a041b

2. libvirt log is attached.

If you need more info, please let me know.

** Affects: nova
 Importance: Undecided
 Status: New


** Tags: libvirt

** Attachment added: "libvirtd log"
   
https://bugs.launchpad.net/bugs/1819569/+attachment/5245583/+files/libvirtd.log

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1819569

Title:
  Swap volumes of vm failed: pivot of disk 'vdb' requires an active copy
  job

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===

  I had one volume, vol-1 attached to two VMs, vm-1 and vm-2, then tried
  to swap vm-1's volume to another volume vol-2, but failed.

  The Cinder 

[Yahoo-eng-team] [Bug 1815844] Re: iscsi multipath dm-N device only used on first volume attachment

[Ryan Beisner]

** Changed in: charm-nova-compute
   Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1815844

Title:
  iscsi multipath dm-N device only used on first volume attachment

Status in OpenStack nova-compute charm:
  Invalid
Status in OpenStack Compute (nova):
  New

Bug description:
  With nova-compute from cloud:xenial-queens and use-multipath=true
  iscsi multipath is configured and the dm-N devices used on the first
  attachment but subsequent attachments only use a single path.

  The back-end storage is a Purestorage array.
  The multipath.conf is attached
  The issue is easily reproduced as shown below:

  jog@pnjostkinfr01:~⟫ openstack volume create pure2 --size 10 --type pure
  +-+--+
  | Field   | Value|
  +-+--+
  | attachments | []   |
  | availability_zone   | nova |
  | bootable| false|
  | consistencygroup_id | None |
  | created_at  | 2019-02-13T23:07:40.00   |
  | description | None |
  | encrypted   | False|
  | id  | e286161b-e8e8-47b0-abe3-4df411993265 |
  | migration_status| None |
  | multiattach | False|
  | name| pure2|
  | properties  |  |
  | replication_status  | None |
  | size| 10   |
  | snapshot_id | None |
  | source_volid| None |
  | status  | creating |
  | type| pure |
  | updated_at  | None |
  | user_id | c1fa4ae9a0b446f2ba64eebf92705d53 |
  +-+--+

  jog@pnjostkinfr01:~⟫ openstack volume show pure2
  ++--+
  | Field  | Value|
  ++--+
  | attachments| []   |
  | availability_zone  | nova |
  | bootable   | false|
  | consistencygroup_id| None |
  | created_at | 2019-02-13T23:07:40.00   |
  | description| None |
  | encrypted  | False|
  | id | e286161b-e8e8-47b0-abe3-4df411993265 |
  | migration_status   | None |
  | multiattach| False|
  | name   | pure2|
  | os-vol-host-attr:host  | cinder@cinder-pure#cinder-pure   |
  | os-vol-mig-status-attr:migstat | None |
  | os-vol-mig-status-attr:name_id | None |
  | os-vol-tenant-attr:tenant_id   | 9be499fd1eee48dfb4dc6faf3cc0a1d7 |
  | properties |  |
  | replication_status | None |
  | size   | 10   |
  | snapshot_id| None |
  | source_volid   | None |
  | status | available|
  | type   | pure |
  | updated_at | 2019-02-13T23:07:41.00   |
  | user_id| c1fa4ae9a0b446f2ba64eebf92705d53 |
  ++--+

  Add the volume to an instance:
  jog@pnjostkinfr01:~⟫ openstack server add volume T1 pure2
  jog@pnjostkinfr01:~⟫ openstack server show T1 
 
  
+-+--+
  | Field   | Value

[Yahoo-eng-team] [Bug 1815051] Re: Bionic netplan render invalid yaml duplicate anchor declaration for nameserver entries

[Ryan Harper]

** Also affects: cloud-init (Ubuntu Cosmic)
   Importance: Undecided
   Status: New

** Also affects: cloud-init (Ubuntu Xenial)
   Importance: Undecided
   Status: New

** Also affects: cloud-init (Ubuntu Bionic)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1815051

Title:
  Bionic netplan render invalid yaml duplicate anchor declaration for
  nameserver entries

Status in cloud-init:
  In Progress
Status in cloud-init package in Ubuntu:
  In Progress
Status in cloud-init source package in Xenial:
  New
Status in cloud-init source package in Bionic:
  New
Status in cloud-init source package in Cosmic:
  New

Bug description:
  The netplan configuration redeclares the nameservers anchor for every
  single section (vlans, bonds), and use the same id for similar entries
  (id001).

  In this specific case the network configuration in maas have a bond0
  with two vlans, bond0.3502 and bond0.3503, and an untagged bond1
  without vlans. The rendered 50-cloud-init.yaml looks like this:

  network:
  version: 2
  ethernets:
  ...
  bonds:
  ...
  bond1:
  ...
  nameservers:  <- anchor declaration here
  addresses:
  - 255.255.255.1
  - 255.255.255.2
  - 255.255.255.3
  - 255.255.255.5
  search:
  - customer.domain
  - maas
  ...
  bondM:
  ...
  nameservers: *id001

 vlans:
  bond0.3502:
  ...
  nameservers:  <- anchor redeclaration here
  addresses:
  - 255.255.255.1
  - 255.255.255.2
  - 255.255.255.3
  - 255.255.255.5
  search:
  - customer.domain
  - maas
  bond0.3503:
  ...
  nameservers: *id001

  As the cloudinit renders an invalid yaml file, the netplan apply
  produces the following error: (due to the anchor redeclaration in the
  vlans section):

 Invalid YAML at /etc/netplan/50-cloud-init.yaml line 118 column 25:
  second occurence

  This render bug prevents us using the untagged bond and the bond with
  the vlans in the same configuration.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1815051/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1815051] Re: Bionic netplan render invalid yaml duplicate anchor declaration for nameserver entries

[Ryan Harper]

** Also affects: cloud-init (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: cloud-init (Ubuntu)
   Importance: Undecided => High

** Changed in: cloud-init (Ubuntu)
   Status: New => In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1815051

Title:
  Bionic netplan render invalid yaml duplicate anchor declaration for
  nameserver entries

Status in cloud-init:
  In Progress
Status in cloud-init package in Ubuntu:
  In Progress
Status in cloud-init source package in Xenial:
  New
Status in cloud-init source package in Bionic:
  New
Status in cloud-init source package in Cosmic:
  New

Bug description:
  The netplan configuration redeclares the nameservers anchor for every
  single section (vlans, bonds), and use the same id for similar entries
  (id001).

  In this specific case the network configuration in maas have a bond0
  with two vlans, bond0.3502 and bond0.3503, and an untagged bond1
  without vlans. The rendered 50-cloud-init.yaml looks like this:

  network:
  version: 2
  ethernets:
  ...
  bonds:
  ...
  bond1:
  ...
  nameservers:  <- anchor declaration here
  addresses:
  - 255.255.255.1
  - 255.255.255.2
  - 255.255.255.3
  - 255.255.255.5
  search:
  - customer.domain
  - maas
  ...
  bondM:
  ...
  nameservers: *id001

 vlans:
  bond0.3502:
  ...
  nameservers:  <- anchor redeclaration here
  addresses:
  - 255.255.255.1
  - 255.255.255.2
  - 255.255.255.3
  - 255.255.255.5
  search:
  - customer.domain
  - maas
  bond0.3503:
  ...
  nameservers: *id001

  As the cloudinit renders an invalid yaml file, the netplan apply
  produces the following error: (due to the anchor redeclaration in the
  vlans section):

 Invalid YAML at /etc/netplan/50-cloud-init.yaml line 118 column 25:
  second occurence

  This render bug prevents us using the untagged bond and the bond with
  the vlans in the same configuration.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1815051/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1813817] [NEW] Max retries exceeded with StaleDataError in standardattributes during live migration

[Ryan Tidwell]

Public bug reported:

I am observing StaleDataError on stable/pike during live migration,
causing live migration to fail. It occurs when attempting to live-
migrate a handful of VM's (5-6 VM's is all it takes) in rapid succession
from the same source to the same target. This quick and dirty script is
able to make the issue appear reliably:

for i in `openstack server list --all-projects --host  -c ID -f
value`; do openstack server migrate $i --live ; done

>From the neutron server logs:

DB exceeded retry limit.: StaleDataError: UPDATE statement on table 
'standardattributes' expected to update 1 row(s); 0 were matched.
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api Traceback (most recent call 
last):
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/oslo_db/api.py",
 line 138, in wrapper
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api return f(*args, **kwargs)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/neutron/db/api.py",
 line 128, in wrapped
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api LOG.debug("Retry wrapper 
got retriable exception: %s", e)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/oslo_utils/excutils.py",
 line 220, in __exit__
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api self.force_reraise()
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/oslo_utils/excutils.py",
 line 196, in force_reraise
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api six.reraise(self.type_, 
self.value, self.tb)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/neutron/db/api.py",
 line 124, in wrapped
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api return f(*dup_args, 
**dup_kwargs)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py",
 line 1346, in update_port
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api mech_context, attrs)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/neutron/plugins/ml2/plugin.py",
 line 354, in _process_port_binding
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api 
db.clear_binding_levels(plugin_context, port_id, original_host)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py",
 line 979, in wrapper
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api return fn(*args, **kwargs)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api self.gen.next()
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py",
 line 1029, in _transaction_scope
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api yield resource
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api self.gen.next()
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py",
 line 655, in _session
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api self.session.flush()
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
 line 2171, in flush
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api self._flush(objects)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
 line 2291, in _flush
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api 
transaction.rollback(_capture_exception=True)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
 line 66, in __exit__
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api compat.reraise(exc_type, 
exc_value, exc_tb)
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
 line 2255, in _flush
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api flush_context.execute()
2019-01-24 09:57:34.959 255478 ERROR oslo_db.api   File 
"/opt/stack/venv/neutron-20181030T130300Z/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py",
 line 389, in 

[Yahoo-eng-team] [Bug 1811446] [NEW] chpasswd: is mangling certain password hashes

[Ryan Harper]

Public bug reported:

#cloud-config

# from 1 files
# part-001

---
chpasswd:
expire: false
list: 'root:$2y$10$8BQjxjVByHA/Ee.O1bCXtO8S7Y5WojbXWqnqYpUW.BrPx/Dlew1Va

'


>From #cloud-init

 Hey there, I'm not sure whether I'm running into a bug or not
 I'm trying to set the password hash for the root user on a system 
using the chpasswd module
 It should match new hash at this line in the module but it doens't 
seem to match
 
https://github.com/cloud-init/cloud-init/blame/master/cloudinit/config/cc_set_passwords.py#L163
 I can confirm this when running it through 
https://regex101.com/r/Nj7VTZ/1
 Then I was thinking, isn't [] for lists of characters rather than 
lists of strings
 Changing it to \$(1|2a|2y|5|6)(\$.+){2} does work
 At least in regex101
 smoser, you any idea, I saw you commited the change: 
https://github.com/cloud-init/cloud-init/commit/21632972df034c200578e1fbc121a07f20bb8774
 marlinc_: i'd think yes. that is a bug for the '2a' and '2y'

** Affects: cloud-init
 Importance: Undecided
 Status: New

** Merge proposal linked:
   https://code.launchpad.net/~marlinc/cloud-init/+git/cloud-init/+merge/361683

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1811446

Title:
  chpasswd: is mangling certain password hashes

Status in cloud-init:
  New

Bug description:
  #cloud-config

  # from 1 files
  # part-001

  ---
  chpasswd:
  expire: false
  list: 'root:$2y$10$8BQjxjVByHA/Ee.O1bCXtO8S7Y5WojbXWqnqYpUW.BrPx/Dlew1Va

  '

  
  From #cloud-init

   Hey there, I'm not sure whether I'm running into a bug or not
   I'm trying to set the password hash for the root user on a system 
using the chpasswd module
   It should match new hash at this line in the module but it doens't 
seem to match
   
https://github.com/cloud-init/cloud-init/blame/master/cloudinit/config/cc_set_passwords.py#L163
   I can confirm this when running it through 
https://regex101.com/r/Nj7VTZ/1
   Then I was thinking, isn't [] for lists of characters rather than 
lists of strings
   Changing it to \$(1|2a|2y|5|6)(\$.+){2} does work
   At least in regex101
   smoser, you any idea, I saw you commited the change: 
https://github.com/cloud-init/cloud-init/commit/21632972df034c200578e1fbc121a07f20bb8774
   marlinc_: i'd think yes. that is a bug for the '2a' and '2y'

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1811446/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1809277] [NEW] nocloud-net doesn't read included network-config file if using seedfrom=URL

[Ryan Harper]

Public bug reported:

Currently nocloud-net does not process network-config file if included
in the seed provided via a URL.  This is by-design, however, now that
several datasources have this model of using an ephemeral DHCP process
to fetch user/metadata, which includes network-configuration and then
apply that; nocloud-net could transition to this model.

** Affects: cloud-init
 Importance: Low
 Status: Triaged

** Changed in: cloud-init
   Importance: Undecided => Low

** Changed in: cloud-init
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1809277

Title:
  nocloud-net doesn't read included network-config file if using
  seedfrom=URL

Status in cloud-init:
  Triaged

Bug description:
  Currently nocloud-net does not process network-config file if included
  in the seed provided via a URL.  This is by-design, however, now that
  several datasources have this model of using an ephemeral DHCP process
  to fetch user/metadata, which includes network-configuration and then
  apply that; nocloud-net could transition to this model.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1809277/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 74747] Re: Default sources.list file has source packages enabled by default

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/74747

Title:
  Default sources.list file has source packages enabled by default

Status in cloud-init:
  Fix Released
Status in apt-setup package in Ubuntu:
  Fix Released
Status in cloud-init package in Ubuntu:
  Fix Released
Status in apt-setup source package in Xenial:
  Fix Released
Status in cloud-init source package in Xenial:
  Confirmed
Status in apt-setup source package in Bionic:
  Fix Released
Status in cloud-init source package in Bionic:
  Confirmed
Status in apt-setup source package in Cosmic:
  Fix Released
Status in cloud-init source package in Cosmic:
  Fix Released

Bug description:
  The default sources.list file has source packages enabled by default,
  this is bad for the average user (especially those on modems) because
  they are very unlikely to use source packages, however they will still
  have the download overhead of the packages list.

  For most people the deb-src lines could simply be commented out by
  default.

  (Bug reported at the behest of Robert Collins)

  Implementing this would probably invalidate bug 301602. See also bug
  987264.

  Mailing list discussion:
  
https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2013-May/thread.html#14503
  
https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2013-July/thread.html#14617

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/74747/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1797231] Re: ec2 integration test failure on changed instance-data.json

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1797231

Title:
  ec2 integration test failure on changed instance-data.json

Status in cloud-init:
  Fix Released

Bug description:
  Integration tests for ec2 expected instance-data.json to contain the
  following keys:

  
  {'ds': {'meta_data': {'network' }}} but meta_data changed to meta-data to 
better represent the raw data received directly from ec2's metadata service.

  
  Update integration tests to use meta_data instead of meta-data.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1797231/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1805854] Re: [feature-request] Add non-x86 Ubuntu EC2 mirrors in to default cloud-init configuration

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1805854

Title:
  [feature-request] Add non-x86 Ubuntu EC2 mirrors in to default cloud-
  init configuration

Status in cloud-init:
  Fix Released

Bug description:
  With the announcement of EC2s A1 instances we'd (CPC) like to be able
  to rely on cloud-init to configure archive mirrors for non-x86
  architectures.

  Currently the AMIs have the mirrors configured using
  /etc/cloud/cloud.cfg.d/95_mirrors.cfg which points to the in region
  archive mirrors for non-x86 architectures.

  ```
  # Add archive mirrors for non-x86 architectures
  system_info:
     package_mirrors:
   - arches: [i386, amd64]
     failsafe:
   primary: http://archive.ubuntu.com/ubuntu
   security: http://security.ubuntu.com/ubuntu
     search:
   primary:
     - http://%(ec2_region)s.ec2.archive.ubuntu.com/ubuntu/
     - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/
     - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/
   security: []
   - arches: [armhf, armel, default]
     failsafe:
   primary: http://ports.ubuntu.com/ubuntu-ports
   security: http://ports.ubuntu.com/ubuntu-ports
     search:
   primary:
     - http://%(ec2_region)s.ec2.ports.ubuntu.com/ubuntu-ports/
     - 
http://%(availability_zone)s.clouds.ports.ubuntu.com/ubuntu-ports/
     - http://%(region)s.clouds.ports.ubuntu.com/ubuntu-ports/
   security: []
  ```

  See http://paste.ubuntu.com/p/QGvGKpKYKb/ for current AMIs that can be
  used for testing.

  So the feature request is that coud-init configure the above mirrors
  automatically without us having to write
  /etc/cloud/cloud.cfg.d/95_mirrors.cfg on the image.

  Please let me know if you would like me to spin up instances for
  testing instead or if you need any further info.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1805854/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1803598] Re: Do not retry polling IMDS for reprovisiondata during timeout

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1803598

Title:
  Do not retry polling IMDS for reprovisiondata during timeout

Status in cloud-init:
  Fix Released

Bug description:
  Azure retries polling IMDS for metadata and reprovisiondata when a
  timeout occurs. However this should not happen when polling for
  reprovisiondata as a timeout could indicate that the IP address has
  changed and that we should do a dhcp instead of polling, which is
  going to result in timeout during every poll.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1803598/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1799338] Re: cloud-init won't reformat NTFS ephemeral drive on SLES 15

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1799338

Title:
  cloud-init won't reformat NTFS ephemeral drive on SLES 15

Status in cloud-init:
  Fix Released

Bug description:
  Commit aa4eeb808 (Paul Meyer 2018-05-23 15:45:39 -0400  710) detects that the 
platform doesn't support NTFS volumes by looking for the appropriate error 
message in the exception it catches. The exact syntax of that error message 
differs between RHEL (the target distro for Paul's merge) and SUSE:
RHEL mount: unknown filesystem type 'ntfs'
SUSE mount: /dev/sdb1: unknown filesystem type 'ntfs'

  As a result, cloud-init on SUSE VMs in Azure doesn't properly detect
  that the distro doesn't support NTFS and thus will not reformat the
  ephemeral volume on Azure.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1799338/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1805201] Re: collect-logs traceback on non-root user

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1805201

Title:
  collect-logs traceback on non-root user

Status in cloud-init:
  Fix Released

Bug description:
  Permissions errors are seen when running collect-logs as non-root user
  cloud-init 18.4.24

  $ cloud-init collect-logs
  Traceback (most recent call last):
File "/usr/bin/cloud-init", line 11, in 
  load_entry_point('cloud-init==18.4', 'console_scripts', 'cloud-init')()
File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 904, in 
main
  get_uptime=True, func=functor, args=(name, args))
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2514, in 
log_time
  ret = func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/cloudinit/cmd/devel/logs.py", line 
125, in handle_collect_logs_args
  collect_logs(args.tarfile, args.userdata, args.verbosity)
File "/usr/lib/python3/dist-packages/cloudinit/cmd/devel/logs.py", line 
113, in collect_logs
  os.path.join(run_dir, 'cloud-init'))
File "/usr/lib/python3.6/shutil.py", line 359, in copytree
  raise Error(errors)
  shutil.Error: [('/run/cloud-init/instance-data-sensitive.json', 
'/tmp/tmphc62cg6h/cloud-init-logs-2018-11-26/run/cloud-init/instance-data-sensitive.json',
 "[Errno 13] Permission denied: 
'/run/cloud-init/instance-data-sensitive.json'")]

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1805201/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1797199] Re: kvm integration test failures due to invalid config-disk path

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1797199

Title:
  kvm integration test failures due to invalid config-disk path

Status in cloud-init:
  Fix Released

Bug description:
  Cosmic and bionic integration tests are failing on nocloud kvm
  platforms due to invalid/strict assertions.

  FAIL: test_instance_data_json_kvm 
(tests.cloud_tests.testcases.modules.set_password_expire.TestPasswordExpire)
  Validate instance-data.json content by nocloud-kvm platform.
  --
  Traceback (most recent call last):
File 
"/var/lib/jenkins/slaves/torkoal/workspace/cloud-init-integration-nocloud-kvm-c/cloud-init/tests/cloud_tests/testcases/base.py",
 line 265, in test_instance_data_json_kvm
  self.assertEqual('config-disk (/dev/vda)', v1_data['subplatform'])
  AssertionError: 'config-disk (/dev/vda)' != 'config-disk (/dev/vdb)'
  - config-disk (/dev/vda)
  ? ^
  + config-disk (/dev/vdb)
  ?

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1797199/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1798117] Re: juju sends "network" top level key to user.network-config in lxd containers

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1798117

Title:
  juju sends "network" top level key to user.network-config in lxd
  containers

Status in cloud-init:
  Fix Released
Status in juju:
  Fix Committed
Status in cloud-init package in Ubuntu:
  Confirmed

Bug description:
  == Short summary ==
  In lxd containers launched by juju, 
  /var/lib/cloud/seed/nocloud-net/network-config has:
   has:
   network:
 config: disabled

  That is invalid content.  Cloud-init assumes content in 'network-config'
  is already namespaced to 'network'.  The correct content would be:

config: disabled

  == Easy recreate ==
  $ lxc launch ubuntu-daily:bionic \
 "--config=user.network-config={'network': {'config': {'disabled'}}}"

  == Longer Info ==
  When looking at bug 1651497, I see containers that run cloud-init
  have errors in a container's cloud-init log 
  (http://paste.ubuntu.com/p/5mKXC8pMwH/) like:
AttributeError: 'NoneType' object has no attribute 'iter_interfaces'
  and
Failed to rename devices: Failed to apply network config names. Found bad 
network config version: None

  After some looking guessing I realized that juju must be attempting to
  disable cloud-init's network configuration via sending the following
  into the nocloud seed (/var/lib/cloud/seed/nocloud-net/network-config)
  via 'user.network-config'.

  cloud-init can clearly handle this better, but juju should not be
  sending invalid configuration.

  
  Related bugs:
   * bug 1651497: iscsid.service fails to start in container, results in failed 
dist-upgrade later on

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: cloud-init 18.3-9-g2e62cb8a-0ubuntu1~18.04.2
  ProcVersionSignature: Ubuntu 4.18.0-8.9-generic 4.18.7
  Uname: Linux 4.18.0-8-generic x86_64
  ApportVersion: 2.20.9-0ubuntu7.4
  Architecture: amd64
  CloudName: NoCloud
  Date: Tue Oct 16 14:33:12 2018
  PackageArchitecture: all
  ProcEnviron:
   TERM=xterm-256color
   PATH=(custom, no user)
   LANG=C.UTF-8
  SourcePackage: cloud-init
  UpgradeStatus: No upgrade log present (probably fresh install)
  cloud-init-log-warnings:
   2018-10-16 14:32:01,706 - stages.py[WARNING]: Failed to rename devices: 
Failed to apply network config names. Found bad network config version: None
   2018-10-16 14:32:01,707 - util.py[WARNING]: failed stage init-local
   AttributeError: 'NoneType' object has no attribute 'version'
   2018-10-16 14:32:02,366 - stages.py[WARNING]: Failed to rename devices: 
Failed to apply network config names. Found bad network config version: None
  user_data.txt:
   #cloud-config
   {}

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1798117/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1799594] Re: Azure - Report ready during preprovisioning as soon as we get the ReprovisionData

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1799594

Title:
  Azure - Report ready during preprovisioning as soon as we get the
  ReprovisionData

Status in cloud-init:
  Fix Released

Bug description:
  When reusing a preprovisioned VM, report ready to Azure fabric as soon
  as we get the reprovision data and the goal state so that we are not
  delayed by the cloud-init stage switch, saving 2-3 seconds. Also
  reduce logging when polling IMDS for reprovision data.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1799594/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1798189] Re: cloud-init query: /run/cloud/instance-data-sensitive.json not generated on upgrade

[Ryan Harper]

This bug is believed to be fixed in cloud-init in version 18.5. If this
is still a problem for you, please make a comment and set the state back
to New

Thank you.

** Changed in: cloud-init
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1798189

Title:
  cloud-init query: /run/cloud/instance-data-sensitive.json not
  generated on upgrade

Status in cloud-init:
  Fix Released

Bug description:
  /run/cloud-init/instance-data.json & instance-data-sensitive.json not
  regenerated on upgrade.

  Between cloud-init from 18.3-9 -> 18.4.0 cloud-init transitioned from
  a single sensitive  /run/cloud-init/instance-data.json  that was read-
  only root to two separate files:  /run/cloud-init/instance-data-
  sensitive.json (root readable) and  /run/cloud-init/instance-data.json
  (world readable).

  cloud-init query subcommand attempts to read the instance-data.json
  when getuid is non-root, and instance-data-sensitive.json when getuid
  is root.

  Since /run/cloud-init/instance-data*json is only regenerated on
  reboot, "cloud-init query" after an upgrade emits the following errors

  # as non-root
  ubuntu@mybox $ cloud-init query --all
  ERROR: Missing instance-data.json file: /run/cloud-init/instance-data.json

  # as root user
  ubuntu@mybox $ sudo cloud-init query --all
  ERROR: Missing instance-data.json file: 
/run/cloud-init/instance-data-sensitive.json

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1798189/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp