SRU Verification script ** Description changed:
[ Impact ] Apport reported bug add invalid bug tags such as `uec-images` which no longer has meaning or `ec2-images` on openstack. Since cloud-init is installed in all these images and detects the correct datasource, leverage cloud-init's instance-data.json or cloud-id [ Test Plan ] 1. Launch LXD container for each series 2. upgrade to cloud-init from -proposed. 3. Execute apport-bug cloud-init and view report to assess that correct tags are present for LXD datasource. 4. replace /run/cloud-init/instance-data.json cloud-id content with examples from openstack, ec2, configdrive and assert appropriate tags match the platform details. - 5. Assert no stray uec-images or invalid ec2-images tags on openstack. - scriptlet TBD: + verification scriptlet (attached): + + #!/bin/bash + set -ex + + cat > setup_proposed.sh <<EOF + #/bin/bash + mirror=http://archive.ubuntu.com/ubuntu + echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list + apt-get update -q + apt-get install -qy cloud-init + EOF + cat > openstack.json <<EOF { - "v1": { - "cloud_id": "openstack", - "cloud_name": "openstack", - "machine": "CloudArchitecture", - "platform": "openstack", - "region": null, - "subplatform": "metadata (http://169.254.169.254)", - } + "ds" : {}, + "v1": { + "cloud_id": "openstack", + "cloud_name": "openstack", + "machine": "CloudArchitecture", + "platform": "openstack", + "region": null, + "subplatform": "metadata (http://169.254.169.254)" + } } EOF cat > ec2.json << EOF { - "v1": { - "cloud_name": "aws", - "distro": "ubuntu", - "distro_release": "jammy", - "distro_version": "22.04", - "instance_id": "i-06b5687b4d7b8595d", - "machine": "x86_64", - "platform": "ec2", - "python_version": "3.10.4", - "region": "us-east-2", - "variant": "ubuntu" - } + "ds": { + "imageId": "ami-123", + "instanceType": "m1.tiny", + "region": "us-east-1" + }, + "v1": { + "cloud_name": "aws", + "distro": "ubuntu", + "distro_release": "jammy", + "distro_version": "22.04", + "instance_id": "i-06b5687b4d7b8595d", + "machine": "x86_64", + "platform": "ec2", + "python_version": "3.10.4", + "region": "us-east-2", + "variant": "ubuntu" + } } EOF for release in focal jammy kinetic lunar; do - lxc launch ubuntu-daily:$release sru-$release - lxc exec sru-f -- cloud-init status --wait --long - lxc exec sru-f -- apport-bug cloud-init - # step through prompts and select view/save report - # inspect tags to assert cloud platform is proper tags where value is non-empty/null - CloudID, CloudName, CloudArchitecture CloudPlatform, CloudRegion, CloudSubPlatform + VM=sru-$release + lxc launch ubuntu-daily:$release $VM + while ! lxc exec $VM -- cloud-init status --wait --long; do + sleep 5 + done + echo --- 1. Generate current apport report, selecting Openstack as cloud. + echo --- step through prompts and select 'K' to keep report + lxc exec $VM -- apport-bug cloud-init + APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport) + lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.orig + lxc exec $VM rm /tmp/$APPORT_FILE - echo ### validate openstack apport-bug - lxc file push openstack.json dev-$release/run/cloud-init/instance-data.json - lxc exec sru-f -- apport-bug cloud-init - # step through prompts and select view/save report - # inspect tags to assert cloud platform is proper tags where value is non-empty/null - CloudID, CloudName, CloudArchitecture CloudPlatform, CloudRegion, + lxc file push setup_proposed.sh $VM/ + lxc exec $VM -- bash /setup_proposed.sh | grep cloud-init - echo ### validate ec2 apport-bug - lxc file push ec2.json dev-$release/run/cloud-init/instance-data.json - lxc exec sru-f -- apport-bug cloud-init - # step through prompts and select view/save report - # inspect tags to assert cloud platform is proper tags where value is non-empty/null - CloudID, CloudName, CloudArchitecture CloudPlatform, CloudRegion, + lxc file push openstack.json $VM/run/cloud-init/instance-data.json + echo --- 2. Generate -proposed apport report which sources openstack instance-data.json, selecting Openstack as cloud. + lxc exec $VM -- apport-bug cloud-init + APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport) + lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.openstack-proposed + lxc exec $VM rm /tmp/$APPORT_FILE + + echo --- 3. Generate -proposed apport report which sources ec2 instance-data, selecting Ec2 as cloud. + lxc file push ec2.json $VM/run/cloud-init/instance-data.json + lxc exec $VM -- apport-bug cloud-init + APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport) + lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.ec2-proposed + lxc exec $VM rm /tmp/$APPORT_FILE + + # redact logs lines for easy diffs + for file in `ls apport*`; do + sed -i '1,/logs.tgz/!d' $file + done + echo --- 4. Inspect diff tags of orig to openstack-proposed report + diff -urN apport-$VM.orig apport-$VM.openstack-proposed || true + echo --- 5. Inspect diff tags of openstack-proposed to ec2-proposed report + diff -urN apport-$VM.openstack-proposed apport-$VM.ec2-proposed || true done + [ Where problems could occur ] - apport-bug could traceback on poor logic preventing simple bug filing at CLI. It could omit tags if unable to process cloud-init/instance-data.json. nothing critical to daily performance, uptime or security + apport-bug could traceback on poor logic preventing simple bug filing at apport CLI. It could omit tags if unable to process cloud-init/instance-data.json. nothing critical to daily performance, uptime or security [ Other Info ] + [Original description] Issues: - Using the presence of cloud-init to flag an image as a cloud image is incorrect now that ubuntu-server includes cloud-init (and ubuntu-core images) - Using the presence of EC2 metadata source is incorrect as many non-EC2 clouds provide EC2 metadata. Thus we have bugs like bug #1722946 that are tagged as an 'ec2-images' bug which are clearly on openstack - Marking all bugs that have cloud-init but no EC2 metadata source as an 'uec-images' bug uses a name that no longer has meaning. Solution: - If cloud-init is present, check for /etc/cloud/build.info to indicate an Ubuntu cloud images, tag as 'cloud-images'. Pull the build_name and serial from that file into the bug comments. - If cloud-init is present, check for the presence of /run/cloud-init/cloud.cfg. Parse this yaml to determine the datasource type. Add the datasource used to the bug comment. I have filed bug #1724626 to ask cloud-init development to surface more information from ds-identify to help ID the cloud so that we can better tag/annotate the bug. There may also be some info we can get to indicate the image ID on more clouds than just AWS. At a minimum I would like to see dsidentify make the EC2 platform it found available for consumers in cloud.cfg. This would allow us to identify AWS EC2 from look-alike datasources so that we can tag a bug as ec2-images for bug really on AWS and add EC2 specific fields to the description/attachments. ** Attachment added: "sru-2025376-verification.sh" https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1724623/+attachment/5682929/+files/sru-2025376-verification.sh -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apport in Ubuntu. https://bugs.launchpad.net/bugs/1724623 Title: Update ubuntu cloud info Status in Apport: Fix Released Status in apport package in Ubuntu: Fix Released Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Focal: Fix Committed Status in cloud-init source package in Jammy: Fix Committed Status in cloud-init source package in Kinetic: Fix Committed Bug description: [ Impact ] Apport reported bug add invalid bug tags such as `uec-images` which no longer has meaning or `ec2-images` on openstack. Since cloud-init is installed in all these images and detects the correct datasource, leverage cloud-init's instance-data.json or cloud-id [ Test Plan ] 1. Launch LXD container for each series 2. upgrade to cloud-init from -proposed. 3. Execute apport-bug cloud-init and view report to assess that correct tags are present for LXD datasource. 4. replace /run/cloud-init/instance-data.json cloud-id content with examples from openstack, ec2, configdrive and assert appropriate tags match the platform details. verification scriptlet (attached): #!/bin/bash set -ex cat > setup_proposed.sh <<EOF #/bin/bash mirror=http://archive.ubuntu.com/ubuntu echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list apt-get update -q apt-get install -qy cloud-init EOF cat > openstack.json <<EOF { "ds" : {}, "v1": { "cloud_id": "openstack", "cloud_name": "openstack", "machine": "CloudArchitecture", "platform": "openstack", "region": null, "subplatform": "metadata (http://169.254.169.254)" } } EOF cat > ec2.json << EOF { "ds": { "imageId": "ami-123", "instanceType": "m1.tiny", "region": "us-east-1" }, "v1": { "cloud_name": "aws", "distro": "ubuntu", "distro_release": "jammy", "distro_version": "22.04", "instance_id": "i-06b5687b4d7b8595d", "machine": "x86_64", "platform": "ec2", "python_version": "3.10.4", "region": "us-east-2", "variant": "ubuntu" } } EOF for release in focal jammy kinetic lunar; do VM=sru-$release lxc launch ubuntu-daily:$release $VM while ! lxc exec $VM -- cloud-init status --wait --long; do sleep 5 done echo --- 1. Generate current apport report, selecting Openstack as cloud. echo --- step through prompts and select 'K' to keep report lxc exec $VM -- apport-bug cloud-init APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport) lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.orig lxc exec $VM rm /tmp/$APPORT_FILE lxc file push setup_proposed.sh $VM/ lxc exec $VM -- bash /setup_proposed.sh | grep cloud-init lxc file push openstack.json $VM/run/cloud-init/instance-data.json echo --- 2. Generate -proposed apport report which sources openstack instance-data.json, selecting Openstack as cloud. lxc exec $VM -- apport-bug cloud-init APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport) lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.openstack-proposed lxc exec $VM rm /tmp/$APPORT_FILE echo --- 3. Generate -proposed apport report which sources ec2 instance-data, selecting Ec2 as cloud. lxc file push ec2.json $VM/run/cloud-init/instance-data.json lxc exec $VM -- apport-bug cloud-init APPORT_FILE=$(lxc exec $VM ls /tmp | grep apport) lxc file pull $VM/tmp/$APPORT_FILE apport-$VM.ec2-proposed lxc exec $VM rm /tmp/$APPORT_FILE # redact logs lines for easy diffs for file in `ls apport*`; do sed -i '1,/logs.tgz/!d' $file done echo --- 4. Inspect diff tags of orig to openstack-proposed report diff -urN apport-$VM.orig apport-$VM.openstack-proposed || true echo --- 5. Inspect diff tags of openstack-proposed to ec2-proposed report diff -urN apport-$VM.openstack-proposed apport-$VM.ec2-proposed || true done [ Where problems could occur ] apport-bug could traceback on poor logic preventing simple bug filing at apport CLI. It could omit tags if unable to process cloud-init/instance-data.json. nothing critical to daily performance, uptime or security [ Other Info ] [Original description] Issues: - Using the presence of cloud-init to flag an image as a cloud image is incorrect now that ubuntu-server includes cloud-init (and ubuntu-core images) - Using the presence of EC2 metadata source is incorrect as many non-EC2 clouds provide EC2 metadata. Thus we have bugs like bug #1722946 that are tagged as an 'ec2-images' bug which are clearly on openstack - Marking all bugs that have cloud-init but no EC2 metadata source as an 'uec-images' bug uses a name that no longer has meaning. Solution: - If cloud-init is present, check for /etc/cloud/build.info to indicate an Ubuntu cloud images, tag as 'cloud-images'. Pull the build_name and serial from that file into the bug comments. - If cloud-init is present, check for the presence of /run/cloud-init/cloud.cfg. Parse this yaml to determine the datasource type. Add the datasource used to the bug comment. I have filed bug #1724626 to ask cloud-init development to surface more information from ds-identify to help ID the cloud so that we can better tag/annotate the bug. There may also be some info we can get to indicate the image ID on more clouds than just AWS. At a minimum I would like to see dsidentify make the EC2 platform it found available for consumers in cloud.cfg. This would allow us to identify AWS EC2 from look-alike datasources so that we can tag a bug as ec2-images for bug really on AWS and add EC2 specific fields to the description/attachments. To manage notifications about this bug go to: https://bugs.launchpad.net/apport/+bug/1724623/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
