Thanks for the feedback. I'm addressing a) in https://code.launchpad.net/~skia/ubuntu-dev-tools/+git/ubuntu-dev-tools/+merge/512558. For b), I can sure add some `noble` call, but I'm not sure this would be testing a new `seeded-in-ubuntu` code path, just testing another index from the server, but I don't believe this is really the point of that test case, otherwise we would need to test every series/arch combination. Anyway, I'm adding an entry.
** Description changed: [ Impact ] The `seeded-in-ubuntu` command, used by the Release Team, is currently backed by an index on UbuntuWire: http://qa.ubuntuwire.org/ubuntu-seeded-packages/seeded.json.gz This index currently has multiple issues: * It doesn't support series and arches, so essentially only returns results for the devel series on amd64 * It seems to be a bit broken, as shown with the `packagekit` case below (in [ Original description ]) [ Test Plan ] Run seeded-in-ubuntu in various situations to cover different combinations among: - * source vs binary package - * series - * architectures - * seeded or not seeded + * source vs binary package + * series + * architectures + * seeded or not seeded Run some other ubuntu-dev-tools commands to make sure the package isn't broken. The following script can help cover these cases: ```bash #!/usr/bin/bash echo "Testing ubuntu-dev-tools's seeded-in-ubuntu command" echo "ubuntu-dev-tools version:" dpkg-query -W ubuntu-dev-tools function run() { - echo " $" $@ - $@ - echo "" + echo " $" $@ + $@ + echo "" } echo "=====================================================================" echo "Testing source package busybox against default series (stonking)." echo "Watch that 'busybox-static' is mostly in 'extra' seed, and 'busybox-initramfs' doesn't exists." run seeded-in-ubuntu busybox echo "=====================================================================" echo "Testing binary package busybox-initramfs against default series (stonking)." echo "Should output that the package is not seeded and may not exist." run seeded-in-ubuntu -b busybox-initramfs echo "=====================================================================" echo "Testing binary package busybox-initramfs against explicit stonking." echo "Should output that the package is not seeded and may not exist." run seeded-in-ubuntu -s stonking -b busybox-initramfs echo "=====================================================================" echo "Testing source package busybox against resolute." echo "Watch that 'busybox-static' is mostly in 'standard' seed, and 'busybox-initramfs' is \"everywhere\"." run seeded-in-ubuntu -s resolute busybox echo "=====================================================================" echo "Testing binary package busybox-initramfs against resolute." echo "Should output that the package is seeded \"everywhere\"." run seeded-in-ubuntu -s resolute -b busybox-initramfs echo "=====================================================================" echo "Testing binary package ubuntu-x1e-settings against resolute amd64." echo "Should output that the package is not seeded and may not exist." run seeded-in-ubuntu -s resolute -b ubuntu-x1e-settings echo "=====================================================================" echo "Testing binary package ubuntu-x1e-settings against resolute arm64." echo "This should be seeded for 'ubuntu' in 'ship-live' and 'server-ship-live'" run seeded-in-ubuntu -s resolute -a arm64 -b ubuntu-x1e-settings echo "=====================================================================" echo "Testing binary package ubuntu-x1e-settings against jammy arm64." echo "Should output that the package is not seeded and may not exist." run seeded-in-ubuntu -s jammy -a arm64 -b ubuntu-x1e-settings + echo "=====================================================================" + echo "Testing binary package s390x-tools against noble s390x." + echo "This should be seeded for everything in 'boot' and 'supported-cloud', and even more seeds for 'ubuntu'" + run seeded-in-ubuntu -s noble -a s390x -b s390x-tools + echo "#####################################################################" echo "Smoke testing a couple of other commands from the package" echo "=====================================================================" run submittodebian --help echo "=====================================================================" run import-bug-from-debian --help echo "=====================================================================" run reverse-depends --help echo "=====================================================================" run ubuntu-build --help echo "=====================================================================" ``` - [ Where problems could occur ] This is a Python script developed on the devel series, meaning a recent Python ecosystem. Backporting it to older release has the risk of hitting a too recent Python feature or API that doesn't work in older ecosystem. Fortunately, this is a self-contained script, not a module/library, so should impact anything else than just being broken. Also, the set of arguments and code paths is fairly low, so it should be humanly doable to reach good exhaustive coverage by following the Test Plan carefully. Finally, this script and package is mostly used by Ubuntu Developers, so shouldn't impact non-technical folks if it happens to break somehow. [ Other info ] Please note that the `seeded-in-ubuntu` command relies on an external service: https://static-reports.ubuntu.com/ If this service is down, a local cache can help, but needs to be populated at least once. With the current difficulties regarding Canonical infrastructure, we might be in a position of not being able to execute the Test Plan swiftly when the package is published in -proposed. [ Original description ] As of now: ❯ seeded-in-ubuntu packagekit gir1.2-packagekitglib-1.0 (from packagekit) is seeded in: ubuntu-core-installer: daily-live libpackagekit-glib2-18 (from packagekit) is seeded in: ubuntu-core-installer: daily-live libpackagekit-glib2-dev (from packagekit) is seeded in: ubuntu-budgie: supported ubuntu: supported packagekit (from packagekit) is seeded in: ubuntu-core-installer: daily-live packagekit-docs (from packagekit) is seeded in: ubuntu-budgie: supported ubuntu: supported but ❯ curl -s https://cdimage.ubuntu.com/ubuntu-server/resolute/daily-live/current/resolute-live-server-amd64.manifest | grep packagekit gir1.2-packagekitglib-1.0 1.3.4-3ubuntu1.1 libpackagekit-glib2-18:amd64 1.3.4-3ubuntu1.1 packagekit 1.3.4-3ubuntu1.1 and ❯ curl -s https://cdimage.ubuntu.com/ubuntu/resolute/daily-live/current/resolute-desktop-amd64.manifest | grep packagekit gir1.2-packagekitglib-1.0 1.3.4-3ubuntu1 gstreamer1.0-packagekit 1.3.4-3ubuntu1 libpackagekit-glib2-18:amd64 1.3.4-3ubuntu1 packagekit 1.3.4-3ubuntu1 and ❯ curl -s https://cdimage.ubuntu.com/ubuntu/stonking/daily-live/current/stonking-desktop-amd64.manifest | grep packagekit gir1.2-packagekitglib-1.0 1.3.5-1ubuntu1 gstreamer1.0-packagekit 1.3.5-1ubuntu1 libpackagekit-glib2-18:amd64 1.3.5-1ubuntu1 packagekit 1.3.5-1ubuntu1 We apparently can't trust `seeded-in-ubuntu` anymore 😢 Maybe a first step would be to get back control over its index, currently hosted and generated who knows how on UbuntuWire: http://qa.ubuntuwire.org/ubuntu-seeded-packages/seeded.json.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2160240 Title: seeded-in-ubuntu is lying To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ubuntu-dev-tools/+bug/2160240/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
