ppa-dev-tools 0.6.0 release

2024-06-04 Thread Bryce Harrington
The `ppa tests` command received a bunch of bug fixes to autopkgtest
trigger URL generation and test result processing, along with some
noteworthy packaging advancements in this 0.6 release.

ppa-dev-tools is a Launchpad API client that lets you create and manage
Personal Package Archives (PPAs) from the command-line, run autopkgtests
against them, and check results.  E.g.:

  $ ppa create my-ppa
  $ dput *.changes ppa:/my-ppa
  $ ppa wait my-ppa
  $ ppa tests my-ppa
  $ ppa destroy my-ppa

There's multiple ways to install:

By Snap...
  $ sudo snap install ppa-dev-tools

By PPA...
  $ sudo add-apt-repository -ys ppa:bryce/ppa-dev-tools
  $ sudo apt-get install ppa-dev-tools

By PIP...
  $ pip install ppa-dev-tools

By source...
  $ git clone https://git.launchpad.net/ppa-dev-tools
  $ cd ppa-dev-tools
  $ sudo python3 ./setup.py install


Notable changes from 0.5 to 0.6
---

With this release comes some significant improvements in packaging of
ppa-dev-tools, including completion of the transition to modern Python
build packaging and dropping of the old setup.py approach.  Of even more
note, ppa-dev-tools is now included in Debian testing, so can now be
installed directly on that operating system; hopefully soon it will be
available from Ubuntu oracular as well.

Meanwhile, the snap packaging has been significantly improved, including
snapshot builds, builds for all supported architectures, better handling
of Launchpad credentials, and a registered alias for the 'ppa' command.

Several irregularities were found in various corner cases with the
parsing, processing, and display of test results for the `ppa tests`
command.  These are fixed and the testsuite expanded to cover a wider
variety of test (mis-)behaviors.  The --package argument now filters the
results as well as the triggers, which will be helpful for users of PPA
containing many packages.

As well, the `ppa tests` command's handling of triggers has received
some fixes that were causing them to not be displayed in some
circumstances, or to be improperly encoded for some package version
numbers.

Thanks to all the contributors to this release: Alberto Contreras,
Alexandre Detiste, Heinrich Schuchardt, Mitchell Dzurick, Nathan Pratta
Teodosio, Simon Chopin and Benjamin Drung.


For more information about ppa-dev-tools, including reporting bugs and
proposing changes via git, please visit the Launchpad project page:

  https://launchpad.net/ppa-dev-tools

Bryce


Issues fixed in this release


* DeprecationWarning: pkg_resources is deprecated as an API. The 
'ppa-dev-tools==0.5.0' distribution was not found
  (LP: #2055636)
* ppa tests --packages still shows test result from other packages
  (LP: #2025483)
* Tests result duplicated
  (LP: #2038649)
* ppa-dev-tools shows failing tests even though they are passing in the log
  (LP: #2043505)
* Duplicated test URLs and no all-proposed URLs
  (LP: #2044608)
* ppa put invalid subcommand
  (LP: #2066884)
* ppa wait waits on cancelled builds
  (LP: #2067268)
* Incomplete packages listing - bionic not included
  (LP: #2038651)
* Try to get results from a serie not available in the ppa
  (LP: #2043595)
* ppa suggests status is a possible argument command
  (LP: #2067271)
* Consider adopting build, poetry or hatch as packaging system
  (LP: #1989617)
* Support subtest status 'FLAKY'
  (LP: #1989650)
* "ppa-dev-tools.ppa tests" fails to trigger build
  (LP: #2049105)
* small help for comfort until --credentials isn't needed anymore
  (LP: #2051235)
* The following error shows as passed: ERROR: testbed failure: unexpected eof 
from the testbed
  (LP: #2065754)


Shortlog of changes since 0.5.1
---

Alberto Contreras (1):
  doc: remove invalid ppa put example

Alexandre Detiste (1):
  replace mock by unittest.mock from standard library

Bryce Harrington (46):
  Include ESM releases in allowed list of triggers.
  results: Add test case for show_results()
  results: Avoid printing excessive duplicate results
  NEWS: Set version
  ppa: Fix the all-proposed link
  ppa: Fix f-string for how to add the ppa
  processes: Include output when raising ReturnCode
  trigger: Refactor object generation into new get_triggers()
  trigger: Refactor trigger display logic to new show_triggers()
  trigger: Rename series_codename to just series
  trigger: Improve Trigger class docs
  tests: Implement test_get_results()
  results: Fix invalid url generation for get_results()
  Rename exception to PpaNotFoundError
  ppa: Use standard error codes for exit conditions
  ppa: Catch exceptions for issues when reading config files
  tests: Expand test cases for test_to_dict()
  tests: Add test case for to_bullet_tree()
  test_result: Add subtest cases to test_to_bullet_tree()
  result: Display status line for failed subtests
  result: Reorder to_bullet_tree()
  

Re: Make proposed available by default? [was: Setting NotAutomatic for hirsute+1-proposed]

2024-06-04 Thread Julian Andres Klode
On Mon, Jun 03, 2024 at 08:49:04PM -0700, Steve Langasek wrote:
> On Fri, May 03, 2024 at 08:43:11AM +0200, Heinrich Schuchardt wrote:
> > On Debian I have seen apt-update downloading diff files. Why don't we use
> > those for Ubuntu
> 
> Disclaimer: I have never benchmarked this, but am going by my observations
> from 20 years ago when this was adopted by Debian.
> 
> In the common case, pdiff files decrease the *data transfer size* for
> downloading indices, but increase the *clock time* it takes to update the
> apt database.  Therefore there has never been evidence that it's a net win
> given modern Internet connections, and no one has ever agitated for this to
> be a priority in Ubuntu/Launchpad.

It's much faster these days. Debian also moved to server-side merged
pdiffs now, so it's just one file to download and apply, whereas before
multiple files were downloaded in the pipeline and then merged locally
before being applied.

And I guess in the very old days it was not merged at any point.

But in any case, this has been brought up a couple of times and
the main issue why we don't have pdiffs for Ubuntu is arguably
that we don't have like 4 dinstall runs a day, but more like 72
launchpad publisher runs, and having 18x as many deltas is quite
expensive both in storage space and calculating them.

Which is why a zsync-style synchronization would be better where
we compress the file block wise and can fetch changed blocks; however
the APT HTTP code is fairly unreliable and that's fairly complex,
and you need to use something like zstd with a custom dictionary
to make this still compress efficiently.

An alternative would be replacing the pdiff format with a tagged
one, where each hunk is associated with a timestamp, such that you
can then ship one larger patch file per day or so, and you only
apply the bits you actually need.

Essentially you just need to keep the previous run's pdiff2 around
to keep pdiffs working on upgrades, so it will be 2 diffs for the
current day, and 1 for each past day or so.

I don't believe there is much value in this for stable releases
though as has been pointed out before. This may change with LTS.

But also we could come up with a much simpler approach where we
just split the -updates/-security pockets into 6 months cycles
(i.e. add 2024H1 or 24.04.1 subdirectories, add some fancy way
to InRelease files to state you need to fetch these subdirectories
too) and then you just fetch the current cycle on update. But
first fetches would be annoying.

But still not convinced that is worth the effort.
-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer  i speak de, en


signature.asc
Description: PGP signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel