Re: [Debian-salsa-ci] Publishing multiple packages with aptly in Salsa CI

2023-09-15 Thread Paul Boddie
On Friday, 15 September 2023 16:33:25 CEST Philip Hands wrote:
>
> For another angle, see:
> 
>   https://salsa.debian.org/philh/user-setup/-/pipelines/576662
> 
> In which I have a `harvest-repos` job that grabs artifacts from `build`
> jobs in other pipelines, and an `aptly-plus` job that's got an added
> `needs: harvest-repos` that can combine the artifacts from its build and
> harvest-repos jobs and lump them all together.
>
> In the resulting aptly repo you can see that it includes both the
> local package (user-setup) under the 'u' directory, and 'grub-installer'
> under the 'g' directory:
>  
> https://salsa.debian.org/philh/user-setup/-/jobs/4671054/artifacts/browse/
aptly/pool/main/

Yes, that was the desired effect: have packages corresponding to several 
source packages being made available in the pool and thus the created package 
repository.

> That's done with:
> 
>   https://salsa.debian.org/installer-team/branch2repo/
> 
> Quite a lot of that is already part of the standard salsa-CI pipeline,
> and my aim is that branch2repo will pretty-much disapear, with its
> components being optional bits of the standard pipeline, and maybe a few
> variable settings.

Indeed. As noted before, my modifications were effectively (1) to keep the 
source repository around to get access to my scripts, (2) to be able to add 
package repositories to the apt configuration, (3) to download packages so 
that aptly can process them.

(1) is just preferable to writing scripts in the horrible YAML syntax and then 
doing fragment inclusion, which I found can lead to opaque error messages. 
Being able to obtain a set of scripts would be quite helpful generally as the 
environment can vary substantially from one kind of job to another.

(2) is something that could be part of the standard job definitions, 
particularly since other jobs like autopkgtest need to know about additional 
repositories in certain cases, brand new packages being one of them.

(3) is just a small enhancement for this specific scenario.

I thought that someone must have done this before, even if the standard 
pipeline didn't support it. I imagine that some coordination would be 
desirable to prevent fragmentation and people like me introducing our own ways 
of addressing this particular need.

Paul




Re: [Debian-salsa-ci] Publishing multiple packages with aptly in Salsa CI

2023-09-15 Thread Philip Hands
Hi,

Someone wrote:

>> However, one other thing I wanted to achieve was to take the complete set of 
>> new packages and to publish them in a single package repository. This would 
>> allow people to install and test the built packages in a more convenient 
>> fashion than asking them to hunt down each built package from job artefacts 
>> or 
>> to build the packages themselves.
>>
>> Obviously, the aptly job in the standard Debian CI pipeline publishes a 
>> single 
>> package (or maybe a collection of packages built from a single source 
>> package), but I wanted to aggregate all packages published by a collection 
>> of 
>> aptly repositories. Fortunately, it seems that this is possible by 
>> augmenting 
>> the existing aptly job definition as shown in the following file:
>> 
>> https://salsa.debian.org/moin-team/moin/-/blob/debian/master/debian/salsa-ci.yml

For another angle, see:

  https://salsa.debian.org/philh/user-setup/-/pipelines/576662

In which I have a `harvest-repos` job that grabs artifacts from `build`
jobs in other pipelines, and an `aptly-plus` job that's got an added
`needs: harvest-repos` that can combine the artifacts from its build and
harvest-repos jobs and lump them all together.

In the resulting aptly repo you can see that it includes both the
local package (user-setup) under the 'u' directory, and 'grub-installer'
under the 'g' directory:

  
https://salsa.debian.org/philh/user-setup/-/jobs/4671054/artifacts/browse/aptly/pool/main/

That's done with:

  https://salsa.debian.org/installer-team/branch2repo/

Quite a lot of that is already part of the standard salsa-CI pipeline,
and my aim is that branch2repo will pretty-much disapear, with its
components being optional bits of the standard pipeline, and maybe a few
variable settings.

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature