Re: debian-installer | salsa-ci: if version-skewed, use the currently available ABI version (!41)

[Philip Hands]

Cyril Brulebois  writes:

> Philip Hands  (2024-01-26):
>> Philip Hands  writes:
>> 
>> > How about using '++'?
>> 
>> After a little more thought I concluded that '+~' is marginally better,
>> so here's that alternative, just in case anyone agrees with me:
>> 
>>   
>> https://salsa.debian.org/philh/debian-installer/-/commit/f5044026b07d8dbf938394193708b33ea811ae2e
>
> Both solutions (and others in the same vein) sound specific enough that
> they shouldn't affect stable releases so feel free to pick whatever you
> like best.

Thanks & Done.

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

Re: debian-installer | salsa-ci: if version-skewed, use the currently available ABI version (!41)

[Cyril Brulebois]

Philip Hands  (2024-01-26):
> Philip Hands  writes:
> 
> > How about using '++'?
> 
> After a little more thought I concluded that '+~' is marginally better,
> so here's that alternative, just in case anyone agrees with me:
> 
>   
> https://salsa.debian.org/philh/debian-installer/-/commit/f5044026b07d8dbf938394193708b33ea811ae2e

Both solutions (and others in the same vein) sound specific enough that
they shouldn't affect stable releases so feel free to pick whatever you
like best.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: debian-installer | salsa-ci: if version-skewed, use the currently available ABI version (!41)

[Philip Hands]

Philip Hands  writes:

> How about using '++'?

After a little more thought I concluded that '+~' is marginally better,
so here's that alternative, just in case anyone agrees with me:

  
https://salsa.debian.org/philh/debian-installer/-/commit/f5044026b07d8dbf938394193708b33ea811ae2e

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

Re: debian-installer | salsa-ci: if version-skewed, use the currently available ABI version (!41)

[Philip Hands]

Hi Cyril,

Cyril Brulebois  writes:

> Hi,
>
> Philip Hands (@philh)  (2024-01-24):
>> Merge request !41 was merged
>> Merge request URL:  
>> Project:Branches: philh/debian-installer:salsa-abi-fixup to 
>> installer-team/debian-installer:master
>> Author: Philip Hands
>
> Commit 931269d99b1595725f41474e3ae00a78c9d274ae breaks builds for 
> stable branches, which are DATE+debXuY. Example on bookworm: DATE
> gets set to 20230607 instead of 20230607+deb12u4.

Ah, well spotted! :-)

I suppose one could use '+ABI' as the delimiter, instead of '+', but
that seems too specific to me.

How about using '++'?

Here's a commit to that effect:

  
https://salsa.debian.org/philh/debian-installer/-/commit/f5044026b07d8dbf938394193708b33ea811ae2e

If that looks OK to you I can push it at master (or will create an MR if
that's preferred for some reason).

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

Re: debian-installer | salsa-ci: if version-skewed, use the currently available ABI version (!41)

[Cyril Brulebois]

Hi,

Philip Hands (@philh)  (2024-01-24):
> Merge request !41 was merged
> Merge request URL: 
> https://salsa.debian.org/installer-team/debian-installer/-/merge_requests/41
> Project:Branches: philh/debian-installer:salsa-abi-fixup to 
> installer-team/debian-installer:master
> Author: Philip Hands

Commit 931269d99b1595725f41474e3ae00a78c9d274ae breaks builds for 
stable branches, which are DATE+debXuY. Example on bookworm: DATE
gets set to 20230607 instead of 20230607+deb12u4.

Of course that's going to be a problem for trixie onwards, but I'd
rather get that sorted out now, and not on the way to the first point
release, which is usually quite busy already.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: salsa CI

[Cyril Brulebois]

Samuel Thibault  (2024-01-23):
> The CI on salsa doesn't manage to build the debian-installer package
> because the signed linux 6.6.13 package is not available yet. Perhaps we
> should turn these build-deps:
> 
>   linux-image-6.6.13-amd64 [amd64],
>   linux-image-6.6.13-arm64 [arm64],
>   linux-image-6.6.13-686 [i386],
>   linux-image-6.6.13-686-pae [i386],
> 
> into
> 
>   linux-image-6.6.13-amd64 [amd64] | linux-image-6.6.13-amd64-unsigned 
> [amd64],
>   linux-image-6.6.13-arm64 [arm64] | linux-image-6.6.13-arm64-unsigned 
> [arm64],
>   linux-image-6.6.13-686 [i386] | linux-image-6.6.13-686-unsigned [i386],
>   linux-image-6.6.13-686-pae [i386] | linux-image-6.6.13-686-pae-unsigned 
> [i386],
> 
> ?

udebs are built from the linux-signed-* packages, so that doesn't help
at all.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: salsa CI

[Philip Hands]

Samuel Thibault  writes:

> For me it's fine for CI to fall back to the previous kernel for most
> jobs of the pipeline. I guess we'd still want a build job in the
> pipeline that sticks with the requested version, so that we notice in
> case that's not working, without breaking the entire CI pipeline.

That's a good idea -- I've done something slightly different, which is
to add an 'ABI-check' job which fails if we are in version-skew:

  
https://salsa.debian.org/philh/debian-installer/-/commit/3b862d27f3ca07e840c0f1a44e42fd26ab69b176

I think that's the finishing touch that makes me happy to merge this.

If you can tell me how the *-unsigned version of the build is actually
supposed to succeed (my attempt failed at the point where it tries to
get hold of the matching modules, where the new version was not yet
available), then I'll happily add that as an additional condition (and
will also check in the check job to see if that's what is going on).

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

Re: salsa CI

[Samuel Thibault]

Philip Hands, le mar. 23 janv. 2024 17:52:57 +0100, a ecrit:
> Samuel Thibault  writes:
> 
> > Philip Hands, le mar. 23 janv. 2024 16:27:12 +0100, a ecrit:
> >> Samuel Thibault  writes:
> >> 
> >> > Hello,
> >> >
> >> > The CI on salsa doesn't manage to build the debian-installer package
> >> > because the signed linux 6.6.13 package is not available yet.
> >> 
> >> Is the thing you want to:
> >>   a) be able to build d-i on salsa even when we're in version skew,
> >> or
> >>   b) do you want to be able to test with the latest version, whether it is 
> >> signed or not?
> >
> > b)
> >
> > Normally the bump in debian-installer comes about the same time as the
> > linux upload. But there is the period between the linux upload and the
> > linux-signed upload during which we don't really know whether we want to
> > bump or not. Adding the alternative between non-signed and signed as I
> > proposed would allow to be fine with either, while making sure it's the
> > signed version which is used on buildds.
> 
> Ah, fair enough.
> 
> I guess in that case I'll need to adjust what I'm doing to detect the
> available versions of kernel that I'm looking for in that patch.
> 
> If you're only worried about builds on salsa-CI, same approach as used
> in my MR ought to work,

Indeed.

It however doesn't fix the build on my laptop without some change :)

> and then one could perhaps control which kernel
> is selected via variables, or perhaps defaulting to the unsigned kernel
> (if available) would work for my use-case too, in which case I could
> just add that as a feature.
> 
> The MR's here BTW: https://deb.li/3hHY2
> 
> BTW would it actually cause you a problem for the build to work, despite
> the kernel being unavailable (e.g. by falling back to the previous
> version)?

For me it's fine for CI to fall back to the previous kernel for most
jobs of the pipeline. I guess we'd still want a build job in the
pipeline that sticks with the requested version, so that we notice in
case that's not working, without breaking the entire CI pipeline.

Samuel

Re: salsa CI

[Philip Hands]

Samuel Thibault  writes:

> Philip Hands, le mar. 23 janv. 2024 16:27:12 +0100, a ecrit:
>> Samuel Thibault  writes:
>> 
>> > Hello,
>> >
>> > The CI on salsa doesn't manage to build the debian-installer package
>> > because the signed linux 6.6.13 package is not available yet.
>> 
>> Is the thing you want to:
>>   a) be able to build d-i on salsa even when we're in version skew,
>> or
>>   b) do you want to be able to test with the latest version, whether it is 
>> signed or not?
>
> b)
>
> Normally the bump in debian-installer comes about the same time as the
> linux upload. But there is the period between the linux upload and the
> linux-signed upload during which we don't really know whether we want to
> bump or not. Adding the alternative between non-signed and signed as I
> proposed would allow to be fine with either, while making sure it's the
> signed version which is used on buildds.

Ah, fair enough.

I guess in that case I'll need to adjust what I'm doing to detect the
available versions of kernel that I'm looking for in that patch.

If you're only worried about builds on salsa-CI, same approach as used
in my MR ought to work, and then one could perhaps control which kernel
is selected via variables, or perhaps defaulting to the unsigned kernel
(if available) would work for my use-case too, in which case I could
just add that as a feature.

The MR's here BTW: https://deb.li/3hHY2

BTW would it actually cause you a problem for the build to work, despite
the kernel being unavailable (e.g. by falling back to the previous
version)?

If so, I guess a variable is required to stop that behaviour.

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

Re: salsa CI

[Samuel Thibault]

Philip Hands, le mar. 23 janv. 2024 16:27:12 +0100, a ecrit:
> Samuel Thibault  writes:
> 
> > Hello,
> >
> > The CI on salsa doesn't manage to build the debian-installer package
> > because the signed linux 6.6.13 package is not available yet.
> 
> Is the thing you want to:
>   a) be able to build d-i on salsa even when we're in version skew,
> or
>   b) do you want to be able to test with the latest version, whether it is 
> signed or not?

b)

Normally the bump in debian-installer comes about the same time as the
linux upload. But there is the period between the linux upload and the
linux-signed upload during which we don't really know whether we want to
bump or not. Adding the alternative between non-signed and signed as I
proposed would allow to be fine with either, while making sure it's the
signed version which is used on buildds.

Samuel

Re: salsa CI

[Philip Hands]

Samuel Thibault  writes:

> Hello,
>
> The CI on salsa doesn't manage to build the debian-installer package
> because the signed linux 6.6.13 package is not available yet.

Is the thing you want to:
  a) be able to build d-i on salsa even when we're in version skew,
or
  b) do you want to be able to test with the latest version, whether it is 
signed or not?

If what you're after a), I have a patch that does that, as you can see
here:

  https://salsa.debian.org/philh/debian-installer/-/jobs/5193962

which works by finding out the version that's actually available, and
using whatever it is (in this case, it used 6.6.11).

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

salsa CI

[Samuel Thibault]

Hello,

The CI on salsa doesn't manage to build the debian-installer package
because the signed linux 6.6.13 package is not available yet. Perhaps we
should turn these build-deps:

linux-image-6.6.13-amd64 [amd64],
linux-image-6.6.13-arm64 [arm64],
linux-image-6.6.13-686 [i386],
linux-image-6.6.13-686-pae [i386],

into

linux-image-6.6.13-amd64 [amd64] | linux-image-6.6.13-amd64-unsigned 
[amd64],
linux-image-6.6.13-arm64 [arm64] | linux-image-6.6.13-arm64-unsigned 
[arm64],
linux-image-6.6.13-686 [i386] | linux-image-6.6.13-686-unsigned [i386],
linux-image-6.6.13-686-pae [i386] | linux-image-6.6.13-686-pae-unsigned 
[i386],

?

Samuel

changed salsa-ci configuration for flash-kernel

[Vagrant Cascadian]

The default salsa-ci pipeline for installer-team projects has always
failed for flash-kernel, probably because flash-kernel builds no
arch:amd64 packages.

I recently merged a salsa-ci configuration from josch and switched
flash-kernel to use a a salsa-ci pipeline that used arm64 runners
instead, so that the pipelines could actually succeed.

Now I have a bunch of green checkmarks for flash-kernel, yay!

I do not know what, if anything, might be missing from this compared to
the usual salsa-ci jobs for installer projects, so I figured it was
worth mentioning here.

Holler if this change broke anything!

live well,
  vagrant


signature.asc
Description: PGP signature

Enabling salsa-CI on more installer-team repos

[Philip Hands]

Hi,

I've been working on making the salsa-CI pipeline[1] work well for
udebs, and now have it setup so that most udeb repos can have this
enabled by simply setting a configuration option[2], without needing to
touch the contents of the repository.

So far I've only done that to repos that I was working on anyway, using
that as an opportunity to make sure that the packages built and the
tests passed etc. but it's gone well in almost all cases recently, so
I'd like to roll it out more widely.

Enabling this on repos that are currently not configured to do CI is
pretty harmless, in that even if it doesn't work for a repo, it will be
no worse than the current state of not having CI running at all.

The most obvious user-visible change that would go along with this is
that people that push things to the repos will start to get emails about
the success/failure of the jobs as they run, which could amount to quite
a few emails in some cases.

Notifications about such things are something that one can tune in
gitlab, and one can use a different email address for notifications, so
they don't have to land in you inbox, or even be sent, as you prefer.

The other aspect of having CI running that's been visible is
IRC notifications to #debian-boot via KGB, which has been a bit of a
problem, because the default setup on quite a few of our repos was to
have job_event notifications selected.  This has been sitting quietly
waiting for anyone to be brave enough to add a debian/salsa-ci.yml file
to the repo. at which point every push generates a metric tonne of
drivel into the channel until someone (e.g. me ;-) ) turns it off.

This has now been fixed in a two-pronged attack:

  1) Ganneff has helpfully setup a new IRC channel where these messages
 can be sent: #di-ci

  2) Today I re-configure almost all installer-team repos to have 2 KGB
 configs, one for the normal push/merge/comment notifications, which
 still go to #debian-boot, the other sending pipeline_event messages
 (currently only success or failure messages) to #di-ci

So, the questions are:

  Is that sufficient to now permit me to enable CI for the rest of the
  installer-team repos?

  Would people prefer that was done in batches (say, 10 repos at a time)
  to see how it goes, or would you prefer to know that (pretty-much) all
  our repos will have CI enabled on them, awaiting your next push (so
  you don't need to configure it yourself)

BTW There are bound to be a few repos where the default setup doesn't
quite fit, and there needs to be a debian/salsa-ci.yml file with some
extra settings. Generally, that's things like disabling tests that were
not going to work for that repo for some reason, or telling it that the
build takes more than an hour (generally not a udeb issue).

Cheers, Phil.

[1] https://salsa.debian.org/salsa-ci-team/pipeline/
[2] Setting the 'CI/CD configuration file' option to:
   trigger_b2r.yml@installer-team/branch2repo
See:  https://wiki.debian.org/DebianInstaller/Salsa-CI
and:  https://salsa.debian.org/installer-team/branch2repo/
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Re: enabling salsa-CI pipeline

[Philip Hands]

Michael Kesper  writes:

>> I'm sure there are ways of scripting the above using the gitlab API, so
>> once I've discovered how I'll think about applying similar changes to
>> all our repos -- if you already know how to prod the API, feel free to
>> tell me.
>
> Did you ask on #salsaci?

Not at all -- I was planning on doing the research when I have the time
to do the updates, and only after having given people a chance to tell
me why they hate the idea.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Re: enabling salsa-CI pipeline

[Michael Kesper]

Hi Phil,

On 12.07.19 15:08, Philip Hands wrote:
> Hi All,
> 
> I just configured salsa-CI pipeline for 'user-setup', and it seems to
> work, so I thought I'd say what was involved so that others can do the
> same for other bits of d-i in a similar manner ... if they get to it
> before I do (and also so I don't forget things next time I do it :-) ).
> 
> The first thing to do is configure the 'Custom CI config path' to be:

A detailed description can be found under:
https://salsa.debian.org/salsa-ci-team/pipeline

[...]
 
> I'm sure there are ways of scripting the above using the gitlab API, so
> once I've discovered how I'll think about applying similar changes to
> all our repos -- if you already know how to prod the API, feel free to
> tell me.

Did you ask on #salsaci?
Afaik there is currently no mailing list re salsa-ci.

Bye
Michael




signature.asc
Description: OpenPGP digital signature

enabling salsa-CI pipeline

[Philip Hands]

Hi All,

I just configured salsa-CI pipeline for 'user-setup', and it seems to
work, so I thought I'd say what was involved so that others can do the
same for other bits of d-i in a similar manner ... if they get to it
before I do (and also so I don't forget things next time I do it :-) ).

The first thing to do is configure the 'Custom CI config path' to be:

  debian/salsa-ci.yml

(which can be found under Settings / CI/CD / General in the web UI)

Then you should ensure that the KGB webhook isn't going to be too
chatty, by unselecting the 'Job Events' setting in the webhook.  We
probably only want to hear about the pipelines when they finish as well,
so adding this to the URL will do that:

  ;pipeline_only_status=failed;pipeline_only_status=success

(these settings are found under Settings / Integrations )

Once that's all correct, you can add the configuration file, thus:

  
https://salsa.debian.org/installer-team/user-setup/blob/f02faef05ecf63cd0817624307839401af81805b/debian/salsa-ci.yml

and you should see it pop into life in IRC.

I'm sure there are ways of scripting the above using the gitlab API, so
once I've discovered how I'll think about applying similar changes to
all our repos -- if you already know how to prod the API, feel free to
tell me.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature