Re: salsa CI

[Cyril Brulebois]

Samuel Thibault  (2024-01-23):
> The CI on salsa doesn't manage to build the debian-installer package
> because the signed linux 6.6.13 package is not available yet. Perhaps we
> should turn these build-deps:
> 
>   linux-image-6.6.13-amd64 [amd64],
>   linux-image-6.6.13-arm64 [arm64],
>   linux-image-6.6.13-686 [i386],
>   linux-image-6.6.13-686-pae [i386],
> 
> into
> 
>   linux-image-6.6.13-amd64 [amd64] | linux-image-6.6.13-amd64-unsigned 
> [amd64],
>   linux-image-6.6.13-arm64 [arm64] | linux-image-6.6.13-arm64-unsigned 
> [arm64],
>   linux-image-6.6.13-686 [i386] | linux-image-6.6.13-686-unsigned [i386],
>   linux-image-6.6.13-686-pae [i386] | linux-image-6.6.13-686-pae-unsigned 
> [i386],
> 
> ?

udebs are built from the linux-signed-* packages, so that doesn't help
at all.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: salsa CI

[Philip Hands]

Samuel Thibault  writes:

> For me it's fine for CI to fall back to the previous kernel for most
> jobs of the pipeline. I guess we'd still want a build job in the
> pipeline that sticks with the requested version, so that we notice in
> case that's not working, without breaking the entire CI pipeline.

That's a good idea -- I've done something slightly different, which is
to add an 'ABI-check' job which fails if we are in version-skew:

  
https://salsa.debian.org/philh/debian-installer/-/commit/3b862d27f3ca07e840c0f1a44e42fd26ab69b176

I think that's the finishing touch that makes me happy to merge this.

If you can tell me how the *-unsigned version of the build is actually
supposed to succeed (my attempt failed at the point where it tries to
get hold of the matching modules, where the new version was not yet
available), then I'll happily add that as an additional condition (and
will also check in the check job to see if that's what is going on).

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

Re: salsa CI

[Samuel Thibault]

Philip Hands, le mar. 23 janv. 2024 17:52:57 +0100, a ecrit:
> Samuel Thibault  writes:
> 
> > Philip Hands, le mar. 23 janv. 2024 16:27:12 +0100, a ecrit:
> >> Samuel Thibault  writes:
> >> 
> >> > Hello,
> >> >
> >> > The CI on salsa doesn't manage to build the debian-installer package
> >> > because the signed linux 6.6.13 package is not available yet.
> >> 
> >> Is the thing you want to:
> >>   a) be able to build d-i on salsa even when we're in version skew,
> >> or
> >>   b) do you want to be able to test with the latest version, whether it is 
> >> signed or not?
> >
> > b)
> >
> > Normally the bump in debian-installer comes about the same time as the
> > linux upload. But there is the period between the linux upload and the
> > linux-signed upload during which we don't really know whether we want to
> > bump or not. Adding the alternative between non-signed and signed as I
> > proposed would allow to be fine with either, while making sure it's the
> > signed version which is used on buildds.
> 
> Ah, fair enough.
> 
> I guess in that case I'll need to adjust what I'm doing to detect the
> available versions of kernel that I'm looking for in that patch.
> 
> If you're only worried about builds on salsa-CI, same approach as used
> in my MR ought to work,

Indeed.

It however doesn't fix the build on my laptop without some change :)

> and then one could perhaps control which kernel
> is selected via variables, or perhaps defaulting to the unsigned kernel
> (if available) would work for my use-case too, in which case I could
> just add that as a feature.
> 
> The MR's here BTW: https://deb.li/3hHY2
> 
> BTW would it actually cause you a problem for the build to work, despite
> the kernel being unavailable (e.g. by falling back to the previous
> version)?

For me it's fine for CI to fall back to the previous kernel for most
jobs of the pipeline. I guess we'd still want a build job in the
pipeline that sticks with the requested version, so that we notice in
case that's not working, without breaking the entire CI pipeline.

Samuel

Re: salsa CI

[Philip Hands]

Samuel Thibault  writes:

> Philip Hands, le mar. 23 janv. 2024 16:27:12 +0100, a ecrit:
>> Samuel Thibault  writes:
>> 
>> > Hello,
>> >
>> > The CI on salsa doesn't manage to build the debian-installer package
>> > because the signed linux 6.6.13 package is not available yet.
>> 
>> Is the thing you want to:
>>   a) be able to build d-i on salsa even when we're in version skew,
>> or
>>   b) do you want to be able to test with the latest version, whether it is 
>> signed or not?
>
> b)
>
> Normally the bump in debian-installer comes about the same time as the
> linux upload. But there is the period between the linux upload and the
> linux-signed upload during which we don't really know whether we want to
> bump or not. Adding the alternative between non-signed and signed as I
> proposed would allow to be fine with either, while making sure it's the
> signed version which is used on buildds.

Ah, fair enough.

I guess in that case I'll need to adjust what I'm doing to detect the
available versions of kernel that I'm looking for in that patch.

If you're only worried about builds on salsa-CI, same approach as used
in my MR ought to work, and then one could perhaps control which kernel
is selected via variables, or perhaps defaulting to the unsigned kernel
(if available) would work for my use-case too, in which case I could
just add that as a feature.

The MR's here BTW: https://deb.li/3hHY2

BTW would it actually cause you a problem for the build to work, despite
the kernel being unavailable (e.g. by falling back to the previous
version)?

If so, I guess a variable is required to stop that behaviour.

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

Re: salsa CI

[Samuel Thibault]

Philip Hands, le mar. 23 janv. 2024 16:27:12 +0100, a ecrit:
> Samuel Thibault  writes:
> 
> > Hello,
> >
> > The CI on salsa doesn't manage to build the debian-installer package
> > because the signed linux 6.6.13 package is not available yet.
> 
> Is the thing you want to:
>   a) be able to build d-i on salsa even when we're in version skew,
> or
>   b) do you want to be able to test with the latest version, whether it is 
> signed or not?

b)

Normally the bump in debian-installer comes about the same time as the
linux upload. But there is the period between the linux upload and the
linux-signed upload during which we don't really know whether we want to
bump or not. Adding the alternative between non-signed and signed as I
proposed would allow to be fine with either, while making sure it's the
signed version which is used on buildds.

Samuel

Re: salsa CI

[Philip Hands]

Samuel Thibault  writes:

> Hello,
>
> The CI on salsa doesn't manage to build the debian-installer package
> because the signed linux 6.6.13 package is not available yet.

Is the thing you want to:
  a) be able to build d-i on salsa even when we're in version skew,
or
  b) do you want to be able to test with the latest version, whether it is 
signed or not?

If what you're after a), I have a patch that does that, as you can see
here:

  https://salsa.debian.org/philh/debian-installer/-/jobs/5193962

which works by finding out the version that's actually available, and
using whatever it is (in this case, it used 6.6.11).

Cheers, Phil.
-- 
Philip Hands -- https://hands.com/~phil


signature.asc
Description: PGP signature

Re: Salsa

[Steve McIntyre]

On Sun, May 13, 2018 at 05:14:49PM +0100, Steve McIntyre wrote:
>On Sun, May 13, 2018 at 06:05:53PM +0200, Cyril Brulebois wrote:
>
>>Can't we just go for that instead, which would have the benefit of
>>matching the source package name?
>>  https://salsa.debian.org/installer-team/installation-guide
>
>Good call, yes. That makes much more sense. I'll happily move it if
>you like, unless you'd rather...?

I've done it, as agreed in irc.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"Further comment on how I feel about IBM will appear once I've worked out
 whether they're being malicious or incompetent. Capital letters are forecast."
 Matthew Garrett, http://www.livejournal.com/users/mjg59/30675.html

Re: Salsa

[Steve McIntyre]

On Sun, May 13, 2018 at 06:05:53PM +0200, Cyril Brulebois wrote:
>Hey,
>
>Looking at names…
>
>Steve McIntyre  (2018-05-09):
>> Yup. In fact, at this point I declare the conversion to be as done as
>> it's going to be. I've just imported these test repos into the
>> installer-team area:
>> 
>>  https://salsa.debian.org/installer-team/d-i
>
>I guess this is fair, even if people might confuse that with the
>(real) debian-installer package/repository.

ACK.

>>  https://salsa.debian.org/installer-team/di-i-manual
>
>This seems to be named this way, now?
>  https://salsa.debian.org/installer-team/d-i-manual

Gah, yes - the name I wrote there was a typo!

>Can't we just go for that instead, which would have the benefit of
>matching the source package name?
>  https://salsa.debian.org/installer-team/installation-guide

Good call, yes. That makes much more sense. I'll happily move it if
you like, unless you'd rather...?

>>  https://salsa.debian.org/installer-team/di-i-archive
>
>Apparently named https://salsa.debian.org/installer-team/d-i-archive
>now, which looks fine if we keep the d-i name for the old svn top-level
>(see first point).

Again, same typo. Late night cut and paste... :-)

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
  Getting a SCSI chain working is perfectly simple if you remember that there
  must be exactly three terminations: one on one end of the cable, one on the
  far end, and the goat, terminated over the SCSI chain with a silver-handled
  knife whilst burning *black* candles. --- Anthony DeBoer

Re: Salsa

[Cyril Brulebois]

Hey,

Looking at names…

Steve McIntyre  (2018-05-09):
> Yup. In fact, at this point I declare the conversion to be as done as
> it's going to be. I've just imported these test repos into the
> installer-team area:
> 
>  https://salsa.debian.org/installer-team/d-i

I guess this is fair, even if people might confuse that with the
(real) debian-installer package/repository.

>  https://salsa.debian.org/installer-team/di-i-manual

This seems to be named this way, now?
  https://salsa.debian.org/installer-team/d-i-manual

Can't we just go for that instead, which would have the benefit of
matching the source package name?
  https://salsa.debian.org/installer-team/installation-guide

>  https://salsa.debian.org/installer-team/di-i-archive

Apparently named https://salsa.debian.org/installer-team/d-i-archive
now, which looks fine if we keep the d-i name for the old svn top-level
(see first point).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Salsa

[Cyril Brulebois]

Steve McIntyre  (2018-05-09):
> AFAICS we still need to (in rough order):
> 
>  1. Disable svn commits
> 
>  2. Tweak the translation workflow to push changes to
>  d-i.git/packages/po, instead of d-i in svn.
> 
>  3. Change the checkouts of d-i on the buildds etc. to use git?

I don't think we have svn checkouts anywhere but dillon. There's nothing
on buildds since we're building d-i from porterboxes. Thinking of it,
they need manual pointing to salsa, and I don't think latest builds
reflected what was in master… I'll go and do that now.

>  4. Update docs to point to the new repos
> 
> Anything else?

Docs are a big part of this transition, especially for translators.
We'll need appropriate communication and possibly hand holding… That
might put a heavy burden on translation coordinators (see Christian's
fears regarding this big transition, and the historical reluctance to
moving off of svn).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Salsa

[Steve McIntyre]

On Sun, May 06, 2018 at 04:56:49PM +0100, Ian Campbell wrote:
>On Sun, 2018-05-06 at 01:43 +0100, Steve McIntyre wrote:
>> OK, so I've tried --aggressive too now, and wow does it make a big
>> difference.
>
>AIUI amongst other things --aggressive forces a full repack of the
>repo, which optimises the delta compression in the pack files. You
>could probably achieve most of the effect with `git repack`. It's
>probably diminishing returns at this point but there are options there
>to make it spend (lots) more time/memory to make things even smaller.

Yup.

>
>>  I've tried using it on the d-i.git and d-i-manual.git
>> repos and the difference is *huge*:
>> 
>> # test2, previous results:
>> $ du -s test*
>> 613888  test1-bare.git
>> 3653432 test2-bare.git
>> 714336  test2-manual-bare.git

$ du -s test4-*
138500  test4-archive-bare.git
174504  test4-bare.git
38112   test4-manual-bare.git

>That's much better ;-)

Yup. In fact, at this point I declare the conversion to be as done as
it's going to be. I've just imported these test repos into the
installer-team area:

 https://salsa.debian.org/installer-team/d-i
 https://salsa.debian.org/installer-team/di-i-manual
 https://salsa.debian.org/installer-team/di-i-archive

I've got a local backup of the svn repos still, and I'll keep them for
a while so if we find I've screwed up we can still recover. I'll keep
these for a couple of months.

I've just added the one commit that's happened in the last week
(r71079 from Holger), and all seems good. I'm just about to add my
changes for .mrconfig etc. now.

AFAICS we still need to (in rough order):

 1. Disable svn commits

 2. Tweak the translation workflow to push changes to
 d-i.git/packages/po, instead of d-i in svn.

 3. Change the checkouts of d-i on the buildds etc. to use git?
 
 4. Update docs to point to the new repos

Anything else?

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
Can't keep my eyes from the circling sky,
Tongue-tied & twisted, Just an earth-bound misfit, I...

Re: Salsa

[Ian Campbell]

On Sun, 2018-05-06 at 01:43 +0100, Steve McIntyre wrote:
> OK, so I've tried --aggressive too now, and wow does it make a big
> difference.

AIUI amongst other things --aggressive forces a full repack of the
repo, which optimises the delta compression in the pack files. You
could probably achieve most of the effect with `git repack`. It's
probably diminishing returns at this point but there are options there
to make it spend (lots) more time/memory to make things even smaller.

>  I've tried using it on the d-i.git and d-i-manual.git
> repos and the difference is *huge*:
> 
> # test2, previous results:
> $ du -s test*
> 613888  test1-bare.git
> 3653432 test2-bare.git
> 714336  test2-manual-bare.git

That's much better ;-)

Ian.

Re: Salsa

[Steve McIntyre]

On Fri, May 04, 2018 at 05:27:47PM +0100, Steve McIntyre wrote:
>On Fri, May 04, 2018 at 05:21:53PM +0100, Ian Campbell wrote:
>
>>If it was a different tool maybe a `git gc --aggressive` will repack
>>(and thus compact/delta-compress) the bigger one? (my hypothesis is
>>that maybe the first run did it automatically and the second didn't)
>
>ACK, that was my initial thought. The first tool does a "git gc" after
>each revision has been converted. The second one doesn't. But I did
>"git gc" and "git repack -a -d" on the second afterwards. It reduced
>from ~3.8G to the ~3.5G you see here.

OK, so I've tried --aggressive too now, and wow does it make a big
difference. I've tried using it on the d-i.git and d-i-manual.git
repos and the difference is *huge*:

# test2, previous results:
$ du -s test*
613888  test1-bare.git
3653432 test2-bare.git
714336  test2-manual-bare.git

# test3, bare clones from new repos after git gc --aggressive:
$ du -s test3-*
261408  test3-bare.git
37908   test3-manual-bare.git

I've continued on and added an extra d-i-archive.git repo, splitting
out lots of older stuff that's already been split into new repos as
Bastian said (see attached config). After "git gc --aggressive" on
each and a bare clone, I now get:

$ du -s test4-*
138500  test4-archive-bare.git
174504  test4-bare.git
38112   test4-manual-bare.git

I think we're getting much closer to a reasonable solution here! I've
uploaded each of these to salsa again, in case anybody else would like
to play with them:

https://salsa.debian.org/93sam/d-i-test4-archive.git
https://salsa.debian.org/93sam/d-i-test4.git
https://salsa.debian.org/93sam/d-i-test4-manual.git

I've not yet applied my patch to d-i-test4.git to allow for building -
see attached.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
< sladen> I actually stayed in a hotel and arrived to find a post-it
  note stuck to the mini-bar saying "Paul: This fridge and
  fittings are the correct way around and do not need altering"
#
# Declare the repositories we know about:
#

create repository d-i.git
end repository

create repository d-i-manual.git
end repository

create repository d-i-archive.git
end repository

#
# Declare the rules
# Note: rules must end in a slash
#

# Ignore this dir:
# Note that rules are applied in order of appearance, so this rule
# must appear before the generic rules
match /people/
end match

match /README
end match

# Split out the manual
match /trunk/manual/
  repository d-i-manual.git
  branch master
end match
match /branches/([^/]+)/manual/
  repository d-i-manual.git
  branch \1
end match
match /tags/([^/]+)/manual/
  repository d-i-manual.git
  branch refs/tags/\1
end match

# /scripts/ and /packages/po/ belong in the main repo
match /trunk/(doc|scripts|packages/po)/
  repository d-i.git
  branch master
  prefix \1/
end match
match /branches/([^/]+)/(doc|scripts|packages/po)/
  repository d-i.git
  branch \1
  prefix \2/
end match
match /tags/([^/]+)/(doc|scripts|packages/po)/
  repository d-i.git
  branch refs/tags/\1
  prefix \2/
end match

# Grab out all of the old content that's now been split ouy to
# separate modules, including anything else under /packages/ that
# we've not already grabbed in /packages/po/
match 
/trunk/(anna|build|doc|installer|kernel.*|lib.*|main-menu|netboot.*|packages-build|partman|retriever|rootskel|tools|utils|win32-loader|packages)/
  repository d-i-archive.git
  branch master
  prefix \1/
end match
match 
/branches/([^/]+)/(anna|build|doc|installer|kernel.*|lib.*|main-menu|netboot.*|packages-build|partman|retriever|rootskel|tools|utils|win32-loader|packages)/
  repository d-i-archive.git
  branch \1
  prefix \2/
end match
match 
/tags/([^/]+)/(anna|build|doc|installer|kernel.*|lib.*|main-menu|netboot.*|packages-build|partman|retriever|rootskel|tools|utils|win32-loader|packages)/
  repository d-i-archive.git
  branch refs/tags/\1
  prefix \2/
end match

# Finally, anything else goes in the main repo. This should also pick up
# bare files like /README and .mrconfig
match /trunk/
  repository d-i.git
  branch master
end match
match /branches/([^/]+)/
  repository d-i.git
  branch \1
end match
match /tags/([^/]+)/
  repository d-i.git
  branch refs/tags/\1
end match

#
#match /project2/trunk/
#  repository project2
#  branch master
#end match
#
# Note how we can use regexp to capture the repository name
#match /([^/]+)/branches/([^/]+)/
#  repository \1
#  branch \2
#end match

# No tag processing
>From bb1df3b30ca83634b9fedb12fb32fb1244c11d9b Mon Sep 17 00:00:00 2001
From: Steve McIntyre 
Date: Fri, 4 May 2018 14:47:17 +0100
Subject: [PATCH] Tweaks for git on salsa

---
 .mrconfig |  2 +-
 scripts/git-setup | 19 ++-
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/.mrconfig b/.mrconfig
index 22067fcb0e..f17260caad 100644
--- a/.mrconfig
+++ b/.mrconfig
@@ -7,7 +7,7 @@
 # see scripts/git-setup.
 
 [.]

Re: Salsa

[Steve McIntyre]

On Fri, May 04, 2018 at 05:21:53PM +0100, Ian Campbell wrote:
>On Thu, 2018-05-03 at 22:10 +0100, Steve McIntyre wrote:
>> The outputs from this run
>> were a surprising amount bigger than my first test repo, as the
>> following bare clones from each will show:
>> 
>> tack:/tmp$ du -s test*
>> 613888  test1-bare.git
>> 3653432 test2-bare.git
>> 714336  test2-manual-bare.git
>> 
>> I've not worked out why yet.
>
>Was this with a completely different tool or the same tool with
>different settings/wrappers?

Different tools.

>If it was a different tool maybe a `git gc --aggressive` will repack
>(and thus compact/delta-compress) the bigger one? (my hypothesis is
>that maybe the first run did it automatically and the second didn't)

ACK, that was my initial thought. The first tool does a "git gc" after
each revision has been converted. The second one doesn't. But I did
"git gc" and "git repack -a -d" on the second afterwards. It reduced
from ~3.8G to the ~3.5G you see here.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
Welcome my son, welcome to the machine.

Re: Salsa

[Ian Campbell]

On Thu, 2018-05-03 at 22:10 +0100, Steve McIntyre wrote:
> The outputs from this run
> were a surprising amount bigger than my first test repo, as the
> following bare clones from each will show:
> 
> tack:/tmp$ du -s test*
> 613888  test1-bare.git
> 3653432 test2-bare.git
> 714336  test2-manual-bare.git
> 
> I've not worked out why yet.

Was this with a completely different tool or the same tool with
different settings/wrappers?

If it was a different tool maybe a `git gc --aggressive` will repack
(and thus compact/delta-compress) the bigger one? (my hypothesis is
that maybe the first run did it automatically and the second didn't)

Ian.

Re: Salsa

[Steve McIntyre]

On Fri, May 04, 2018 at 03:35:46PM +0200, Bastian Blank wrote:
>On Fri, May 04, 2018 at 02:08:40PM +0100, Steve McIntyre wrote:
>> On Fri, May 04, 2018 at 10:44:00AM +0200, Bastian Blank wrote:
>> >Does that mean you export all the old cruft from
>> >/trunk/{build,kernel,tools,utils} to this new git repo?
>> So far, yes. As I said, "For now, I've not filtered
>> any branches or anything". If people are sure that this is junk and
>> can be dropped, that's easily done.
>
>I would assume only manual, packages/po and scripts have any value to
>be preserved.  Everything else was already exported into different a
>gazillion different git repos.

ACK, fair enough. I was wondering if/how that had happened. I'll
re-run and dump that lot off into a "d-i-archive" repo to see how it
looks.

>For .mrconfig the history is pretty pointless.

Sure, but it's tiny - hardly worth bothering about. :-)

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"I've only once written 'SQL is my bitch' in a comment. But that code 
 is in use on a military site..." -- Simon Booth

Re: Salsa

[Bastian Blank]

On Fri, May 04, 2018 at 02:08:40PM +0100, Steve McIntyre wrote:
> On Fri, May 04, 2018 at 10:44:00AM +0200, Bastian Blank wrote:
> >Does that mean you export all the old cruft from
> >/trunk/{build,kernel,tools,utils} to this new git repo?
> So far, yes. As I said, "For now, I've not filtered
> any branches or anything". If people are sure that this is junk and
> can be dropped, that's easily done.

I would assume only manual, packages/po and scripts have any value to
be preserved.  Everything else was already exported into different a
gazillion different git repos.

For .mrconfig the history is pretty pointless.

Bastian

-- 
A Vulcan can no sooner be disloyal than he can exist without breathing.
-- Kirk, "The Menagerie", stardate 3012.4

Re: Salsa

[Steve McIntyre]

On Thu, May 03, 2018 at 10:10:28PM +0100, Steve McIntyre wrote:

...

>tack:/tmp$ du -s test*
>613888  test1-bare.git
>3653432 test2-bare.git
>714336  test2-manual-bare.git
>
>I've not worked out why yet. In case people might find them useful (or
>maybe find time to have a look!), I've pushed these new test repos to
>salsa too:
>
>  https://salsa.debian.org/93sam/d-i-test2
>  https://salsa.debian.org/93sam/d-i-test2-manual
>
>Suggestions on what else we might want to separate or prune here would
>be helpful. I don't really like the idea of losing our history. We
>could maybe prune old branches, but I'm not sure it'll save much. Or
>am I worrying too much about the repo sizes already?

Just to confirm - I managed to build d-i last night using the
d-i-test2 repo as a base, just needing to tweak .mrconfig (trivial)
and scripts/git-setup (less trivial). I've not tested the output, but
it all looked sane.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
Dance like no one's watching. Encrypt like everyone is.
 - @torproject

Re: Salsa

[Steve McIntyre]

On Fri, May 04, 2018 at 10:44:00AM +0200, Bastian Blank wrote:
>On Thu, May 03, 2018 at 10:10:28PM +0100, Steve McIntyre wrote:
>> match /trunk/
>>   repository d-i.git
>>   branch master
>> end match
>
>Does that mean you export all the old cruft from
>/trunk/{build,kernel,tools,utils} to this new git repo?

So far, yes. As I said, "For now, I've not filtered
any branches or anything". If people are sure that this is junk and
can be dropped, that's easily done.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"Yes, of course duct tape works in a near-vacuum. Duct tape works
 anywhere. Duct tape is magic and should be worshipped."
   -― Andy Weir, "The Martian"

Re: Salsa

[Bastian Blank]

On Thu, May 03, 2018 at 10:10:28PM +0100, Steve McIntyre wrote:
> match /trunk/
>   repository d-i.git
>   branch master
> end match

Does that mean you export all the old cruft from
/trunk/{build,kernel,tools,utils} to this new git repo?

Bastian

-- 
Each kiss is as the first.
-- Miramanee, Kirk's wife, "The Paradise Syndrome",
   stardate 4842.6

Re: Salsa

[Steve McIntyre]

YA update

On Wed, May 02, 2018 at 02:52:09PM +0100, Steve McIntyre wrote:
>Update again...
>
>On Sun, Apr 29, 2018 at 01:12:51PM +0100, Steve McIntyre wrote:
>>
>>So, an initial run of svn2git just ignoring the people directory took
>>11.5h here, and gave me a .git dir of ~680M. That's quite
>>big. I'm re-doing it now with an "authors" file in place, to get
>>something more usable.
>
>If anybody would like to play with this, I've just uploaded it to
>
>https://salsa.debian.org/93sam/d-i-test1
>
>and I'll leave it up there for now.
>
>>Discussing with KiBi on IRC last night, we're thinking that it's
>>probably worth splitting the manual off into a separate
>>project/repo. I'll try doing that too, and see what we get.
>
>To make things go much faster, I grabbed a copy of the svn repo
>directly and I've been running with that. It goes *much* more quickly
>due to the latency reduction on each revision checkout, but it
>reliably fails with:
>
>Name does not refer to a filesystem directory: Failure opening
>'/trunk/installer/build/pkg-lists/netboot/mipsel/sb1-swarm-bn.cfg':
>'/trunk/installer/build/pkg-lists/netboot/mipsel' is not a directory
>in filesystem '48c42b26-1dd6-0310-b98f-a58d8bce7237' at
>/usr/share/perl5/Git/SVN/Ra.pm line 312
>
>at r35516. Joy. I've not modified the svn data files in any way, and
>this worked from alioth...
>
>Having looked online, I find various recommendations to avoid using
>this version of svn2git (which is a simple wrapper around
>git-svn). I'm now trying the svn2git tool the KDE people used for
>migration:
>
>  https://github.com/svn-all-fast-export/svn2git.git

After some fighting with config, I've used this tool with the attached
configs - a rules to control what goes where, and a mapping file for
username -> name/email lookups.

It is *massively* faster than the first tool, something like a factor
of 15-20x. That makes it much more feasible to run this a few times
with different configs, to compare results. For now, I've not filtered
any branches or anything, but I've ignored /people and /README and
moved the manual out into a separate repo. The outputs from this run
were a surprising amount bigger than my first test repo, as the
following bare clones from each will show:

tack:/tmp$ du -s test*
613888  test1-bare.git
3653432 test2-bare.git
714336  test2-manual-bare.git

I've not worked out why yet. In case people might find them useful (or
maybe find time to have a look!), I've pushed these new test repos to
salsa too:

  https://salsa.debian.org/93sam/d-i-test2
  https://salsa.debian.org/93sam/d-i-test2-manual

Suggestions on what else we might want to separate or prune here would
be helpful. I don't really like the idea of losing our history. We
could maybe prune old branches, but I'm not sure it'll save much. Or
am I worrying too much about the repo sizes already?

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
You lock the door
And throw away the key
There's someone in my head but it's not me 
93sam = Steve McIntyre <93...@debian.org>
adioe3-guest = Armin Beširović 
adn = Mohammed Adnène Trojette 
adn-guest = Mohammed Adnène Trojette 
adorosh-guest = Andrei Darashenka 
adrianorg = Adriano Rafael Gomes 
adrianorg-guest = Adriano Rafael Gomes 
agx = Guido Guenther 
aiet-guest = Aiet Kolkhi  
aigarius = Aigars Mahinovs 
ajenbo-guest = Anders Jenbo 
ajt = Anthony Towns 
akumar = Kumar Appaiah 
albbas-guest = Børre Gaup 
alphix-guest = David Härdeman 
amanpreet-guest = A S Alam 
amp-guest = Andrei Popescu 
andersee = Erik B. Andersen 
andread-guest = Andreas Dahl 
andreas = Andreas Schuldei 
andred-guest = André Dahlqvist 
andrelop = Andre Luis Lopes 
anmar-guest = Anmar Oueja 
aph = Adam Di Carlo 
appaji = Giridhar Appaji Nag Yasa 
appaji-guest = Y Giridhar Appaji Nag  
arangel-guest = Arangel Angov 
arashb-guest = Arash Bijanzadeh 
arief-guest = Arief S Fitrianto 
arjunaraoc-guest = Arjuna Rao Chavala 
athornton-guest = Adam Thornton 
aurel32 = Aurelien Jarno 
baptiste-guest = Baptiste Jammet 
barbier = Denis Barbier 
bcollins = Ben Collins 
bdale = Bdale Garbee 
benh = Ben Hutchings 
berserker-guest = Pavel Piatruk 
bibar-guest = Denis ARNAUD 
blade = Eduard 

Re: Salsa

[Ben Hutchings]

On Wed, 2018-05-02 at 14:52 +0100, Steve McIntyre wrote:
[...]
> Having looked online, I find various recommendations to avoid using
> this version of svn2git (which is a simple wrapper around
> git-svn). I'm now trying the svn2git tool the KDE people used for
> migration:
> 
>   https://github.com/svn-all-fast-export/svn2git.git

This seems to be the right tool for weird Subversion repos - and
Subversion makes it easy to do weird things by accident!  I used it for
converting the kernel repo.  I had to do a small amount of post-
processing with git filter-branch.

If you need examples of dealing with weird stuff, see
https://salsa.debian.org/kernel-team/d-k-conversion

Ben.

-- 
Ben Hutchings
One of the nice things about standards is that
there are so many of them.


signature.asc
Description: This is a digitally signed message part

Re: Salsa

[Steve McIntyre]

Update again...

On Sun, Apr 29, 2018 at 01:12:51PM +0100, Steve McIntyre wrote:
>
>So, an initial run of svn2git just ignoring the people directory took
>11.5h here, and gave me a .git dir of ~680M. That's quite
>big. I'm re-doing it now with an "authors" file in place, to get
>something more usable.

If anybody would like to play with this, I've just uploaded it to

https://salsa.debian.org/93sam/d-i-test1

and I'll leave it up there for now.

>Discussing with KiBi on IRC last night, we're thinking that it's
>probably worth splitting the manual off into a separate
>project/repo. I'll try doing that too, and see what we get.

To make things go much faster, I grabbed a copy of the svn repo
directly and I've been running with that. It goes *much* more quickly
due to the latency reduction on each revision checkout, but it
reliably fails with:

Name does not refer to a filesystem directory: Failure opening
'/trunk/installer/build/pkg-lists/netboot/mipsel/sb1-swarm-bn.cfg':
'/trunk/installer/build/pkg-lists/netboot/mipsel' is not a directory
in filesystem '48c42b26-1dd6-0310-b98f-a58d8bce7237' at
/usr/share/perl5/Git/SVN/Ra.pm line 312

at r35516. Joy. I've not modified the svn data files in any way, and
this worked from alioth...

Having looked online, I find various recommendations to avoid using
this version of svn2git (which is a simple wrapper around
git-svn). I'm now trying the svn2git tool the KDE people used for
migration:

  https://github.com/svn-all-fast-export/svn2git.git

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"...In the UNIX world, people tend to interpret `non-technical user'
 as meaning someone who's only ever written one device driver." -- Daniel Pead

Re: Salsa

[Steve McIntyre]

Update...

On Sat, Apr 28, 2018 at 11:52:42PM +0100, Steve McIntyre wrote:
>
>Yup. I'm now working on the svn2git conversion for the top-level d-i
>svn repo. It's taking a while with ~70k commits to resolve! I'm
>explicitly ignoring the "people" dir for now, as most of those
>personal brances are ancient:
>
>$ svn ls -v svn://svn.debian.org/svn/d-i/people/ 
>  69848 adrianor  Apr 22  2015 ./
>  40600 xam   Sep 12  2006 alphix-guest/
>  43936 fjp   Jan 06  2007 barbier/
>  57427 bubulle   Jan 23  2009 bubulle/
>  50510 cjwatson  Dec 18  2007 cjwatson/
>  41238 eddyp-gu  Oct 04  2006 eddyp/
>  62833 fjp   Apr 01  2010 fjp/
>  40623 huggieSep 13  2006 huggie/
>  21996 jbailey   Sep 24  2004 jbailey/
>  60699 joeyh Sep 09  2009 joeyh/
>  48862 lunar Aug 03  2007 lunar/
>  41604 sferriol  Oct 12  2006 ninou/
>  40190 joeyh Aug 24  2006 otavio/
>  42674 pgeyleg-  Nov 17  2006 pema/
>  51136 philh Feb 01  2008 philh/
>  49104 rmh   Sep 01  2007 rmh/
>  69848 adrianor  Apr 22  2015 sandbox/
>  17800 sjogren   Jul 16  2004 sjogren/
>  63663 aurel32   Jul 06  2010 slackydeb/
>  41094 stappers  Sep 29  2006 stappers/
>  35463 tbm   Mar 16  2006 tbm/
>  65626 cjwatson  Nov 15  2010 waldi/
>  66442 wouterMar 13  2011 wouter/
>  41260 fjp   Oct 05  2006 xam/
>  49051 zinoviev  Aug 27  2007 zinoviev/
>
>If people want their old personal svn branches, shout now or they can
>grab them from the archive...

So, an initial run of svn2git just ignoring the people directory took
11.5h here, and gave me a .git dir of ~680M. That's quite
big. I'm re-doing it now with an "authors" file in place, to get
something more usable.

Discussing with KiBi on IRC last night, we're thinking that it's
probably worth splitting the manual off into a separate
project/repo. I'll try doing that too, and see what we get.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"...In the UNIX world, people tend to interpret `non-technical user'
 as meaning someone who's only ever written one device driver." -- Daniel Pead

Re: Salsa

[Steve McIntyre]

On Fri, Apr 27, 2018 at 04:20:26AM +0200, Cyril Brulebois wrote:
>Hi,
>
>I've just checked the list you provided below with what's in my local
>copy, deployed with .mrconfig:
>
>Steve McIntyre  (2018-04-26):
>> bak.debootstrap
>
>=> deleted as mentioned earlier.
>
>> d-i
>
>=> can go away (alioth generated for all git-based projects)?

Yup. It's empty, so I've killed it. We'll want it anyway for the
top-level d-i git repo, see below!

>> daily-build-logs
>
>=> repository is likely corrupted, not sure it's recoverable; it might
>   make sense to scratch it since its main purpose is to make it easy
>   to find what broke when daily builds regress. Most useful short term
>   anyway, IMHO.

Nod. Looks totally broken, so the import to salsa didn't work. Deleted
the salsa project.

>> kadit-playbooks
>> kadit
>
>=> both not mentioned on purpose

Nod.

>So I think we're (almost) good here.

Yup. I'm now working on the svn2git conversion for the top-level d-i
svn repo. It's taking a while with ~70k commits to resolve! I'm
explicitly ignoring the "people" dir for now, as most of those
personal brances are ancient:

$ svn ls -v svn://svn.debian.org/svn/d-i/people/ 
  69848 adrianor  Apr 22  2015 ./
  40600 xam   Sep 12  2006 alphix-guest/
  43936 fjp   Jan 06  2007 barbier/
  57427 bubulle   Jan 23  2009 bubulle/
  50510 cjwatson  Dec 18  2007 cjwatson/
  41238 eddyp-gu  Oct 04  2006 eddyp/
  62833 fjp   Apr 01  2010 fjp/
  40623 huggieSep 13  2006 huggie/
  21996 jbailey   Sep 24  2004 jbailey/
  60699 joeyh Sep 09  2009 joeyh/
  48862 lunar Aug 03  2007 lunar/
  41604 sferriol  Oct 12  2006 ninou/
  40190 joeyh Aug 24  2006 otavio/
  42674 pgeyleg-  Nov 17  2006 pema/
  51136 philh Feb 01  2008 philh/
  49104 rmh   Sep 01  2007 rmh/
  69848 adrianor  Apr 22  2015 sandbox/
  17800 sjogren   Jul 16  2004 sjogren/
  63663 aurel32   Jul 06  2010 slackydeb/
  41094 stappers  Sep 29  2006 stappers/
  35463 tbm   Mar 16  2006 tbm/
  65626 cjwatson  Nov 15  2010 waldi/
  66442 wouterMar 13  2011 wouter/
  41260 fjp   Oct 05  2006 xam/
  49051 zinoviev  Aug 27  2007 zinoviev/

If people want their old personal svn branches, shout now or they can
grab them from the archive...

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"I've only once written 'SQL is my bitch' in a comment. But that code 
 is in use on a military site..." -- Simon Booth

Re: Salsa

[Cyril Brulebois]

Hi,

Steve McIntyre  (2018-04-26):
> On Thu, Apr 26, 2018 at 04:10:40AM +0200, Cyril Brulebois wrote:
> >> bak.debootstrap
> >
> >That one should probably go away?
> 
> Yup, good call. Didn't see that one in the list, I just did '*.git'
> basically. Now deleted.

Thanks!

> >I've updated .mrconfig to point to the new location; I'll probably
> >remove the “deleted = true” entries as they are for packages in the
> >attic; and those are doubly deprecated as they're going to be
> >archived now anyway.
> 
> Yup.

Done. Current mr status outputs this last line:
| mr update: finished (100 ok)

I've also updated all Vcs-* fields in the master branch of each
repository.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Salsa

[Cyril Brulebois]

Hi,

I've just checked the list you provided below with what's in my local
copy, deployed with .mrconfig:

Steve McIntyre  (2018-04-26):
> bak.debootstrap

=> deleted as mentioned earlier.

> d-i

=> can go away (alioth generated for all git-based projects)?

> daily-build-logs

=> repository is likely corrupted, not sure it's recoverable; it might
   make sense to scratch it since its main purpose is to make it easy
   to find what broke when daily builds regress. Most useful short term
   anyway, IMHO.

> kadit-playbooks
> kadit

=> both not mentioned on purpose


So I think we're (almost) good here.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Salsa

[Steve McIntyre]

On Thu, Apr 26, 2018 at 01:04:55AM +0100, Steve McIntyre wrote:
>On Tue, Apr 24, 2018 at 11:16:04PM +0200, Cyril Brulebois wrote:
>>Hi,
>>
>>Chris Boot  (2018-04-24):
>>> I didn't receive much feedback from the -boot team since I created the
>>> Salsa project and did the initial member import, but I also didn't
>>> remind people about it after those first few emails.
>>
>>Yes, sorry for not having been more proactive on this topic. :(
>>
>>> FWIW I have been using GitLab both personally and at work very
>>> successfully for quite a while now with my generally Git-centric
>>> workflow. I have no recent experience with conversions from other VCS,
>>> and I do understand some people's concerns about whether GitLab is the
>>> best fit for Debian, but I strongly believe that whilst it may require
>>> a bit of a mindset readjustment it's the best thing out there for us.
>>
>>That's very fine. :) I was more wondering about tips and tricks for the
>>actual migration; but apparently Steve is happy to help with setting
>>things up with packages, so I would only have to deal with adapting the
>>l10n commit robot for new URLs.
>
>I've now migrated all 103(!) of the non-attic d-i repos across from
>alioth to salsa, following the pattern
>
> https://anonscm.debian.org/cgit/d-i/$REPO.git
>
>   to
>
> https://salsa.debian.org/installer-team/$REPO

Forgot last night; I've just pushed a merge request for AliothRewriter
for the 102 repos too. anonscm.d.o URLs should point to salsa shortly.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"This dress doesn't reverse." -- Alden Spiess

Re: Salsa

[Steve McIntyre]

On Thu, Apr 26, 2018 at 04:10:40AM +0200, Cyril Brulebois wrote:
>Steve McIntyre  (2018-04-26):
>
>> In each case, the old alioth repo will now reject pushes. The new
>> repos are each set up to use KGB to announce changes in #debian-boot
>> as previously. I've not (yet) spent any time on updating all the
>> debian/control files to point at salsa, or updated mr configs or
>> anything else.
>> 
>> The repos involved are:
>> 
>> […]
>> bak.debootstrap
>
>That one should probably go away?

Yup, good call. Didn't see that one in the list, I just did '*.git'
basically. Now deleted.

>I've updated .mrconfig to point to the new location; I'll probably remove the
>“deleted = true” entries as they are for packages in the attic; and those are
>doubly deprecated as they're going to be archived now anyway.

Yup.

>URLs on dillon (d-i.debian.org host) have been updated to point to salsa.
>
>A key has been generated there specifically for salsa, ~/.gitconfig has been
>updated to rewrite https into ssh to push using ssh, and I've set up a
>di-l10n-guest account as previously on alioth. I'm told we could be using
>deploy keys, but from a quick glance they seem to work on a project level
>(rather than on the group leve), and I'm not toying with salsa's API right
>away. Also, https://wiki.debian.org/Salsa/Doc#Deployment_keys is in FIXME
>mode. ;) If the di-l10n-guest account remains needed, we should adjust the
>mail address from one of mine to something shared across several people (not
>sure about using debian-boot@ for that…).

ACK.

>A manual l10n-sync script seems to be working just fine, given the log
>and the commits popping up on IRC. The only issue seems to be this, which is
>likely an issue with the package itself:
>| Everything up-to-date
>| - preseed
>|   - Run debconf-updatepo... failed.
>
>
>Migrating away from SVN is still an open task though.

Right. We should get that done next.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
  Armed with "Valor": "Centurion" represents quality of Discipline,
  Honor, Integrity and Loyalty. Now you don't have to be a Caesar to
  concord the digital world while feeling safe and proud.

Re: Salsa

[Christian PERRIER]

> I've updated .mrconfig to point to the new location; I'll probably remove the
> “deleted = true” entries as they are for packages in the attic; and those are
> doubly deprecated as they're going to be archived now anyway.
> 
> URLs on dillon (d-i.debian.org host) have been updated to point to salsa.

I guess I should mimic this in order to update my local copy (the one
where I build packages), right?

And figure out how I can now push my changes back to salsa (but,
again, I guess that looking at what you did on dillon under the d-i
role account is probably the thing to do)

/me in n00b mode:-)



signature.asc
Description: PGP signature

Re: Salsa

[Cyril Brulebois]

Hey,

Steve McIntyre  (2018-04-26):
> I've now migrated all 103(!) of the non-attic d-i repos across from
> alioth to salsa, following the pattern
> 
>  https://anonscm.debian.org/cgit/d-i/$REPO.git
> 
>to
> 
>  https://salsa.debian.org/installer-team/$REPO

Thanks!

> In each case, the old alioth repo will now reject pushes. The new
> repos are each set up to use KGB to announce changes in #debian-boot
> as previously. I've not (yet) spent any time on updating all the
> debian/control files to point at salsa, or updated mr configs or
> anything else.
> 
> The repos involved are:
> 
> […]
> bak.debootstrap

That one should probably go away?


I've updated .mrconfig to point to the new location; I'll probably remove the
“deleted = true” entries as they are for packages in the attic; and those are
doubly deprecated as they're going to be archived now anyway.

URLs on dillon (d-i.debian.org host) have been updated to point to salsa.

A key has been generated there specifically for salsa, ~/.gitconfig has been
updated to rewrite https into ssh to push using ssh, and I've set up a
di-l10n-guest account as previously on alioth. I'm told we could be using
deploy keys, but from a quick glance they seem to work on a project level
(rather than on the group leve), and I'm not toying with salsa's API right
away. Also, https://wiki.debian.org/Salsa/Doc#Deployment_keys is in FIXME
mode. ;) If the di-l10n-guest account remains needed, we should adjust the
mail address from one of mine to something shared across several people (not
sure about using debian-boot@ for that…).

A manual l10n-sync script seems to be working just fine, given the log
and the commits popping up on IRC. The only issue seems to be this, which is
likely an issue with the package itself:
| Everything up-to-date
| - preseed
|   - Run debconf-updatepo... failed.


Migrating away from SVN is still an open task though.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Salsa

[Steve McIntyre]

On Tue, Apr 24, 2018 at 11:16:04PM +0200, Cyril Brulebois wrote:
>Hi,
>
>Chris Boot  (2018-04-24):
>> I didn't receive much feedback from the -boot team since I created the
>> Salsa project and did the initial member import, but I also didn't
>> remind people about it after those first few emails.
>
>Yes, sorry for not having been more proactive on this topic. :(
>
>> FWIW I have been using GitLab both personally and at work very
>> successfully for quite a while now with my generally Git-centric
>> workflow. I have no recent experience with conversions from other VCS,
>> and I do understand some people's concerns about whether GitLab is the
>> best fit for Debian, but I strongly believe that whilst it may require
>> a bit of a mindset readjustment it's the best thing out there for us.
>
>That's very fine. :) I was more wondering about tips and tricks for the
>actual migration; but apparently Steve is happy to help with setting
>things up with packages, so I would only have to deal with adapting the
>l10n commit robot for new URLs.

I've now migrated all 103(!) of the non-attic d-i repos across from
alioth to salsa, following the pattern

 https://anonscm.debian.org/cgit/d-i/$REPO.git

   to

 https://salsa.debian.org/installer-team/$REPO

In each case, the old alioth repo will now reject pushes. The new
repos are each set up to use KGB to announce changes in #debian-boot
as previously. I've not (yet) spent any time on updating all the
debian/control files to point at salsa, or updated mr configs or
anything else.

The repos involved are:

anna
apt-setup
arcboot-installer
babelbox
bak.debootstrap
base-installer
bterm-unifont
busybox
cdebconf-entropy
cdebconf-terminal
cdebconf
cdrom-checker
cdrom-detect
cdrom-retriever
choose-mirror
clock-setup
console-setup
d-i
daily-build-logs
debian-installer-launcher
debian-installer-netboot-images
debian-installer-utils
debian-installer
debootstrap
desktop-chooser
devicetype-detect
dh-di
efi-reader
elilo-installer
finish-install
flash-kernel
grub-installer
hw-detect
installation-locale
installation-report
iso-scan
kadit-playbooks
kadit
kbd-chooser
kernel-wedge
kickseed
libdebian-installer
lilo-installer
live-installer
localechooser
lowmem
lvmcfg
main-menu
mdcfg
media-retriever
mklibs
mountmedia
net-retriever
netboot-assistant
netcfg
network-console
nobootloader
oldsys-preseed
os-prober
partconf
partman-auto-crypto
partman-auto-lvm
partman-auto-raid
partman-auto
partman-base
partman-basicfilesystems
partman-basicmethods
partman-btrfs
partman-crypto
partman-efi
partman-ext3
partman-iscsi
partman-jfs
partman-lvm
partman-md
partman-multipath
partman-nbd
partman-newworld
partman-partitioning
partman-prep
partman-swapfile
partman-target
partman-ufs
partman-xfs
partman-zfs
pkgsel
prep-installer
preseed
quik-installer
rescue
rootskel-gtk
rootskel
s390-dasd
s390-netdevice
s390-sysconfig-writer
s390-zfcp
tzsetup
udpkg
usb-discover
user-setup
win32-loader
yaboot-installer
zipl-installer

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"Because heaters aren't purple!" -- Catherine Pitt

Re: Salsa

[Christian PERRIER]

Quoting Cyril Brulebois (k...@debian.org):

> > I think I will have to leave the Salsa -boot project to you and the
> > rest of the team going forwards.
> 
> Thanks for having shown interest and having set up a project so much in
> advance, and sorry again for not having jumped on that train at the
> time…


On my side, well, I'm watching the Salsa train from the platform and I
have to admit that I'm not active at all in the Salsa move.

While I very well understand reasons behind Alioth's shutdown, it
breaks so many established workflows of mine that, since the beginning,
I decided that I would be a follower on that topic. I never wanted to
enter arguments on -devel about this, but I think this move is way too
disruptive, with regards of the *real* situation of manpower in Debian.

If I can adapt my workflows to changes needed by the move to Salsa,
then I'm fine with it, that's mostly how I  follow things.

As of now, I know that l10n workflows *will* be affected, that's obvious.



signature.asc
Description: PGP signature

Re: Salsa

[Cyril Brulebois]

Hi,

Chris Boot  (2018-04-24):
> I didn't receive much feedback from the -boot team since I created the
> Salsa project and did the initial member import, but I also didn't
> remind people about it after those first few emails.

Yes, sorry for not having been more proactive on this topic. :(

> FWIW I have been using GitLab both personally and at work very
> successfully for quite a while now with my generally Git-centric
> workflow. I have no recent experience with conversions from other VCS,
> and I do understand some people's concerns about whether GitLab is the
> best fit for Debian, but I strongly believe that whilst it may require
> a bit of a mindset readjustment it's the best thing out there for us.

That's very fine. :) I was more wondering about tips and tricks for the
actual migration; but apparently Steve is happy to help with setting
things up with packages, so I would only have to deal with adapting the
l10n commit robot for new URLs.

While I'm on that topic, short brain dump: I've created it already
(di-l10n-guest), pointing to a mail address of mine for the time being;
and I've requested access to the group.

> I also must admit that I haven't had a huge amount of time lately to
> work on Debian things, much to my regret. From now on for a while I
> will have even *less* time, but with no regret now as my first child
> was born yesterday! (Mother and child doing very well, thanks, and
> email to -private will come shortly!)

Congratz. :)

> I think I will have to leave the Salsa -boot project to you and the
> rest of the team going forwards.

Thanks for having shown interest and having set up a project so much in
advance, and sorry again for not having jumped on that train at the
time…


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Salsa

[Chris Boot]

On 24/04/18 02:29, Cyril Brulebois wrote:
> Hi Chris,
> 
> Chris Boot  (2018-01-22):
>> I think it would be helpful to start using Salsa for some of our repos.
>>
>> I would like to move my personal busybox work-in-progress repo to Salsa;
>> I know nothing prevents me from doing that but it feels like everything
>> would be more joined-up if the main busybox repo was also in Salsa and
>> in a debian-boot team/group.
>>
>> Does anyone have any objection if I create a d-i/boot team on Salsa?
>> What should it be called? Should its membership just be copied from the
>> Alioth team? Alternatively, would it be preferable to use the "Debian"
>> group given we have such a large membership anyway?
>>
>> axhn: any objection to moving busybox into Salsa?
> 
> Do you have any feedback regarding your salsa attempts?
> 
> Clock's ticking and it would be nice to migrate soon, so any gathered
> experience is welcome.

Hi Cyril,

I didn't receive much feedback from the -boot team since I created the
Salsa project and did the initial member import, but I also didn't
remind people about it after those first few emails.

FWIW I have been using GitLab both personally and at work very
successfully for quite a while now with my generally Git-centric
workflow. I have no recent experience with conversions from other VCS,
and I do understand some people's concerns about whether GitLab is the
best fit for Debian, but I strongly believe that whilst it may require a
bit of a mindset readjustment it's the best thing out there for us.

I also must admit that I haven't had a huge amount of time lately to
work on Debian things, much to my regret. From now on for a while I will
have even *less* time, but with no regret now as my first child was born
yesterday! (Mother and child doing very well, thanks, and email to
-private will come shortly!)

I think I will have to leave the Salsa -boot project to you and the rest
of the team going forwards.

Cheers,
Chris

-- 
Chris Boot
bo...@debian.org



signature.asc
Description: OpenPGP digital signature

Re: Salsa

[Cyril Brulebois]

Hi Chris,

Chris Boot  (2018-01-22):
> I think it would be helpful to start using Salsa for some of our repos.
> 
> I would like to move my personal busybox work-in-progress repo to Salsa;
> I know nothing prevents me from doing that but it feels like everything
> would be more joined-up if the main busybox repo was also in Salsa and
> in a debian-boot team/group.
> 
> Does anyone have any objection if I create a d-i/boot team on Salsa?
> What should it be called? Should its membership just be copied from the
> Alioth team? Alternatively, would it be preferable to use the "Debian"
> group given we have such a large membership anyway?
> 
> axhn: any objection to moving busybox into Salsa?

Do you have any feedback regarding your salsa attempts?

Clock's ticking and it would be nice to migrate soon, so any gathered
experience is welcome.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Salsa

[Bastian Blank]

On Mon, Jan 22, 2018 at 06:52:01PM +0100, Christian PERRIER wrote:
> Quoting Cyril Brulebois (k...@debian.org):
> > If you have time/motivation to explore salsa.debian.org for d-i things,
> > that's more than welcome!
> > > Does anyone have any objection if I create a d-i/boot team on Salsa?
> > Not from me.
> I will then need to convert the last remaining SVN parts (mostly l10n
> stuff, which was never gitified.for lazyness reasons). 

Okay, that would be:
- the manual
- the package po files
- the scripts

> which means updates to the scripts that process them (l10n-sync).

Yeah.

Does it need changes if we just start to move the package repos to
salsa, apart from the authentication for the sync process?

Bastian

-- 
Conquest is easy. Control is not.
-- Kirk, "Mirror, Mirror", stardate unknown

Re: Salsa

[Joerg Jaspert]

On 14930 March 1977, Chris Boot wrote:
> - I (bootc) am listed as an Owner because I created the team. I can't
>   demote myself. I'm happy to be demoted, but it might be best to do
>   that after I've migrated the remaining users.

JFTR, yes you can, but only if other Owners exist.

-- 
bye, Joerg

Re: Salsa

[Bastian Blank]

On Sat, Jan 27, 2018 at 11:46:30AM +, Chris Boot wrote:
> I have added the following people to the project as members:
> { Alioth Role / Position } => { GitLab Role }
> Admin => Owner
> Senior Developer => Master
> Developer => Developer
> 
> I haven't added "Junior Developers" yet because I'm not convinced that
> GitLab has an equivalent. The next level down from Developer is
> Reporter. Shall I just add Junior Developers as Developers?

It always depends on what you think this people should be allowed to do.
On Alioth all where member of the d-i group and could do anything with
the git repositories.  On Salsa there are actually different permissions
to the repos attached.

> I also haven't added Translators. Should they also be Developers?
> Alternatively, we can create a sub-team for translators if finer-grained
> access to repos is required.

It may make sense to have sub-groups for such special topics.

> - I haven't made any attempt to add -guest users, even those who might
>   now be Debian Developers but who stuck with their -guest accounts on
>   Alioth.

We can now give out access to single repos.

> - I haven't been able to add the various DD (emeritus) people who are
>   still on the Alioth project, for hopefully obvious reasons.

This is by choice.  Disabled DD accounts are also blocked from Salsa.

Bastian

-- 
Beam me up, Scotty, there's no intelligent life down here!

Re: Salsa

[Chris Boot]

On 23/01/18 15:17, Chris Boot wrote:
> On 22/01/18 17:23, Bastian Blank wrote:
>> On Mon, Jan 22, 2018 at 05:28:43PM +0100, Cyril Brulebois wrote:
>>> Chris Boot  (2018-01-22):
 What should it be called?
>>>
>>> Good question. d-i looks good to me, and would match the current group
>>> on alioth. debian-boot is historical and I think we should keep only
>>> the list named this way (along with IRC).
>>
>> Or just use "installer-team".
> 
> I don't mind d-i-team or installer-team or even debian-installer-team,
> but the self-service interface always adds "-team" to the end. Not
> having -team on the end presumably means asking the admin folks to do it
> manually for us, so it's probably best avoided unless we really want to.

I went for "installer-team" in the end. If people feel strongly about
it, I'm sure we can get it renamed before we start using it properly.

 Should its membership just be copied from the Alioth team?
>>> If possible, that would look good to me. Not sure about non-DD accounts
>>> support though (I've had too little time to keep track of salsa things,
>>> which seemed to be fast moving).
>>
>> This just needs to be done by hand.
> 
> Indeed, non-DD accounts need to register at
> https://signup.salsa.debian.org/ and then a DD needs to add them to the
> relevant team/project with appropriate permissions.

I have added the following people to the project as members:

{ Alioth Role / Position } => { GitLab Role }
Admin => Owner
Senior Developer => Master
Developer => Developer

I haven't added "Junior Developers" yet because I'm not convinced that
GitLab has an equivalent. The next level down from Developer is
Reporter. Shall I just add Junior Developers as Developers?

I also haven't added Translators. Should they also be Developers?
Alternatively, we can create a sub-team for translators if finer-grained
access to repos is required.

Notes:

- I (bootc) am listed as an Owner because I created the team. I can't
  demote myself. I'm happy to be demoted, but it might be best to do
  that after I've migrated the remaining users.

- I haven't made any attempt to add -guest users, even those who might
  now be Debian Developers but who stuck with their -guest accounts on
  Alioth.

- I haven't been able to add the various DD (emeritus) people who are
  still on the Alioth project, for hopefully obvious reasons.

Please let me know your thoughts about the above, I'm more than happy to
continue the manual work of importing people into the project.

Cheers,
Chris

-- 
Chris Boot
bo...@debian.org



signature.asc
Description: OpenPGP digital signature

Re: Salsa

[Bastian Blank]

On Thu, Jan 25, 2018 at 08:43:57PM +0100, Philip Hands wrote:
> 'installer-team' would be my top choice too (given that the -team bit
> seems to be necessary to accommodate the fact that gitlab uses the same
> namespace for teams and users).

It is possible to get different names, esp for teams who already have a
Debian user.

Bastian

-- 
Without facts, the decision cannot be made logically.  You must rely on
your human intuition.
-- Spock, "Assignment: Earth", stardate unknown

Re: Salsa

[Philip Hands]

On Thu, 25 Jan 2018, Chris Boot  wrote:
> On 23/01/18 15:17, Chris Boot wrote:
>> On 22/01/18 17:23, Bastian Blank wrote:
>>> On Mon, Jan 22, 2018 at 05:28:43PM +0100, Cyril Brulebois wrote:
 Chris Boot  (2018-01-22):
> What should it be called?

 Good question. d-i looks good to me, and would match the current group
 on alioth. debian-boot is historical and I think we should keep only
 the list named this way (along with IRC).
>>>
>>> Or just use "installer-team".
>> 
>> I don't mind d-i-team or installer-team or even debian-installer-team,
>> but the self-service interface always adds "-team" to the end. Not
>> having -team on the end presumably means asking the admin folks to do it
>> manually for us, so it's probably best avoided unless we really want to.
>
> Hi all,
>
> Any further thoughts about this?
>
> Of the simple options, "installer-team" feels like the most appropriate
> to me.

'installer-team' would be my top choice too (given that the -team bit
seems to be necessary to accommodate the fact that gitlab uses the same
namespace for teams and users).

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Re: Salsa

[Chris Boot]

On 23/01/18 15:17, Chris Boot wrote:
> On 22/01/18 17:23, Bastian Blank wrote:
>> On Mon, Jan 22, 2018 at 05:28:43PM +0100, Cyril Brulebois wrote:
>>> Chris Boot  (2018-01-22):
 What should it be called?
>>>
>>> Good question. d-i looks good to me, and would match the current group
>>> on alioth. debian-boot is historical and I think we should keep only
>>> the list named this way (along with IRC).
>>
>> Or just use "installer-team".
> 
> I don't mind d-i-team or installer-team or even debian-installer-team,
> but the self-service interface always adds "-team" to the end. Not
> having -team on the end presumably means asking the admin folks to do it
> manually for us, so it's probably best avoided unless we really want to.

Hi all,

Any further thoughts about this?

Of the simple options, "installer-team" feels like the most appropriate
to me.

Best regards,
Chris

-- 
Chris Boot
bo...@debian.org

GPG: 8467 53CB 1921 3142 C56D  C918 F5C8 3C05 D9CE 

Re: Salsa

[Chris Boot]

On 22/01/18 17:23, Bastian Blank wrote:
> On Mon, Jan 22, 2018 at 05:28:43PM +0100, Cyril Brulebois wrote:
>> Chris Boot  (2018-01-22):
>>> I think it would be helpful to start using Salsa for some of our repos.
>>>
>>> I would like to move my personal busybox work-in-progress repo to Salsa;
>>> I know nothing prevents me from doing that but it feels like everything
>>> would be more joined-up if the main busybox repo was also in Salsa and
>>> in a debian-boot team/group.
>>
>> If you have time/motivation to explore salsa.debian.org for d-i things,
>> that's more than welcome!

Well, I was going to do so only as far as my involvement goes. Since I'm
only really involved in busybox, I was looking at bootstrapping the team
and putting busybox there.

I don't want to barge in and push anyone else into things they may not
yet be ready for, so I was planning on leaving the rest to other people.

[snip]

>>> What should it be called?
>>
>> Good question. d-i looks good to me, and would match the current group
>> on alioth. debian-boot is historical and I think we should keep only
>> the list named this way (along with IRC).
> 
> Or just use "installer-team".

I don't mind d-i-team or installer-team or even debian-installer-team,
but the self-service interface always adds "-team" to the end. Not
having -team on the end presumably means asking the admin folks to do it
manually for us, so it's probably best avoided unless we really want to.

>>> Should its membership just be copied from the Alioth team?
>> If possible, that would look good to me. Not sure about non-DD accounts
>> support though (I've had too little time to keep track of salsa things,
>> which seemed to be fast moving).
> 
> This just needs to be done by hand.

Indeed, non-DD accounts need to register at
https://signup.salsa.debian.org/ and then a DD needs to add them to the
relevant team/project with appropriate permissions.

[snip comments about SVN migrations and using the Debian umbrella]

Cheers,
Chris

-- 
Chris Boot
bo...@debian.org

Re: Salsa

[Christian PERRIER]

Quoting Cyril Brulebois (k...@debian.org):

> If you have time/motivation to explore salsa.debian.org for d-i things,
> that's more than welcome!
> 
> > Does anyone have any objection if I create a d-i/boot team on Salsa?
> 
> Not from me.

I will then need to convert the last remaining SVN parts (mostly l10n
stuff, which was never gitified.for lazyness reasons). 

which means updates to the scripts that process them (l10n-sync).



signature.asc
Description: PGP signature

Re: Salsa

[Bastian Blank]

On Mon, Jan 22, 2018 at 05:28:43PM +0100, Cyril Brulebois wrote:
> Chris Boot  (2018-01-22):
> > I think it would be helpful to start using Salsa for some of our repos.
> > 
> > I would like to move my personal busybox work-in-progress repo to Salsa;
> > I know nothing prevents me from doing that but it feels like everything
> > would be more joined-up if the main busybox repo was also in Salsa and
> > in a debian-boot team/group.
> 
> If you have time/motivation to explore salsa.debian.org for d-i things,
> that's more than welcome!
> 
> > Does anyone have any objection if I create a d-i/boot team on Salsa?
> 
> Not from me.
> 
> > What should it be called?
> 
> Good question. d-i looks good to me, and would match the current group
> on alioth. debian-boot is historical and I think we should keep only
> the list named this way (along with IRC).

Or just use "installer-team".

> > Should its membership just be copied from the Alioth team?
> If possible, that would look good to me. Not sure about non-DD accounts
> support though (I've had too little time to keep track of salsa things,
> which seemed to be fast moving).

This just needs to be done by hand.

> Not necessary for busybox AFAICT, but we'll need to have that later when
> moving all repositories there: we need to have access for the l10n robot
> (including write access), working from dillon.debian.org these days.

What about the stuff still in SVN?

> > Alternatively, would it be preferable to use the "Debian" group given
> > we have such a large membership anyway?
> I'm not sure. ISTR having seen people mention on IRC that views weren't
> too practical when projects are under the Debian umbrella, because
> everything is listed altogether? Maybe a separate group would be best?

No, the Debian group is for single repositories, not such large lumbs.

Bastian

-- 
Captain's Log, star date 21:34.5...

Re: Salsa

[Cyril Brulebois]

Hi,

Chris Boot  (2018-01-22):
> I think it would be helpful to start using Salsa for some of our repos.
> 
> I would like to move my personal busybox work-in-progress repo to Salsa;
> I know nothing prevents me from doing that but it feels like everything
> would be more joined-up if the main busybox repo was also in Salsa and
> in a debian-boot team/group.

If you have time/motivation to explore salsa.debian.org for d-i things,
that's more than welcome!

> Does anyone have any objection if I create a d-i/boot team on Salsa?

Not from me.

> What should it be called?

Good question. d-i looks good to me, and would match the current group
on alioth. debian-boot is historical and I think we should keep only
the list named this way (along with IRC).

> Should its membership just be copied from the Alioth team?

If possible, that would look good to me. Not sure about non-DD accounts
support though (I've had too little time to keep track of salsa things,
which seemed to be fast moving).

Not necessary for busybox AFAICT, but we'll need to have that later when
moving all repositories there: we need to have access for the l10n robot
(including write access), working from dillon.debian.org these days.

> Alternatively, would it be preferable to use the "Debian" group given
> we have such a large membership anyway?

I'm not sure. ISTR having seen people mention on IRC that views weren't
too practical when projects are under the Debian umbrella, because
everything is listed altogether? Maybe a separate group would be best?


Thanks for leading the way!

Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Salsa

[Domenico Andreoli]

On Mon, Jan 22, 2018 at 10:10:32AM +, Chris Boot wrote:
> Hi folks,

Hi,

> 
> I think it would be helpful to start using Salsa for some of our repos.

you are reading my mind...

maybe rename it to Debian Installer?

thanks,
Domenico

-- 
3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13