Re: Reducing allowed Vcs for packaging?

[Sean Whitton]

Hello,

On Mon 27 Mar 2023 at 12:02AM +01, Steve McIntyre wrote:

> I think you're *reaching* here. I know of quite a few projects where
> they consider their CI setup to be an intergral part of project
> development. Should we therefore declare that "preferred form of
> modification" could include all of the github or gitlab
> infrastructure?

Something that I have in mind in particular is longer commit messages.
These can be as important as code comments, and sometimes having them
attached to commits is more useful than just finding somewhere to put
them in the code.  Indeed, Emacs exports all these as CHANGELOG files in
its release tarballs.

-- 
Sean Whitton

Re: Reducing allowed Vcs for packaging?

[G. Branden Robinson]

At 2023-03-26T13:56:55-0700, Sean Whitton wrote:
> On Sat 04 Mar 2023 at 10:58PM +01, Joerg Jaspert wrote:
> > But git or svn or even sccs and rcs is NOT, in any way, preferred
> > for of modification. Only one way of storage and handling some
> > metadata.
> 
> This is Debian's official position, but it's debateable.
> 
> There is the preferred form for modification for an individual *file*,
> and that probably doesn't include version control.  But the preferred
> form for modifying a *project* arguably does.

I wonder if, after twelve years, we have any empirical data regarding
whether Red Hat's stance, that change sets (discretized by the process
of commits to VCSes) are not constitutive of the "preferred form for
modifying a copyrighted work" when that is the form invariably used by
those performing the modifications, frustrated Oracle's aims, or whether
this deliberately anemic interpretation of the GPL was a perfomative
weakening of copyleft to little or no commercial advantage.

https://lwn.net/Articles/432012/

Regards,
Branden


signature.asc
Description: PGP signature

Re: Reducing allowed Vcs for packaging?

[Steve McIntyre]

Sean Whitton wrote:
>
>On Sat 04 Mar 2023 at 10:58PM +01, Joerg Jaspert wrote:
>
>> On 16792 March 1977, Adrian Bunk wrote:
>>
>>> for the contents of packages in the archive the ftp team requires that
>>> everything is in the preferred form of modification.
>>
>> Yes. Of course.
>>
>> But git or svn or even sccs and rcs is NOT, in any way, preferred for of
>> modification. Only one way of storage and handling some metadata.
>
>This is Debian's official position, but it's debateable.
>
>There is the preferred form for modification for an individual *file*,
>and that probably doesn't include version control.  But the preferred
>form for modifying a *project* arguably does.

I think you're *reaching* here. I know of quite a few projects where
they consider their CI setup to be an intergral part of project
development. Should we therefore declare that "preferred form of
modification" could include all of the github or gitlab
infrastructure?

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
< sladen> I actually stayed in a hotel and arrived to find a post-it
  note stuck to the mini-bar saying "Paul: This fridge and
  fittings are the correct way around and do not need altering"

Re: Reducing allowed Vcs for packaging?

[Sean Whitton]

Hello,

On Sat 04 Mar 2023 at 10:58PM +01, Joerg Jaspert wrote:

> On 16792 March 1977, Adrian Bunk wrote:
>
>> for the contents of packages in the archive the ftp team requires that
>> everything is in the preferred form of modification.
>
> Yes. Of course.
>
> But git or svn or even sccs and rcs is NOT, in any way, preferred for of
> modification. Only one way of storage and handling some metadata.

This is Debian's official position, but it's debateable.

There is the preferred form for modification for an individual *file*,
and that probably doesn't include version control.  But the preferred
form for modifying a *project* arguably does.

-- 
Sean Whitton

Re: Reducing allowed Vcs for packaging?

[Sean Whitton]

Hello,

On Sat 04 Mar 2023 at 07:25PM +02, Adrian Bunk wrote:

> for the contents of packages in the archive the ftp team requires that
> everything is in the preferred form of modification.
>
> It is therefore surprising that you as member of the ftp team declare
> that there is no requirement at all that the packages themselves that
> get uploaded to the archive are in the preferred form of modification
> as long as the preferred form of modification is in salsa.

That is not what I meant.

We certainly want some mechanism to enforce a correspondence between
preferred forms for modification and binary packages, so that we know we
have the former for every one of the latter.

dgit-repos and dak's archive of source packages both serve this purpose.

-- 
Sean Whitton

Re: Reducing allowed Vcs for packaging?

[Adrian Bunk]

On Sat, Mar 04, 2023 at 07:43:37PM +, Scott Kitterman wrote:
> On March 4, 2023 5:25:35 PM UTC, Adrian Bunk  wrote:
> >On Wed, Mar 01, 2023 at 05:54:38PM -0700, Sean Whitton wrote:
> >> 
> >> This is a matter of perspective.  The fact that dak doesn't store git
> >> histories and send them out to mirrors is an implementation detail, to
> >> me.  salsa and dgit-repos are both just as significant Debian archives,
> >> even if they're not what we refer to when we write "Debian archive".
> >
> >for the contents of packages in the archive the ftp team requires that 
> >everything is in the preferred form of modification.
> >
> >It is therefore surprising that you as member of the ftp team declare 
> >that there is no requirement at all that the packages themselves that 
> >get uploaded to the archive are in the preferred form of modification
> >as long as the preferred form of modification is in salsa.
>...
> Putting something in a git repository doesn't make a particular file more or 
> less the preferred form of modification.  It's the same form.  IMO you are 
> conflating two separate concepts here.  I don't find  Sean's perspective at 
> all surprising.

In proper git workflows the metadata is in git only, and cannot be 
included in what is being exported for upload to the Debian archive.

Example:
https://salsa.debian.org/haskell-team/git-annex/-/blob/master/debian/patches/debian-changes

> Scott K

cu
Adrian

Re: Reducing allowed Vcs for packaging?

[Joerg Jaspert]

On 16792 March 1977, Adrian Bunk wrote:

for the contents of packages in the archive the ftp team requires that 
everything is in the preferred form of modification.


Yes. Of course.

But git or svn or even sccs and rcs is NOT, in any way, preferred for of 
modification. Only one way of storage and handling some metadata.


--
bye, Joerg

Re: Reducing allowed Vcs for packaging?

[Scott Kitterman]

On March 4, 2023 5:25:35 PM UTC, Adrian Bunk  wrote:
>On Wed, Mar 01, 2023 at 05:54:38PM -0700, Sean Whitton wrote:
>> Hello,
>
>Hi Sean,
>
>> On Sun 26 Feb 2023 at 11:38PM +02, Adrian Bunk wrote:
>> 
>> > On Sun, Feb 26, 2023 at 09:57:34PM +0100, Diederik de Haas wrote:
>> >> On Sunday, 26 February 2023 20:06:26 CET Adrian Bunk wrote:
>> >>...
>> >> > For anything in Debian, the package sources in Debian would not
>> >> > disappear when a repository (or salsa) disappears.
>> >>
>> >> Question as I don't know: is that only the package change that gets 
>> >> uploaded
>> >> to the Debian archive, or is there also a place where the (git) history 
>> >> of the
>> >> changes leading up to a new upload gets stored?
>> >>
>> >> To use an analogy: I'd like that not only the 'destination' is preserved, 
>> >> but
>> >> also the lead up to the destination.
>> >
>> > What goes into the Debian archive is based on tarballs and quilt patches.
>> > Nothing is stored there except the files you upload.
>> 
>> This is a matter of perspective.  The fact that dak doesn't store git
>> histories and send them out to mirrors is an implementation detail, to
>> me.  salsa and dgit-repos are both just as significant Debian archives,
>> even if they're not what we refer to when we write "Debian archive".
>
>for the contents of packages in the archive the ftp team requires that 
>everything is in the preferred form of modification.
>
>It is therefore surprising that you as member of the ftp team declare 
>that there is no requirement at all that the packages themselves that 
>get uploaded to the archive are in the preferred form of modification
>as long as the preferred form of modification is in salsa.
>
>Maintainers are now permitted to clone the upstream git tree, take one 
>commit as upstream, work on top of that, and then upload this without 
>the kludge of pristine-tar or restrictions due to quilt.
>
>Formats 1.0 or 3.0 (native) will be the natural formats generated for
>the Debian archive.
>
>Format 3.0 (quilt) will be another option, where a generated tarball is 
>uploaded as upstream source (as is already required by the ftp team for 
>repacks) plus one debian/patches/debian.patch containing all changes to 
>the upstream sources.
>
>Generating one quilt patch per commit that changes the upstream sources
>would not always be technically possible due to limitations of quilt.

Putting something in a git repository doesn't make a particular file more or 
less the preferred form of modification.  It's the same form.  IMO you are 
conflating two separate concepts here.  I don't find  Sean's perspective at all 
surprising.

Scott K

Re: Reducing allowed Vcs for packaging?

[Adrian Bunk]

On Wed, Mar 01, 2023 at 05:54:38PM -0700, Sean Whitton wrote:
> Hello,

Hi Sean,

> On Sun 26 Feb 2023 at 11:38PM +02, Adrian Bunk wrote:
> 
> > On Sun, Feb 26, 2023 at 09:57:34PM +0100, Diederik de Haas wrote:
> >> On Sunday, 26 February 2023 20:06:26 CET Adrian Bunk wrote:
> >>...
> >> > For anything in Debian, the package sources in Debian would not
> >> > disappear when a repository (or salsa) disappears.
> >>
> >> Question as I don't know: is that only the package change that gets 
> >> uploaded
> >> to the Debian archive, or is there also a place where the (git) history of 
> >> the
> >> changes leading up to a new upload gets stored?
> >>
> >> To use an analogy: I'd like that not only the 'destination' is preserved, 
> >> but
> >> also the lead up to the destination.
> >
> > What goes into the Debian archive is based on tarballs and quilt patches.
> > Nothing is stored there except the files you upload.
> 
> This is a matter of perspective.  The fact that dak doesn't store git
> histories and send them out to mirrors is an implementation detail, to
> me.  salsa and dgit-repos are both just as significant Debian archives,
> even if they're not what we refer to when we write "Debian archive".

for the contents of packages in the archive the ftp team requires that 
everything is in the preferred form of modification.

It is therefore surprising that you as member of the ftp team declare 
that there is no requirement at all that the packages themselves that 
get uploaded to the archive are in the preferred form of modification
as long as the preferred form of modification is in salsa.

Maintainers are now permitted to clone the upstream git tree, take one 
commit as upstream, work on top of that, and then upload this without 
the kludge of pristine-tar or restrictions due to quilt.

Formats 1.0 or 3.0 (native) will be the natural formats generated for
the Debian archive.

Format 3.0 (quilt) will be another option, where a generated tarball is 
uploaded as upstream source (as is already required by the ftp team for 
repacks) plus one debian/patches/debian.patch containing all changes to 
the upstream sources.

Generating one quilt patch per commit that changes the upstream sources
would not always be technically possible due to limitations of quilt.

> Sean Whitton

cu
Adrian

Re: Reducing allowed Vcs for packaging?

[Sean Whitton]

Hello,

On Sun 26 Feb 2023 at 11:38PM +02, Adrian Bunk wrote:

> On Sun, Feb 26, 2023 at 09:57:34PM +0100, Diederik de Haas wrote:
>> On Sunday, 26 February 2023 20:06:26 CET Adrian Bunk wrote:
>>...
>> > For anything in Debian, the package sources in Debian would not
>> > disappear when a repository (or salsa) disappears.
>>
>> Question as I don't know: is that only the package change that gets uploaded
>> to the Debian archive, or is there also a place where the (git) history of 
>> the
>> changes leading up to a new upload gets stored?
>>
>> To use an analogy: I'd like that not only the 'destination' is preserved, but
>> also the lead up to the destination.
>
> What goes into the Debian archive is based on tarballs and quilt patches.
> Nothing is stored there except the files you upload.

This is a matter of perspective.  The fact that dak doesn't store git
histories and send them out to mirrors is an implementation detail, to
me.  salsa and dgit-repos are both just as significant Debian archives,
even if they're not what we refer to when we write "Debian archive".

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: Reducing allowed Vcs for packaging?

[Sean Whitton]

Hello,

On Mon 27 Feb 2023 at 12:17AM +01, Diederik de Haas wrote:

> But AFAIK the Debian Xen Team does use dgit (not surprising given dgit's
> maintainer (and author?)) ... and that drives me insane.
> I'm very sure that is due to me not understanding the concepts/idea/etc, but I
> can't wrap my head around it and it feels *to me* like it constantly rewrites
> the (git) history and then does a `git push -f` ... every time.
> I once referenced a patch by number (and a short description iirc) and on the
> next push, that patch had a different number, thus messing up my commit msg.
>
> The most confusing thing for/to me is that it completely rearranges the commit
> sequence, so I can't follow the changes that happened over time.
> Right now in https://salsa.debian.org/xen-team/debian-xen/-/commits/master
> HEAD~30 (and the 2 commits before that) are the most recent (and to me the
> most relevant), but HEAD~9 was made 8 YEARS ago.
>
> I may learn dgit in time, but that'll probably be a (long) while.

This is not dgit.  This is git-debrebase, an experimental tool.

Russ's comments about dgit are not at all connected with the
(legitimate) issues with git-debrebase you describe.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: Reducing allowed Vcs for packaging?

[Jelmer Vernooij]

On Sun, Feb 26, 2023 at 02:24:26PM +0100, Bastian Germann wrote:
> During the last weeks I had a look at the Vcs situation in Debian. Currently,
> there are eight possible systems allowed and one might specify several of 
> them for
> one package. No package makes use of several Vcs references and frankly I do 
> not
> see why this was supported in the first place.
> 
> For the allowed systems the situation in unstable is the following:
> arch is used by 2 packages pointing to bad URLs: #1025510, 1025511.
> bzr is used by ~50 packages, half of which point to bad URLs.
> cvs is used by 3 packages, 2 of which point to bad URLs: #1031312, #1031313.
> svn is used by ~130 packages, many of which point to bad URLs.
> darcs, mtn, and hg are not used.

> 
> We can see: The Vcs wars are over; with git there is a clear winner and in my
> opinion, we should remove the possibility to use most of them for package
> maintenance. It is one additional barrier to get into package maintenance and
> we should remove the barriers that are not necessary.

What about packages that don't use a Vcs today? There are far more of those 
today
than that are using non-standard Vcs repositories.

The number of packages that's using non-standards Vcs repositories is declining
gradually anyway (0.4% of the archive today). What does dropping the Vcs-* 
headers
achieve, besides making it even harder to work with these packages?

As somebody who uses Vcs-Bzr for some of the Bzr packages, I'd be on board
with mandating Git (because that gets us consistency in being able to work with
every package)  - but I don't see the point of dropping support for other
Vcs-* headers without doing that.

Cheers,

Jelmer

Re: Reducing allowed Vcs for packaging?

[Holger Levsen]

On Sun, Feb 26, 2023 at 10:25:21AM -0700, Sean Whitton wrote:
> Why don't we just fix all those packacges, instead of changing any
> documents?  Is there anyone who actually wants to introduce new packages
> not using git?  I'm not so sure.

mostly agreed, i'm just sure there will be very few people who want that,
though for the scope of developers-reference I think those should be ignored.

that said, dev-ref currently only explicitly mentions vcs-git.


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

The mark of a civilized man is the ability to look at a column of numbers and
weep. (Bertrand Russell)


signature.asc
Description: PGP signature

Re: Reducing allowed Vcs for packaging?

[Simon McVittie]

On Mon, 27 Feb 2023 at 00:17:41 +0100, Diederik de Haas wrote:
> Russ Allbery  wrote:
> > dgit maintains a history of every package in Debian.
> 
> AFAIK the Debian Xen Team does use dgit (not surprising given dgit's 
> maintainer (and author?)) ... and that drives me insane.
> I'm very sure that is due to me not understanding the concepts/idea/etc, but 
> I 
> can't wrap my head around it and it feels *to me* like it constantly rewrites 
> the (git) history and then does a `git push -f` ... every time.

In most practical maintainer workflows, the Debian patch series
fundamentally *does* get rebased every time there is a new upstream
release (if not more often), so the various packaging workflows (including
at least gbp-pq and git-debrebase, possibly also git-dpm) are ways to
reconcile that with wanting the history to be fast-forwarding.

dgit is at least partially workflow-agnostic, but as a central design
principle it had to choose exactly one representation to be the canonical
one, and the one its authors chose was "patches-applied", meaning that
any Debian-specific changes to the upstream source code of a package are
already pre-applied in the version you get from dgit.

Many (perhaps most?) Debian packages in git use the "patches-unapplied"
layout compatible with quilt or `gbp pq`, in which the Debian-specific
changes in the public VCS only exist in debian/patches and are not
pre-applied to the upstream source code (similar to what we used to do
in cvs and svn); if using `gbp pq` (as I do), the patches do temporarily
get imported into git as a patch-queue branch, which gets rebased, but
that patch-queue branch isn't intended to be pushed anywhere. Examples
of this include everything maintained by the Perl, Python, GNOME and
systemd teams, and many others.

For packages that use the common patches-unapplied workflow, the
representation in dgit and the representation used by the package's
maintainer are not the same (dgit refers to this as "split brain mode").
Packages that I maintain, like dbus and flatpak, might be useful examples
for this. The representation you get from Salsa is the way I prefer to
work with the package, and the representation you get from `dgit clone`
is the way the dgit maintainers would prefer it to be represented. dgit
reconciles these two views of the world automatically for me when I
upload, so I don't need to do anything differently. See dgit-maint-gbp(7)
for more details from the dgit side. Using that workflow is a way I can
be nice to dgit users without changing how I interact with debian/patches
and without requiring my co-maintainers to do anything differently,
so I do that.

I suspect that the Xen packaging is using a different workflow that is
less focused on the "patches-unapplied" tree than mine is, perhaps based
on git-dpm or git-debrebase.

smcv

Re: Reducing allowed Vcs for packaging?

[Adrian Bunk]

On Sun, Feb 26, 2023 at 11:42:25PM +0100, Diederik de Haas wrote:
>...
> The reason that I'm such a proponent of that is that 3 weeks or 3 months from 
> now, there's a reasonable chance that you (the author/committer) does no 
> longer remember the details of that commit.
> In 3+ years that will be close to 0.
> AFAIK actual mind reading does not exist, so someone else surely wouldn't 
> know.
> 
> I've already encountered several cases where the commit was 10+ years old and 
> the commit msg what "Disable setting X" and looking at the diff, I can see 
> the 
> X was indeed disabled. But nothing more.
> But now I want to enable setting X. But I have no context to know why that 
> would be a bad idea, or actually a good idea *now*, or what will break as a 
> consequence of my enabling X. All I can do is enable X and keep an eye out 
> for 
> bug reports.
> 
> I think that's what you want to achieve with 'better' changelogs is something 
> similar. I think the git commits are a better place as it's easier to make 
> finer grained distinctions and it's more directly linked to the changes.
>...

Where applicable:
You can add comments in debian/rules.
You can write long descriptions in debian/patches/*.patch
That's even more directly linked to the changes.

You can also write 5 line changelog entries.

The "if" and "where" are likely far less related than you hope:

People tend to be either terse or verbose,
not terse in the package but verbose in git commits.

And when trying to improve verbosity, this shouldn't be only
in the git metadata outside the package.

> Cheers,
>   Diederik

cu
Adrian

Re: Reducing allowed Vcs for packaging?

[Diederik de Haas]

[please CC me in replies as I'm not subscribed to d-devel]

Russ Allbery  wrote:
> Diederik de Haas  writes:
> > Question as I don't know: is that only the package change that gets
> > uploaded to the Debian archive, or is there also a place where the (git)
> > history of the changes leading up to a new upload gets stored?
> 
> dgit maintains a history of every package in Debian.  If you use dgit to
> make your upload, that will include the full Git history of the changes,
> regardless of where you store your Git repository.  If you don't, then it
> only has the uploads to work with, so each package upload is a new commit
> and the detailed breakdown of the work between those uploads is not
> available via dgit.
> 
> But it does provide at least upload-level granularity as a Git history for
> every package, even those maintained with no version control system at
> all, with the option for each package maintainer of opting into providing
> more.

I'm not new to git or Debian, but I have ~0 experience with packaging (and I 
want to change that with id3lib).
Therefor I don't have (much) experience with the packaging tools (in general).

But AFAIK the Debian Xen Team does use dgit (not surprising given dgit's 
maintainer (and author?)) ... and that drives me insane.
I'm very sure that is due to me not understanding the concepts/idea/etc, but I 
can't wrap my head around it and it feels *to me* like it constantly rewrites 
the (git) history and then does a `git push -f` ... every time.
I once referenced a patch by number (and a short description iirc) and on the 
next push, that patch had a different number, thus messing up my commit msg.

The most confusing thing for/to me is that it completely rearranges the commit 
sequence, so I can't follow the changes that happened over time.
Right now in https://salsa.debian.org/xen-team/debian-xen/-/commits/master 
HEAD~30 (and the 2 commits before that) are the most recent (and to me the 
most relevant), but HEAD~9 was made 8 YEARS ago.

I may learn dgit in time, but that'll probably be a (long) while.

But as I described in another reply to this thread, the upload-level 
granularity is mostly useless *to me* as it usually only (tersely) describes 
the *what*, but not the *why*.

signature.asc
Description: This is a digitally signed message part.

Re: Reducing allowed Vcs for packaging?

[Diederik de Haas]

On Sunday, 26 February 2023 22:38:51 CET Adrian Bunk wrote:
> On Sun, Feb 26, 2023 at 09:57:34PM +0100, Diederik de Haas wrote:
> > On Sunday, 26 February 2023 20:06:26 CET Adrian Bunk wrote:
> >...
> >
> > > For anything in Debian, the package sources in Debian would not
> > > disappear when a repository (or salsa) disappears.
> > 
> > Question as I don't know: is that only the package change that gets
> > uploaded to the Debian archive, or is there also a place where the (git)
> > history of the changes leading up to a new upload gets stored?
> > 
> > To use an analogy: I'd like that not only the 'destination' is preserved,
> > but also the lead up to the destination.
> 
> What goes into the Debian archive is based on tarballs and quilt patches.
> Nothing is stored there except the files you upload.

Thanks, I thought/'feared' so.

> But what do you expect to get from the git history?
> 
> There is no requirement that any history in addition to what is in
> the Debian archive exists at all, or any guarantee that what is in
> some git tree somewhere is actually the same as what is in the
> Debian archive. And git history might just be one commit per upload.
> 
> I would rather like to see people write useful changelog entries
> that will still be useful in 25 years instead of
>   * New upstream version (Closes: #1234567, #1234568, #1234569, ...
> or
>   * Add R³
> than hoping that git metadata would contain anything more useful
> than that for such packages.

What you described and what I mostly see on changelog entries is:
What was changed.

What I rarely see is *why*

I'm a big proponent of the following git commit structure, which then 
automatically becomes the git history:
primary commit msg: what has changed
secondary commit msg: why that change was made including context, 
deliberations made during that change/commit and alternative solutions 
considered and why the committer didn't choose for those.

The secondary commit msg can be LONG and I like that as it also shows the 
author/committer pays attention to such things.
The (upstream) kernel commit history is a treasure trove of excellent commit 
messages.

The reason that I'm such a proponent of that is that 3 weeks or 3 months from 
now, there's a reasonable chance that you (the author/committer) does no 
longer remember the details of that commit.
In 3+ years that will be close to 0.
AFAIK actual mind reading does not exist, so someone else surely wouldn't 
know.

I've already encountered several cases where the commit was 10+ years old and 
the commit msg what "Disable setting X" and looking at the diff, I can see the 
X was indeed disabled. But nothing more.
But now I want to enable setting X. But I have no context to know why that 
would be a bad idea, or actually a good idea *now*, or what will break as a 
consequence of my enabling X. All I can do is enable X and keep an eye out for 
bug reports.

I think that's what you want to achieve with 'better' changelogs is something 
similar. I think the git commits are a better place as it's easier to make 
finer grained distinctions and it's more directly linked to the changes.

FTR: I don't *expect* to see those extended commit messages in those old SVN 
repos, but I do hope I can at least derive _some_ information from the various 
commits itself.

When I was using my own SVN repo I started doing that and I know that I 
benefited from that a couple of months later. I should have a backup of that 
somewhere and I'm quite curious if I restore it (and convert it to git) if I 
can still construct enough detail/history/context from those commit msgs.
(As described in the debian-qa ML post, that repo should be 15+ y/o ?)

Cheers,
  Diederik

signature.asc
Description: This is a digitally signed message part.

Re: Reducing allowed Vcs for packaging?

[Adrian Bunk]

On Sun, Feb 26, 2023 at 09:57:34PM +0100, Diederik de Haas wrote:
> On Sunday, 26 February 2023 20:06:26 CET Adrian Bunk wrote:
>...
> > For anything in Debian, the package sources in Debian would not
> > disappear when a repository (or salsa) disappears.
> 
> Question as I don't know: is that only the package change that gets uploaded 
> to the Debian archive, or is there also a place where the (git) history of 
> the 
> changes leading up to a new upload gets stored?
> 
> To use an analogy: I'd like that not only the 'destination' is preserved, but 
> also the lead up to the destination. 

What goes into the Debian archive is based on tarballs and quilt patches.
Nothing is stored there except the files you upload.

But what do you expect to get from the git history?

There is no requirement that any history in addition to what is in
the Debian archive exists at all, or any guarantee that what is in
some git tree somewhere is actually the same as what is in the
Debian archive. And git history might just be one commit per upload.

I would rather like to see people write useful changelog entries
that will still be useful in 25 years instead of
  * New upstream version (Closes: #1234567, #1234568, #1234569, ...
or
  * Add R³
than hoping that git metadata would contain anything more useful
than that for such packages.

cu
Adrian

Re: Reducing allowed Vcs for packaging?

[Russ Allbery]

Diederik de Haas  writes:

> Question as I don't know: is that only the package change that gets
> uploaded to the Debian archive, or is there also a place where the (git)
> history of the changes leading up to a new upload gets stored?

dgit maintains a history of every package in Debian.  If you use dgit to
make your upload, that will include the full Git history of the changes,
regardless of where you store your Git repository.  If you don't, then it
only has the uploads to work with, so each package upload is a new commit
and the detailed breakdown of the work between those uploads is not
available via dgit.

But it does provide at least upload-level granularity as a Git history for
every package, even those maintained with no version control system at
all, with the option for each package maintainer of opting into providing
more.

-- 
Russ Allbery (r...@debian.org)  

Re: Reducing allowed Vcs for packaging?

[Diederik de Haas]

On Sunday, 26 February 2023 20:06:26 CET Adrian Bunk wrote:
> On Sun, Feb 26, 2023 at 07:25:57PM +0100, Diederik de Haas wrote:
> > Apart from me not liking proprietary systems in general and M$ GitHub in
> > particular, you also run the risk of things disappearing entirely without
> > any notice and without any recourse.
> 
> Perhaps tomorrow some company like Oracle decides to buy GitLab Inc.,
> and then Oracle GitLab stops the current Freemium business model
> effective immediately.

That is a concern afaic, but the critical difference for me is that (all) the 
*data* is on Debian infrastructure.

> For anything in Debian, the package sources in Debian would not
> disappear when a repository (or salsa) disappears.

Question as I don't know: is that only the package change that gets uploaded 
to the Debian archive, or is there also a place where the (git) history of the 
changes leading up to a new upload gets stored?

To use an analogy: I'd like that not only the 'destination' is preserved, but 
also the lead up to the destination. 

signature.asc
Description: This is a digitally signed message part.

Re: Reducing allowed Vcs for packaging?

[Adrian Bunk]

On Sun, Feb 26, 2023 at 07:25:57PM +0100, Diederik de Haas wrote:
>...
> Apart from me not liking proprietary systems in general and M$ GitHub in 
> particular, you also run the risk of things disappearing entirely without any 
> notice and without any recourse.

Perhaps tomorrow some company like Oracle decides to buy GitLab Inc.,
and then Oracle GitLab stops the current Freemium business model
effective immediately.

Would anyone be able to provide security support for the stale free 
version, or would salsa be shutdown with the next CVE?

> C.I.P. https://github.com/community/community/discussions/48173
> where VundleVim (vim plugin manager) disappeared 'out of the blue'.
> (that vundlevim isn't packaged for Debian is irrelevant)

For anything in Debian, the package sources in Debian would not 
disappear when a repository (or salsa) disappears.

cu
Adrian

Re: Reducing allowed Vcs for packaging?

[Diederik de Haas]

On Sunday, 26 February 2023 17:59:52 CET Bill Allombert wrote:
> > During the last weeks I had a look at the Vcs situation in Debian.
> > 
> > For the allowed systems the situation in unstable is the following:
> > ...
> > svn is used by ~130 packages, many of which point to bad URLs.
> > 
> > We can see: The Vcs wars are over; with git there is a clear winner and in
> > my opinion, we should remove the possibility to use most of them for
> > package maintenance. It is one additional barrier to get into package
> > maintenance and we should remove the barriers that are not necessary.

I have something to say about this too, but I'll do it in a P.S. as that's not 
the main point I want to make ...

> People that use e.g. subversion could just remove the Vcs-svn field and
> pretend that they do not use any VCS. What does that gain us ?

I'm currently working on doing a mass-migration from (old) SVN repos to git.
See https://lists.debian.org/debian-qa/2023/01/msg00031.html for details.

As for this specific point: that obsolete/non-working (svn) URL are actually 
useful for me to figure out which need to be converted.

> I have sympathy for maintainers that use the same VCS as usptream. I have
> sympathy for upstream of a VCS to use that VCS instead of GIT.
> 
> ... Unfortunately some projects I work with did not convert their whole
> history to GIT and I find useful that Debian still provide subversion and
> mercurial to access older commits (and dead projects history).

One response to my debian-qa mail contained this:

> use "gbp import-dscs" to recreate some partial history based on
> snapshot.debian.org and I would not put any more effort than this.

That would probably be easier/faster, but I'm hoping to (indeed) preserve all 
(or at least as much as possible) of the original packaging commit history.

Diederik

PS: I would also like that all packaging data is available on Salsa
For some it's not available at all (but apparently permitted per Policy 
§5.6.26), which makes it hard(er then needed) if someone wants to contribute.
And in other cases it's contained in an external (proprietary) system like 
GitHub, which is not under Debian's control.
Apart from me not liking proprietary systems in general and M$ GitHub in 
particular, you also run the risk of things disappearing entirely without any 
notice and without any recourse.
C.I.P. https://github.com/community/community/discussions/48173
where VundleVim (vim plugin manager) disappeared 'out of the blue'.
(that vundlevim isn't packaged for Debian is irrelevant)

signature.asc
Description: This is a digitally signed message part.

Re: Reducing allowed Vcs for packaging?

[Sean Whitton]

Hello,

On Sun 26 Feb 2023 at 02:24PM +01, Bastian Germann wrote:

> Hi!
>
> During the last weeks I had a look at the Vcs situation in Debian. Currently,
> there are eight possible systems allowed and one might specify several of 
> them for
> one package. No package makes use of several Vcs references and frankly I do 
> not
> see why this was supported in the first place.
>
> For the allowed systems the situation in unstable is the following:
> arch is used by 2 packages pointing to bad URLs: #1025510, 1025511.
> bzr is used by ~50 packages, half of which point to bad URLs.
> cvs is used by 3 packages, 2 of which point to bad URLs: #1031312, #1031313.
> svn is used by ~130 packages, many of which point to bad URLs.
> darcs, mtn, and hg are not used.
>
> We can see: The Vcs wars are over; with git there is a clear winner and in my
> opinion, we should remove the possibility to use most of them for package
> maintenance. It is one additional barrier to get into package maintenance and
> we should remove the barriers that are not necessary.
>
> I would like to suggest removing the possibility to specify several systems 
> and
> removing all systems except bzr, svn, and git, while deprecating bzr and 
> possibly svn.
> This means solving the four listed bugs and convincing the cvsd maintainer to
> switch or drop the Vcs-Cvs reference. Then, the Debian Developer's Reference
> should specify the changes in §6.2.5 and whatever parses Vcs-* in 
> debian/control
> should be adapted to do the specified thing.
>
> Finally, we can drop the orphaned packages {cvs,mercurial}-buildpackage
> (see #1026433) and add deprecation notices in brz-debian and svn-buildpackage.

Why don't we just fix all those packacges, instead of changing any
documents?  Is there anyone who actually wants to introduce new packages
not using git?  I'm not so sure.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: Reducing allowed Vcs for packaging?

[Bill Allombert]

On Sun, Feb 26, 2023 at 02:24:26PM +0100, Bastian Germann wrote:
> Hi!
> 
> During the last weeks I had a look at the Vcs situation in Debian. Currently,
> there are eight possible systems allowed and one might specify several of 
> them for
> one package. No package makes use of several Vcs references and frankly I do 
> not
> see why this was supported in the first place.
> 
> For the allowed systems the situation in unstable is the following:
> arch is used by 2 packages pointing to bad URLs: #1025510, 1025511.
> bzr is used by ~50 packages, half of which point to bad URLs.
> cvs is used by 3 packages, 2 of which point to bad URLs: #1031312, #1031313.
> svn is used by ~130 packages, many of which point to bad URLs.
> darcs, mtn, and hg are not used.
> 
> We can see: The Vcs wars are over; with git there is a clear winner and in my
> opinion, we should remove the possibility to use most of them for package
> maintenance. It is one additional barrier to get into package maintenance and
> we should remove the barriers that are not necessary.

People that use e.g. subversion could just remove the Vcs-svn field and pretend
that they do not use any VCS. What does that gain us ?

I have sympathy for maintainers that use the same VCS as usptream. I have 
sympathy for 
upstream of a VCS to use that VCS instead of GIT.

... Unfortunately some projects I work with did not convert their whole history 
to GIT 
and I find useful that Debian still provide subversion and mercurial to access 
older
commits (and dead projects history).

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Re: Reducing allowed Vcs for packaging?

[Bastian Germann]

Am 26.02.23 um 17:26 schrieb Adrian Bunk:

I do not get your point what we would gain if the cvsd maintainer
drops the Vcs-Cvs reference while continuing to maintain the package
in cvs.


That would be a prerequisite to drop Vcs-Cvs support.
It is the last package that points to a working CVS URL.

Re: Reducing allowed Vcs for packaging?

[Adrian Bunk]

On Sun, Feb 26, 2023 at 02:24:26PM +0100, Bastian Germann wrote:
> Hi!
> 
> During the last weeks I had a look at the Vcs situation in Debian. Currently,
> there are eight possible systems allowed and one might specify several of 
> them for
> one package. No package makes use of several Vcs references and frankly I do 
> not
> see why this was supported in the first place.

Policy §5.6.26 says it is not permitted.

> For the allowed systems the situation in unstable is the following:
> arch is used by 2 packages pointing to bad URLs: #1025510, 1025511.
> bzr is used by ~50 packages, half of which point to bad URLs.
> cvs is used by 3 packages, 2 of which point to bad URLs: #1031312, #1031313.
> svn is used by ~130 packages, many of which point to bad URLs.
> darcs, mtn, and hg are not used.
> 
> We can see: The Vcs wars are over; with git there is a clear winner and in my
> opinion, we should remove the possibility to use most of them for package
> maintenance. It is one additional barrier to get into package maintenance and
> we should remove the barriers that are not necessary.

One barrier is that our work is based around tarballs and quilt,
instead of using upstream git trees and commits.

> I would like to suggest removing the possibility to specify several systems 
> and
> removing all systems except bzr, svn, and git, while deprecating bzr and 
> possibly svn.
> This means solving the four listed bugs and convincing the cvsd maintainer to
> switch or drop the Vcs-Cvs reference. Then, the Debian Developer's Reference
> should specify the changes in §6.2.5 and whatever parses Vcs-* in 
> debian/control
> should be adapted to do the specified thing.

Policy §5.6.26 would be the primary definition you want to change.

Not using any Vcs for maintaining packages in Debian stays permitted, 
and I do not get your point what we would gain if the cvsd maintainer 
drops the Vcs-Cvs reference while continuing to maintain the package
in cvs.

In practice e.g. tracker.d.o seems to support Vcs-Bzr but not Vcs-Cvs,
and there is no requirement for tools to drop working support for
something that is no longer specified.

Vcs-Browser is Vcs agnostic and would stay permitted for any kind of Vcs,
including ones never listed in Policy.

> Thanks for any comments,
> Bastian

cu
Adrian

Re: Reducing allowed Vcs for packaging?

[Andrey Rakhmatullin]

On Sun, Feb 26, 2023 at 02:24:26PM +0100, Bastian Germann wrote:
> During the last weeks I had a look at the Vcs situation in Debian. Currently,
> there are eight possible systems allowed
I see a difference between (dis)allowing a VCS in the Vcs-* fields and
(dis)allowing maintainers to store packages in a VCS.

> We can see: The Vcs wars are over; with git there is a clear winner
True.

> and in my opinion, we should remove the possibility to use most of them
> for package maintenance.
Which is not done by restricting Vcs-* values.

> It is one additional barrier to get into package maintenance
I don't think anything mentioned here is a barrier.
Even if you imagine someone looking at the list of allowed Vcs-* values to
choose which VCS to use without any context about any of these VCSes, we
have since recently "Almost all packages in Debian that use a version
control system use Git; if you create a new package, using Git is a good
idea simply because it's the system that contributors will be familiar
with." in devref 6.2.5.2.

> I would like to suggest removing the possibility to specify several systems 
> and
> removing all systems except bzr, svn, and git, while deprecating bzr and 
> possibly svn.
> This means solving the four listed bugs and convincing the cvsd maintainer to
> switch or drop the Vcs-Cvs reference. Then, the Debian Developer's Reference
> should specify the changes in §6.2.5 
The main place for this is Policy 5.6.26, not devref (I don't know what
does devref mean when saying "currently the following systems are
supported by the package tracking system").

> and whatever parses Vcs-* in debian/control should be adapted to do the 
> specified thing.
Do you mean "by dropping support for ones that are no longer allowed"?
This is code cleanup in the best case and breaking support for looking at
the relevant field in old packages in the worst one.

> Finally, we can drop the orphaned packages {cvs,mercurial}-buildpackage
> (see #1026433) and add deprecation notices in brz-debian and svn-buildpackage.
This can be done independently.

Re: Reducing allowed Vcs for packaging?

[Brian Thompson]

On Sun, 2023-02-26 at 14:24 +0100, Bastian Germann wrote:
> Hi!
> 
> During the last weeks I had a look at the Vcs situation in Debian. Currently,
> there are eight possible systems allowed and one might specify several of them
> for
> one package. No package makes use of several Vcs references and frankly I do
> not
> see why this was supported in the first place.
> 
> For the allowed systems the situation in unstable is the following:
> arch is used by 2 packages pointing to bad URLs: #1025510, 1025511.
> bzr is used by ~50 packages, half of which point to bad URLs.
> cvs is used by 3 packages, 2 of which point to bad URLs: #1031312, #1031313.
> svn is used by ~130 packages, many of which point to bad URLs.
> darcs, mtn, and hg are not used.
> 
> We can see: The Vcs wars are over; with git there is a clear winner and in my
> opinion, we should remove the possibility to use most of them for package
> maintenance. It is one additional barrier to get into package maintenance and
> we should remove the barriers that are not necessary.
> 
> I would like to suggest removing the possibility to specify several systems
> and
> removing all systems except bzr, svn, and git, while deprecating bzr and
> possibly svn.
> This means solving the four listed bugs and convincing the cvsd maintainer to
> switch or drop the Vcs-Cvs reference. Then, the Debian Developer's Reference
> should specify the changes in §6.2.5 and whatever parses Vcs-* in
> debian/control
> should be adapted to do the specified thing.
> 
> Finally, we can drop the orphaned packages {cvs,mercurial}-buildpackage
> (see #1026433) and add deprecation notices in brz-debian and svn-buildpackage.
> 
> Thanks for any comments,
> Bastian
> 

It does seem like the VCS wars are over--for now.  Who knows what type of
situation could force our hand to use a different VCS than Git?  That future is
hard to imagine but is certainly a possibility.  I think I'm understanding your
point that it would make the package maintainers lives' easier, and it does
sound like some of the packages using non-Git VCS are rotting, at least in that
regard.  However, I would be hesitant to remove support for the other VCS
systems.  

From my experience, whenever software engineers are actively limited for
seemingly no or little gain, they tend to turn their attention elsewhere.  Also,
ripping out logic to support 7 other VCS systems stifles creativity and puts us
down a one-way street.  Sure, you could argue that adding that logic back in
wouldn't be hard, but then why remove it in the first place?  Wouldn't it be
more prudent just to update the non-Git packages to use Git?   That's going to
have to be done anyway for a lot of them (or not).  

My point is, the gain doesn't seem to be larger than the possible (and not that
probable in the near future) cost.  Admittedly, it's difficult to gauge the
costs of these things.

-- 
Sincerely,

Brian T


publickey - brianrobt@pm.me - c8f2ec48.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature