Re: git vs dfsg tarballs
On Fri, Dec 7, 2018 at 7:48 PM Enrico Weigelt wrote: > Have there been any cases where those files have been in the > upstream VCS ? I don't recall any such case. I assume most of the rejects from NEW would have this issue. > For the case where certain parts shouldn't be built/shipped due to > policy, this can - and IMHO should - be handled with changes within > the VCS, instead of having tarballs laying around w/o any clear > history and no indication how exactly it was created from upstream. See Files-Excluded in Debian policy. > Actually, since about a decade, I'm not doing any code changes outside > git, and I'm building packages only directly from git. Frankly, I don't > see any reason why that can't be the standard case. The world is much less simple than that. There are people who don't use version control systems, there are people who don't use git, there are people who use version control systems that git cannot interface with and there are people who invent new version control systems that are meant to be better than git. -- bye, pabs https://wiki.debian.org/PaulWise
Re: git vs dfsg tarballs
On 2018-12-07 12:48 +0100, Enrico Weigelt, metux IT consult wrote: > On 21.11.18 04:22, Paul Wise wrote: > > Actually, since about a decade, I'm not doing any code changes outside > git, and I'm building packages only directly from git. Frankly, I don't > see any reason why that can't be the standard case. Just because you like a tool doesn't mean everyone does. I know I understand patches and tarballs much better than I understand git, so I prefer to use the former. To answer your original question about DFSG differences, the difference between upstream and the DFSG is normally defined by 'Files-Excluded' in the copyright file and compression/name transformations in the watch file. It is for my packages, anyway. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Re: git vs dfsg tarballs
On 21.11.18 04:22, Paul Wise wrote: > I don't think Andreas was talking about applying the DFSG but about > files we don't have permission to distribute at all. Have there been any cases where those files have been in the upstream VCS ? I don't recall any such case. For the case where certain parts shouldn't be built/shipped due to policy, this can - and IMHO should - be handled with changes within the VCS, instead of having tarballs laying around w/o any clear history and no indication how exactly it was created from upstream. Actually, since about a decade, I'm not doing any code changes outside git, and I'm building packages only directly from git. Frankly, I don't see any reason why that can't be the standard case. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: git vs dfsg tarballs
On Tue, Nov 20, 2018 at 6:50 PM Jonathan Dowland wrote: > It was widely done on alioth (by pulling in upstream branches to git > repos) and I imagine is common on Salsa, too. In practise we are not > applying the DFSG to the content of the VCS, the wiki, or really much > except the archive. I don't think Andreas was talking about applying the DFSG but about files we don't have permission to distribute at all. -- bye, pabs https://wiki.debian.org/PaulWise
Re: git vs dfsg tarballs
On Mon, Nov 19, 2018 at 08:17:03PM +0100, Andreas Metzler wrote: the main reason for having +dfsg versions is that non-distributable stuff is removed. Distributing these files in a Debian hosted GIT repository would not be workable, would it? It was widely done on alioth (by pulling in upstream branches to git repos) and I imagine is common on Salsa, too. In practise we are not applying the DFSG to the content of the VCS, the wiki, or really much except the archive. -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Jonathan Dowland ⢿⡄⠘⠷⠚⠋⠀ https://jmtd.net ⠈⠳⣄ Please do not CC me, I am subscribed to the list.
Re: git vs dfsg tarballs
Enrico Weigelt, metux IT consult wrote: > I'm often seeing packagers directly putting dfsg'ed trees into their git > repos, w/o any indication how the tree was actually created from the > original releases. [...] > My preferred way (except for rare cases where upstream history is > extremely huge - like mozilla stuff) would be just branching at the > upstream's release tag and adding commits for removing the non-dfsg > files ontop of that. [...] Hello, the main reason for having +dfsg versions is that non-distributable stuff is removed. Distributing these files in a Debian hosted GIT repository would not be workable, would it? cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure'
Re: git vs dfsg tarballs
On 19.11.18 17:36, Ian Jackson wrote: Hi, > I am saying that for packages whose Debian maintainer follow those> > recommendations, much of what you want would be straightforward - or,> anyway a lot easier. So I was plugging my recommendations. Unfortunately, those packages I'm coping w/ don't really follow that. Kodi is an really unpleasant example: * unclear orig<->dfsg relationship (I'll have to analyze them one by one and adapt my import scripts) * very non-linear history (eg. new upstream trees, sometimes even completely unrelated branches, directly merged down into the deb branch) * lot's of patches against non-existing files * rules trying to touch missing source files/directories. >> Here're some examples on how my deb branches look like:> > Not sure what you >> mean by `your deb branches', Those who add the debian/* stuff, and possibly other patches. In my model, they're always linear descendants of the corresponding upstream release tag. > but looking at what> Debian gives you:> >> * canonical ref names> > dgit > (dgit clone, dgit fetch) will give you this, regardless of the> maintainer's behaviour. hmm, looks like good start. But it doesn't really look easy to clone from different distros and specific or yet unreleased versions. and one of my main problems remains unresolved: linear history ontop of the upstream's release tag. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: git vs dfsg tarballs
Enrico Weigelt, metux IT consult writes ("Re: git vs dfsg tarballs"):
> On 19.11.18 13:52, Ian Jackson wrote:
> > I think that most of the workflows recommended in these manpages
> >
> >
> > https://manpages.debian.org/stretch-backports/dgit/dgit-maint-gbp.7.en.html
> >
> > https://manpages.debian.org/stretch-backports/dgit/dgit-maint-merge.7.en.html
> >
> > https://manpages.debian.org/stretch-backports/dgit/dgit-maint-debrebase.7.en.html
>
> Yet complicated for me (especially regarding automating/CI).
I'm sorry, I think you have misunderstood my point. I wasn't
suggesting that *you* should follow the recommendations in those
manpages.
I am saying that for packages whose Debian maintainer follow those
recommendations, much of what you want would be straightforward - or,
anyway a lot easier. So I was plugging my recommendations.
I was also inviting comment from you as a downstream, if there are
ways recommendations (and tools such as dgit) could be improved.
> Here're some examples on how my deb branches look like:
Not sure what you mean by `your deb branches', but looking at what
Debian gives you:
> * canonical ref names
dgit (dgit clone, dgit fetch) will give you this, regardless of the
maintainer's behaviour.
> * always based on the corresponding upstream's release tag
A maintainer who uses dgit and follows the recommendations in the
dgit-maint-*(7) manpages will give you this.
So I think you should be plugging dgit to maintainers, like I am :-).
> * changes directly as git commits - no text-based patches whatsoever
dgit will pretty much give you this, regardless of the maintainer's
behavior, because it will automatically convert the `text based
patches' into git commits so the git commits are there.
> * generic changes below the deb-specific ones
Again, dgit will give you this.
> I'm currently helping myself w/ lots of mappings and import scripts,
> but I'd like to get rid of maintaining all these little pieces.
One of dgit's objectives is to make the work of downstreams easier.
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: git vs dfsg tarballs
On 19.11.18 13:52, Ian Jackson wrote: > Clearly the transformation on the *tree* can't be reversible because > in the usual case it is deleting things. So you'll need the history. It certain can be, if you know the exact orig commit. Maybe I wasn't really clear here: I wanna do a fully automatic import into a git history (optimally, by just having package name and version). > With most gitish workflows, the corresponding pre-dfsg upstream > *commit* can be found with `git-merge-base', assuming you have some > uploaded (or pushed) Debian commit and a suitable upstream branch. It's not entirely trivial, if the maintainers are doing wild merges. (eg. w/ kodi). Even worse: reconstructing the change history ontop of some given upstream release is pretty complicated and manual. Merging down from upstream into packaging branch (instead of just a simple rebase) turns out as bad idea here. >> My preferred way (except for rare cases where upstream history is >> extremely huge - like mozilla stuff) would be just branching at the >> upstream's release tag and adding commits for removing the non-dfsg >> files ontop of that. From that branching the debianized branch, >> where all patches are directly applied in git. > > I think that most of the workflows recommended in these manpages > > https://manpages.debian.org/stretch-backports/dgit/dgit-maint-gbp.7.en.html > > https://manpages.debian.org/stretch-backports/dgit/dgit-maint-merge.7.en.html > > https://manpages.debian.org/stretch-backports/dgit/dgit-maint-debrebase.7.en.html Yet complicated for me (especially regarding automating/CI). Here're some examples on how my deb branches look like: https://github.com/oss-qm/flatbuffers/commits/debian/maint-1.9.0 https://github.com/oss-qm/go/commits/debian/maint-1.11.1 * canonical ref names * always based on the corresponding upstream's release tag * changes directly as git commits - no text-based patches whatsoever * generic changes below the deb-specific ones While gbp can help a bit here and there, it still far away from an fully-automated process. I'm currently helping myself w/ lots of mappings and import scripts, but I'd like to get rid of maintaining all these little pieces. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: git vs dfsg tarballs
Enrico Weigelt, metux IT consult writes ("git vs dfsg tarballs"):
> Can we agree on some auomatically reproducable (and inversable)
> transformation process from orig to dfsg tree
Clearly the transformation on the *tree* can't be reversible because
in the usual case it is deleting things. So you'll need the history.
With most gitish workflows, the corresponding pre-dfsg upstream
*commit* can be found with `git-merge-base', assuming you have some
uploaded (or pushed) Debian commit and a suitable upstream branch.
> My preferred way (except for rare cases where upstream history is
> extremely huge - like mozilla stuff) would be just branching at the
> upstream's release tag and adding commits for removing the non-dfsg
> files ontop of that. From that branching the debianized branch,
> where all patches are directly applied in git.
I think that most of the workflows recommended in these manpages
https://manpages.debian.org/stretch-backports/dgit/dgit-maint-gbp.7.en.html
https://manpages.debian.org/stretch-backports/dgit/dgit-maint-merge.7.en.html
https://manpages.debian.org/stretch-backports/dgit/dgit-maint-debrebase.7.en.html
ought to have the property I describe above, which I think is
sufficient for you ?
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
git vs dfsg tarballs
Hi folks, I'm often seeing packagers directly putting dfsg'ed trees into their git repos, w/o any indication how the tree was actually created from the original releases. As I'm doing all patching exclusively via git (no text-based patches anymore - adding my changes ontop the upstream release tag and then rebasing for new releases), this (amongst other problems like wild merges) is quite a challenge for efficient (heavily automatic) handling. Can we agree on some auomatically reproducable (and inversable) transformation process from orig to dfsg tree My preferred way (except for rare cases where upstream history is extremely huge - like mozilla stuff) would be just branching at the upstream's release tag and adding commits for removing the non-dfsg files ontop of that. From that branching the debianized branch, where all patches are directly applied in git. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
