Hi Matthijs, On Wed, May 13, 2020 at 08:33:22PM +0200, Matthijs Kooijman wrote: > Package: git-buildpackage > Version: 0.9.15 > Severity: normal > > Hi Guido, > > I'm trying to figure out whether I need pristine-tar or not, and I can't > quite figure it out based on the documentation. > > I understand that, when upstream supplies a tarball, you can use > `gbp import-orig --pristine-tar` to import the tarball along with > pristine-tar data, and then `gbp buildpackage --pristine-tar` will > recreate the pristine tarball again (if needed). > > However, what if there is no upstream tarball and sources are > merged using git directly? Is having a pristine tarball still relevant? > I can imagine that when multiple people check out the packaging repo and > build a package, they want to have the exact same tarball, so it would > make sense to: > - When a tarball is first created, commit pristine-tar data for it. > - When a tarball is created later, use the pristine-tar info to > recreate an identical tarball. > > Or does the `git archive` used to create tarballs from git already > produce consistent tarballs (since there is no upstream tarball to > match, this would be sufficient)? I could not find anything in the git > archive manpage to suggest this, nor any details about what > `pristine-tar` actually *does* to reason about this. A quick try shows > that gbp does indeed manage to reproduce the same tarball twice, but > that gives no guarantees about the same result with different git > versions and different machines, of course.
git-archive *currently* produces stable tarballs (if we feed it correctly which was not always the case (see #851645) - there's no guarantee it'll stay this way forever but it works since several years now so when upstream uses git you can avoid using pristine-tar *if* you don't use multiple tarballs (that's currently a limitation within gbp). > I was further confused by the gbp buildpackage manpage. It says: > > > When gbp buildpackage doesn't find a suitable upstream tarball it will > > create one either using pristine-tar™ or git archive. These options > > determine how the tarball is created: > > This is a bit confusing, I had expected that `git archive` would create a > tarball and then pristine-tar would be used to make it pristine. I later > read the `pristine-tar` manpage and noticed that pristine-tar actually > creates the files from git directly (combining upstream commit and the > pristine-tar commit), so no git archive involved. > > Also, it says: > > > --git-pristine-tar > > Use pristine-tar when generating the upstream tarball if it doesn't > > exist. If this mode is enabled the --git-upstream-tag, > > --git-upstream-tree options have no effect. > > More confusion: If gbp-buildpackage is supposed to generate a > pristine-tar delta when the tarball is first created from git, I would > think I needed to pass `--git-pristine-tar`, but *also* > `--git-upstream-tag` (or `--git-upstream-tree`) to allow gbp to locate > the right upstream commit to base the tarball on? When building pristine-tar *commits* you need a tree (thus --git-upstream-tag or similar) when building a *tarball* pristine-tar can figure it out on it's own (that's where the above manpage snippet appplies to). > Later, I learned from the `pristine-tar` manpage that the pristine-tar > commit actually contains the git tree id of the commit used to create > it, so when recreating the tarball, pristine-tar can find the right > commit on its own. > > But when creating the tarball the first time, `gbp buildpackage` *does* > require the upstream commit, right? yes, see above. > Or is the workflow to, on the first build for a new upstream version, > run buildpackage without `--pristine-tar` to create the tarball and then > run `pristine-tar` manually? I don't think so, since there is also `gbp > --git-pristine-tar-commit`: > > > --git-pristine-tar-commit > > Commit the pristine-tar delta to the pristine-tar branch if a new > > tarball was generated and the pristine-tar data isn't already there. > > Or does this option maybe disable `--git-pristine-tar` when there is no > pristine-tar data yet, and thus "reactivate" the `--git-upstream-*` > options? On first builds (when upstream uses git) you can just do: gbp buildpackage --git-pristine-tar --git-pristine-tar-commit .... that works well but seems a bit confusing to some people so one can also split this apart: # Create upstream tarbal using git-archive: gbp export-orig --no-pristine-tar # Commit the result back to the pristine-tar branch gbp pristine-tar commit ../upstream-tarball_0.1.0.tar.xz > If so (and if pristine-tar is still needed without upstream tarballs), > the workflow could be to always run: > > gbp --git-pristine-tar --git-pristine-tar-commit > --git-upstream-tag=v%(version)s > > And that would then *create* a tarball and pristine-tar data when no > pristine-tar data is present yet, or *use* it (and ignore upstream-tag) > when pristine-tar data *is* present? A quick shows that this might > indeed work like this. Yes. The problem with always passing `--git-pristine-tar-commit` is just that it can cause problems with team maintenance since it will create different pristine-tar *commmits* for different people when somebody forgot to update the pristine-tar branch (since e.g. the author is different) and that can look confusing when pushing later on. > I would be grateful if you could clarify some of this for me. Updating > the docs would be even better, but maybe I can find some time to have a > look at that if you can clarify things here first :-) I'd be very happy to apply patches that clarify this! > Somewhat related: Is git archive even used at all? I ran 0.9.15 with > `--git-verbose`, which seems to print all individual git commands ran, > but I did not see git archive in there? Or is it just not printed? Yes, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=851645 - it's not printed since it's using a different invocation (which by itself might be worth fixing). Hope this helps, -- Guido