Hi Matthijs,
On Wed, May 13, 2020 at 08:33:22PM +0200, Matthijs Kooijman wrote:
> Package: git-buildpackage
> Version: 0.9.15
> Severity: normal
>
> Hi Guido,
>
> I'm trying to figure out whether I need pristine-tar or not, and I can't
> quite figure it out based on the documentation.
>
> I understand that, when upstream supplies a tarball, you can use
> `gbp import-orig --pristine-tar` to import the tarball along with
> pristine-tar data, and then `gbp buildpackage --pristine-tar` will
> recreate the pristine tarball again (if needed).
>
> However, what if there is no upstream tarball and sources are
> merged using git directly? Is having a pristine tarball still relevant?
> I can imagine that when multiple people check out the packaging repo and
> build a package, they want to have the exact same tarball, so it would
> make sense to:
> - When a tarball is first created, commit pristine-tar data for it.
> - When a tarball is created later, use the pristine-tar info to
> recreate an identical tarball.
>
> Or does the `git archive` used to create tarballs from git already
> produce consistent tarballs (since there is no upstream tarball to
> match, this would be sufficient)? I could not find anything in the git
> archive manpage to suggest this, nor any details about what
> `pristine-tar` actually *does* to reason about this. A quick try shows
> that gbp does indeed manage to reproduce the same tarball twice, but
> that gives no guarantees about the same result with different git
> versions and different machines, of course.
git-archive *currently* produces stable tarballs (if we feed it
correctly which was not always the case (see #851645) - there's no
guarantee it'll stay this way forever but it works since several years
now so when upstream uses git you can avoid using pristine-tar *if* you
don't use multiple tarballs (that's currently a limitation within gbp).
> I was further confused by the gbp buildpackage manpage. It says:
>
> > When gbp buildpackage doesn't find a suitable upstream tarball it will
> > create one either using pristine-tar™ or git archive. These options
> > determine how the tarball is created:
>
> This is a bit confusing, I had expected that `git archive` would create a
> tarball and then pristine-tar would be used to make it pristine. I later
> read the `pristine-tar` manpage and noticed that pristine-tar actually
> creates the files from git directly (combining upstream commit and the
> pristine-tar commit), so no git archive involved.
>
> Also, it says:
>
> > --git-pristine-tar
> > Use pristine-tar when generating the upstream tarball if it doesn't
> > exist. If this mode is enabled the --git-upstream-tag,
> > --git-upstream-tree options have no effect.
>
> More confusion: If gbp-buildpackage is supposed to generate a
> pristine-tar delta when the tarball is first created from git, I would
> think I needed to pass `--git-pristine-tar`, but *also*
> `--git-upstream-tag` (or `--git-upstream-tree`) to allow gbp to locate
> the right upstream commit to base the tarball on?
When building pristine-tar *commits* you need a tree (thus
--git-upstream-tag or similar) when building a *tarball* pristine-tar
can figure it out on it's own (that's where the above manpage snippet
appplies to).
> Later, I learned from the `pristine-tar` manpage that the pristine-tar
> commit actually contains the git tree id of the commit used to create
> it, so when recreating the tarball, pristine-tar can find the right
> commit on its own.
>
> But when creating the tarball the first time, `gbp buildpackage` *does*
> require the upstream commit, right?
yes, see above.
> Or is the workflow to, on the first build for a new upstream version,
> run buildpackage without `--pristine-tar` to create the tarball and then
> run `pristine-tar` manually? I don't think so, since there is also `gbp
> --git-pristine-tar-commit`:
>
> > --git-pristine-tar-commit
> > Commit the pristine-tar delta to the pristine-tar branch if a new
> > tarball was generated and the pristine-tar data isn't already there.
>
> Or does this option maybe disable `--git-pristine-tar` when there is no
> pristine-tar data yet, and thus "reactivate" the `--git-upstream-*`
> options?
On first builds (when upstream uses git) you can just do:
gbp buildpackage --git-pristine-tar --git-pristine-tar-commit ....
that works well but seems a bit confusing to some people so one can
also split this apart:
# Create upstream tarbal using git-archive:
gbp export-orig --no-pristine-tar
# Commit the result back to the pristine-tar branch
gbp pristine-tar commit ../upstream-tarball_0.1.0.tar.xz
> If so (and if pristine-tar is still needed without upstream tarballs),
> the workflow could be to always run:
>
> gbp --git-pristine-tar --git-pristine-tar-commit
> --git-upstream-tag=v%(version)s
>
> And that would then *create* a tarball and pristine-tar data when no
> pristine-tar data is present yet, or *use* it (and ignore upstream-tag)
> when pristine-tar data *is* present? A quick shows that this might
> indeed work like this.
Yes. The problem with always passing `--git-pristine-tar-commit` is just
that it can cause problems with team maintenance since it will create
different pristine-tar *commmits* for different people when somebody
forgot to update the pristine-tar branch (since e.g. the author is
different) and that can look confusing when pushing later on.
> I would be grateful if you could clarify some of this for me. Updating
> the docs would be even better, but maybe I can find some time to have a
> look at that if you can clarify things here first :-)
I'd be very happy to apply patches that clarify this!
> Somewhat related: Is git archive even used at all? I ran 0.9.15 with
> `--git-verbose`, which seems to print all individual git commands ran,
> but I did not see git archive in there? Or is it just not printed?
Yes, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=851645 - it's
not printed since it's using a different invocation (which by itself
might be worth fixing).
Hope this helps,
-- Guido
