Control: forcemerge 882511 1068483
Hi!
After replying to Adrian's report, I recalled there being a previous one
that was similar, and then recalled that I had an even older branch that
implemented a potential solution for this. See below.
On Thu, 2017-11-23 at 16:23:29 +0100, Ximin Luo wrote:
> Package: dpkg-dev
> Version: 1.19.0.4
> Severity: wishlist
> Tags: patch
> dpkg-buildpackage currently does not automatically list the source .dsc nor
> its hash in the call to dpkg-genbuildinfo when doing a binary-only build. This
> is understandable because in a binary-only build, dpkg-buildpackage does not
> have any concept of a source package and therefore does not know (and cannot
> verify) if the working tree was actually generated from any .dsc or not.
>
> However, the caller knows this information, and it is useful for reproducible
> builds to track exactly which (i.e. hash-wise) source code generates which
> binary packages. So it should be possible for the caller to tell
> dpkg-buildpackage, "yes please do include the .dsc hash in the buildinfo, I am
> telling you it is correct, you can assume this safely".
>
> Tools like sbuild/pbuilder could then do this, as well as users or rebuilders.
>
> The attached patch implements this in the simplest way possible. It allows the
> caller to run something like:
>
> $ dpkg-buildpackage --no-sign -b --buildinfo-option=--build=full
>
> The resulting $pkg_$ver_$arch.buildinfo then contains the .dsc and its hash.
>
> However this requires the caller to know which option to pass, which would
> either be
>
> --buildinfo-option=--build=full
> --buildinfo-option=--build=any,source
> --buildinfo-option=--build=all,source
>
> depending on whether the original build request (to dpkg-buildpackage) was a
> -b, -B, or -A.
>
> For this reason, it may be better (more usable) to add a
> --force-source-in-buildinfo
> flag (or similar name) and when this is switched on, do this instead:
>
> -push @buildinfo_opts, "--build=$build_types" if
> build_has_none(BUILD_DEFAULT);
> +push @buildinfo_opts, "--build=$build_types,source" if
> build_has_none(BUILD_DEFAULT);
>
> Let me know if you like this idea and I'll be happy to implement that instead
> of
> the attached patch.
The problem with this solution is that it is prone do accidental use,
as it is very easy for a user to unknowingly have recreated the sources
from a locally extracted tree (be that modified or not).
On Sat, 2024-04-06 at 02:57:40 +0200, Guillem Jover wrote:
> On Sat, 2024-04-06 at 02:56:02 +0300, Adrian Bunk wrote:
> > Package: dpkg-dev
> > Version: 1.22.6
> > Severity: normal
> > X-Debbugs-Cc: reproducible-bui...@lists.alioth.debian.org
>
> > A thought I already wrote in a recent debian-devel discussion:
> >
> > In theory source package filenames should be eternally and globally
> > unique, but in practice there are cornercases where this assumption
> > might break like for example:
> > - *stable-security does not currently have a copy of the sources
> > in the main archive, one always have to upload the source archive
> > there and this might accidentally be a different orig.tar
> > - dak does not keep an eternal history of everything it ever knew,
> > e.g. RM and later re-NEW of a source version might have a different
> > source .orig.tar or even different sources for a Debian revision
> > - Debian and Ubuntu might have different orig.tar for the same version,
> > if Ubuntu updated a package before Debian did, or with packages
> > were development is completely independent in Debian and Ubuntu
> > (e.g. OpenStack, KDE)
> >
> > The reason for different files might be as trivial as "git archive"
> > not always producing the same output when running in different
> > environments, e.g. the autogenerated tarball for a git tag on Github
> > might have different checksums depending on whether it is downloaded
> > today or next year despite identical contents due to slightly
> > different gzip compression.
> >
> > Should buildinfo files contain the hashes of the source package,
> > to clearly define what sources have been used?
>
> Ideally? Yes, and I think we considered that at the time when we
> introduced the .buildinfo files. Although a ref to the .dsc does get
> included if the build is also creating the source package.
>
> The problem is that when dpkg-buildpackage is not building the source
> package, there is no guarantee the source package is going to be
> present, or that if it is present it matches what is currently being
> built from the working directory.
I've now finished the change I had in that branch, which implements
support so that dpkg-buildpackage can be passed a .dsc or a source-dir,
and in the former will first extract it, and for both then it will
change directory to the source tree. If it got passed a .dsc then it
will instruct dpkg-genbuildinfo to include a ref to it.
Which I think accomplishes the requested behavior in a safe way? I've
attached what I've got, which I'm planning on merging for 1.22.7. I'll
probably split that into two commits though before merging.
Thanks,
Guillem
From 972a9630fdb25705ca011c9b6b9c8a0a75bca6ea Mon Sep 17 00:00:00 2001
From: Guillem Jover <guil...@debian.org>
Date: Wed, 17 Aug 2016 00:54:47 +0200
Subject: [PATCH] dpkg-buildpackage: Add support for building from a .dsc or
dir
This adds support to build directly from a source package .dsc, or to
specify a source directory to use for the build. In the first case we
will extract the source package. For both .dsc and directory we will
change directory to the source tree.
For the .dsc case, we can then tell dpkg-genbuildinfo to include
references to the source package, as we can ascertain the provenance of
the source tree.
Closes: #882511
---
man/dpkg-buildpackage.pod | 15 +++++++---
scripts/dpkg-buildpackage.pl | 54 ++++++++++++++++++++++++++++++++++--
2 files changed, 62 insertions(+), 7 deletions(-)
diff --git a/man/dpkg-buildpackage.pod b/man/dpkg-buildpackage.pod
index a3dc324de..7e74ce8b7 100644
--- a/man/dpkg-buildpackage.pod
+++ b/man/dpkg-buildpackage.pod
@@ -27,21 +27,26 @@ dpkg-buildpackage - build binary or source packages from sources
=head1 SYNOPSIS
-B<dpkg-buildpackage>
-[I<option>...]
+B<dpkg-buildpackage> [I<option>...] [B<-->] [I<filename>.dsc|I<directory>]
=head1 DESCRIPTION
B<dpkg-buildpackage>
is a program that automates the process of building a Debian package.
-It
-consists of the following steps:
+
+The I<filename>.dsc and I<directory> arguments are
+supported since dpkg 1.22.7.
+
+It consists of the following steps:
=over
=item B<1.>
It runs the B<preinit> hook before reading any source file.
+If a B<.dsc> file has been specified it unpacks it and changes directory to it,
+if a I<directory> has been specified it changes directory to it,
+otherwise it expects the current directory to contain the source tree.
It prepares the build environment by setting various environment
variables (see B<ENVIRONMENT>), runs the B<init> hook, and calls
B<dpkg-source --before-build> (unless B<-T> or B<--target>
@@ -87,6 +92,8 @@ It runs the B<buildinfo>
hook and calls B<dpkg-genbuildinfo> to generate a B<.buildinfo> file.
Several B<dpkg-buildpackage> options are forwarded to
B<dpkg-genbuildinfo>.
+If a I<.dsc> was specified, then it will be referenced in the generated
+B<.buildinfo> file, as we can ascertain the provenance of the source tree.
=item B<7.>
diff --git a/scripts/dpkg-buildpackage.pl b/scripts/dpkg-buildpackage.pl
index df2edded9..8cf857600 100755
--- a/scripts/dpkg-buildpackage.pl
+++ b/scripts/dpkg-buildpackage.pl
@@ -64,7 +64,7 @@ later for copying conditions. There is NO warranty.
sub usage {
printf g_(
-'Usage: %s [<option>...]')
+'Usage: %s [<option>...] [--] [<filename.dsc>|<directory>]')
. "\n\n" . g_(
'Options:
--build=<type>[,...] specify the build <type>: full, source, binary,
@@ -173,6 +173,7 @@ my $parallel;
my $parallel_force = 0;
my $checkbuilddep = 1;
my $check_builtin_builddep = 1;
+my $source;
my @source_opts;
my $check_command = $ENV{DEB_CHECK_COMMAND};
my @check_opts;
@@ -200,6 +201,7 @@ my $changedby;
my $desc;
my $buildinfo_file;
my @buildinfo_opts;
+my $buildinfo_with_dsc = 0;
my $changes_file;
my @changes_opts;
my @hook_names = qw(
@@ -408,8 +410,14 @@ while (@ARGV) {
} elsif (/^-R(.*)$/ or /^--rules-file=(.*)$/) {
my $arg = $1;
@debian_rules = split ' ', $arg;
- } else {
+ } elsif ($_ eq '--') {
+ $source = shift @ARGV;
+ last;
+ } elsif (/^-/) {
usageerr(g_('unknown option or argument %s'), $_);
+ } else {
+ $source = $_;
+ last;
}
}
@@ -486,6 +494,40 @@ if ($build_opts->has('terse')) {
set_build_profiles(@build_profiles) if @build_profiles;
+# Handle specified source trees.
+if (defined $source) {
+ if (-d $source) {
+ chdir $source
+ or syserr(g_('cannot change directory to %s'), $source);
+ } elsif (-f $source) {
+ require Dpkg::Source::Package;
+
+ run_cmd('dpkg-source', @source_opts, '--extract', $source);
+
+ my %srcopt = (
+ no_check => 0,
+ no_overwrite_dir => 1,
+ require_valid_signature => 0,
+ require_strong_checksums => 0,
+ );
+ my $srcpkg = Dpkg::Source::Package->new(
+ filename => $source,
+ options => \%srcopt,
+ );
+
+ my $srcdir = $srcpkg->get_basename();
+ $srcdir =~ s/_/-/g;
+
+ chdir $srcdir
+ or syserr(g_('cannot change directory to %s'), $srcdir);
+
+ # We can now let dpkg-genbuildinfo know that we can include the .dsc
+ # in the .buildinfo file as we handled it ourselves, and what we are
+ # building matches the source we extracted it from.
+ $buildinfo_with_dsc = 1;
+ }
+}
+
my $changelog = changelog_parse();
my $ctrl = Dpkg::Control::Info->new();
@@ -682,7 +724,13 @@ if (build_has_any(BUILD_BINARY)) {
$buildinfo_file //= "../$pva.buildinfo";
-push @buildinfo_opts, "--build=$build_types" if build_has_none(BUILD_DEFAULT);
+my $buildinfo_buildtypes;
+if (build_has_none(BUILD_DEFAULT) || $buildinfo_with_dsc) {
+ $buildinfo_buildtypes = $build_types;
+ $buildinfo_buildtypes .= ",source" if $buildinfo_with_dsc;
+}
+
+push @buildinfo_opts, "--build=$buildinfo_buildtypes" if $buildinfo_buildtypes;
push @buildinfo_opts, "--admindir=$admindir" if $admindir;
push @buildinfo_opts, "-O$buildinfo_file" if $buildinfo_file;
--
2.43.0
