Hi Gregor,
I reverted one single line in your patch because this would break one of
my test cases namely
Vcs-Svn: svn://svn.debian.org/debian-med/trunk/packages/saint/trunk/
and does not fit my intention. You tried to check whether upstream might
contain a dir /^$pkg\w*$newversion$excludesuffix\.orig$/ which is most
probably never the case (why should upstream name something ".orig"?).
The intention of my check was to verify whether upstream source contains
something like a reasonable directory containing
/^$pkg\w*$newversion$/
and do not try to undirty the tarball if this is the case. It is true
that in this specific case we fail to name the directory ".orig" ...
but that's IMHO something we could hardly control.
I attached another Git formated patch and repeat my question whether
it might make sense to directly commit to the devscripts repository.
Kind regards
Andreas.
On Sat, Aug 25, 2012 at 02:59:33PM +0200, gregor herrmann wrote:
> On Fri, 24 Aug 2012 16:38:13 +0200, Andreas Tille wrote:
>
> > in a (bit longish) thread on debian-devel@l.d.o[1] there was some
> > discussion about enabling uscan to remove files from upstream archives
> > according to some information given in some control file. There was no
> > real consensus about what control file to use. The implementation below
> > is based on using debian/copyright but is easy to switch to other files
> > in case some other consensus might be reached.
>
> Thanks!
>
> > The attached patch does the following:
>
> Attached are some patches based on devscipts+git plus your original
> patch.
>
> The important one is
> 0003-Rename-directory-in-renamed-tarball-to-pkg-newversio.patch which
> renames the directory in the new tarball to
> $pkg - $newversion $excludesuffix .orig
> as recommended in DevRef 6.7.8.2.
>
>
> Cheers,
> gregor
>
>
> --
> .''`. Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
> : :' : Debian GNU/Linux user, admin, and developer - http://www.debian.org/
> `. `' Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
> `- NP: Ry Cooder: Let's Work Together
> From 34d41b8480c3467304383df3633c03ce6c5f34e2 Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gre...@debian.org>
> Date: Sat, 25 Aug 2012 14:12:10 +0200
> Subject: [PATCH 1/5] whitespace
>
> ---
> scripts/uscan.pl | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index e118142..1ea4489 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -2099,7 +2099,7 @@ sub get_main_source_dir($$$) {
> print "-- Dirty tarball found.\n" if $verbose;
> if ( $main_source_dir ) { # if tarball is dirty but does contain a
> $pkg-$newversion dir we will not undirty but leave it as is
> print "-- No idea how to create proper tarball structure - leaving
> as is.\n" if $verbose;
> - return $tempdir;
> + return $tempdir;
> }
> print "-- Move files to subdirectory $pkg-$newversion.\n" if $verbose;
> $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion ;
> @@ -2108,7 +2108,7 @@ sub get_main_source_dir($$$) {
> unless ($file =~ /^\.\.?/) {
> # move("${tempdir}/$file", $main_source_dir) or die("Unable to
> move ${tempdir}/$file directory $main_source_dir\n");
> unless ( move("${tempdir}/$file", $main_source_dir) ) {
> - # HELP: why can't perl move not move directories????
> + # HELP: why can't perl move not move directories????
> print "Perl move seems to be not able to `
> move(\"${tempdir}/$file\", $main_source_dir) ` ... trying system mv\n" if
> $debug;
> system( "mv ${tempdir}/$file $main_source_dir" ) ;
> }
> --
> 1.7.10.4
>
> From 67b13a72c801fb7419254784e443a90506819bc4 Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gre...@debian.org>
> Date: Sat, 25 Aug 2012 14:19:48 +0200
> Subject: [PATCH 2/5] s/fuffix/suffix/g
>
> ---
> scripts/uscan.pl | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index 1ea4489..cd69217 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -1483,9 +1483,9 @@ EOF
> if ( $nfiles_before == $nfiles_after ) {
> print "-- Source tree remains identical - no need for
> repacking.\n" if $verbose;
> } else {
> - my $excludefuffix = '+dfsg' ;
> + my $excludesuffix = '+dfsg' ;
> my $suffix = 'gz' ;
> - my $newfile_base_dfsg =
> "${pkg}_${newversion}${excludefuffix}.orig.tar.$suffix" ;
> + my $newfile_base_dfsg =
> "${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix" ;
> system("cd $tempdir; GZIP='-n -9' tar --owner=root
> --group=root --mode=a+rX -czf \"$absdestdir/$newfile_base_dfsg\"
> $globpattern") == 0
> or die("Excluding files failed (could not create
> tarball)\n");
> $symlink = 'no' # prevent symlinking or renaming
> --
> 1.7.10.4
>
> From d40d483fb870b1a8efea2b2c77a70cf3358f3fb5 Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gre...@debian.org>
> Date: Sat, 25 Aug 2012 14:29:50 +0200
> Subject: [PATCH 3/5] Rename directory in renamed tarball to $pkg -
> $newversion $excludesuffix .orig
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> DevRef ยง 6.7.8.2
> ---
> scripts/uscan.pl | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index cd69217..cb29f44 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -75,7 +75,7 @@ sub dehs_die ($);
> sub dehs_output ();
> sub quoted_regex_replace ($);
> sub safe_replace ($$);
> -sub get_main_source_dir($$$);
> +sub get_main_source_dir($$$$);
>
> sub usage {
> print <<"EOF";
> @@ -1465,7 +1465,8 @@ EOF
> system('unzip', '-q', '-a', '-d', $tempdir,
> "$destdir/$newfile_base") == 0
> or die("Repacking from zip to tar.gz failed (could not
> unzip)\n");
> }
> - my $main_source_dir = get_main_source_dir($tempdir, $pkg,
> $newversion);
> + my $excludesuffix = '+dfsg' ;
> + my $main_source_dir = get_main_source_dir($tempdir, $pkg,
> $newversion, $excludesuffix);
> unless ( -d $main_source_dir ) {
> print STDERR "Error: $main_source_dir is no directory";
> }
> @@ -1483,7 +1484,6 @@ EOF
> if ( $nfiles_before == $nfiles_after ) {
> print "-- Source tree remains identical - no need for
> repacking.\n" if $verbose;
> } else {
> - my $excludesuffix = '+dfsg' ;
> my $suffix = 'gz' ;
> my $newfile_base_dfsg =
> "${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix" ;
> system("cd $tempdir; GZIP='-n -9' tar --owner=root
> --group=root --mode=a+rX -czf \"$absdestdir/$newfile_base_dfsg\"
> $globpattern") == 0
> @@ -2070,8 +2070,8 @@ sub safe_replace($$) {
> }
> }
>
> -sub get_main_source_dir($$$) {
> - my ($tempdir, $pkg, $newversion) = @_;
> +sub get_main_source_dir($$$$) {
> + my ($tempdir, $pkg, $newversion, $excludesuffix) = @_;
> my $fcount = 0;
> my $main_source_dir = '' ;
> my $any_dir = '' ;
> @@ -2083,7 +2083,7 @@ sub get_main_source_dir($$$) {
> $fcount++;
> if ( -d $tempdir.'/'.$file ) {
> $any_dir = $tempdir . '/' . $file ;
> - $main_source_dir = $any_dir if ( $file =~
> /^$pkg\w*$newversion$/i ) ;
> + $main_source_dir = $any_dir if ( $file =~
> /^$pkg\w*$newversion$excludesuffix\.orig$/i ) ;
> }
> }
> }
> @@ -2092,7 +2092,7 @@ sub get_main_source_dir($$$) {
> }
> if ( $fcount == 1 and $any_dir ) {
> # Unusual base dir in tarball - should be rather something like
> ${pkg}-${newversion}
> - $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion ;
> + $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion .
> $excludesuffix . '.orig';
> move($any_dir, $main_source_dir) or die("Unable to move $any_dir
> directory $main_source_dir\n");
> return $main_source_dir ;
> }
> @@ -2102,7 +2102,7 @@ sub get_main_source_dir($$$) {
> return $tempdir;
> }
> print "-- Move files to subdirectory $pkg-$newversion.\n" if $verbose;
> - $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion ;
> + $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion .
> $excludesuffix . '.orig';
> mkdir($main_source_dir) or die("Unable to create temporary source
> directory $main_source_dir\n");
> foreach my $file (@files) {
> unless ($file =~ /^\.\.?/) {
> --
> 1.7.10.4
>
> From 482540af9ac9e8783a1ae85eac63965f60a2cbbd Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gre...@debian.org>
> Date: Sat, 25 Aug 2012 14:41:18 +0200
> Subject: [PATCH 4/5] add support for .jar
>
> ---
> scripts/uscan.pl | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index cb29f44..0bab138 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -1410,7 +1410,7 @@ EOF
> $newfile_base = $newfile_base_gz;
> }
>
> - if ($repack and $newfile_base =~ /^(.*)\.zip$/) {
> + if ($repack and $newfile_base =~ /^(.*)\.(zip|jar)$/) {
> print "-- Repacking from zip to .tar.gz\n" if $verbose;
>
> system('command -v unzip >/dev/null 2>&1') >> 8 == 0
> @@ -1422,12 +1422,12 @@ EOF
> my $hidden = ".[!.]*";
> my $absdestdir = abs_path($destdir);
> system('unzip', '-q', '-a', '-d', $tempdir, "$destdir/$newfile_base")
> == 0
> - or die("Repacking from zip to tar.gz failed (could not unzip)\n");
> + or die("Repacking from zip or jar to tar.gz failed (could not
> unzip)\n");
> if (defined glob("$tempdir/$hidden")) {
> $globpattern .= " $hidden";
> }
> system("cd $tempdir; GZIP='-n -9' tar --owner=root --group=root
> --mode=a+rX -czf \"$absdestdir/$newfile_base_gz\" $globpattern") == 0
> - or die("Repacking from zip to tar.gz failed (could not create
> tarball)\n");
> + or die("Repacking from zip or jar to tar.gz failed (could not create
> tarball)\n");
> unlink "$destdir/$newfile_base";
> $newfile_base = $newfile_base_gz;
> }
> @@ -1463,7 +1463,7 @@ EOF
> system('command -v unzip >/dev/null 2>&1') >> 8 == 0
> or die("unzip binary not found. This would serve as
> fallback because tar just failed.\n");
> system('unzip', '-q', '-a', '-d', $tempdir,
> "$destdir/$newfile_base") == 0
> - or die("Repacking from zip to tar.gz failed (could not
> unzip)\n");
> + or die("Repacking from zip or jar to tar.gz failed (could
> not unzip)\n");
> }
> my $excludesuffix = '+dfsg' ;
> my $main_source_dir = get_main_source_dir($tempdir, $pkg,
> $newversion, $excludesuffix);
> --
> 1.7.10.4
>
> From f85b9c16db8e388cd6a62fbe8e8aabe9ef082c99 Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gre...@debian.org>
> Date: Sat, 25 Aug 2012 14:52:01 +0200
> Subject: [PATCH 5/5] Output messages about the files-excluding action.
>
> ---
> scripts/uscan.pl | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index 0bab138..ba458ea 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -1443,6 +1443,7 @@ EOF
> }
> }
>
> + my $excludesuffix = '+dfsg';
> if ( !$no_exclusion ) {
> my $data = Dpkg::Control::Hash->new();
> $data->load('debian/copyright');
> @@ -1465,7 +1466,6 @@ EOF
> system('unzip', '-q', '-a', '-d', $tempdir,
> "$destdir/$newfile_base") == 0
> or die("Repacking from zip or jar to tar.gz failed (could
> not unzip)\n");
> }
> - my $excludesuffix = '+dfsg' ;
> my $main_source_dir = get_main_source_dir($tempdir, $pkg,
> $newversion, $excludesuffix);
> unless ( -d $main_source_dir ) {
> print STDERR "Error: $main_source_dir is no directory";
> @@ -1488,7 +1488,7 @@ EOF
> my $newfile_base_dfsg =
> "${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix" ;
> system("cd $tempdir; GZIP='-n -9' tar --owner=root
> --group=root --mode=a+rX -czf \"$absdestdir/$newfile_base_dfsg\"
> $globpattern") == 0
> or die("Excluding files failed (could not create
> tarball)\n");
> - $symlink = 'no' # prevent symlinking or renaming
> + $symlink = 'files-excluded' # prevent symlinking or renaming
> }
> }
> }
> @@ -1519,6 +1519,8 @@ EOF
> print " and symlinked $renamed_base to it\n";
> } elsif ($symlink eq 'rename') {
> print " and renamed it as $renamed_base\n";
> + } elsif ($symlink eq 'files-excluded') {
> + print " and removed files from it in
> ${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix\n";
> }
> } elsif ($dehs) {
> my $msg = "Successfully downloaded updated package $newfile_base";
> @@ -1527,6 +1529,8 @@ EOF
> $msg .= " and symlinked $renamed_base to it";
> } elsif ($symlink eq 'rename') {
> $msg .= " and renamed it as $renamed_base";
> + } elsif ($symlink eq 'files-excluded') {
> + $msg .= " and removed files from it in
> ${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix\n";
> } else {
> $dehs_tags{'target'} = $newfile_base;
> }
> @@ -1537,6 +1541,8 @@ EOF
> print " and symlinked $renamed_base to it\n";
> } elsif ($symlink eq 'rename') {
> print " and renamed it as $renamed_base\n";
> + } elsif ($symlink eq 'files-excluded') {
> + print " and removed files from it in
> ${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix\n";
> }
> }
> last;
> --
> 1.7.10.4
>
--
http://fam-tille.de
>From ba580704fab55b47f706b0e76e1a6bd867ebb095 Mon Sep 17 00:00:00 2001
From: Andreas Tille <ti...@debian.org>
Date: Sun, 26 Aug 2012 15:20:46 +0200
Subject: [PATCH] Undo one part of Gregor's changes. It is not safe to try to
undirty the upstream tarball if there is something that
looks like a reasonable directory.
---
scripts/uscan.pl | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/uscan.pl b/scripts/uscan.pl
index 7155f2b..f575a0f 100755
--- a/scripts/uscan.pl
+++ b/scripts/uscan.pl
@@ -2089,7 +2089,9 @@ sub get_main_source_dir($$$$) {
$fcount++;
if ( -d $tempdir.'/'.$file ) {
$any_dir = $tempdir . '/' . $file ;
- $main_source_dir = $any_dir if ( $file =~ /^$pkg\w*$newversion$excludesuffix\.orig$/i ) ;
+ # check whether there is some dir in upstream source which looks reasonable
+ # If such dir exists, we do not try to undirty the directory structure
+ $main_source_dir = $any_dir if ( $file =~ /^$pkg\w*$newversion$/i ) ;
}
}
}
--
1.7.10.4
