tags 628779 + patch
thanks
> Aha, I caught it in action. Looks like possibly content-autonegotiation
The cause is definitely an out of sync parent mirror (They've ignored my
previous requests to run their mirror properly). However I get free
access to it, so I'm prepared to live with the pain.
The Package lists are failing verification against Release, but
debmirror isn't checking the Packages lists after download, only
before-hand. Thus it notices Packages.gz is invalid after downloading
it, downloads Packages too, pulls bz2 due to autonegotiation, and then
tries to parse this.
Easy way to reproduce these issues:
Put a mirror behind a normally configured apache (with content
autonegotiation enabled) Break the Packages files. debmirror from it.
Attached are two patches, both a bit hacky, to work-around this:
0001-Detect-errors-when-parsing-Packages.patch:
Abort when running into a parse error, and refuse to clean up if there
were errors.
0002-Before-download-Package-list-files-are-checked-with-.patch:
Before download, Package list files are checked with prepended $tempdir.
At download time, Package list files are verified without it. Store both
forms in file_lists.
SR
--
Stefano Rivera
http://tumbleweed.org.za/
H: +27 21 465 6908 C: +27 72 419 8559 UCT: x3127
From 5c6e9506e63f5f818e812e936a011a049cbef2c2 Mon Sep 17 00:00:00 2001
From: Stefano Rivera <stef...@rivera.za.net>
Date: Mon, 6 Jun 2011 09:18:52 +0200
Subject: [PATCH 1/2] Detect errors when parsing Packages
---
debmirror | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/debmirror b/debmirror
index ae50245..ce641c5 100755
--- a/debmirror
+++ b/debmirror
@@ -1071,6 +1071,7 @@ say("Parsing Packages and Sources files ...");
die "$file: $!" if $!;
}
my ($filename)=m/^Filename:\s+(.*)/im;
+ die "Error encountered while parsing $file." if !defined($filename);
$filename=~s:/+:/:; # remove redundant slashes in paths
my ($deb_section)=m/^Section:\s+(.*)/im;
my ($deb_priority)=m/^Priority:\s+(.*)/im;
@@ -1398,7 +1399,7 @@ if (! $do_dry_run) {
}
# Post mirror cleanup.
-cleanup_unknown_files() if ($post_cleanup && ! $debmarshal);
+cleanup_unknown_files() if ($post_cleanup && ! $debmarshal && $num_errors == 0);
# Mirror cleanup for directories.
if (! $use_cache && ($pre_cleanup || $post_cleanup)) {
--
1.7.4.4
From c9c758a208cba6e42023136a60a3d46cdb4df629 Mon Sep 17 00:00:00 2001
From: Stefano Rivera <stef...@rivera.za.net>
Date: Mon, 6 Jun 2011 12:04:21 +0200
Subject: [PATCH 2/2] Before download, Package list files are checked with
prepended $tempdir. At download time, Package list
files are verified without it. Store both forms in
file_lists.
---
debmirror | 5 ++++-
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/debmirror b/debmirror
index ce641c5..520083e 100755
--- a/debmirror
+++ b/debmirror
@@ -906,9 +906,12 @@ foreach my $dist (keys %distset) {
$checksum_type=$1;
}
elsif (/^ / && defined $checksum_type) {
- my ($checksum, $size, $filename) = /^ +([a-z0-9]+) +(\d+) +(.*)$/;
+ my ($checksum, $size, $filename) = /^ +([a-z0-9]+) +(\d+) +(.*)$/;
$file_lists{"$tempdir/dists/$dist/$filename"}{$checksum_type} = $checksum;
$file_lists{"$tempdir/dists/$dist/$filename"}{size} = $size;
+ # At download time, files are verified without $tempdir
+ $file_lists{"dists/$dist/$filename"}{$checksum_type} = $checksum;
+ $file_lists{"dists/$dist/$filename"}{size} = $size;
}
}
close RELEASE;
--
1.7.4.4
