Your message dated Mon, 06 Aug 2018 09:34:11 +0000
with message-id <[email protected]>
and subject line Bug#629922: fixed in debmirror 1:2.30
has caused the Debian Bug report #629922,
regarding deals badly with unreliable proxies
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
629922: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=629922
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: debmirror
Version: 1:2.14~bpo60+1
Severity: normal
Hi,
At one particular customer, where there is a firewall which blocks just
about everything, I need to go through a somewhat paranoid proxy.
Because of the way this proxy is set up, the chain has to be something
like this:
putty SSH tunnel to server running Debian -> cntlm running on Windows
machine -> active directory-authenticated proxy server running in
another country -> (possibly more things) -> public Debian mirror
I run debmirror like so:
debmirror --section=main,contrib --ignore-release-gpg -p
--host=ftp.uk.debian.org --method=http --proxy=http://localhost:3128/
--dist=squeeze --arch=amd64 --nosource --nocleanup --progress
--ignore-small-errors --exclude=clamav-testfiles --rsync-extra=none <directory>
(later on it's also run without the nocleanup, and with some other
options added, but that's irrelevant for this bug report)
This works for the most part, except for one detail:
Something in the above chain (likely the proxy server on which I have no
access) seems to be configured to abort large downloads (starting at
files of somewhere between 50 and 60MiB). Repeating the download as a
simple HTTP GET at that point doesn't fix it; the proxy just notices it
has an object from the requested URL already in its cache, and will
happily produce it -- truncating the download at the exact same place.
However, if the download is then repeated with a Range: header so that
it only requests the missing parts of the file, the proxy will behave
correctly and produce the missing parts.
Unfortunately, debmirror doesn't even try to do that. If a download is
truncated, the already-downloaded file range is removed(!) and the
mirror run is aborted.
When this happens, I have to use wget to download the problematic file
manually (which will get the truncated part with a normal GET, notice
the file isn't complete, and send another GET request with a Range:
header so that the file is completed, which then (usually) succeeds. I
then need to restart debmirror, where it will continue on for a few more
files, until it encounters another file over 50-60MiB where it will fail
again.
It would be great if debmirror could retry downloading if files are
truncated. Alternatively, if that's too complex to implement, it would
be okay if there was an option for me to have it soldier on downloading
even in the face of truncated downloads, possibly with a file containing
errors that I can then inspect (and possibly retry downloading them
manually with wget or something).
Thanks,
--
This end should point toward the ground if you want to go to space.
If it starts pointing toward space you are having a bad problem and you
will not go to space today.
-- http://xkcd.com/1133/
--- End Message ---
--- Begin Message ---
Source: debmirror
Source-Version: 1:2.30
We believe that the bug you reported is fixed in the latest version of
debmirror, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to [email protected],
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Colin Watson <[email protected]> (supplier of updated debmirror package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing [email protected])
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Mon, 06 Aug 2018 10:07:54 +0100
Source: debmirror
Binary: debmirror
Architecture: source
Version: 1:2.30
Distribution: unstable
Urgency: medium
Maintainer: Colin Watson <[email protected]>
Changed-By: Colin Watson <[email protected]>
Description:
debmirror - Debian partial mirror script, with ftp and package pool support
Closes: 629922
Changes:
debmirror (1:2.30) unstable; urgency=medium
.
* Make the error message when mkdir fails more helpful.
* Restart partial HTTP downloads (closes: #629922).
* Use /usr/share/dpkg/pkg-info.mk rather than equivalent hand-written
code.
* Upgrade to debhelper v9.
Checksums-Sha1:
7e7e84558ab305565ffe7d4639a7534518b2bafa 1665 debmirror_2.30.dsc
bbc5a6ee85f1218423ceacaa7cd1163b3914f52b 53668 debmirror_2.30.tar.xz
e5465d16d000b06d41f8e38afafc0aa446cf18ce 7062 debmirror_2.30_source.buildinfo
Checksums-Sha256:
6050ef7fc220580f8981befe704903c731e0d8de4191a69b2d91ee091701bd0d 1665
debmirror_2.30.dsc
7e164a4265c60c8c28813e50c7dcd60113a3cbe352220e785be8f3b8ae276ee3 53668
debmirror_2.30.tar.xz
3876fc7e6011e00fd8c5bfc0fb7d7d8c5ee4c52af120dd92b44b5b35bc87b282 7062
debmirror_2.30_source.buildinfo
Files:
c60ea567e4eb0154d6f340154f212c71 1665 net optional debmirror_2.30.dsc
093d4647c1585b2ff46fa1d0ac0fa1db 53668 net optional debmirror_2.30.tar.xz
88cd52af8e43a5820291a92b835851b7 7062 net optional
debmirror_2.30_source.buildinfo
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEErApP8SYRtvzPAcEROTWH2X2GUAsFAltoEAwACgkQOTWH2X2G
UAs4txAAvJtgldFqZXA+CUzNf1/VXb1VR6L6+36c4+XxAqZekvvwpOhGET0JX21Q
i7SKbN+ExYzLs9A8GhTRXAjxUw61RInX/1I4G1NiCNfxasKxD90mq/ixc/itP0BL
C5KeD0m9dBmywvepNfDplFZ9Sf1YREcRR8Z4uS2Cj+hbY2vHRxIpIFUB3cvAqHuV
H0+mct8e00BS21zT3tGb3bjAqEmSng9O2JVaKEz9Eei09YUxZ/jaCTZ448BTM/AZ
kD+OyA8bIxONUzBsYppmJTS4tqs//v2M7f0XEG/AzQwnEh4KaTlLnSkQbsazWcey
NJDTU0VJioypVaWF/nREz9iK5j6YB0Zk8rvE3v9kegkIdx5Oxr0IqErC6KYVCjA5
ON7TRKoGF/yT00sFIpwm+zLtSk2h+oVSG4riRi+l5+v9xChggAOpuh86eJccj5ov
FhKJxXIwC9c1bAUOhFVyVPR9k0D+6ZBrNb3cNHAX/XyOFFXJmMaeJppMqv22TyRM
lXRcMjkYeni2VNWNBDsRrsoIp7fmirNm1tE368JWcUj38zAiYFyRved0KiUqDaHG
MR6RDuytBFPGP2oovTEbZEIjjQXfpSacLrUmZ/zS5zHmBt4/JzgIG2qr0iAFpD/N
7Uh2YI2BYNRhDKMKph8dGoqWsyQ0L84owpuHQ+l/pI7dkK/x8CM=
=2vcB
-----END PGP SIGNATURE-----
--- End Message ---
