Bug#878085: simple-cdd: Creating the mirror is failing due to missing files

[David Touzeau]

Thanks, fixed for our side " ftp.fr.debian.org/debian"



-Message d'origine-
De : Vagrant Cascadian [mailto:vagr...@debian.org] 
Envoyé : lundi 9 octobre 2017 21:21
À : Andre Rodier ; David Touzeau 
Cc : 878...@bugs.debian.org
Objet : Re: Bug#878085: simple-cdd: Creating the mirror is failing due to
missing files

Control: merge 877987 878085
Control: reassign 878085 mirrors
Control: affects 878085 simple-cdd

On 2017-10-09, Andre Rodier wrote:
> When running simple-cdd --do-mirror, the program fails with one 
> missing file. I have tried with multiple mirrors, but the file is 
> missing on all of them.

It appears to be a problem on the mirrors.


> simple-cdd --do-mirror --locale en_US --dist stable
>
> [...]
>
> 2017-10-09 18:09:27,290 DEBUG skipping download: 
> /home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedicat
> ion-5.0.txt checksum matched
> 2017-10-09 18:09:27,290 DEBUG skipping download: 
> /home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedicat
> ion-5.0.vi.txt checksum matched
> 2017-10-09 18:09:27,290 DEBUG downloading: 
> /home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedicat
> ion-9.0.bn.txt
> Traceback (most recent call last):

Several doc/dedication/dedication-9.0.* files present in extrafiles[1] are
missing from the mirrors. Since simple-cdd uses extrafiles to determine
which files to download, missing files cause it to fail.

You can workaround the issue by setting mirror_files='README tools/' in
profiles/default.conf until it gets fixed.

I've mentioned this on the #debian-mirrors IRC channel, and waldi fixed it;
it may take several hours to propegate.

[1] https://cdn-aws.deb.debian.org/debian/extrafiles


live well,
  vagrant

Bug#878085: simple-cdd: Creating the mirror is failing due to missing files

[Vagrant Cascadian]

Control: merge 877987 878085
Control: reassign 878085 mirrors
Control: affects 878085 simple-cdd

On 2017-10-09, Andre Rodier wrote:
> When running simple-cdd --do-mirror, the program fails with one
> missing file. I have tried with multiple mirrors, but the file is
> missing on all of them.

It appears to be a problem on the mirrors.


> simple-cdd --do-mirror --locale en_US --dist stable
>
> [...]
>
> 2017-10-09 18:09:27,290 DEBUG skipping download: 
> /home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedication-5.0.txt
>  checksum matched
> 2017-10-09 18:09:27,290 DEBUG skipping download: 
> /home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedication-5.0.vi.txt
>  checksum matched
> 2017-10-09 18:09:27,290 DEBUG downloading: 
> /home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedication-9.0.bn.txt
> Traceback (most recent call last):

Several doc/dedication/dedication-9.0.* files present in extrafiles[1]
are missing from the mirrors. Since simple-cdd uses extrafiles to
determine which files to download, missing files cause it to fail.

You can workaround the issue by setting mirror_files='README tools/' in
profiles/default.conf until it gets fixed.

I've mentioned this on the #debian-mirrors IRC channel, and waldi fixed
it; it may take several hours to propegate.

[1] https://cdn-aws.deb.debian.org/debian/extrafiles


live well,
  vagrant


signature.asc
Description: PGP signature

Bug#878085: simple-cdd: Creating the mirror is failing due to missing files

[Andre Rodier]

Package: simple-cdd
Version: 0.6.5
Severity: important

Dear Maintainer,

*** Reporter, please consider answering these questions, where appropriate ***

When running simple-cdd --do-mirror, the program fails with one missing file. I 
have tried with multiple mirrors, but the file is missing on all of them.

Below an example on what is happening.

---
simple-cdd --do-mirror --locale en_US --dist stable

[...]

2017-10-09 18:09:27,290 DEBUG skipping download: 
/home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedication-5.0.txt
 checksum matched
2017-10-09 18:09:27,290 DEBUG skipping download: 
/home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedication-5.0.vi.txt
 checksum matched
2017-10-09 18:09:27,290 DEBUG downloading: 
/home/seequestor/Projects/simple-cdd/tmp/mirror/doc/dedication/dedication-9.0.bn.txt
Traceback (most recent call last):
  File "/usr/bin/simple-cdd", line 658, in 
scdd.build_mirror()
  File "/usr/bin/simple-cdd", line 270, in build_mirror
self.run_tool("mirror", tool)
  File "/usr/bin/simple-cdd", line 367, in run_tool
tool.run()
  File "/usr/lib/python3/dist-packages/simple_cdd/tools/mirror_wget.py", line 
93, in run
_download(x["url"], x["absname"], checksums=extrafile_sums, 
relname=x["relname"])
  File "/usr/lib/python3/dist-packages/simple_cdd/tools/mirror_wget.py", line 
55, in _download
request.urlretrieve(url, filename=output)
  File "/usr/lib/python3.5/urllib/request.py", line 188, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 504, in error
result = self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 696, in http_error_302
return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

---

*** End of the template - remove these template lines ***


-- System Information:
Debian Release: 9.1
  APT prefers stable
  APT policy: (990, 'stable'), (500, 'stable-updates'), (500, 'testing')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8), 
LANGUAGE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages simple-cdd depends on:
ii  dctrl-tools 2.24-2+b1
ii  debian-cd   3.1.20
ii  lsb-release 9.20161125
ii  python3 3.5.3-1
ii  python3-simple-cdd  0.6.5
ii  reprepro5.1.1-1
ii  rsync   3.1.2-1
ii  wget1.18-5

Versions of packages simple-cdd recommends:
ii  dose-distcheck  5.0.1-8

Versions of packages simple-cdd suggests:
ii  qemu-kvm 1:2.8+dfsg-6+deb9u3
ii  qemu-system  1:2.8+dfsg-6+deb9u3

-- no debconf information