Bug#823339: apt fails from the last three days saying Clearsigned file isn't valid

2016-05-03 Thread shirish शिरीष 
in-line :-

On 03/05/2016, David Kalnischkies  wrote:
>
>
> The bad InRelease files should be in partial/, perhaps their content can
> tell you what this something is. Looks for me like a proxy (maybe from
> your ISP) running amok.



Surprisingly, now on another run it worked. But on the one it had
failed, I did take a look
at the directory and I had the Release Files with the FAILED written
in front of them.

I'm sorry I didn't save them. On a successful run, partial is empty as
I just found out now.

> Do you get something reasonable if you browse to
> 'http://httpredir.debian.org/debian/dists/testing/InRelease'?
> If you download it with wget/curl?

When you are saying browse to the URL you have shared, you mean
through the browser, right. If you are asking that, then sure did that
and it took me to page  where all the Inrelease files with checksum
are there.

Wget was also successful  :-

[$] wget http://httpredir.debian.org/debian/dists/testing/InRelease
 [3:59:26]
--2016-05-04 03:59:29--
http://httpredir.debian.org/debian/dists/testing/InRelease
Resolving httpredir.debian.org (httpredir.debian.org)... 5.153.231.35,
128.31.0.66
Connecting to httpredir.debian.org
(httpredir.debian.org)|5.153.231.35|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://debian-mirror.sakura.ne.jp/debian/dists/testing/InRelease
[following]
--2016-05-04 03:59:30--
http://debian-mirror.sakura.ne.jp/debian/dists/testing/InRelease
Resolving debian-mirror.sakura.ne.jp (debian-mirror.sakura.ne.jp)...
133.242.8.20
Connecting to debian-mirror.sakura.ne.jp
(debian-mirror.sakura.ne.jp)|133.242.8.20|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 219854 (215K)
Saving to: ‘InRelease’

InRelease
100%[===>]
214.70K   113KB/sin 1.9s

2016-05-04 03:59:33 (113 KB/s) - ‘InRelease’ saved [219854/219854]

So yeah, that worked.

>
> In any case, I don't see much potential for improvement from the apt
> side here as "garbage in, garbage out", but perhaps if you can shine
> some light on what is going on there might be something so I am not
> closing downright… but I am damn close.
>
I am guessing you should close it, I would try to remember if and when
the next time
I encounter it and try to get you that info. from
/var/lib/apt/lists/partial . It was news to me that partial had SUID
bit.

Thank you for replying so quickly.

> Best regards
>
> David Kalnischkies

-- 
  Regards,
  Shirish Agarwal  शिरीष अग्रवाल
  My quotes in this email licensed under CC 3.0
http://creativecommons.org/licenses/by-nc/3.0/
http://flossexperiences.wordpress.com
EB80 462B 08E1 A0DE A73A  2C2F 9F3D C7A4 E1C4 D2D8

Bug#823339: apt fails from the last three days saying Clearsigned file isn't valid

2016-05-03 Thread David Kalnischkies 

Control: forcemerge -1 823340

On Tue, May 03, 2016 at 06:53:46PM +, shirish शिरीष wrote:
> root@debian:~# apt -o Debug::Acquire::http=yes update
> 0% [Connecting to httpredir.debian.org (128.31.0.66)]GET
> /debian/dists/testing/InRelease HTTP/1.1
> Host: httpredir.debian.org
> Cache-Control: max-age=0
> Accept: text/*
> User-Agent: Debian APT-HTTP/1.3 (1.2.11)
> 
> 
> Answer for: http://httpredir.debian.org/debian/dists/testing/InRelease
> HTTP/1.1 200 OK
> Content-Length: 244
> Connection: close
> Cache-Control: no-cache
> Expires: -1
> Content-Type: text/html
> 
> Get:1 http://httpredir.debian.org/debian testing InRelease [244 B]
> Err:1 http://httpredir.debian.org/debian testing InRelease
>   Clearsigned file isn't valid, got 'NOSPLIT' (does the network
> require authentication?)

The characteristics of the httpredir.debian.org mirror is that it uses
http redirects to push you to a mirror near you, so an HTTP 301/302/307
response code is expected.

What you get instead is "something" which answers 200 with a normal html
page. That is very much not expected and no surprise that apt fails to
work with the received data.


The bad InRelease files should be in partial/, perhaps their content can
tell you what this something is. Looks for me like a proxy (maybe from
your ISP) running amok.

Do you get something reasonable if you browse to
'http://httpredir.debian.org/debian/dists/testing/InRelease'?
If you download it with wget/curl?


In any case, I don't see much potential for improvement from the apt
side here as "garbage in, garbage out", but perhaps if you can shine
some light on what is going on there might be something so I am not
closing downright… but I am damn close.


Best regards

David Kalnischkies


signature.asc
Description: PGP signature

Bug#823339: apt fails from the last three days saying Clearsigned file isn't valid

2016-05-03 Thread shirish शिरीष 
Package: apt
Version: 1.2.11
Severity: important

Dear Maintainer,
For the last 2-3 days I've been unable to update the index. Today I
attempted to debug this and came across this :-

root@debian:~# apt -o Debug::Acquire::http=yes update
0% [Connecting to httpredir.debian.org (128.31.0.66)]GET
/debian/dists/testing/InRelease HTTP/1.1
Host: httpredir.debian.org
Cache-Control: max-age=0
Accept: text/*
User-Agent: Debian APT-HTTP/1.3 (1.2.11)


Answer for: http://httpredir.debian.org/debian/dists/testing/InRelease
HTTP/1.1 200 OK
Content-Length: 244
Connection: close
Cache-Control: no-cache
Expires: -1
Content-Type: text/html

Get:1 http://httpredir.debian.org/debian testing InRelease [244 B]
Err:1 http://httpredir.debian.org/debian testing InRelease
  Clearsigned file isn't valid, got 'NOSPLIT' (does the network
require authentication?)
0% [Connecting to httpredir.debian.org (128.31.0.66)]GET
/debian/dists/unstable/InRelease HTTP/1.1
Host: httpredir.debian.org
Cache-Control: max-age=0
Accept: text/*
User-Agent: Debian APT-HTTP/1.3 (1.2.11)


Answer for: http://httpredir.debian.org/debian/dists/unstable/InRelease
HTTP/1.1 200 OK
Content-Length: 244
Connection: close
Cache-Control: no-cache
Expires: -1
Content-Type: text/html

Get:2 http://httpredir.debian.org/debian unstable InRelease [244 B]
Err:2 http://httpredir.debian.org/debian unstable InRelease
  Clearsigned file isn't valid, got 'NOSPLIT' (does the network
require authentication?)
0% [Connecting to httpredir.debian.org (128.31.0.66)]GET
/debian/dists/experimental/InRelease HTTP/1.1
Host: httpredir.debian.org
Cache-Control: max-age=0
Accept: text/*
User-Agent: Debian APT-HTTP/1.3 (1.2.11)


Answer for: http://httpredir.debian.org/debian/dists/experimental/InRelease
HTTP/1.1 200 OK
Content-Length: 244
Connection: close
Cache-Control: no-cache
Expires: -1
Content-Type: text/html

Get:3 http://httpredir.debian.org/debian experimental InRelease [244 B]
Err:3 http://httpredir.debian.org/debian experimental InRelease
  Clearsigned file isn't valid, got 'NOSPLIT' (does the network
require authentication?)
Fetched 732 B in 36s (19 B/s)
Reading package lists... Done
E: Failed to fetch
http://httpredir.debian.org/debian/dists/testing/InRelease
Clearsigned file isn't valid, got 'NOSPLIT' (does the network require
authentication?)
E: Failed to fetch
http://httpredir.debian.org/debian/dists/unstable/InRelease
Clearsigned file isn't valid, got 'NOSPLIT' (does the network require
authentication?)
E: Failed to fetch
http://httpredir.debian.org/debian/dists/experimental/InRelease
Clearsigned file isn't valid, got 'NOSPLIT' (does the network require
authentication?)
E: Some index files failed to download. They have been ignored, or old
ones used instead.

I tried the following things :-

a. Went to /var/lib/apt/lists/ and rm everything underneath it.
/var/lib/apt/lists$ sudo rm -rf *

b. Then ran sudo apt update in one instance, sudo apt-get update and
sudo aptitude update in three different instances. Each time before
starting the instance, make sure to remove if there is any cruft under
/var/lib/apt/lists but in all the three instances, got the issue as
above.

I am able to browse pages normally, am able to download http files
from data lockers, file vaults etc. but for reason the update is not
happening.

Am open to debugging it more if somebody has more suggestions (apart
from changing the httpredir mirror as it has served me quite well till
the recent past i.e. till last 2-3 days.)

Look forward to a solution or at least a way to find more info.

-- Package-specific info:

-- apt-config dump --

APT "";
APT::Architecture "amd64";
APT::Build-Essential "";
APT::Build-Essential:: "build-essential";
APT::Install-Recommends "1";
APT::Install-Suggests "0";
APT::Sandbox "";
APT::Sandbox::User "_apt";
APT::Authentication "";
APT::Authentication::TrustCDROM "true";
APT::NeverAutoRemove "";
APT::NeverAutoRemove:: "^firmware-linux.*";
APT::NeverAutoRemove:: "^linux-firmware$";
APT::NeverAutoRemove:: "^linux-image-4\.4\.0-1-amd64$";
APT::NeverAutoRemove:: "^linux-image-4\.5\.0-1-amd64$";
APT::NeverAutoRemove:: "^linux-headers-4\.4\.0-1-amd64$";
APT::NeverAutoRemove:: "^linux-headers-4\.5\.0-1-amd64$";
APT::NeverAutoRemove:: "^linux-image-extra-4\.4\.0-1-amd64$";
APT::NeverAutoRemove:: "^linux-image-extra-4\.5\.0-1-amd64$";
APT::NeverAutoRemove:: "^linux-signed-image-4\.4\.0-1-amd64$";
APT::NeverAutoRemove:: "^linux-signed-image-4\.5\.0-1-amd64$";
APT::NeverAutoRemove:: "^kfreebsd-image-4\.4\.0-1-amd64$";
APT::NeverAutoRemove:: "^kfreebsd-image-4\.5\.0-1-amd64$";
APT::NeverAutoRemove:: "^kfreebsd-headers-4\.4\.0-1-amd64$";
APT::NeverAutoRemove:: "^kfreebsd-headers-4\.5\.0-1-amd64$";
APT::NeverAutoRemove:: "^gnumach-image-4\.4\.0-1-amd64$";
APT::NeverAutoRemove:: "^gnumach-image-4\.5\.0-1-amd64$";
APT::NeverAutoRemove:: "^.*-modules-4\.4\.0-1-amd64$";
APT::NeverAutoRemove:: "^.*-modules-4\.5\.0-1-amd64$";
APT::NeverAutoRemove:: "^.*-kernel-4\.4\.0-1-amd64$";