Re: howto avoid "apt-get update" going guru?
On 2016-07-05 06:32, Harald Dunkel wrote:
> Hi folks,
>
> this morning I found "apt-get update" getting stuck due to an
> unresponsive host:
>
> # cat /etc/apt/sources.list
> deb http://ftp.debian.org/debian sid main contrib non-free
> deb-src http://ftp.debian.org/debian sid main contrib non-free
>
> # apt-get update
> Err:1 http://ftp.debian.org/debian sid InRelease
> Could not connect to klecker-ftp.debian.org:80 (130.89.148.12), connection
> timed out [IP: 2001:6b0:e:2018::173 80]
> Reading package lists... Done
> W: Failed to fetch http://ftp.debian.org/debian/dists/sid/InRelease Could
> not connect to klecker-ftp.debian.org:80 (130.89.148.12), connection timed
> out [IP: 2001:6b0:e:2018::173 80]
> W: Some index files failed to download. They have been ignored, or old ones
> used instead.
>
>
> I didn't mention klecker-ftp anywhere in my config files.
> Its not on the round-robin list for ftp.debian.org either:
>
> # host ftp.debian.org
> ftp.debian.org has address 130.239.18.173
> ftp.debian.org has address 130.239.18.165
> ftp.debian.org has IPv6 address 2001:6b0:e:2018::165
> ftp.debian.org has IPv6 address 2001:6b0:e:2018::173
> ftp.debian.org mail is handled by 0 .
Was this command on the same box as the problematic apt-get update command?
Long time after the command was issued? ("this morning found" suggests a
time between the two).
> What would you suggest to avoid this kind of problem?
klecker is serving ftp.d.o.
http://anonscm.debian.org/cgit/mirror/dsa-auto-dns.git/tree/services/ftp.debian.org.service
From my view
brother ~$ host ftp.debian.org
ftp.debian.org has address 130.89.148.12
ftp.debian.org mail is handled by 0 .
And by now my view have switched to use the ACC provided mirrors just as
your example.
On the other hand the long standing recommendation is to not use ftp.d.o:
https://wiki.debian.org/ftp.debian.org#line-22
The concluding answer to your question is probably "use another
hostname". Either a ftp.xx.d.o host or the geo dns based:
http://httpredir.debian.org
--
brother
http://sis.bthstudent.se
signature.asc
Description: OpenPGP digital signature
Re: howto avoid "apt-get update" going guru?
On 07/05/16 06:32, Harald Dunkel wrote: > Hi folks, > > this morning I found "apt-get update" getting stuck due to an > unresponsive host: > Sorry, this was supposed to go to debian-user. Regards Harri
Re: howto avoid "apt-get update" going guru?
Hi Martin,
On 07/05/16 10:09, Martin Bagge / brother wrote:
> On 2016-07-05 06:32, Harald Dunkel wrote:
>>
>> I didn't mention klecker-ftp anywhere in my config files.
>> Its not on the round-robin list for ftp.debian.org either:
>>
>> # host ftp.debian.org
>> ftp.debian.org has address 130.239.18.173
>> ftp.debian.org has address 130.239.18.165
>> ftp.debian.org has IPv6 address 2001:6b0:e:2018::165
>> ftp.debian.org has IPv6 address 2001:6b0:e:2018::173
>> ftp.debian.org mail is handled by 0 .
>
> Was this command on the same box as the problematic apt-get update command?
Yes.
> Long time after the command was issued? ("this morning found" suggests a
> time between the two).
>
No, the "host" was running on the same machine while apt-get
was waiting.
>> What would you suggest to avoid this kind of problem?
>
> klecker is serving ftp.d.o.
>
> http://anonscm.debian.org/cgit/mirror/dsa-auto-dns.git/tree/services/ftp.debian.org.service
>
> From my view
> brother ~$ host ftp.debian.org
> ftp.debian.org has address 130.89.148.12
> ftp.debian.org mail is handled by 0 .
>
> And by now my view have switched to use the ACC provided mirrors just as
> your example.
>
> On the other hand the long standing recommendation is to not use ftp.d.o:
> https://wiki.debian.org/ftp.debian.org#line-22
>
I see. "ftp.debian.org" is in my sources.list since ages.
Probably I missed the note introducing this new host.
Maybe it would help to provide recommended settings for
sources.list within /usr/share/base-files ?
Thanx very much
Harri
Re: howto avoid "apt-get update" going guru?
]] Martin Bagge / brother > On 2016-07-05 06:32, Harald Dunkel wrote: > > > # apt-get update > > Err:1 http://ftp.debian.org/debian sid InRelease > > Could not connect to klecker-ftp.debian.org:80 (130.89.148.12), > > connection timed out [IP: 2001:6b0:e:2018::173 80] > > Reading package lists... Done > > W: Failed to fetch http://ftp.debian.org/debian/dists/sid/InRelease > > Could not connect to klecker-ftp.debian.org:80 (130.89.148.12), > > connection timed out [IP: 2001:6b0:e:2018::173 80] > > W: Some index files failed to download. They have been ignored, or old ones > > used instead. We had some networking trouble for klecker today. > > I didn't mention klecker-ftp anywhere in my config files. > > Its not on the round-robin list for ftp.debian.org either: It's the rdns entry for the IP you're connecting to. > The concluding answer to your question is probably "use another > hostname". Either a ftp.xx.d.o host or the geo dns based: > http://httpredir.debian.org I'd not actively recommend people use httpredir.debian.org as it's somewhat sporadically maintained. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are
Re: howto avoid "apt-get update" going guru?
Tollef Fog Heen wrote: > I'd not actively recommend people use httpredir.debian.org as it's > somewhat sporadically maintained. Do you have any more details on that? Does a better alternative exist? I still have hopes that someday the d-i mirror question becomes an expert-level question for people with a local mirror (assuming we don't also someday have automatic local mirror discovery). - Josh Triplett
Re: howto avoid "apt-get update" going guru?
Josh, On 5 July 2016 at 14:53, Josh Triplett wrote: > Tollef Fog Heen wrote: >> I'd not actively recommend people use httpredir.debian.org as it's >> somewhat sporadically maintained. > > Do you have any more details on that? There was a discussion[1] on "debian-project" mailing list a few months ago. The first message is about shutting it down, but there were other exchanged messages that you might find interesting. Regards, Tiago. [1]: https://lists.debian.org/debian-project/2016/04/msg00012.html -- Tiago "Myhro" Ilieve Blog: https://blog.myhro.info/ GitHub: https://github.com/myhro LinkedIn: https://br.linkedin.com/in/myhro Montes Claros - MG, Brasil
Re: howto avoid "apt-get update" going guru?
]] Josh Triplett > Tollef Fog Heen wrote: > > I'd not actively recommend people use httpredir.debian.org as it's > > somewhat sporadically maintained. > > Do you have any more details on that? Does a better alternative exist? I personally recommend using deb.debian.org. > I still have hopes that someday the d-i mirror question becomes an > expert-level question for people with a local mirror (assuming we don't > also someday have automatic local mirror discovery). Yeah, that'd be great. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are
Re: howto avoid "apt-get update" going guru?
Tollef Fog Heen wrote: > I personally recommend using deb.debian.org. That works nicely, thanks! Seems to have decent performance. I couldn't find any announcement or documentation of this, other than that on the site itself, though I did find a use of it in a recent announcement of dbgsym packages. Does the CDN this uses download and cache packages on first request? Because I noticed when testing it that if I requested a package reasonably unlikely to have already been fetched, it would hang at "0% [Waiting for headers]" for a long time (minutes). But if I reattempted that same package later, it would download just fine. - Josh Triplett
Re: howto avoid "apt-get update" going guru?
On Jul 06, Tollef Fog Heen wrote: > I personally recommend using deb.debian.org. I do not, since it does not have local nodes in my country. -- ciao, Marco signature.asc Description: PGP signature
Re: howto avoid "apt-get update" going guru?
]] Josh Triplett > Tollef Fog Heen wrote: > > I personally recommend using deb.debian.org. > > That works nicely, thanks! Seems to have decent performance. > > I couldn't find any announcement or documentation of this, other than > that on the site itself, though I did find a use of it in a recent > announcement of dbgsym packages. It's somewhat in beta yet. I should probably write up an announcement about it. > Does the CDN this uses download and cache packages on first request? > Because I noticed when testing it that if I requested a package > reasonably unlikely to have already been fetched, it would hang at "0% > [Waiting for headers]" for a long time (minutes). But if I reattempted > that same package later, it would download just fine. This was a bug and should be fixed now. (It downloads on first request, but it streams, so there should not be a big initial delay.) -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are
deb.debian.org [was: Re: howto avoid "apt-get update" going guru?]
[Please CC me on replies.] Tollef Fog Heen wrote: > ]] Josh Triplett > > Tollef Fog Heen wrote: > > > I personally recommend using deb.debian.org. > > > > That works nicely, thanks! Seems to have decent performance. > > > > I couldn't find any announcement or documentation of this, other than > > that on the site itself, though I did find a use of it in a recent > > announcement of dbgsym packages. > > It's somewhat in beta yet. I should probably write up an announcement > about it. Ah, that makes sense. I look forward to the announcement. When you make the announcement, can you include a link to the details of the CDN, such as the extent of its caching servers? That would help people determine if using it will likely produce good results for them. > > Does the CDN this uses download and cache packages on first request? > > Because I noticed when testing it that if I requested a package > > reasonably unlikely to have already been fetched, it would hang at "0% > > [Waiting for headers]" for a long time (minutes). But if I reattempted > > that same package later, it would download just fine. > > This was a bug and should be fixed now. (It downloads on first request, > but it streams, so there should not be a big initial delay.) Out of curiosity, what was the bug? - Josh Triplett
Re: deb.debian.org [was: Re: howto avoid "apt-get update" going guru?]
On Thu, Jul 07, 2016 at 11:03:16PM -0700, Josh Triplett wrote: > Tollef Fog Heen wrote: > > ]] Josh Triplett > > > Tollef Fog Heen wrote: > > > > I personally recommend using deb.debian.org. > > > > > > That works nicely, thanks! Seems to have decent performance. > > Ah, that makes sense. I look forward to the announcement. > > When you make the announcement, can you include a link to the details of > the CDN, such as the extent of its caching servers? That would help > people determine if using it will likely produce good results for them. The locality of this CDN seems to be... not the best. In Poland, there's 11 mirrors (according to choose-mirror 2.69), and httpredir.debian.net always gives me one of those. Not the closest one network- or geography- wise, but in a country the size of Poland, that's good enough. deb.debian.org on the other hand, trying from two locations over three ISPs: Starogard Gdański/Netia, Starogard Gdański/UPC, Gdańsk/Limes: * IPv6: Amsterdam or London * IPv4: MIT, San Francisco, London -- An imaginary friend squared is a real enemy.
Re: deb.debian.org [was: Re: howto avoid "apt-get update" going guru?]
]] Josh Triplett > [Please CC me on replies.] > > Tollef Fog Heen wrote: > > ]] Josh Triplett > > > Tollef Fog Heen wrote: > > > > I personally recommend using deb.debian.org. > > > > > > That works nicely, thanks! Seems to have decent performance. > > > > > > I couldn't find any announcement or documentation of this, other than > > > that on the site itself, though I did find a use of it in a recent > > > announcement of dbgsym packages. > > > > It's somewhat in beta yet. I should probably write up an announcement > > about it. > > Ah, that makes sense. I look forward to the announcement. > > When you make the announcement, can you include a link to the details of > the CDN, such as the extent of its caching servers? That would help > people determine if using it will likely produce good results for them. Yeah, though what you actually want to check is whether it is faster for them or not, rather than base it on just geographical distance. > > > Does the CDN this uses download and cache packages on first request? > > > Because I noticed when testing it that if I requested a package > > > reasonably unlikely to have already been fetched, it would hang at "0% > > > [Waiting for headers]" for a long time (minutes). But if I reattempted > > > that same package later, it would download just fine. > > > > This was a bug and should be fixed now. (It downloads on first request, > > but it streams, so there should not be a big initial delay.) > > Out of curiosity, what was the bug? A bug in the VCL generator which caused an early return before the flag to enable streaming was set. I added a workaround which makes it so we no longer hit the bug. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are
Re: deb.debian.org [was: Re: howto avoid "apt-get update" going guru?]
On Fri, Jul 08, 2016 at 08:56:37AM +0200, Adam Borowski wrote: > On Thu, Jul 07, 2016 at 11:03:16PM -0700, Josh Triplett wrote: > > Tollef Fog Heen wrote: > > > ]] Josh Triplett > > > > Tollef Fog Heen wrote: > > > > > I personally recommend using deb.debian.org. > > > > > > > > That works nicely, thanks! Seems to have decent performance. > > > > Ah, that makes sense. I look forward to the announcement. > > > > When you make the announcement, can you include a link to the details of > > the CDN, such as the extent of its caching servers? That would help > > people determine if using it will likely produce good results for them. > > The locality of this CDN seems to be... not the best. > > In Poland, there's 11 mirrors (according to choose-mirror 2.69), and > httpredir.debian.net always gives me one of those. Not the closest one > network- or geography- wise, but in a country the size of Poland, that's > good enough. > > deb.debian.org on the other hand, trying from two locations over three ISPs: > Starogard Gdański/Netia, Starogard Gdański/UPC, Gdańsk/Limes: > * IPv6: Amsterdam or London > * IPv4: MIT, San Francisco, London Just to confirm, did you use SRV records for this check as apt does, or did you check the path to deb.debian.org's A or records directly? If I ping deb.debian.org directly, it resolves to various mirrors, with ping times ranging from 20ms to 200ms away. The SRV records, though, point to a CDN server that reliably gets 20ms. (I don't actually know the benefit of using SRV records here.) - Josh Triplett
