Re: apt-torrent (WAS: Re: apt PARALLELISM)

2006-01-09 Thread Nathanael Nerode
It'll take me some time to find a new, and more appropriate home for apt-torrent. The Debian archive (experimental distribution) would be a *very* appropriate home. It won't provide a testbed package seeder or place to download .torrent files, but that can be done later (and by any number

Re: apt-torrent (WAS: Re: apt PARALLELISM)

2006-01-09 Thread Adam Heath
On Mon, 9 Jan 2006, Arnaud Kyheng wrote: Hello all and Happy New Year, Thanks to George, apt-torrent has been mentioned in the Debian Devel list :o) I've just noticed it, and the fun part of this discovery, is that I also found why my ISP has closed sianka.free.fr: Too much hits since

apt-torrent (WAS: Re: apt PARALLELISM)

2006-01-08 Thread Arnaud Kyheng
Hello all and Happy New Year, Thanks to George, apt-torrent has been mentioned in the Debian Devel list :o) I've just noticed it, and the fun part of this discovery, is that I also found why my ISP has closed sianka.free.fr: Too much hits since the latest Debian Weekly News, and the new

Re: apt-torrent (WAS: Re: apt PARALLELISM)

2006-01-08 Thread Christian Perrier
I've just noticed it, and the fun part of this discovery, is that I also found why my ISP has closed sianka.free.fr: Too much hits since the latest Debian Weekly News, and the new apt-torrent 0.3.1-1 package ! The solution is simple: get it in the Debian archive..:) -- To UNSUBSCRIBE,

Re: apt PARALLELISM

2006-01-06 Thread Michelle Konzack
Am 2005-12-27 16:25:10, schrieb Bernd Eckenfels: In article [EMAIL PROTECTED] you wrote: apt-get update apt-get --download-only upgrade It would make more sense to send out the DIFFs to the packages.gz, so you dont actually need to download the packages file every five minutes. But if

Re: apt PARALLELISM

2005-12-27 Thread Michelle Konzack
Am 2005-12-21 16:32:20, schrieb Goswin von Brederlow: I have 10240kBit downstream and get way less from security.debian.org. Especialy when there is a security release of X or latex. They are two possibilitys: [EMAIL PROTECTED] subscribe with an seperated E-Mail and track it. Check the E-Mail

Re: apt PARALLELISM

2005-12-27 Thread Bernd Eckenfels
In article [EMAIL PROTECTED] you wrote: [EMAIL PROTECTED] subscribe with an seperated E-Mail and track it. Check the E-Mail in a delay of 5 minutes. Write a script (we do not want to download Packages.gz, if there is no Pakage of interest) which check, whether the new package is installed on

Re: apt PARALLELISM

2005-12-23 Thread Goswin von Brederlow
Olaf van der Spek [EMAIL PROTECTED] writes: On 12/21/05, Goswin von Brederlow [EMAIL PROTECTED] wrote: Who need PARALELISM and who has a bandwidth of more then 8 MBit? I have 10240kBit downstream and get way less from security.debian.org. Especialy when there is a security release of X or

Re: apt PARALLELISM

2005-12-21 Thread Goswin von Brederlow
Michelle Konzack [EMAIL PROTECTED] writes: Am 2005-12-12 13:23:01, schrieb Goswin von Brederlow: Actualy one thing apt could do: [EMAIL PROTECTED]:~% host security.debian.org security.debian.org A 82.94.249.158 security.debian.org A 128.101.80.133 security.debian.org

Re: apt PARALLELISM

2005-12-21 Thread Goswin von Brederlow
Michelle Konzack [EMAIL PROTECTED] writes: Am 2005-12-06 09:53:43, schrieb Ivan Adams: Hi again, in my case: I have slow internet connection. BUT I have friends with the same ^^^ connection in my local area network, who have apt-proxy. My goal is: When I need

Re: apt PARALLELISM

2005-12-21 Thread Olaf van der Spek
On 12/21/05, Goswin von Brederlow [EMAIL PROTECTED] wrote: Who need PARALELISM and who has a bandwidth of more then 8 MBit? I have 10240kBit downstream and get way less from security.debian.org. Especialy when there is a security release of X or latex. But parallel downloads won't solve

Re: apt PARALLELISM

2005-12-21 Thread Henrique de Moraes Holschuh
On Wed, 21 Dec 2005, Olaf van der Spek wrote: On 12/21/05, Goswin von Brederlow [EMAIL PROTECTED] wrote: Who need PARALELISM and who has a bandwidth of more then 8 MBit? I have 10240kBit downstream and get way less from security.debian.org. Especialy when there is a security release of X

Re: apt PARALLELISM

2005-12-20 Thread Michelle Konzack
Am 2005-12-05 16:11:35, schrieb Joe Smith: This person is requesting parallel downloads from multiple servers. So basicly during package download, if there are three full and up-to-date mirrors in sources.list, there should be simulatious downloads of different packages from all three

Re: apt PARALLELISM

2005-12-20 Thread Michelle Konzack
Am 2005-12-06 09:53:43, schrieb Ivan Adams: Hi again, in my case: I have slow internet connection. BUT I have friends with the same ^^^ connection in my local area network, who have apt-proxy. My goal is: When I need to install new system (Debian) on new user,

Re: apt PARALLELISM

2005-12-20 Thread Michelle Konzack
Am 2005-12-12 13:23:01, schrieb Goswin von Brederlow: Actualy one thing apt could do: [EMAIL PROTECTED]:~% host security.debian.org security.debian.org A 82.94.249.158 security.debian.org A 128.101.80.133 security.debian.org A 194.109.137.218 Why not open 3

Re: apt PARALLELISM

2005-12-20 Thread Michelle Konzack
Am 2005-12-12 17:06:28, schrieb Bas Zoetekouw: But what would you gain from that? In my experience, the mirrors are fast enough to saturate anything but the fastest (100Mb) links. FullACK! - OK, I have currently only an E3 in Paris, but if all goes right I will get my own FiberOptic

Re: apt PARALLELISM

2005-12-14 Thread Martijn van Oosterhout
2005/12/13, Henrique de Moraes Holschuh [EMAIL PROTECTED]: Time to devise a way to teach it about that, then. HOW to do it is the big problem, though. How should one deal with round-robin DNS mirrors which are supposed to be equal, but are not. What are the failure modes to cater for?

Re: apt PARALLELISM

2005-12-14 Thread Robert Lemmen
On Wed, Dec 14, 2005 at 01:23:17PM +0100, Martijn van Oosterhout wrote: ISTM the easiest would be for apt to lookup the hostname itself and treat the single entry as a list of entires, one for each possible address the hostname can resolve to. If one fails, try the next. apt already does that

Re: apt PARALLELISM

2005-12-14 Thread Claus Färber
Olaf van der Spek [EMAIL PROTECTED] schrieb/wrote: That's not true. Suppose you've only got 3 users. If each user connects to one (different) mirror, he gets 1/1 of that mirror's bandwidth. If each user connects to each mirror, he only gets 1/3 of that mirror's bandwidth. They could get 1/1

Re: apt PARALLELISM

2005-12-14 Thread Olaf van der Spek
On 13 Dec 2005 15:56:00 +0100, Claus Färber [EMAIL PROTECTED] wrote: Olaf van der Spek [EMAIL PROTECTED] schrieb/wrote: That's not true. Suppose you've only got 3 users. If each user connects to one (different) mirror, he gets 1/1 of that mirror's bandwidth. If each user connects to each

Re: apt PARALLELISM

2005-12-14 Thread Wesley J. Landaker
On Wednesday 14 December 2005 14:41, Olaf van der Spek wrote: On 13 Dec 2005 15:56:00 +0100, Claus Färber [EMAIL PROTECTED] wrote: Olaf van der Spek [EMAIL PROTECTED] schrieb/wrote: That's not true. Suppose you've only got 3 users. If each user connects to one (different) mirror, he gets

Re: apt PARALLELISM

2005-12-13 Thread Henning Makholm
Scripsit Henrique de Moraes Holschuh [EMAIL PROTECTED] THis is not something that would bother anyone if it is a single user... but if you have 10k users doing that, often close enough in time, well, things should get MUCH worse as far as I can see. If they are doing this at random times in

Re: apt PARALLELISM

2005-12-13 Thread Olaf van der Spek
On 12/13/05, Henning Makholm [EMAIL PROTECTED] wrote: Assume a situation where mirror bandwidth is the limiting factor, and imagine a world with 3 mirrors. Say that during a certain time of the day 600 users each minute start to download updated x.org packages. Either they can do their

Re: apt PARALLELISM

2005-12-13 Thread Henning Makholm
Scripsit Olaf van der Spek [EMAIL PROTECTED] On 12/13/05, Henning Makholm [EMAIL PROTECTED] wrote: Alternatively each user can spread his load over all three servers; his download now takes 5 minutes, and each server _still_ sees 600*5 = 3000 active connections at any time. Thus _all_ users

Re: apt PARALLELISM

2005-12-12 Thread Ivan Adams
Thanks ...We don't want them to open multiple connections even to MULTIPLE servers...

Re: apt PARALLELISM

2005-12-12 Thread Martijn van Oosterhout
2005/12/12, Henrique de Moraes Holschuh [EMAIL PROTECTED]: We don't want them to open multiple connections even to MULTIPLE servers... That's odd though, because apt *does* open connections to multiple servers all the time. To fetch packages lists, or if a package is only available on one of the

Re: apt PARALLELISM

2005-12-12 Thread Goswin von Brederlow
Wouter Verhelst [EMAIL PROTECTED] writes: On Sun, Dec 11, 2005 at 02:45:50AM +0100, Marco d'Itri wrote: On Dec 11, Charles Fry [EMAIL PROTECTED] wrote: But if multiple URLs could satisfactorily serve requests for a single repository, only one of them is currently used. Which is fine,

Re: apt PARALLELISM

2005-12-12 Thread Ivan Adams
Can have option in /etc/apt/apt.conf or apt-get (install || upgrade) with -? some char here for parallelism.And by default can be disabled ...

Re: apt PARALLELISM

2005-12-12 Thread Goswin von Brederlow
Martijn van Oosterhout [EMAIL PROTECTED] writes: 2005/12/12, Henrique de Moraes Holschuh [EMAIL PROTECTED]: We don't want them to open multiple connections even to MULTIPLE servers... That's odd though, because apt *does* open connections to multiple servers all the time.

Re: apt PARALLELISM

2005-12-12 Thread Henrique de Moraes Holschuh
On Mon, 12 Dec 2005, Martijn van Oosterhout wrote: 2005/12/12, Henrique de Moraes Holschuh [EMAIL PROTECTED]: We don't want them to open multiple connections even to MULTIPLE servers... That's odd though, because apt *does* open connections to multiple servers all the time. To fetch

Re: apt PARALLELISM

2005-12-12 Thread Marco d'Itri
On Dec 12, Goswin von Brederlow [EMAIL PROTECTED] wrote: Why not open 3 connections one to each host? Why do? Or at least fall back to the other IPs if the first one gives an error? I hope that this already happens... -- ciao, Marco signature.asc Description: Digital signature

Re: apt PARALLELISM

2005-12-12 Thread Marco d'Itri
On Dec 12, Henrique de Moraes Holschuh [EMAIL PROTECTED] wrote: use more than a single server by itself... I didn't think it common practice for large mirror to configure multi-megabyte windows... TCP/IP windows? Or user bw shaping? He means the TCP window size, and it *is* common practice

Re: apt PARALLELISM

2005-12-12 Thread Henrique de Moraes Holschuh
On Mon, 12 Dec 2005, Marco d'Itri wrote: On Dec 12, Henrique de Moraes Holschuh [EMAIL PROTECTED] wrote: use more than a single server by itself... I didn't think it common practice for large mirror to configure multi-megabyte windows... TCP/IP windows? Or user bw shaping? He means

Re: apt PARALLELISM

2005-12-12 Thread Henning Makholm
Scripsit Henrique de Moraes Holschuh [EMAIL PROTECTED] 1. We care about a large lot of people a lot more than we care for an individual's downloading speed 2. Thus we try to keep the mirror load down, and downloading hundreds of megabytes using multiple connections to multiple sources

Re: apt PARALLELISM

2005-12-12 Thread Wouter Verhelst
On Mon, Dec 12, 2005 at 12:52:09PM +0100, Goswin von Brederlow wrote: Wouter Verhelst [EMAIL PROTECTED] writes: True, but that's not what's being asked here. If multiple URLs could serve requests for a single repository---i.e., if you've got both deb http://ftp1.CC.debian.org/debian

Re: apt PARALLELISM

2005-12-12 Thread Henrique de Moraes Holschuh
On Mon, 12 Dec 2005, Henning Makholm wrote: As far as I read the proposal, it is about downloading _different_ files from different mirrors - if you have 25 packages to get for your 'apt-get update' operation, download 5 packages from each of 5 different servers, with one connection to each

Re: apt PARALLELISM

2005-12-12 Thread Ivan Adams
As far as I read the proposal, it is about downloading _different_files from different mirrors - if you have 25 packages to get for your 'apt-get update' operation, download 5 packages from each of 5different servers, with one connection to each server active at atime.That is what I mean ...

Re: apt PARALLELISM

2005-12-12 Thread Olaf van der Spek
On 12/12/05, Ivan Adams [EMAIL PROTECTED] wrote: As far as I read the proposal, it is about downloading _different_ files from different mirrors - if you have 25 packages to get for your 'apt-get update' operation, download 5 packages from each of 5 different servers, with one connection

Re: apt PARALLELISM

2005-12-12 Thread Ivan Adams
My goal is using more bandwidth with apt-proxy servers on my friends, who have other internet connection.And I want to download first packet from my internet and second packet at the same time from the apt-proxy with my friend internet connection. Now, I can only download all packets from my

Re: apt PARALLELISM

2005-12-12 Thread Linas Zvirblis
Ivan Adams wrote: My goal is using more bandwidth with apt-proxy servers on my friends, who have other internet connection. And I want to download first packet from my internet and second packet at the same time from the apt-proxy with my friend internet connection. Now, I can only download

Re: apt PARALLELISM

2005-12-12 Thread Bas Zoetekouw
Hi Ivan! You wrote: As far as I read the proposal, it is about downloading _different_ files from different mirrors - if you have 25 packages to get for your 'apt-get update' operation, download 5 packages from each of 5 different servers, with one connection to each server

Re: apt PARALLELISM

2005-12-12 Thread Simon Richter
Hi, Bas Zoetekouw wrote: But what would you gain from that? In my experience, the mirrors are fast enough to saturate anything but the fastest (100Mb) links. I think the idea is a) load-balancing over multiple DSL lines b) checking a bunch of apt-proxy servers whether they can provide

Re: apt PARALLELISM

2005-12-12 Thread Joey Hess
Marco d'Itri wrote: Or at least fall back to the other IPs if the first one gives an error? I hope that this already happens... apt doesn't know anything about round robin dns, and especially with secure apt, if one mirror gets out of sync things break horribly. This recently happened with

Re: apt PARALLELISM

2005-12-12 Thread Henrique de Moraes Holschuh
On Mon, 12 Dec 2005, Joey Hess wrote: Marco d'Itri wrote: Or at least fall back to the other IPs if the first one gives an error? I hope that this already happens... apt doesn't know anything about round robin dns, and especially with secure apt, if one mirror gets out of sync things

Re: apt PARALLELISM

2005-12-11 Thread Wouter Verhelst
On Sun, Dec 11, 2005 at 02:45:50AM +0100, Marco d'Itri wrote: On Dec 11, Charles Fry [EMAIL PROTECTED] wrote: But if multiple URLs could satisfactorily serve requests for a single repository, only one of them is currently used. Which is fine, because we do not want people to open multiple

Re: apt PARALLELISM

2005-12-11 Thread Marco d'Itri
On Dec 11, Wouter Verhelst [EMAIL PROTECTED] wrote: True, but that's not what's being asked here. If multiple URLs could serve requests for a single repository---i.e., if you've got both deb http://ftp1.CC.debian.org/debian unstable main and deb http://ftp2.CC.debian.org/debian unstable

Re: apt PARALLELISM

2005-12-11 Thread Henrique de Moraes Holschuh
On Sun, 11 Dec 2005, Marco d'Itri wrote: On Dec 11, Charles Fry [EMAIL PROTECTED] wrote: But if multiple URLs could satisfactorily serve requests for a single repository, only one of them is currently used. Which is fine, because we do not want people to open multiple connections to the

Re: apt PARALLELISM

2005-12-10 Thread Cosimo Alfarano
On Tue, Dec 06, 2005 at 09:09:39AM -0200, Henrique de Moraes Holschuh wrote: I doubt very much so parallel downloads will be added to apt. Could be added Release-file based. Parallelize only unrelated pools, and disabled by default to avoid conflict wiht apt-proxy (if any). ie: Debian and Foo

Re: apt PARALLELISM

2005-12-10 Thread Stephen Gran
This one time, at band camp, Cosimo Alfarano said: On Tue, Dec 06, 2005 at 09:09:39AM -0200, Henrique de Moraes Holschuh wrote: I doubt very much so parallel downloads will be added to apt. Could be added Release-file based. Parallelize only unrelated pools, and disabled by default to avoid

Re: apt PARALLELISM

2005-12-10 Thread Charles Fry
Seperate repositiories are already parallelized, which is what I think you are asking for here. But if multiple URLs could satisfactorily serve requests for a single repository, only one of them is currently used. Ideally, the amount of parallelism could match the number of redundant URLs

Re: apt PARALLELISM

2005-12-10 Thread Marco d'Itri
On Dec 11, Charles Fry [EMAIL PROTECTED] wrote: But if multiple URLs could satisfactorily serve requests for a single repository, only one of them is currently used. Which is fine, because we do not want people to open multiple connections to the same server. -- ciao, Marco signature.asc

Re: apt PARALLELISM

2005-12-06 Thread Henrique de Moraes Holschuh
On Tue, 06 Dec 2005, Ivan Adams wrote: I have slow internet connection. BUT I have friends with the same connection in my local area network, who have apt-proxy. My goal is: When I need to install new system (Debian) on new user, or dist-upgrade on entire system, I need the unstable packets

Re: apt PARALLELISM

2005-12-06 Thread George Danchev
On Tuesday 06 December 2005 13:09, Henrique de Moraes Holschuh wrote: On Tue, 06 Dec 2005, Ivan Adams wrote: I have slow internet connection. BUT I have friends with the same connection in my local area network, who have apt-proxy. My goal is: When I need to install new system (Debian) on

Re: apt PARALLELISM

2005-12-06 Thread Henrique de Moraes Holschuh
On Tue, 06 Dec 2005, George Danchev wrote: apt-torrent seems to approach that too: http://sianka.free.fr/documentation.html Now, THAT is something nice. BitTorrent won't overload the mirrors ever. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the

apt PARALLELISM

2005-12-05 Thread Ivan Adams
Hi, I'll be so glad to see next (or one of the next) version of the apt package, to have PARALLELISM.For example if I have more than one row pointing to one stage (stable testing or unstable) in /etc/apt/sources.list, apt to get one packet from one server, and at the same time to get second packet

Re: apt PARALLELISM

2005-12-05 Thread Olaf van der Spek
On 12/5/05, Ivan Adams [EMAIL PROTECTED] wrote: Example: (/etc/apt/sources.list) deb http://ftp.en.debian.org/debian main stable contrib non-free deb http://ftp.de.debian.org/debian main stable contrib non-free in this case the stable packages will be ONLY downloaded from first server from

Re: apt PARALLELISM

2005-12-05 Thread Joe Smith
Olaf van der Spek [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] On 12/5/05, Ivan Adams [EMAIL PROTECTED] wrote: Example: (/etc/apt/sources.list) deb http://ftp.en.debian.org/debian main stable contrib non-free deb http://ftp.de.debian.org/debian main stable contrib non-free in

Re: apt PARALLELISM

2005-12-05 Thread Olaf van der Spek
On 12/5/05, Joe Smith [EMAIL PROTECTED] wrote: Olaf van der Spek [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] On 12/5/05, Ivan Adams [EMAIL PROTECTED] wrote: Example: (/etc/apt/sources.list) deb http://ftp.en.debian.org/debian main stable contrib non-free deb

Re: apt PARALLELISM

2005-12-05 Thread Joe Smith
Olaf van der Spek [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] On 12/5/05, Joe Smith [EMAIL PROTECTED] wrote: Olaf van der Spek [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] On 12/5/05, Ivan Adams [EMAIL PROTECTED] wrote: Example: (/etc/apt/sources.list) deb

Re: apt PARALLELISM

2005-12-05 Thread Henrique de Moraes Holschuh
On Mon, 05 Dec 2005, Joe Smith wrote: Olaf van der Spek [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] On 12/5/05, Ivan Adams [EMAIL PROTECTED] wrote: Example: (/etc/apt/sources.list) deb http://ftp.en.debian.org/debian main stable contrib non-free deb

Re: apt PARALLELISM

2005-12-05 Thread Romain Beauxis
Le Mardi 6 Décembre 2005 02:50, Joe Smith a écrit : Now it is useless for users where the bottleneck is on their end. Well, it can also be usefull in case of a broken mirror can't it? Romain -- Not even the dog That piss against the wall of Babylon, Shall escape his judgement

Re: apt PARALLELISM

2005-12-05 Thread Henrique de Moraes Holschuh
On Tue, 06 Dec 2005, Romain Beauxis wrote: Le Mardi 6 Décembre 2005 02:50, Joe Smith a écrit : Now it is useless for users where the bottleneck is on their end. Well, it can also be usefull in case of a broken mirror can't it? apt already handles that and skips to the next mirror. --

Re: apt PARALLELISM

2005-12-05 Thread Ivan Adams
Hi again,in my case:I have slow internet connection. BUT I have friends with the same connection in my local area network, who have apt-proxy.My goal is: When I need to install new system (Debian) on new user, or dist-upgrade on entire system, I need the unstable packets from site. In this case I