Arnaud Kyheng wrote: | Hello, | | I love the Debian project, and I have worked on a new development for | it: Apt-Torrent :)
Thank you for your contribution. However, I looked at doing something similar to this a little while ago and found that bittorrent is not very well suited for doing package downloads. Of the ~15k binary packages in Debian about 87% are under 1 meg in size and 98% are under 10 megs. The bit-torrent protocol works best on files significantly larger than this. Also, the protocol is not as efficient as it could be for the server hosting the .torrent file, which means it scales quite poorly when there are lots of requests for small files, as would be the case for Debian packages.
I don't agree with the little package problem with Bittorrent. With Bittornado I'm using as a backend, the super-seeder option answer to this problem since if the package is already well available on the network, it'll not answer to the client but let it download from peers.
Furthermore the .torrent files are very small ~2k and could be easily cached by ISPs.
And also, if you think that the tracker overhead not worth a 5k package, you could as well split the downloading system in two. I mean put only >= 5k packages files on the apt-torrent server and let the others be fetched directly in http.
This can be done easily since apt-torrent is fetching the Packages.gz as usual. I mean I could add a special header in the Packages.gz description to tell the proxy where to download the package direct-http, or apt-torrent-server for example.
However, I do feel that having a p2p backend to apt is a very interesting and feasible distribution method. There is a lot of structure in the way Debian lays out its archive, from the Package files to the .deb's themselves, which can be exploited to make this very efficient. There has also been a myriad of research papers exploring overlay networks and application level multicast which could be adopted to form a package network that would provide advantages over our current mirror infrastructure. (I've actually done a little work on exploring this, but haven't gotten very far due to time constraints.)
My original idea was to save bandwidth of the Debian server, and improve the downloading speed of the packages for users that are even far of a mirror. I found that the Bittorrent was really mature and will fit well. In the future, I could as well use GNUnet as a backend :)
With apt-torrent you may also reduce the number of proximity servers. We don't need extreme latency response for 2k .torrent files, and except if the mirror is an ISP, it'll not overload the ISPs peers, by Bittorent scheme. Furthermore it'll load more the ISPs cache with the .torrent files.
With lesser proximity servers/seeders, it might also save rsync bandwidth needed to sync Debian repository servers.
Arnaud
