Re: 404's
Matt Zimmerman wrote: On Mon, Apr 30, 2001 at 02:00:18PM -0700, Adam McKenna wrote: rsync --exclude Packages* debian/pool rsync --delete debian/pool (If old packages are even deleted) It is probably a good idea to hold off the Sources* and Release* files too and then at the very end run a --delete-after. This is aproximately what I have started using... rsync -vaH --stats --partial --exclude Packages* \ --exclude Sources* --exclude Release* host::debian debian/ rsync -vaH --stats --partial --delete --delete-after \ host::debian debian/ with this rsync implementation, will cause nearly every file in the archive to be checksum-compared twice instead of once. No, rsync will only checksum files by default when they are being sent or if the timestamp and/or file sizes do not match. Rsync will only checksum every file if you hand it the -c option. The only problem with this is getting the whole file list twice which takes some io. I am looking at the possibility of using cvsup (yes that sounds strange, but trust me) for mirroring because it uses the same algorithm as rsync and is much faster when it comes to getting the file list. Rsync will wait until the entire file list has been received, whereas cvsup will immediately grab the first file on the list which saves a lot of time in many cases. - Doug
404's
It seems like an easy way to prevent the following would be to update the Packages.gz file LAST, after syncing up the other files, IE: rsync --exclude Packages* debian/pool rsync --delete debian/pool (If old packages are even deleted) --Adam [EMAIL PROTECTED]:~$ sudo apt-get upgrade Reading Package Lists... Done Building Dependency Tree... Done The following packages have been kept back dnsutils sysklogd 96 packages upgraded, 0 newly installed, 0 to remove and 2 not upgraded. Need to get 24.4MB/57.0MB of archives. After unpacking 1949kB will be used. Do you want to continue? [Y/n] Err http://http.us.debian.org unstable/main rpm 4.0.2-6 404 Not Found [IP: 192.25.206.10 80] Err http://http.us.debian.org unstable/main libpopt0 1.6.2-5 404 Not Found [IP: 192.25.206.10 80] Err http://http.us.debian.org unstable/main lilo 1:21.7.1-4 404 Not Found [IP: 192.25.206.10 80] Err http://http.us.debian.org unstable/main modutils 2.4.5-1 404 Not Found [IP: 192.25.206.10 80] Err http://http.us.debian.org unstable/main libmysqlclient10 3.23.37-3 404 Not Found [IP: 192.25.206.10 80] Err http://http.us.debian.org unstable/main librpm0 4.0.2-6 404 Not Found [IP: 192.25.206.10 80] Err http://http.us.debian.org unstable/main libxaw6 4.0.3-1 404 Not Found [IP: 192.25.206.10 80] Err http://http.us.debian.org unstable/main libxaw7 4.0.3-1 404 Not Found [IP: 192.25.206.10 80] -- Adam McKenna [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: 404's
On Mon, Apr 30, 2001 at 02:00:18PM -0700, Adam McKenna wrote: It seems like an easy way to prevent the following would be to update the Packages.gz file LAST, after syncing up the other files, IE: rsync --exclude Packages* debian/pool rsync --delete debian/pool (If old packages are even deleted) As I pointed out in another message, this will cause additional disk space on the mirrors (the number of changed packages each day seems to be increasing steadily), and also, with this rsync implementation, will cause nearly every file in the archive to be checksum-compared twice instead of once. I doubt the additional resource consumption is acceptable to all of the mirror operators. If it is, go for it. -- - mdz
