RE: [Cooker] Proper rsync syntax for Cooker downloads
> I noticed this myself actually. When I did the first rsync download to > an empty directory, I got the rpms and not RPMS as one would expect. > I am downloading to a FAT32 mounted partition - this may have something ^ do not do this! It is not a file system :-) > to do with it . . . not case sensitive I guess. Even worse. FAT has 2 seconds time resolution, so you may get interesting things here as well. Get yourself decent file system :-) Seriously, it is (known) Linux bug. It does not create long filenames for 8.3 files vfat, and then it automatically *lowers* short filenames (for long filenames case is preserved). Sorry, get yourself decent file system :-( If you cannot repartition - consider creating loop device. Regards, -andrej
RE: [Cooker] Proper rsync syntax for Cooker downloads
I noticed this myself actually. When I did the first rsync download to an empty directory, I got the rpms and not RPMS as one would expect. I am downloading to a FAT32 mounted partition - this may have something to do with it . . . not case sensitive I guess. here is the mount table and path as on my machine: [root@amda7v cooker]# mount /dev/hde6 on / type reiserfs (rw) none on /proc type proc (rw) devfs on /dev type devfs (rw) none on /dev/pts type devpts (rw,mode=0620) none on /dev/shm type tmpfs (rw) /dev/hde8 on /home type reiserfs (rw) /dev/hde1 on /mnt/win_c type vfat (rw,nosuid,nodev,umask=0,codepage=850,iocharset=iso8859-1) /dev/hde5 on /mnt/win_d type vfat (rw,nosuid,nodev,umask=0,codepage=850,iocharset=iso8859-1) /dev/hde9 on /mnt/win_e type vfat (rw,nosuid,nodev,umask=0,codepage=850,iocharset=iso8859-1) /dev/hde10 on /mnt/win_f type vfat (rw,nosuid,nodev,umask=0,codepage=850,iocharset=iso8859-1) /dev/hde11 on /mnt/win_g type vfat (rw,nosuid,nodev,umask=0,codepage=850,iocharset=iso8859-1) /dev/hda5 on /mnt/win_h type vfat (rw,nosuid,nodev,umask=0,codepage=850,iocharset=iso8859-1) /proc/bus/usb on /proc/bus/usb type usbdevfs (rw,devmode=0664,devgid=43) The path to my rpms is: [root@amda7v Mandrake]# pwd /mnt/win_h/cooker/Mandrake [root@amda7v Mandrake]# ls base/ mdkinst/ rpms/ rpms2/ share/ [root@amda7v Mandrake]# Strange huh? R.Fox On 28 Aug 2001 16:15:43 +0400, Borsenkow Andrej wrote: > > > > Strange. > > > > I have tried the --delete option with not so nice results. Every time > I > > update the tree with rsync (and previously with wget) I end up > > eventually with lots of older (outdated) files. > > > > I tried the following line which basically blew away many "good" files > > (not just older files!): > > > > rsync -auvPn --delete ftp.sunet.se::Mandrake-devel/cooker/i586/ . > > > > Here's a snip of the output: > > deleting Mandrake/rpms/kernel22-source-2.2.19-17mdk.i586.rpm > > Is it correct? It should be RPMS (capitals). > > -andrej >
RE: [Cooker] Proper rsync syntax for Cooker downloads
> > Strange. > > I have tried the --delete option with not so nice results. Every time I > update the tree with rsync (and previously with wget) I end up > eventually with lots of older (outdated) files. > > I tried the following line which basically blew away many "good" files > (not just older files!): > > rsync -auvPn --delete ftp.sunet.se::Mandrake-devel/cooker/i586/ . > > Here's a snip of the output: > deleting Mandrake/rpms/kernel22-source-2.2.19-17mdk.i586.rpm Is it correct? It should be RPMS (capitals). -andrej
Re: [Cooker] Proper rsync syntax for Cooker downloads
Strange. I have tried the --delete option with not so nice results. Every time I update the tree with rsync (and previously with wget) I end up eventually with lots of older (outdated) files. I tried the following line which basically blew away many "good" files (not just older files!): rsync -auvPn --delete ftp.sunet.se::Mandrake-devel/cooker/i586/ . Here's a snip of the output: deleting Mandrake/rpms/kernel22-source-2.2.19-17mdk.i586.rpm deleting Mandrake/rpms/kernel22-smp-2.2.19-17mdk.i586.rpm deleting Mandrake/rpms/kernel22-secure-2.2.19-17mdk.i586.rpm deleting Mandrake/rpms/kernel22-2.2.19-17mdk.i586.rpm deleting Mandrake/rpms/kernel-source-2.4.8-12mdk.i586.rpm deleting Mandrake/rpms/kernel-smp-2.4.8-12mdk.i586.rpm deleting Mandrake/rpms/kernel-pcmcia-cs-2.4.8-12mdk.i586.rpm deleting Mandrake/rpms/kernel-linus2.4-2.4.8-1mdk.i586.rpm deleting Mandrake/rpms/kernel-linus2.2-2.2.19-5mdk.i586.rpm deleting Mandrake/rpms/kernel-headers-2.4.8-12mdk.i586.rpm deleting Mandrake/rpms/kernel-enterprise-2.4.8-12mdk.i586.rpm deleting Mandrake/rpms/kernel-doc-2.4.8-12mdk.i586.rpm deleting Mandrake/rpms/kernel-2.4.8-12mdk.i586.rpm ... Something is not right here. Thx, R. Fox On 28 Aug 2001 11:04:37 +1000, Ron Stodden wrote: > Lonnie Borntreger wrote: > > > > This thread has been interesting. I've been mirroring using the > > following command: > > > > mirrordir -v \ > > ftp://sunsite.uio.no/linux/Mandrake-devel/cooker/i586 \ > > /home/Mandrake-devel/cooker/i586 > > > > I'd like to try rsync, since I'm thinking that it would be more > > bandwidth efficient. So far no luck. Here is what I tried (first try > > as a "dry run" ... "-n"): > > > > rsync -avz -r -n --delete --partial --progress --update \ > > sunsite.uio.no:/linux/Mandrake-devel/cooker/i586 \ > > /home/Mandrake-devel/cooker/i586 > > Try: > > rsync -avlHP --delete --partial --progress --stats \ > rsync://sunsite.uio.no:873/Mandrake-devel/cooker/i586 \ > /home/Mandrake-devel/cooker/i586 > > Or better still use our popular and faster package for the whole > Mandrake downloading job (select from: contrib, 8.0-tree, cooker, 8.0 > updates, unsupported, freq ISOs, release ISOs, beta ISOs), available > from my web site: > > http://www.ains.net.au/~ronst/ > > This includes the ability to specify selective downloads, initially > set to English only (save over 200 MB on a tree download). > > -- > Ron. [au] > > Kindly note my new email address: [EMAIL PROTECTED] > and new web site: http://www.ains.net.au/~ronst/ >
Re: [Cooker] Proper rsync syntax for Cooker downloads
So sprach »Randy Kramer« am 2001-08-27 um 17:39:51 -0400 : > instead of one. Sometimes it needs one colon, sometimes it needs two, > there is a logical reason but I don't remember what it is (has to do > with protocols used) -- I always seem to need two. because with this syntax, you need two. however, you can also write rsync:/sunsite.uio.no/linu, in which case you only need one. Alexander Skwar -- How to quote: http://learn.to/quote (german) http://quote.6x.to (english) Homepage: http://www.digitalprojects.com | http://www.iso-top.de iso-top.de - Die günstige Art an Linux Distributionen zu kommen Uptime: 22 hours 24 minutes
Re: [Cooker] Proper rsync syntax for Cooker downloads
So sprach »Lonnie Borntreger« am 2001-08-27 um 15:59:13 -0500 : > sunsite.uio.no:/linux/Mandrake-devel/cooker/i586 \ sunsite.uio.no:: Alexander Skwar -- How to quote: http://learn.to/quote (german) http://quote.6x.to (english) Homepage: http://www.digitalprojects.com | http://www.iso-top.de iso-top.de - Die günstige Art an Linux Distributionen zu kommen Uptime: 22 hours 24 minutes
Re: [Cooker] Proper rsync syntax for Cooker downloads
Lonnie Borntreger wrote: > > This thread has been interesting. I've been mirroring using the > following command: > > mirrordir -v \ > ftp://sunsite.uio.no/linux/Mandrake-devel/cooker/i586 \ > /home/Mandrake-devel/cooker/i586 > > I'd like to try rsync, since I'm thinking that it would be more > bandwidth efficient. So far no luck. Here is what I tried (first try > as a "dry run" ... "-n"): > > rsync -avz -r -n --delete --partial --progress --update \ > sunsite.uio.no:/linux/Mandrake-devel/cooker/i586 \ > /home/Mandrake-devel/cooker/i586 Try: rsync -avlHP --delete --partial --progress --stats \ rsync://sunsite.uio.no:873/Mandrake-devel/cooker/i586 \ /home/Mandrake-devel/cooker/i586 Or better still use our popular and faster package for the whole Mandrake downloading job (select from: contrib, 8.0-tree, cooker, 8.0 updates, unsupported, freq ISOs, release ISOs, beta ISOs), available from my web site: http://www.ains.net.au/~ronst/ This includes the ability to specify selective downloads, initially set to English only (save over 200 MB on a tree download). -- Ron. [au] Kindly note my new email address: [EMAIL PROTECTED] and new web site: http://www.ains.net.au/~ronst/
Re: [Cooker] Proper rsync syntax for Cooker downloads
On Mon, 2001-08-27 at 16:47, Sergio Korlowsky wrote: > First... do not use -z all files are already compressed... most sites > recomend to turn this off, as it is cpu intensive. > > secondly not need to set the complete path use the module instead... > rsync -av (no z) -n (as this is your first try use) --delete-after --progress > --update sunsite.uio.no::Mandrake-devel/cooker/i586 /home/whatever-dir/cooker > > And this is what I actually use: > > rsync -a -vvv --progress --partial --update --delete > ftp.sunet.se::Mandrake-devel/cooker/i586 /home/admin/cooker > > Sergio Korlowsky Based on your input, I used: rsync -avv --delete --partial --progress --update \ sunsite.uio.no::Mandrake-devel/cooker/i586 /home/Mandrake-devel/cooker It worked beautifully. Thanks. -- TTFN, Lonnie Borntreger -- -- -- Our mission is to authoritatively promote timely methods of empowerment so that we may endeavor to collaboratively supply inexpensive opportunities to exceed customer expectations
Re: [Cooker] Proper rsync syntax for Cooker downloads
On Monday 27 August 2001 03:59 pm, you wrote: > This thread has been interesting. I've been mirroring using the > following command: > > mirrordir -v \ > ftp://sunsite.uio.no/linux/Mandrake-devel/cooker/i586 \ > /home/Mandrake-devel/cooker/i586 > > I'd like to try rsync, since I'm thinking that it would be more > bandwidth efficient. So far no luck. Here is what I tried (first try > as a "dry run" ... "-n"): > > rsync -avz -r -n --delete --partial --progress --update \ > sunsite.uio.no:/linux/Mandrake-devel/cooker/i586 \ > /home/Mandrake-devel/cooker/i586 > > All I see is: > > poll: protocol failure in circuit setup > unexpected EOF in read_timeout > > Am I doing something wrong, or does this site not support rsync? First... do not use -z all files are already compressed... most sites recomend to turn this off, as it is cpu intensive. secondly not need to set the complete path use the module instead... rsync -av (no z) -n (as this is your first try use) --delete-after --progress --update sunsite.uio.no::Mandrake-devel/cooker/i586 /home/whatever-dir/cooker And this is what I actually use: rsync -a -vvv --progress --partial --update --delete ftp.sunet.se::Mandrake-devel/cooker/i586 /home/admin/cooker Sergio Korlowsky -- SedeComp Comunicaciones Internet Solutions MandrakeSoft's VAR and System Integrator mailto:[EMAIL PROTECTED] OpenPGP key available on:http://www.keyserver.net/en/ |--| Current Linux kernel 2.4.8-5mdk uptime: 18 hours 45 minutes.
Re: [Cooker] Proper rsync syntax for Cooker downloads
Lonnie, Until someone who know what they're doing responds, try two colons (::) instead of one. Sometimes it needs one colon, sometimes it needs two, there is a logical reason but I don't remember what it is (has to do with protocols used) -- I always seem to need two. hth, Randy Kramer Lonnie Borntreger wrote: > rsync -avz -r -n --delete --partial --progress --update \ > sunsite.uio.no:/linux/Mandrake-devel/cooker/i586 \ > /home/Mandrake-devel/cooker/i586
Re: [Cooker] Proper rsync syntax for Cooker downloads
This thread has been interesting. I've been mirroring using the following command: mirrordir -v \ ftp://sunsite.uio.no/linux/Mandrake-devel/cooker/i586 \ /home/Mandrake-devel/cooker/i586 I'd like to try rsync, since I'm thinking that it would be more bandwidth efficient. So far no luck. Here is what I tried (first try as a "dry run" ... "-n"): rsync -avz -r -n --delete --partial --progress --update \ sunsite.uio.no:/linux/Mandrake-devel/cooker/i586 \ /home/Mandrake-devel/cooker/i586 All I see is: poll: protocol failure in circuit setup unexpected EOF in read_timeout Am I doing something wrong, or does this site not support rsync? -- TTFN, Lonnie Borntreger -- -- -- Our mission is to authoritatively promote timely methods of empowerment so that we may endeavor to collaboratively supply inexpensive opportunities to exceed customer expectations
Re: [Cooker] Proper rsync syntax for Cooker downloads
On Monday 27 August 2001 08:59 am, you wrote: > > > For me, wget -m keeps the file dates, even under w2k... > > > > Across time zones? Good news if so. > > For ftp transfers? Impossible. HTTP sends time in GMT so it is no > problem. FTP has no notion of local time, so it is impossible by > definition (some clients allow you to configure offset on per-server > basis, but forget daylight saving time). > > -andrej I use gftp to mirror most anything... but has the problem of leving behind 'old' packages, so you have to run compare windows in the tools option. So, I went back to rsync with the --progress --partial --delete --update enabled and forget about partial (incomplete packages) or deleting by hand old packages. rsync its easy and clean to use. I only use wget for downloading complete web sites. so I guess each one of them has its own advantages and dis-advantages. sk -- SedeComp Comunicaciones Internet Solutions MandrakeSoft's VAR and System Integrator mailto:[EMAIL PROTECTED] OpenPGP key available on:http://www.keyserver.net/en/ |--| Current Linux kernel 2.4.8-5mdk uptime: 16 hours 25 minutes.
Re: [Cooker] Proper rsync syntax for Cooker downloads
guran wrote: > > rsync has the following option that was originally asked for: > > --size-only > Normally rsync will skip any files that are already > the same length and have the same time-stamp. With > the --size-only option files will be skipped if > they have the same size, regardless of timestamp. > This is useful when starting to use rsync after > using another mirroring system which may not pre > serve timestamps exactly. Thanks! -- Ron. [au]
Re: [Cooker] Proper rsync syntax for Cooker downloads
Hi rsync has the following option that was originally asked for: --size-only Normally rsync will skip any files that are already the same length and have the same time-stamp. With the --size-only option files will be skipped if they have the same size, regardless of timestamp. This is useful when starting to use rsync after using another mirroring system which may not pre serve timestamps exactly. regards guran
RE: [Cooker] Proper rsync syntax for Cooker downloads
> > > For me, wget -m keeps the file dates, even under w2k... > > Across time zones? Good news if so. > For ftp transfers? Impossible. HTTP sends time in GMT so it is no problem. FTP has no notion of local time, so it is impossible by definition (some clients allow you to configure offset on per-server basis, but forget daylight saving time). -andrej
Re: [Cooker] Proper rsync syntax for Cooker downloads
[EMAIL PROTECTED] wrote: > For me, wget -m keeps the file dates, even under w2k... Across time zones? Good news if so. -- Ron. [au]
Re: [Cooker] Proper rsync syntax for Cooker downloads
> No, it will download it all again, because wget is a defective > downloader in that it does not preserve file dates - rsync therefore > determines the server file to be a new version. Never speak so quick : wget is somewhat old, so there is much chances you are "the defect"! For me, wget -m keeps the file dates, even under w2k... /\ | José Jorge <[EMAIL PROTECTED]>| | TEKLYNX International http://www.teklynx.com | \/
Re: [Cooker] Proper rsync syntax for Cooker downloads
"R.I.P. Deaddog" wrote: > > On Mon, 27 Aug 2001, Ron Stodden wrote: > > > No, it will download it all again, because wget is a defective > > downloader in that it does not preserve file dates - rsync therefore > > determines the server file to be a new version. > > To preserve file dates in wget, I think -m option may do it, but not so > sure. rsync uses a file last modified dating scheme that takes into account time zones. I speculate that the file timestamps must match when normalised to the same time zone. I do not know how rsync determines the server time zone (but it does). AFAIK, wget has no timezone awareness. -- Ron. [au]
Re: [Cooker] Proper rsync syntax for Cooker downloads
On Mon, 27 Aug 2001, Ron Stodden wrote: > No, it will download it all again, because wget is a defective > downloader in that it does not preserve file dates - rsync therefore > determines the server file to be a new version. To preserve file dates in wget, I think -m option may do it, but not so sure. Abel
Re: [Cooker] Proper rsync syntax for Cooker downloads
yOn 26 Aug 2001, Robert Fox wrote: > I appreciate the feedback so far - there appears to be several tools to > use (wget,rsync,fmirror,mirror,lftp,etc.) > > It's a tad confusing trying to figure out the ideal tool, let alone the > correct syntax and parms to use. > rsync is better in my experience. > Can I rsync on top of the wget tree I downloaded? > Yes. Because wget does not preserve mtime (does it nowadays?) rsync will check every file. You may want to try -I first to ignore file times in comparison. The simplest syntax to download cooker with rsync is rsync -av --delete rsync://ftp.sunet.se/Mandrake-devel/cooker/i586/ /local/mirror/cooker/i586/ This will mirror everything and delete files not found on server. There were several scripts that rename local files to take advantage of rsync algorithm. In my experience it does not bring much. Where rsync is *very* good - is downloading of hdlist. Instead of 12MB every time it ends up only in several KB - it is really only difference between old and new. -andrej
Re: [Cooker] Proper rsync syntax for Cooker downloads
I appreciate the feedback so far - there appears to be several tools to use (wget,rsync,fmirror,mirror,lftp,etc.) It's a tad confusing trying to figure out the ideal tool, let alone the correct syntax and parms to use. Can I rsync on top of the wget tree I downloaded? Thanks again. RF On 26 Aug 2001 15:08:13 +0200, Eivind Eriksen wrote: > On Sun, 26 Aug 2001, Ron Stodden wrote: > > > Robert Fox wrote: > > > > > > I have been using "wget -rm" to fetch the Cooker tree - but if I do it often >enough, I end up with a bunch of older files on the downloaded branch > > > (because the older version is not removed) > > > > > > Is rsync or mirror better? If so, what is the recommended download syntax for >using rsync or mirror? > > > > rsync is much better, and a few of us have done a lot of good work > > packaging rsync for use with the Mandrake RPM naming conventions. > > > > Download the package from my web site: > > > > http://www.ains.net.au/~ronst/ > > > > and stop having to worry - it's much faster and better and quietly > > deletes any local files not on the server. > > > > > > You may also use lftp with mirror -e: > > lftp ftp.uninett.no > cd linux/Mandrake-devel/cooker/i586 > mirror -e -v > > >
Re: [Cooker] Proper rsync syntax for Cooker downloads
On Sun, 26 Aug 2001, Ron Stodden wrote: > Robert Fox wrote: > > > > I have been using "wget -rm" to fetch the Cooker tree - but if I do it often >enough, I end up with a bunch of older files on the downloaded branch > > (because the older version is not removed) > > > > Is rsync or mirror better? If so, what is the recommended download syntax for >using rsync or mirror? > > rsync is much better, and a few of us have done a lot of good work > packaging rsync for use with the Mandrake RPM naming conventions. > > Download the package from my web site: > > http://www.ains.net.au/~ronst/ > > and stop having to worry - it's much faster and better and quietly > deletes any local files not on the server. > > You may also use lftp with mirror -e: lftp ftp.uninett.no cd linux/Mandrake-devel/cooker/i586 mirror -e -v
Re: [Cooker] Proper rsync syntax for Cooker downloads
Robert Fox wrote: > > I have been using "wget -rm" to fetch the Cooker tree - but if I do it often enough, >I end up with a bunch of older files on the downloaded branch > (because the older version is not removed) > > Is rsync or mirror better? If so, what is the recommended download syntax for using >rsync or mirror? rsync is much better, and a few of us have done a lot of good work packaging rsync for use with the Mandrake RPM naming conventions. Download the package from my web site: http://www.ains.net.au/~ronst/ and stop having to worry - it's much faster and better and quietly deletes any local files not on the server. -- Ron. [au]
