Re: Rsync sends again already existing files
You should also read about --inplace. Without it --no-whole-file you are telling it to do all the extra data diffing only to write out an entire new file anyway (just using data from source and target to create it). On 6/30/23 21:29, Selva Nair via rsync wrote: So this disable a lot of interest in Rsync :-( Isn't there a way to disable "--whole-file"? "--no-whole-file" should do it though for local copies, forcing delta transfer is not going to speed up anything in most cases. Selva -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync sends again already existing files
> > So this disable a lot of interest in Rsync :-( Isn't there a way to > disable > "--whole-file"? > "--no-whole-file" should do it though for local copies, forcing delta transfer is not going to speed up anything in most cases. Selva -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync sends again already existing files
Stephane Ascoet via rsync wrote: > Kevin Korb le 29/06/2023 22:43: > > Are you so sure rsync actually copies the file? It should > > correct the timestamp and tell you it did. > > Of that what it should do! But I'm sure not: the target is a very > low-quality-and-performance USB key ... less than 3 kb/s. And > looking the FS during the transfer, I saw the temporary files > being written. And since I was using --backup, the files has been > backuped... only to be replaced by exactly the same thing :-( > > So this disable a lot of interest in Rsync :-( Isn't there a way > to disable "--whole-file"? Short answer: Last I heard, no, for (what rsync sees as) "local" transfers. And I agree that there should be, at least when the target is a flash device -- but it might not help much with your situation. Longer answer, with details: rsync believes that, for any transfer which it sees as local (and this includes NFS, CIFS, etc. which are mapped into the local namespace although the data may in fact be accessed over a network), overwriting the destination -- even with what may be the same data -- is at least as fast as reading it first to determine whether it needs to be overwritten. This belief was usually accurate before the advent of read-mostly technologies (like flash). However, when the target is on a flash device, --whole-file creates a couple of problems: * Most flash devices, including but not limited to USB "thumb drives," can be read substantially faster than they can be written. * Even if there were no speed difference, overwriting -- even with the same data -- typically imposes wear on the flash device. (There may be a few which internally do a read-before-write, and avoid overwriting the same data.) As to your situation, I suspect you may be dealing more with transfer speed (via a USB 1.1 port?) than with read/write speed differences. The rsync assumption that reading is no faster than writing may well be accurate with this "low-quality-and-performance USB key." -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync sends again already existing files
Kevin Korb le 29/06/2023 22:43:
-i, -v, and --progress all only affect the output.
Bonjour, of course I know ;-)
adds a header and footer and --progress of course adds the per-file
progress bar.
Thanks, that what I wanted to know, so I keep them all.
as those 2 options are very different and I don't really see why they
should go together.
I fully agree but I don't like long options ;-p
Are you so sure rsync actually copies the file? It should correct the timestamp
and tell you it did.
Of that what it should do! But I'm sure not: the target is a very
low-quality-and-performance USB key. I had to do it on another computer witch
write on it a little less slow, otherwise I would have last forever with less
than 3 kb/s. And looking the FS during the transfer, I saw the temporary files
being written. And since I was using --backup, the files has been backuped...
only to be replaced by exactly the same thing :-(
So this disable a lot of interest in Rsync :-( Isn't there a way to disable
"--whole-file"?
Even if, I must admit, the source problem is the changes in timestamps. It's a
mystery, it seems that there was an exact one-hour difference each time. So I
suppose it's related to timezone, but how can this be? Is it the fact that the
source is in Vfat(but almost always used under GNU/Linux) and target in Ext2?
However, "date -r {file}" was saying "GMT+2" in both cases... I don't know and
don't have time to investigate on this.
Another strange thing, regarding this mailing list this time: I'm subscribed in
digest mode, but this doesn't prevent some mails to be missing. For example, in
the thread "copy to destination only new files" one week ago, I got the two
from s...@shch262.com but not the one from Robin Lee Powell
--
Sincerely, Stephane
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync sends again already existing files
Am 29.06.23 um 22:31 schrieb Stephane Ascoet via rsync: Kevin Korb le 29/06/2023 04:52: --itemize-changes will cause rsync to tell you what it thinks is Hi, thank you so much! Today I used a little different way of doing it, and another computer, and the behaviour is the same. It seems that the reason is a different timestamp. So the whole file is replaced just for this? I've always read that Rsync was using an advanced checksum mecanism instead of timestamps-only? Are you so sure rsync actually copies the file? It should correct the timestamp and tell you it did. smime.p7s Description: S/MIME Cryptographic Signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync sends again already existing files
On 6/29/23 16:31, Stephane Ascoet via rsync wrote: Kevin Korb le 29/06/2023 04:52: --itemize-changes will cause rsync to tell you what it thinks is Hi, thank you so much! Today I used a little different way of doing it, and another computer, and the behaviour is the same. It seems that the reason is a different timestamp. So the whole file is replaced just for this? I've always read that Rsync was using an advanced checksum mecanism instead of timestamps-only? Your rsync isn't networking. That means --whole-file is forced making rsync work pretty much like cp. If rsync was networking it would determine what is different about the files then use the existing version + changes to make a new file (this is just extra effort when rsync isn't networking). different. Also, -z is counter-productive when rsync isn't networking. Thanks again. It seems that "--progress" is redondant with "-P"(but it's because I use a list of rsync commands skeletons, and some doesn't have "-P" and I want to be sure to always have the progress indication). But, since I will now always use "-i", do "-v" and "--progress" add something to "-i"? -i, -v, and --progress all only affect the output. Once -i is in play -v adds a header and footer and --progress of course adds the per-file progress bar. -P is just --partial and --progress. I am not a fan of -P as those 2 options are very different and I don't really see why they should go together. -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync sends again already existing files
Kevin Korb le 29/06/2023 04:52: --itemize-changes will cause rsync to tell you what it thinks is Hi, thank you so much! Today I used a little different way of doing it, and another computer, and the behaviour is the same. It seems that the reason is a different timestamp. So the whole file is replaced just for this? I've always read that Rsync was using an advanced checksum mecanism instead of timestamps-only? different. Also, -z is counter-productive when rsync isn't networking. Thanks again. It seems that "--progress" is redondant with "-P"(but it's because I use a list of rsync commands skeletons, and some doesn't have "-P" and I want to be sure to always have the progress indication). But, since I will now always use "-i", do "-v" and "--progress" add something to "-i"? -- Sincerely, Stephane Ascoet -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync sends again already existing files
--itemize-changes will cause rsync to tell you what it thinks is different. Also, -z is counter-productive when rsync isn't networking. On 6/28/23 22:28, Stephane Ascoet via rsync wrote: Hi, /media/clec1enextsanstampon/gigamopourcdas4/ contains an old copy of files of /media/mo. I'm updating with "cd /media/clec1enextsanstampon/ && rsync --backup --backup-dir=/media/clec1enextsanstampon/elementssupprimesdepourcdas4/ --del -hPrStvXz --progress /media/mo/ /media/clec1enextsanstampon/gigamopourcdas4/" But unmodified and already in place files in /media/clec1enextsanstampon/gigamopourcdas4/ are re-sended from /media/mo instead of being skipped, why? -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
