Re: Does rsync verify its writes?
Robin is right. Read back verification has lost its meaning since old days when we used extremely unreliable media or transports. Who writes to floppies anymore? In those old days you could/should use any copy program with a -v verify flag. But read-back wore down your floppy faster, so in the end... Transfer by TCP and like protocols sense errors and do repair "on the fly". (Re-sent packages happen, and you never even notice.) If not correctable (only if connection breaks) you really receive an error that is sensed and passed through all layers to the application. And if this is not really weird programming, the message will be caught by the user or calling program. The same is true for modern hardware. Your s.m.a.r.t HDD informs the system of problems. Since 1980s or so I NEVER had an error displayed by a direct (immediate) verification read. At some LATER time your aged CD might become unreadable... To make it short: Technical failure WILL be noticed by modern systems. It is admin's job to react. Nowadays 99% of data loss is neglect in programming or user attention. And the last may even occur with a verification message. Do you write your script so you REALLY receive an error if one happens? Even operating worn/aged HW is not remedied by read back, but in fact a human neglect. Still, flushing all volatile memory at the end of a write process is a good idea. A proper sync does that, a read back does not. Au contraire, it will happily read cached data. Am 17.02.23 um 17:51 schrieb Robin Lee Powell via rsync: That's not the same as a read-back write verification. I believe that in general, rsync assumes that the disk actually wrote whatever it was told to write. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Does rsync verify its writes?
That's not the same as a read-back write verification. I believe that in general, rsync assumes that the disk actually wrote whatever it was told to write. However, a second pass with --checksum will, in fact, read the entirety of both files; if a --checksum run doesn't actually transfer anything, you can safely conclude the files are byte-for-byte[1] identical (or you have a serious low-level problem where the disk is lying to you, which rsync is definitely not going to be able to fix). [1]: Technically they might not be byte-for-byte identical if you somehow get identical hashing blocks, but Schneier once compared that sort of thing to the survival time of a mouse on the surface of the sun, so :D On Fri, Feb 17, 2023 at 11:18:59AM +0100, anubis23 via rsync wrote: > Hi, > > you can use the switch --fsync to verify the transfer. From the manpage: > > --fsync > Cause the receiving side to fsync each finished file. > This may slow down the transfer, but can help to > provide peace of mind when updating critical files. > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Does rsync verify its writes?
Hi, you can use the switch --fsync to verify the transfer. From the manpage: --fsync Cause the receiving side to fsync each finished file. This may slow down the transfer, but can help to provide peace of mind when updating critical files. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Does rsync verify its writes?
Note that it won't actually tell you that the file was written wrongly just that it is different. In fact, you can only even know that it might be detecting a broken file with --itemize-changes and look for files with a checksum difference but not a timestamp difference. Though there are legitimate causes for that. On 7/12/22 17:26, Paul Slootman via rsync wrote: On Tue 12 Jul 2022, Kevin Korb via rsync wrote: Rsync does not verify writes. --checksum doesn't verify anything. Sounds like you want a file verification tool. The simplest would be md5sum. Running rsync --checksum directly after transferring your files will verify that the files are written correctly (if the source hasn't changed in the meantime). That might help to give some peace of mind. Paul On 7/12/22 02:31, Mark Filipak via rsync wrote: Hello. Does rsync verify its writes? Re, 'info rsync'. Maybe I just being stupid, but there's no mention of verification in the 'DESCRIPTION' section, so despite the words in the 'OPTIONS' section, '-c, --checksum' topic (which I may be misinterpreting), I assume rsync does not verify except for the checksum directive. I admit that I'm paranoid. ;-) ...Please clarify. Regards, and Thanks, Mark Filipak. -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Does rsync verify its writes?
On Tue 12 Jul 2022, Kevin Korb via rsync wrote: > Rsync does not verify writes. --checksum doesn't verify anything. Sounds > like you want a file verification tool. The simplest would be md5sum. Running rsync --checksum directly after transferring your files will verify that the files are written correctly (if the source hasn't changed in the meantime). That might help to give some peace of mind. Paul > On 7/12/22 02:31, Mark Filipak via rsync wrote: > > Hello. Does rsync verify its writes? > > > > Re, 'info rsync'. > > > > Maybe I just being stupid, but there's no mention of verification in the > > 'DESCRIPTION' section, so despite the words in the 'OPTIONS' section, > > '-c, --checksum' topic (which I may be misinterpreting), I assume rsync > > does not verify except for the checksum directive. > > > > I admit that I'm paranoid. ;-) ...Please clarify. > > > > Regards, and Thanks, > > Mark Filipak. > > > > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone:(407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. ke...@futurequest.net (work) > Orlando, Floridak...@sanitarium.net (personal) > Web page: https://sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Does rsync verify its writes?
Rsync does not verify writes. --checksum doesn't verify anything. Sounds like you want a file verification tool. The simplest would be md5sum. On 7/12/22 02:31, Mark Filipak via rsync wrote: Hello. Does rsync verify its writes? Re, 'info rsync'. Maybe I just being stupid, but there's no mention of verification in the 'DESCRIPTION' section, so despite the words in the 'OPTIONS' section, '-c, --checksum' topic (which I may be misinterpreting), I assume rsync does not verify except for the checksum directive. I admit that I'm paranoid. ;-) ...Please clarify. Regards, and Thanks, Mark Filipak. -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Does rsync verify its writes?
Hello. Does rsync verify its writes? Re, 'info rsync'. Maybe I just being stupid, but there's no mention of verification in the 'DESCRIPTION' section, so despite the words in the 'OPTIONS' section, '-c, --checksum' topic (which I may be misinterpreting), I assume rsync does not verify except for the checksum directive. I admit that I'm paranoid. ;-) ...Please clarify. Regards, and Thanks, Mark Filipak. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
