propagate atimes with rsync-2.6.6
Dear Wayne. Thank you very much for your extensive comments on the our patch. I will try and take them into account and mend our patch accordingly. Unfortunately, I can only do this during my leisure time which is already very limited so it might take some time. But it would be very nice if this feature could make it into the reference version. We consider it as very useful if not necessary for back-ups of systems where the access to files by employees is monitored. That's especially relevant if the usage of a database should be assessed. We also thought of extending the changes such that rsync would also transfer atimes if the modification times are the same but we dropped that for the moment since the hardlink-feature allows this already. I have received an email from Emile LeBlanc who supports the development of these features very much. The initiative for our changes came from a company with daily back-ups of their database. Maybe, this feature may be useful to other people too. Maybe, we could merge the two patches to create a nice new feature. That would be great. Thank you very much, Christian -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
propagate atimes with rsync-2.6.6 (fwd)
Dear Martin Pool. We regularly use rsync for making backups of our file systems but we have noticed that the atimes are not transferred with the files and are also always updated on the sender's side. Therefore, we have created a modified version of rsync based on rsync-2.6.6 protocol version 29 which transfers the access times with the transferred files and also allows to preserve the access times on the sending side. The transfer of the atimes is done by default while the preservation of the source atimes is switched on by an additional option (-X --atime-preserve). To guarantee compatibility with previous protocol versions, we have introduced a new protocol version (30). If the remote protocol version is older than 30, rsync behaves as before. The changes made to the reference version are small and kept in accordance with the original style. The affected files are: flist.c generator.c options.c proto.h rsync.1 rsync.c rsync.h rsync3.txt sender.c and util.c Please find the corresponding patch attached to this email. We have thouroughly tested our new protocol under Linux and also compiled it under Unix (Aix). It works as expected under Linux while some of the changes were redundant under Unix. We consider the preservation of the source access times as useful since this information might be worth being backed-up as well. We hope our approach will find your attention and would be glad if it were included in the next update of rsync. Please let us know about your suggestions for improvements. Yours truly, Christian Konz diff -uNr rsync-2.6.6/flist.c rsync-2.6.6_patch/flist.c --- rsync-2.6.6/flist.c 2005-07-07 21:49:14.0 +0200 +++ rsync-2.6.6_patch/flist.c 2006-01-07 14:04:38.0 +0100 @@ -54,6 +54,7 @@ extern int implied_dirs; extern int copy_links; extern int copy_unsafe_links; +extern int remote_protocol; extern int protocol_version; extern int sanitize_paths; extern int orig_umask; @@ -314,6 +315,7 @@ { unsigned short flags; static time_t modtime; + static time_t acctime; static mode_t mode; static int64 dev; static dev_t rdev; @@ -329,7 +331,7 @@ if (!file) { write_byte(f, 0); - modtime = 0, mode = 0; + modtime = 0, acctime = 0, mode = 0; dev = 0, rdev = makedev(0, 0); rdev_major = 0; uid = 0, gid = 0; @@ -379,6 +381,8 @@ else modtime = file->modtime; +acctime = file->acctime; + #ifdef SUPPORT_HARD_LINKS if (file->link_u.idev) { if (file->F_DEV == dev) { @@ -431,6 +435,8 @@ write_longint(f, file->length); if (!(flags & XMIT_SAME_TIME)) write_int(f, modtime); + if (remote_protocol >= 30) + write_int(f, acctime); if (!(flags & XMIT_SAME_MODE)) write_int(f, to_wire_mode(mode)); if (preserve_uid && !(flags & XMIT_SAME_UID)) { @@ -506,6 +512,7 @@ unsigned short flags, int f) { static time_t modtime; + static time_t acctime; static mode_t mode; static int64 dev; static dev_t rdev; @@ -524,7 +531,7 @@ struct file_struct *file; if (!flist) { - modtime = 0, mode = 0; + modtime = 0, acctime = 0, mode = 0; dev = 0, rdev = makedev(0, 0); rdev_major = 0; uid = 0, gid = 0; @@ -578,6 +585,10 @@ file_length = read_longint(f); if (!(flags & XMIT_SAME_TIME)) modtime = (time_t)read_int(f); + if (remote_protocol >= 30) + acctime = (time_t)read_int(f); + else + acctime = time(NULL); if (!(flags & XMIT_SAME_MODE)) mode = from_wire_mode(read_int(f)); @@ -630,6 +641,7 @@ file->flags = 0; file->modtime = modtime; + file->acctime = acctime; file->length = file_length; file->mode = mode; file->uid = uid; @@ -882,6 +894,7 @@ file->flags = flags; file->modtime = st.st_mtime; + file->acctime = st.st_atime; file->length = st.st_size; file->mode = st.st_mode; file->uid = st.st_uid; diff -uNr rsync-2.6.6/generator.c rsync-2.6.6_patch/generator.c --- rsync-2.6.6/generator.c 2005-07-28 21:06:03.0 +0200 +++ rsync-2.6.6_patch/generator.c 2005-11-25 11:33:33.0 +0100 @@ -917,6 +917,13 @@ fnamecmpbuf, 1, itemizing && verbose > 1, code) == 0) { + if (set_modtime(fname,file->modtime, + file->acctime) != 0) { +
Re: propagate atimes with rsync-2.6.6
On Thu, 2006-01-26 at 01:12 -0800, Wayne Davison wrote: > On Tue, Jan 24, 2006 at 05:46:36PM +0100, Christian Konz T3 wrote: > > Therefore, we have created a modified version of rsync based on > > rsync-2.6.6 protocol version 29 which transfers the access times with > > the transferred files and also allows to preserve the access times on > > the sending side. > > - Your patch's optional preservation of the access times on the > sending side is done by forcing both the access time and the modify > time to the values that were acquired at the start of a file's > transfer.[...] My personal stance on file times is as follows. A file's mtime is a useful counterpart to its data because it identifies the version of the file; two files with identical names and mtimes are very likely to have identical data (as rsync's quick check assumes). Thus, mtimes should usually be kept with data and preserved when a file's data is copied. On the other hand, atime and ctime have hardly any use to normal users and should be available only to the owner of the filesystem for forensic purposes, if at all. I especially have trouble with atime because "reading shouldn't write". Thus, all my computer's filesystems are mounted noatime, and all my files on my computer and others' computers carry the ext2 attribute A. Enough digression. While I don't have any desire to copy atimes to my backups, others might want this feature, so we might as well add an option to rsync. On the other hand, the issue of changing back or leaving undisturbed the atimes of the source files has been discussed extensively on the tar mailing list. The thread starts here: http://lists.gnu.org/archive/html/bug-tar/2005-09/msg00035.html Summary: For a while, tar has supported setting atimes back with utime(2). The new O_NOATIME option to open(2) provides a better way: one can open a file one owns without disturbing its atime. However, O_NOATIME only works on some kernels; on others, it is silently ignored. I think tar can now be told to use one method or the other or to try and figure out whether O_NOATIME works and then act accordingly. -- Matt McCutchen, ``hashproduct'' [EMAIL PROTECTED] -- http://mysite.verizon.net/hashproduct/ -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: propagate atimes with rsync-2.6.6 (fwd)
On Tue, Jan 24, 2006 at 05:46:36PM +0100, Christian Konz T3 wrote: > Therefore, we have created a modified version of rsync based on > rsync-2.6.6 protocol version 29 which transfers the access times with > the transferred files and also allows to preserve the access times on > the sending side. First -- thanks for the patch. You've unfortunately duplicated some work that was already done (see the file patches/atimes.diff in the rsync source), but your version does do things a little differently: - The atimes.diff patch does not require a protocol bump because the new behavior is not enabled by default; it is turned on via the --atimes (-U) option. (I personally don't think that preserving the atime should happen by default, not even with -a.) - The atimes.diff patch omits directories from its access-time preservation (since it is very likely that the atime on directories will change outside of rsync's control and some systems don't like the atime to be forced on a directory). - The atimes.diff patch adds a new XMIT_SAME_ATIME flag that avoids sending the same atime value twice in a row (just as the regular code uses XMIT_SAME_TIME for the mtime value). - The atimes.diff patch changes the output of --itemize-changes when --atimes is specified to indicate which files needed an access-time change. - Your patch has an option for leaving the access time unchanged on the source where the atimes.diff patch does not (though see below for my comments on this). The main reason that the --atimes option has not made it into the main source is that I was not willing to increase the amount of per-file memory needed to make room for a value that most people didn't need (since the preservation of access times is not commonly required). However, I recently modified the atimes.diff to make it only use extra memory when the --atimes option was actually specified, so I am considering adding the option to the next version (its inclusion now mainly depends on if I consider this memory-saving design too much of a kluge or not). Here are some comments on your patch: - Your code should not be checking the remote_protocol variable, but the protocol_version variable. This is because the negotiated protocol version can differ from the maximum remote value (mainly by the use of the --protocol option). - You have code that sets the atime on a file that was hard-linked via --link-dest. This is probably the right choice (and one that the atimes.diff patch fails to consider): it can mean that older version(s) of identical files can get the access time updated, but it's probably not an important enough of a difference to force a new version to be copied into the destination (rather than hard-linked to a --link-dest version). - Your change in set_perms() only sets the atime if the mtime differs. - Your patch's optional preservation of the access times on the sending side is done by forcing both the access time and the modify time to the values that were acquired at the start of a file's transfer. This can be unsafe if someone managed to change the file during the time that the file was being sent. You should at least re-stat the file at the end of the transfer to make sure that the modify time hasn't changed (and skip the update if it has). Even then, I'd be uneasy about this method of trying to leave the access time unchanged on the source because there would still be a chance that it would obliterate an updated mtime. Overall you did a very nice job of changing the code. Thanks again for helping out. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html