Hi, I am at the moment using rsync to move quite a big set of backups from one machine to another. The source filesystem is xfs; the target filesystem is btrfs.
For various reasons I have been stopping the rsync part way through and re-starting. I have noticed that a large number of files are transferred over and over and I can't work out why. Example: sudo rsync -iPva \ --inplace \ --numeric-ids \ --delete \ /data/backup/rsnapshot/daily.0/cacti/ \ root@koff:/data/backup/rsnapshot/daily.0/cacti/ ... <f..t.og... var/www/index.html 5,258 100% 5.78kB/s 0:00:00 (xfr#1276, to-chk=1/43437) If I run the rsync command again, thousands of lines of output will appear again, all showing itemized changes for 't' and sometimes 'p', 'o' and 'g'. Notably, var/www/index.html will keep appearing in the list. Let's have a look at that file. Source: $ stat /data/backup/rsnapshot/daily.0/cacti/var/www/index.html File: /data/backup/rsnapshot/daily.0/cacti/var/www/index.html Size: 5258 Blocks: 16 IO Block: 4096 regular file Device: fd05h/64773d Inode: 354337 Links: 37 Access: (0644/-rw-r--r--) Uid: ( 33/www-data) Gid: ( 33/www-data) Access: 2022-02-03 04:53:12.115719681 +0000 Modify: 2006-07-14 16:42:37.000000000 +0000 Change: 2022-01-01 17:31:28.553758359 +0000 Birth: - Destination: $ stat /data/backup/rsnapshot/daily.0/cacti/var/www/index.html File: /data/backup/rsnapshot/daily.0/cacti/var/www/index.html Size: 5258 Blocks: 16 IO Block: 4096 regular file Device: 26h/38d Inode: 7534065 Links: 26 Access: (0644/-rw-r--r--) Uid: ( 33/www-data) Gid: ( 33/www-data) Access: 2022-01-25 20:40:09.930960486 +0000 Modify: 2006-07-14 16:42:37.000000000 +0000 Change: 2022-02-03 21:45:44.194559899 +0000 Birth: 2022-01-25 20:40:09.930960486 +0000 When rsync considers times as being different, it means mtime, right? Yet these files have identical mtimes. They also have identical uid, gid and permissions. I would not expect this and other files like it to keep being listed for change over and over again. I can tell by the summary that the actual contents of the files weren't sent, so at least it didn't try to send all the data again. But even if rsync did consider these files to have different mtime/uid/gid, should it not have written that and be happy at next run? rsync versions: Source: $ rsync --version rsync version 3.1.2 protocol version 31 Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others. Web site: http://rsync.samba.org/ Capabilities: 64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints, socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace, append, ACLs, xattrs, iconv, symtimes, prealloc Destination: $ rsync --version rsync version 3.2.3 protocol version 31 Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others. Web site: https://rsync.samba.org/ Capabilities: 64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints, socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes, batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv, symtimes, prealloc, stop-at, no crtimes Optimizations: SIMD, asm, openssl-crypto Checksum list: xxh128 xxh3 xxh64 (xxhash) md5 md4 none Compress list: zstd lz4 zlibx zlib none What am I missing? Thanks, Andy -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html