Hi all, rsync typically detects file modifications by comparing exact modification time and size of each identically named file in the source and destination locations.
An rsync backup can be surreptitiously corrupted by modifying a source file's content while keeping its size and modification time the same. If some program is doing this then one way for rsync to detect the modification is by appending the --checksum option. This requires every file in the source and destination to be fully read. This can be many orders of magnitude slower than a "quick check". If you have been using rsync without --checksum to back up your Debian partitions then your backups are likely corrupted. Some packages are being generated with files containing exactly the same size and timestamps even though the contents of those files are different. This is how the data corruption arises: 1. You use rsync to back up your OS partition. 2. You perform package upgrades. Some files are replaced with different content but have the same size and modification time. 3. You use rsync to back up your OS partition to the same destination. The modified files with the same size and modification time are skipped. Your backup is now corrupt (you have old files mixed with new files). Here's a recent example of a package upgrade causing this: <https://packages.debian.org/testing/amd64/libqt5concurrent5/download> <https://packages.debian.org/sid/amd64/libqt5concurrent5/download> If you're tracking unstable you may have recently upgraded from libqt5concurrent5_5.7.1+dfsg-3_amd64.deb to libqt5concurrent5_5.7.1+dfsg-3+b1_amd64.deb (a subsequent binary non- maintainer upload). Here are the contents of both packages: $ dpkg -c libqt5concurrent5_5.7.1+dfsg-3_amd64.deb drwxr-xr-x root/root 0 2017-01-12 04:14 ./ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/ -rw-r--r-- root/root 27352 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/ -rw-r--r-- root/root 1196 2016-12-01 21:17 ./usr/share/doc/libqt5concurrent5/LGPL_EXCEPTION.txt -rw-r--r-- root/root 18232 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/changelog.Debian.gz -rw-r--r-- root/root 3792 2016-12-01 21:17 ./usr/share/doc/libqt5concurrent5/changelog.gz -rw-r--r-- root/root 103466 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/copyright drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/overrides/ -rw-r--r-- root/root 230 2017-01-12 04:14 ./usr/share/lintian/overrides/libqt5concurrent5 lrwxrwxrwx root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5 -> libQt5Concurrent.so.5.7.1 lrwxrwxrwx root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7 -> libQt5Concurrent.so.5.7.1 $ dpkg -c libqt5concurrent5_5.7.1+dfsg-3+b1_amd64.deb drwxr-xr-x root/root 0 2017-01-12 04:14 ./ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/ -rw-r--r-- root/root 27352 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/ -rw-r--r-- root/root 1196 2016-12-01 21:17 ./usr/share/doc/libqt5concurrent5/LGPL_EXCEPTION.txt -rw-r--r-- root/root 252 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/changelog.Debian.amd64.gz -rw-r--r-- root/root 18232 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/changelog.Debian.gz -rw-r--r-- root/root 3792 2016-12-01 21:17 ./usr/share/doc/libqt5concurrent5/changelog.gz -rw-r--r-- root/root 103466 2017-01-12 04:14 ./usr/share/doc/libqt5concurrent5/copyright drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/ drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/overrides/ -rw-r--r-- root/root 230 2017-01-12 04:14 ./usr/share/lintian/overrides/libqt5concurrent5 lrwxrwxrwx root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5 -> libQt5Concurrent.so.5.7.1 lrwxrwxrwx root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7 -> libQt5Concurrent.so.5.7.1 In particular notice the shared library /usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 has an identical size and timestamp. However: (testing) $ sha256sum ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 aef7b3d108ba5c686279fa2dbd63bb7916b1bb79ec69faeab55ea8f3b85725df ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 (sid) $ sha256sum ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 052a748ebc2ab3e65e8b25cab3515a73762450b7c84308dad157c1f677e21b0a ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 Why is a 27 January recompilation of the source package purporting to have the same modification time as the original binary package distributed 16 days earlier? It is now also clear I have previously encountered this with some X shared libraries. I suggest an rsync --checksum test on your backups (using --itemize-changes and --dry-run to check for modifications without making changes to the destination). Regards, Adam Warner