As a user I have run into this with X11 files myself. I use rsnapshot to backup the root partition to a location on /home mounted on its own much larger partition before I do upgrades. A while back when xorg was crashing a lot I had to restore from this backup. I routinely use "debsums -ca" after an upgrade to check OS files against their hashes and I discovered this strange occurrence with files having the same date/time and size but different hashes. Luckily I have a server running apt-cacher-ng so I had the older deb files to re-install to fix the problem.
So as a user this a BIG pain in the rear. I also tried the rsync checksum option but as you noted it is much more time consuming. Are these files actually equivalent but with their code segments scrambled for security? I was actually running xorg with the "bad" files for a while before I found them with debsums. From a user point of view this is totally inexcusable. I hope that a solution can be found to fix this ... even if it is just incrementing the second by one. *...Bob* On 01/28/2017 08:11 AM, Adam Warner wrote: > Hi all, > > rsync typically detects file modifications by comparing exact > modification time and size of each identically named file in the source > and destination locations. > > An rsync backup can be surreptitiously corrupted by modifying a source > file's content while keeping its size and modification time the same. > > If some program is doing this then one way for rsync to detect the > modification is by appending the --checksum option. This requires every > file in the source and destination to be fully read. This can be many > orders of magnitude slower than a "quick check". > > If you have been using rsync without --checksum to back up your Debian > partitions then your backups are likely corrupted. > > Some packages are being generated with files containing exactly the > same size and timestamps even though the contents of those files are > different. This is how the data corruption arises: > > 1. You use rsync to back up your OS partition. > 2. You perform package upgrades. Some files are replaced with different > content but have the same size and modification time. > 3. You use rsync to back up your OS partition to the same destination. > The modified files with the same size and modification time are > skipped. Your backup is now corrupt (you have old files mixed with new > files). > > Here's a recent example of a package upgrade causing this: > > <https://packages.debian.org/testing/amd64/libqt5concurrent5/download> > <https://packages.debian.org/sid/amd64/libqt5concurrent5/download> > > If you're tracking unstable you may have recently upgraded from > libqt5concurrent5_5.7.1+dfsg-3_amd64.deb to > libqt5concurrent5_5.7.1+dfsg-3+b1_amd64.deb (a subsequent binary non- > maintainer upload). > > Here are the contents of both packages: > $ dpkg -c libqt5concurrent5_5.7.1+dfsg-3_amd64.deb > drwxr-xr-x root/root 0 2017-01-12 04:14 ./ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/ > -rw-r--r-- root/root 27352 2017-01-12 04:14 > ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/ > drwxr-xr-x root/root 0 2017-01-12 04:14 > ./usr/share/doc/libqt5concurrent5/ > -rw-r--r-- root/root 1196 2016-12-01 21:17 > ./usr/share/doc/libqt5concurrent5/LGPL_EXCEPTION.txt > -rw-r--r-- root/root 18232 2017-01-12 04:14 > ./usr/share/doc/libqt5concurrent5/changelog.Debian.gz > -rw-r--r-- root/root 3792 2016-12-01 21:17 > ./usr/share/doc/libqt5concurrent5/changelog.gz > -rw-r--r-- root/root 103466 2017-01-12 04:14 > ./usr/share/doc/libqt5concurrent5/copyright > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/overrides/ > -rw-r--r-- root/root 230 2017-01-12 04:14 > ./usr/share/lintian/overrides/libqt5concurrent5 > lrwxrwxrwx root/root 0 2017-01-12 04:14 > ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5 -> libQt5Concurrent.so.5.7.1 > lrwxrwxrwx root/root 0 2017-01-12 04:14 > ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7 -> > libQt5Concurrent.so.5.7.1 > > $ dpkg -c libqt5concurrent5_5.7.1+dfsg-3+b1_amd64.deb > drwxr-xr-x root/root 0 2017-01-12 04:14 ./ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/lib/x86_64-linux-gnu/ > -rw-r--r-- root/root 27352 2017-01-12 04:14 > ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/doc/ > drwxr-xr-x root/root 0 2017-01-12 04:14 > ./usr/share/doc/libqt5concurrent5/ > -rw-r--r-- root/root 1196 2016-12-01 21:17 > ./usr/share/doc/libqt5concurrent5/LGPL_EXCEPTION.txt > -rw-r--r-- root/root 252 2017-01-12 04:14 > ./usr/share/doc/libqt5concurrent5/changelog.Debian.amd64.gz > -rw-r--r-- root/root 18232 2017-01-12 04:14 > ./usr/share/doc/libqt5concurrent5/changelog.Debian.gz > -rw-r--r-- root/root 3792 2016-12-01 21:17 > ./usr/share/doc/libqt5concurrent5/changelog.gz > -rw-r--r-- root/root 103466 2017-01-12 04:14 > ./usr/share/doc/libqt5concurrent5/copyright > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/ > drwxr-xr-x root/root 0 2017-01-12 04:14 ./usr/share/lintian/overrides/ > -rw-r--r-- root/root 230 2017-01-12 04:14 > ./usr/share/lintian/overrides/libqt5concurrent5 > lrwxrwxrwx root/root 0 2017-01-12 04:14 > ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5 -> libQt5Concurrent.so.5.7.1 > lrwxrwxrwx root/root 0 2017-01-12 04:14 > ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7 -> > libQt5Concurrent.so.5.7.1 > > In particular notice the shared library > /usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 has an identical > size and timestamp. > > However: > (testing) $ sha256sum ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > aef7b3d108ba5c686279fa2dbd63bb7916b1bb79ec69faeab55ea8f3b85725df > ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > (sid) $ sha256sum ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > 052a748ebc2ab3e65e8b25cab3515a73762450b7c84308dad157c1f677e21b0a > ./usr/lib/x86_64-linux-gnu/libQt5Concurrent.so.5.7.1 > > Why is a 27 January recompilation of the source package purporting to > have the same modification time as the original binary package > distributed 16 days earlier? > > It is now also clear I have previously encountered this with some X > shared libraries. > > I suggest an rsync --checksum test on your backups (using > --itemize-changes and --dry-run to check for modifications without > making changes to the destination). > > Regards, > Adam Warner > >
