The purpose of this note is to report more information about this bug.
(1) I suspect this is the same bug that I reported earlier in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=781894.
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=781894>
That earlier report provides a link to some sample data which
demonstrates that the same discrepancy *always* appears in the resulting
squashed filesystem, regardless of the version of squashfs-tools used,
the compressor specified, the number of processors specified (or any
other command-line options that I have tried), and even the number of
virtual or real processor cores present on the machine.
(2) I have confirmed that the bug still exists in the alpha 4.3-2
version of squashfs-tools, which I downloaded from
https://packages.debian.org/sid/squashfs-tools
(3) Since I still need some sort of work-around for the problem, I have
tried, unsuccessfully, to confirm the result here described by
Andreas Krüger <andreas.krue...@famsik.de>
<https://bugs.debian.org/cgi-bin/pkgreport.cgi?submitter=andreas.krueger%40famsik.de>
I understood (perhaps wrongly) that he reports or at least suspects that
the discrepancy problem does not occur in the context of a
single-processor machine or a single-processor VM. However, using such
processing contexts, my own results are that the same discrepancy always
occurs regardless. In one such trial, I used BIOS options to disable
all but one core and hyperthreading within that core. In another trial,
I used a VirtualBox VM with access to only one core. The result is
always discrepant in the same manner, regardless.
Therefore, contrary to Andreas's conjecture, I'm seeing no evidence that
the bug is due to a race condition between processes. The discrepancy
always looks like this:
original recovered from squashfs
contents differ: byte # 6614941696 0x73 s 0x00
contents differ: byte # 6614941697 0x74 t 0x00
contents differ: byte # 6614941700 0x73 s 0x00
contents differ: byte # 6614941701 0x65 e 0x00
contents differ: byte # 6614941702 0x61 a 0x00
etc ... the rest of the squashed file continues to be bytes of value
0x00.
Steve Newcomb
On Tue, 09 Jun 2015 09:58:05 +0200 =?UTF-8?B?QW5kcmVhcyBLcsO8Z2Vy?= wrote:
> Package: squashfs-tools
> Version: 1:4.2+20130409-2
> Severity: normal
>
> Hello,
>
> I'm using squashfs for full backups of my laptop, from a (quiet)
> LVM snapshot. I run sha256sum checksums on each file, then run
>
> mksquashfs /freeze /backup/$timestamp.backup.squashfs \
> -always-use-fragments -processors 2
>
> then mount the resulting squashfs file system and compare the checksums.
>
> Over the years, I've seen checksum errors occasionally.
>
> I have recently started to use virtual machines, so I now have more
> very large files in my backup. Since then, the number of problems
> seemed to have increased substantially
>
> The current backup saw two of these. Interestingly, in both
> cases, the problem is very close to the end of the file. For the
> record: In either case, the file length, according to ls -l, is
> the same for the original and the copy in the squashfs.
>
> One file is 2776104960 bytes long, a plain "cmp" run finds the
> first inconsistency at byte 2776018945, a mere 86015 bytes from
> the end.
>
> The other file is 7831814144 bytes long, a plain "cmp" finds the
> first inconsistency at byte 7831683073, a mere 131071 bytes from
> the end.
>
> Regards, and thank you for providing fine software
>
> Andreas
>
>
> -- System Information:
> Debian Release: 8.1
> APT prefers stable-updates
> APT policy: (500, 'stable-updates'), (500, 'stable')
> Architecture: amd64 (x86_64)
> Foreign Architectures: i386
>
> Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
> Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/dash
> Init: systemd (via /run/systemd/system)
>
> Versions of packages squashfs-tools depends on:
> ii libc6 2.19-18
> ii liblzma5 5.1.1alpha+20120614-2+b3
> ii liblzo2-2 2.08-1.2
> ii zlib1g 1:1.2.8.dfsg-2+b1
>
> squashfs-tools recommends no packages.
>
> squashfs-tools suggests no packages.
>
