Hello Antonio Thanks for the informative reply.
On 23/01/2025 01:54, Antonio Diaz Diaz wrote: > No, it is not normal. I always use level -9 and never had a problem. If > level -9 has a problem, I'm very interested in fixing it. > > Level -9 may run out of memory on some machines, but if it happens it is > clearly diagnosed. > > Level -9 uses more memory than level -6, and is therefore more frequently > affected by RAM errors. I recommend to repeat the exact command where you > used level -9 and compare the resulting archive with the faulty one: > - If they are identical, it is a bug in tarlz. If this is the case, please, > send the output of 'lzip -tvvvv'. > - If they are different and the second is correct, it may be a bug (data > race), or a memory problem. > - If they are different but both are faulty, it may be also a bug (data > race), or a memory problem. In this case try creating the archive with one > thread (--threads=1) to discard a data race. So it looks like your suspicion was correct, in that it [must surely be/is] a memory/RAM error (it is interesting to know that this is still a pertinent hardware issue on modern machines). The machine I am running this on has 128GB RAM, so quantity/resources is never an issue. Anyway, I tried creating a fresh archive with -9 again today, and could not reproduce the issue. This would add further weight to it being a RAM issue since the machine was powered off and obviously the live state entirely different from 24 hours previous. I ran lzip -tvvvv on the resultant achive as you suggested, and got this output: files.tar.lz: dict 32 MiB, 1.331:1, 75.16% ratio, 24.84% saved. CRC 436ED216, 66670080 out, 50107520 in. ok files.tar.lz: dict 32 MiB, 1.008:1, 99.24% ratio, 0.76% saved. CRC 4C463A53, 40073728 out, 39767927 in. ok files.tar.lz: dict 32 MiB, 1.132:1, 88.37% ratio, 11.63% saved. CRC C640AE66, 319841792 out, 282650901 in. ok files.tar.lz: dict 32 MiB, 1.279:1, 78.18% ratio, 21.82% saved. CRC DAC64363, 67428352 out, 52712237 in. ok files.tar.lz: dict 32 MiB, 1.100:1, 90.95% ratio, 9.05% saved. CRC 177B3C4F, 89001472 out, 80946035 in. ok files.tar.lz: dict 32 MiB, 1.473:1, 67.88% ratio, 32.12% saved. CRC 37D5EEB6, 67553280 out, 45854099 in. ok files.tar.lz: dict 32 MiB, 1.755:1, 56.99% ratio, 43.01% saved. CRC 937F7099, 65571840 out, 37372377 in. ok files.tar.lz: dict 32 MiB, 0.998:1, 100.20% ratio, -0.20% saved. CRC D8D5FFD8, 67117056 out, 67252004 in. ok files.tar.lz: dict 32 MiB, 44.039:1, 2.27% ratio, 97.73% saved. CRC 7E5629BD, 67116032 out, 1524000 in. ok files.tar.lz: dict 32 MiB, 1.726:1, 57.92% ratio, 42.08% saved. CRC 94F7AA52, 69840896 out, 40453016 in. ok files.tar.lz: dict 32 MiB, 1.254:1, 79.73% ratio, 20.27% saved. CRC 7CD97C5E, 66978304 out, 53402326 in. ok files.tar.lz: dict 32 MiB, 1.424:1, 70.21% ratio, 29.79% saved. CRC 261712D3, 67108864 out, 47114950 in. ok files.tar.lz: dict 20 MiB, 1.571:1, 63.66% ratio, 36.34% saved. CRC 0256CCC7, 19752960 out, 12575533 in. ok files.tar.lz: dict 32 MiB, 1.055:1, 94.78% ratio, 5.22% saved. CRC E6D8309F, 562729984 out, 533335546 in. ok files.tar.lz: dict 32 MiB, 1.236:1, 80.91% ratio, 19.09% saved. CRC 1C66D6C4, 62746112 out, 50765045 in. ok files.tar.lz: dict 32 MiB, 1.825:1, 54.79% ratio, 45.21% saved. CRC BB6BC655, 67108352 out, 36765408 in. ok files.tar.lz: dict 32 MiB, 1.256:1, 79.61% ratio, 20.39% saved. CRC 700E4C9C, 84463104 out, 67237671 in. ok files.tar.lz: dict 1408 KiB, 4.658:1, 21.47% ratio, 78.53% saved. CRC 295576E3, 1430016 out, 307011 in. ok files.tar.lz: dict 4 KiB, 23.273:1, 4.30% ratio, 95.70% saved. CRC EFB5AF2E, 1024 out, 44 in. ok So no issues. So I can only assume it is a hardware issue rather than bug with tarlz. One minor curiosity, though I assume it is probably just to do with the threading model (?): creating the archive with the -v flag so it shows the files being added, stopped producing any console output after a while, until the process successfully completed. This doesn't really matter, but it is nice to be able to watch the files being added. I can of course watch the progress by monitoring the growing size of the new archive on the filesystem. > But remember that it does not matter how safe is the format if the archive > is created corrupt because of a bug in the tool or a faulty RAM. Check > everything twice at creation time. Thanks for the advice - from now on I'm going to always run the lzip '-tvvvv' command on the resultant created archive. I'll probably also just leave it at default compression, since the saving was minimal. I presume I am safe in assuming that if that command completes all ok, and every checksum is correct, then the archive integrity CAN be trusted :-) Best regards Aren.
