Hi Antonio/wrotycz Re: memory errors, I've installed 'memtester', so I'll give that a try to see if it produces anything interesting. Given that generally speaking I've never had any issues on my PC and have used it for years, any error rate if it occurs must be minor/benign, but it does highlight the need for caution particularly when backing up precious data into compressed archives, for example. Will be interesting, anyway.
On 24/01/2025 00:13, Antonio Diaz Diaz wrote: > Memory errors happen even in the most expensive hardware. They almost ruined > the mission of the Mars Science Laboratory[1]. I guess that in a machine > with 128 GiB of RAM without ECC, a bit flip may happen almost daily[2]. > Maybe you should test your RAM to get an idea of how reliable it is. (And it > still could be a bug in tarlz). I dug into my actual hardware using the tool 'inxi' (command "inxi -m -xxx") and the output showed that my Corsair memory is all "synchronous unbuffered (unregistered)" UDIMMs, so I presume that means no error correction capability. In any case, this "false positive" (i.e. no bug, seems that way) has been enormously useful/beneficial: I've now modified my script to make it much more robust as a result, whereas had nothing wrong ever happened, I'd have likely blithely (and incorrectly) just relied on occasional listing and spot-checking and assumed everything was perfect. So it now does the following steps: 1. Compresses the tarlz archive with maximum compression. 2. Immediately runs tarlx -df <archive> to diff it against all the original files. 3. runs lzip -tv <archive> to further verify the integrity of the archive. If step (2) runs with no resultant output (i.e. no diff's found), and step (3) results in verifying as 'ok', I take that to be a pretty strong indication that the data has been correctly/safely archived. --- Thanks for the help/insight. One suggestion I have is you might wish to append/add a small section into the tarlz documentation as a quick checklist for "verifying your archive integrity", or "recommended best practices for very important data", perhaps even with an short example sh/bash script showing how the tarlz -df and lzip -tv can be used to verify an archive. Since the primary use-case for this tool is to generate extremely robust archives that have both error checking and a level of recoverability (plus extremely fast decompression, etc.), a few pointers/tips/examples in the documentation here might nudge potential users in the right direction and also highlight the real unique proposition this tool presents versus more conventional/established archive/compression candidates that are currently used. All of this might help to increase adoption of this excellent format. Anyway, I'll be using tarlz for all of my long term archive needs going forward. Best regards Aren.
