On Sunday, 26 September 2021 13:25:24 BST Ramon Fischer wrote: > Addendum: > > To complete the list. Here the parallel implementation of "lzip": > > "plzip": https://www.nongnu.org/lzip/plzip.html > > -Ramon > > On 26/09/2021 14:23, Ramon Fischer wrote: > > In addition to this, you may want to use the parallel implementations > > of "gzip", "xz", "bzip2" or the new "zstd" (zstandard), which are > > "pigz"[1], "pixz"[2], "pbzip2"[3], or "zstmt" (within package > > "app-arch/zstd")[4] in order to increase performance: > > > > $ cd <path_to_mounted_backup_partition> > > $ for tar_archive in *.tar; do pixz "${tar_archive}"; done > > > > -Ramon > > > > [1] > > * https://www.zlib.net/pigz/ > > > > [2] > > * https://github.com/vasi/pixz > > > > [3] > > * https://launchpad.net/pbzip2 > > * http://compression.ca/pbzip2/ > > > > [4] > > * https://facebook.github.io/zstd/ > > > > On 26/09/2021 13:36, Simon Thelen wrote: > >> [2021-09-26 11:57] Peter Humphrey <pe...@prh.myzen.co.uk> > >> > >>> part text/plain 382 > >>> Hello list, > >> > >> Hi, > >> > >>> I have an external USB-3 drive with various system backups. There > >>> are 350 .tar > >>> files (not .tar.gz etc.), amounting to 2.5TB. I was sure I wouldn't > >>> need to > >>> compress them, so I didn't, but now I think I'm going to have to. Is > >>> there a > >>> reasonably efficient way to do this? I have 500GB spare space on > >>> /dev/sda, and > >>> the machine runs constantly. > >> > >> Pick your favorite of gzip, bzip2, xz or lzip (I recommend lzip) and > >> then: > >> mount USB-3 /mnt; cd /mnt; lzip * > >> > >> The archiver you chose will compress the file and add the appropriate > >> extension all on its own and tar will use that (and the file magic) to > >> find the appropriate decompresser when you want to extract files later > >> (you can use `tar tf' to test if you want).
Thank you both. Now, as it's a single USB-3 drive, what advantage would a parallel implementation confer? I assume I'd be better compressing from external to SATA, then writing back, or is that wrong? Or, I could connect a second USB-3 drive to a different interface, then read from one and write to the other, with or without the SATA between. -- Regards, Peter.
