Phako wrote:
$ time lziprecover -v -Fc2% archive.tar.lz
archive.tar.lz.fec: 300_754_928 bytes, 300_482_560 fec bytes, 655 blocks
real 407m19,994s
user 171m18,431s
sys 27m58,918s
I tried to find what cause all this wait, htop show lziprecover
continuously reading at full speed, with the disk I/O reaching 95-100%
during all that time and vmstat report a very high disk reading usage
but very low swap activity.
Thanks for this new interesting data. It suggests a workaround until I can
implement the mmapping of the protected file by parts.
Reed-Solomon is basically matrix multiplication; the whole protected file
must be multiplied once per FEC block. With four threads, four FEC blocks
are created in parallel. In your case, mmap may need to read the protected
file (archive.tar.lz) 166 times.
Therefore, if you run lziprecover with as many threads as FEC blocks, the
protected file will be read just 3 times (one for each chksum packet plus
one for all the FEC packets). I have just run the command below on a machine
with 2 processors and 8 GiB of RAM (no swap), and it took a little more than
2 hours. (You may specify a large number of threads. They will be adjusted
automatically to the number of FEC blocks).
$ time lziprecover -v -Fc2% -n 2048 archive.tar.lz
archive.tar.lz.fec: 320_594_048 bytes, 320_339_968 fec bytes, 611 blocks
real 132m45.193s
user 194m39.860s
sys 3m44.510s
Hope this helps,
Antonio.
