Hans, That is a great idea. However, can I be sure the file is completely written to disk before tar prints the filename? It seems to print the filename first. Could that not lead to a race condition, or the consumer reaching the "end" of the file before tar has finished writing it?
On Tue, Mar 29, 2011 at 11:20 AM, Hans Schou <[email protected]> wrote: > On Tue, 29 Mar 2011, Jay Hacker wrote: > >> I have a large gzipped tar archive containing many small files; just >> untarring it takes a lot of time and space. I'd like to be able to process >> each file in the archive, ideally without untarring the whole thing first, >> and I'd like to process several files in parallel. Is there a recipe for >> this with GNU Parallel? > > tar xvf big-file.tar.gz | parallel echo "Proc this file {}" > > Parallel will start when the first file is untared. > > /hans > -- > Horsebakken 78, DK-2400 København NV >
