Hi Parallel gurus,
I am running into an disk space issue when I run a parallel command (GNU
parallel 20140322).
The pseudo code is as defined below:
---
echo 300k lines <pipe> parallel "script.bash {}"
--—
The Bash script perform a dig command, some pure Bash instructions and write a
single line of 50 to 100 characters to stdout.
Nothing crash except the /tmp partition is reported to use several GB of data
while the script is running.
I first tried --tmpdir to a larger partition, but after a couple of hours, 15
GB were used.
I interrupted the execution and I assume Parallel trapped properly the signal
to cleanup the temporary directory. I got back the 15 GB.
Note: I was unable to see any temporary file in the tmpdir directory.
Any idea what could cause such a big temporary buffer output usage?
Thanks,
Antoine
