On Sun, Mar 20, 2011 at 12:28 PM, Sebastian Eiser <[email protected]> wrote:
> I'm new to using "parallel", but so far I made very good experience > with it on our 16 core machine. Thank you very much for this useful > tool! You can repay by: * Post the intro video on forums/blogs/Twitter/Facebook/Linkedin * Request or write a review for your favourite magazine * Request or build a package for your favourite distribution (if it is not there already) * Invite me for your next conference (Contact http://ole.tange.dk) * (Have your company) donate to FSF https://my.fsf.org/donate/ > Unfortunately I don't know more details about the error, but from the > output I found a few hints > 1. some file names from "{}" seem to get f*cked up: : > Using --dry-run this does not occur, all {}s are substituted > correctly. I initially thought the file names are too long, but using > shorter file names didn't help. > 2. then, I sometimes get this warning: > Use of uninitialized value in hash element at > /home/seiser/alt/usr/bin/parallel line 3284, <STDIN> line 2542 > Use of uninitialized value in hash element at > /home/seiser/alt/usr/bin/parallel line 3287, <STDIN> line 2542 > 3. and finally, it tells me this (without exiting): > ETA: 1670s 3854left 0.43avg local:16/1131/100%/0.4s This should not > happen. Contact <[email protected]>. at parallel line 3384. > > It seems that the problem doesn't occur if I reduce the number of > inputs, e.g. not using 5000 files, but just 1000 seems to be fine. > Then, just hint 1. and 2. occur. * Do you need to run for all the years or will it fail if you just run for one year? * Will it fail if you remove --eta (--eta is alpha quality) * Will it fail if you prepend 'echo' to /home/seiser/src/gtproduct-omi-old? * Can you run a simpler command and still make it fail? * Does it fail the same place or does it fail in different places every time? * Does it fail if you use -j1? * Are your files local or on a network file system? * Can you make a set of dummy files that fails too? Maybe a copy of the original files but with no content: cp -rs /the/dir my_dir; find my_dir -type l | parallel rm {} '&&' touch {} /Ole
