All,
I have a bunch (~200) small (1K to 100K) binary files that I want to
'cat' into a larger file. I usually use "cat pe* > diag", but this
takes considerable time on the Lustre file system we are using. I am
exploring using GNU parallel for this task but have run into some
difficulties. Basically the resulting diag file only contains one of
the input files.
I've tried the following variations.
parallel "cat {} >diag_amsua_n18_03.2011041700" ::: pe*
parallel cat {} ">"diag_amsua_n18_03.2011041700 ::: pe*
ls pe* | parallel cat {} ">"diag_amsua_n18_03.2011041700
ls pe* | parallel -j4 -k cat {} ">"diag_amsua_n18_03.2011041700
ls pe* | parallel -k cat {} ">"diag_amsua_n18_03.2011041700
parallel -j4 -k "cat {} >diag_amsua_n18_03.2011041700" ::: pe*
Should this work? Any ideas what I'm doing wrong?
Thanks
parallel --version
GNU parallel 20110622
on
SLESS11
perl --version
This is perl, v5.10.0 built for x86_64-linux-thread-multi
--
Dan Kokron
Global Modeling and Assimilation Office
NASA Goddard Space Flight Center
Greenbelt, MD 20771
[email protected]
Phone: (301) 614-5192
Fax: (301) 614-5304
