Ole, On Wed, Dec 14, 2016 at 2:36 PM, Ole Tange <[email protected]> wrote: > But if you can somehow replace the record separator, then you can use > --recend. > > Given your input this might work: > > parallel --pipe --recend '"\n" > > assuming a good part of the records have a last column with newlines.
Thank you for your help. I can't assume the last column will always have newlines, but your suggestion with --recend gave me an idea to do something like: cat file.csv | php reformat.php | parallel --pipe --recend '@@@' --remove-rec-sep wc reformat.php is a PHP script that reads the csv and writes it out to stdout with @@@ in-between each record. That seems to work great except I don't believe I can use --pipepart with this method because the csv with @@@ is generated on the fly. I would have to save the reformatted csv file to disk. Do you have any thoughts to get around that? If not, generating a new csv file in this format would also be ok for my use case. Thanks again. Ryan
