I'm trying to process a big messy csv file with the --pipe option but I keep
getting errors such as unexpected EOF while looking for matching `"' or awk:
cmd. line:1: ^ unexpected newline or end of string. I suspect I'm not using
the quoting correctly. 2 of the commands I've tried:
$ cat big.csv | parallel --pipe --files 'awk -v FS="\",\"" '{print $1, $3, $4,
$5, $9, $14}' | grep -v "#" | sed -e '1d' -e 's/\"//g' -e 's/\/\/\//\t/g' | cut
-f1-6,11 | sed -e 's/\/\//\t/g' -e 's/ /\t/g'' | parallel -Xj1 sort -k1 {} ';'
rm {} > big_modified_parallel.csv
$ cat big.csv | parallel --pipe --files 'awk -v FS="\",\"" '{OFS="\\t\"\; print
$1, $3, $4, $5, $9, $14}' | grep -v "#" | sed -e '1d' -e 's/\"//g' -e
's/\/\/\//\t/g' | cut -f1-6,11 | sed -e 's/\/\//\t/g' -e 's/ /\t/g'' | parallel
-Xj1 sort -k1 {} ';' rm {} > big_modified_parallel.csv
Can someone point out where I might have made a mistake?
Thank you.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
