See the 'prankb' and 'bslash' modifiers from my mailing list posting a few
months ago (or at
https://github.com/moon-chilled/j/blob/master/parallel/p.ijs). (I really
ought to polish these up into a library at some point.) The gist is, if you
have N cores, to slice up 'rows' into N chunks and
ps. remove "echo #" , it was nice when the loop runs just one pass.
The test file contained
id,x,xx
4,,t1
5,t2,t1
6,t1,
--
For information about J forums see http://www.jsoftware.com/forums.htm
I've got a sparse csv with shape 1183748 2141. Where there is data, it
has usually tally 2, and probably never longer than 7.
My usual methods run out of memory, something like
([: <;._2 ,&',');._2 CR-.~LF_separated_csv
I tried sparse array to store index numbers of data, with data as a
preal
