I need to run something similar due to a large number of text files that I have. They are too large to load into memory at one-time, let alone multiple files at the same time. I find that pmap() works very well here.
First, you should wrap your for loop in a function. In general you should block your code with functions in Julia. Second, can you provide a determiner to the split function? Third, you may not need 32 procs for this job. There's overhead associated with parallel processing. This stackoverflow post has some more information that might be useful: http://stackoverflow.com/questions/21890893/reading-csv-in-julia-is-slow-compared-to-python/35120894?noredirect=1#comment66827279_35120894
