On Thursday, 21 January 2016 at 09:39:30 UTC, data pulverizer
wrote:
R takes about half as long to read the file. Both read the data
in the "equivalent" type format. Am I doing something incorrect
here?
CsvReader hasn't been compared and optimized from other CSV
readers. It does have allocation for the parsed string (even if
it isn't changed) and it does a number of validation checks.
You may get some improvement disabling the CSV validation, but
again this wasn't tested for performance.
csvReader!(string,Malformed.ignore)(str)
Generally people recommend using GDC/LCD if you need resulting
executable performance, but csvReader being slower isn't the most
surprising.
Before submitting my library to phobos I had started a CSV reader
that would do no allocations and instead return the string slice.
This wasn't completed and so it never had performance testing
done against it. It could very well be slower.
https://github.com/JesseKPhillips/JPDLibs/blob/csvoptimize/csv/csv.d
My original CSV parser was really slow because I parsed the
string twice.
