Thank you both, JLD was indeed what I needed, it's got a lot quicker since
I last used it I think.
Serialize is much faster now too which is great.
Serializing strings was indeed unreasonably slow. I just pushed a
commit that should be a significant improvement.
On Wed, May 21, 2014 at 7:17 AM, Tim Holy wrote:
> Samuel, rewriting in C is almost never necessary, because Julia is as fast as
> C. It's just that some implementations are faster,
Samuel, rewriting in C is almost never necessary, because Julia is as fast as
C. It's just that some implementations are faster, and some slower.
I don't want to dissuade you in any way from improving the performance of
serialize, but you might consider looking at HDF5/JLD.
--Tim
On Tuesday, M
Thanks for that, yes you're write I was being dumb.
Just to give an example of how slow, reading a 48mb csv files with a
mixture of strings and numbers with DataFrames's readtable, then writing it
gives:
*julia> **@time writetable("data.csv", r)*
elapsed time: 3.539380236 seconds (77981180
Regarding this comment:
> Even just serializing a single string profile shows masses of calls in
inference.jl. Surely this shouldn't be necessary?
You'll need to run it twice to get useful results; the first time, the
code must be compiled and therefore many calls are indeed made to inference.
Serlialize is very slow for string.
Is there any way the to speed it up at present, or an alternative way to
save data structures which is significantly faster?
Are there any plans to speed up string serialization in Julia? Is there an
easy low hanging fruit which needs work and could speed thi
