Kenneth Hoste ha scritto:
[...]
However, as I posted yesterday, I've been able to circumvent the issue
by rethinking my data type, i.e. using
the ~18K movie IDs as key instead of the 480K user IDs, which radically
limits the overhead...
Well, but what if you really need the original data structure, for
better data processing?
That way, I'm able to fit the data set in <700M of memory, without
having to reorganize the raw data.
The uvector package implements a vector of unboxed types, and has an
snocU operation, to append an element to the array.
I don't know how efficient it is, however.
By the way, about uvector: it has a Stream data type, and you can
build a vector from a stream.
Thanks for letting me know, I'll keep this in mind.
Let me know if there are performance improvements.
Arrays are one of the few things I dislike in Haskell, and all the
available array/vector packages cause me some confusion.
Regards Manlio
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
