Re: [Haskell-cafe] binary package: memory problem decoding an IntMap

Manlio Perillo ha scritto: Hi. I'm having memory problems decoding a big IntMap. The data structure is: IntMap (UArr (Word16 :*: Word8)) There are 480189 keys, and a total of 100480507 elements (Netflix Prize). The size of the encoded (and compressed) data is 184 MB. When I load data from

Re: [Haskell-cafe] binary package: memory problem decoding an IntMap

Excerpts from Manlio Perillo's message of Sun Apr 05 22:41:57 +0200 2009: Manlio Perillo ha scritto: Hi. I'm having memory problems decoding a big IntMap. The data structure is: IntMap (UArr (Word16 :*: Word8)) There are 480189 keys, and a total of 100480507 elements

Re: [Haskell-cafe] binary package: memory problem decoding an IntMap

Manlio Perillo ha scritto: [...] It seems there is a problem with tuples, too. I have a: [(Word16, UArr (Word32 :*:* Word8))] This eats more memory than it should, since tuples are decoded lazily. My bad, sorry. I simply solved by using a strict consumer (foldl' instead of foldl).

[Haskell-cafe] binary package: memory problem decoding an IntMap

Hi. I'm having memory problems decoding a big IntMap. The data structure is: IntMap (UArr (Word16 :*: Word8)) There are 480189 keys, and a total of 100480507 elements (Netflix Prize). The size of the encoded (and compressed) data is 184 MB. When I load data from the Netflix Prize data set,