Re: [Haskell-cafe] Debugging ByteString and Data.Binary.Get memory usage

2013-08-29 Thread Bob Ippolito
foldl' is the right way to simulate the sequential IO action, foldr would be doing it in reverse (and for large enough input will stack overflow). On Thu, Aug 29, 2013 at 9:33 PM, Kyle Hanson wrote: > Thanks Bob, > > I made it foldr because it was meant to simulate the sequential IO action > th

Re: [Haskell-cafe] Debugging ByteString and Data.Binary.Get memory usage

2013-08-29 Thread Kyle Hanson
Thanks Bob, I made it foldr because it was meant to simulate the sequential IO action that my server uses to populate the Map. I found the problem to be that I need to force the map to evaluate so adding a little $! fixed the problem -- Kyle Hanson On Thu, Aug 29, 2013 at 9:09 PM, Bob Ippoli

Re: [Haskell-cafe] Debugging ByteString and Data.Binary.Get memory usage

2013-08-29 Thread Bob Ippolito
Building a map with foldr seems unwise, have you tried doing it with fromListWith instead? Or foldl'? In either case, since you don't even put the map into WHNF, none of the computation is done at all in either case until the first lookup. On Thu, Aug 29, 2013 at 3:35 PM, Kyle Hanson wrote: > O

Re: [Haskell-cafe] Debugging ByteString and Data.Binary.Get memory usage

2013-08-29 Thread Johan Tibell
A good starting point is to estimate how much space you think the data should take using e.g. http://blog.johantibell.com/2011/06/memory-footprints-of-some-common-data.html If you do that, is the actual space usage close to what you expected? On Thu, Aug 29, 2013 at 5:35 PM, Kyle Hanson wrote

[Haskell-cafe] Debugging ByteString and Data.Binary.Get memory usage

2013-08-29 Thread Kyle Hanson
OK I have a bunch of BSON documents that I convert to ByteStrings, put in a Map, and write to a socket based on the response. I noticed some high memory usage (in the GBs) so I decided to investigate. I simplified my problem into a small program that demonstrates clearer what is happening. I wrot