Dear haskellers, I'd like assistance figuring out a strange space leak using zlib package.
To make it easier to reproduce I've come up with the following snippet that pretty much resumes up the problem I'm trying to solve: > -- omitting imports and function signatures > encode = compress . L.fromChunks > > main = do { hSetBinaryMode stdout True > ; hSetBinaryMode stdin True > ; hSetBuffering stdin NoBuffering > ; hSetBuffering stdout NoBuffering > ; loop [] > } > where loop buff > | length buff == 64 = L.hPut stdout (encode buff) >> loop [] > | otherwise = do { eof <- hIsEOF stdin > ; when (not eof) (fmap (: buff) (B.hGetSome > stdin 512) >>= loop) > } N.B.: Removing the `compress' function from the above code also removes the space leak. Now, feeding the above program with ~8GB worth of binary data: > $ uname -a Linux mephisto.localhost.localdomain 3.8.7-1-ARCH #1 SMP PREEMPT Sat Apr 13 09:01:47 CEST 2013 x86_64 GNU/Linux > $ ghc --version The Glorious Glasgow Haskell Compilation System, version 7.6.3 > $ ghc -W -Wall -rtsopts --make -O2 test.hs > $ sudo dd if=/dev/sda bs=4K count=2048K | ./test +RTS -M1M -s >/dev/null ... 8589934592 bytes (8.6 GB) copied, 277.263 s, 31.0 MB/s ... 2 MB total memory in use (0 MB lost due to fragmentation) ... Which is fine. However, monitoring the RSS size is a different story: > $ while pidof test; do ps -o rss= -p $(pidof test); done | tail 16967 25620 16967 25628 16967 25628 16967 25628 16967 # ~16M 0 I know the RSS usually overestimates the memory consumption but the problem is that it is forever growing. The following I found very intriguing: * `+RTS -hc` gives me no hint about whats wrong [at least I couldn't see one]; * `+RTS -M1M` doesn't produce an error; * removing the `compress' functions makes the problem disappear; I couldn't figure these out and I don't think this is matter of strictness, though. Has anyone seen this before? Thanks in advance, ~dsouza _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe