I am trying to play with iteratee making parser for squid log files, but found
that my code do not run in constant space when it tries to process compressed
log files. So i simplified my code down to this snippet:
import Data.ByteString (ByteString)
import Data.Iteratee as I
import Data.Iteratee.Char
import Data.Iteratee.ZLib
import System
main = do
args <- getArgs
let fname = args !! 0
let blockSize = read $ args !! 1
fileDriver (leak blockSize) fname >>= print
leak :: Int -> Iteratee ByteString IO ()
leak blockSize = joinIM $ enumInflate GZip defaultDecompressParams chunkedRead
where
consChunk :: Iteratee ByteString IO String
consChunk = (joinI $ I.take blockSize I.length) >>= return . show
chunkedRead :: Iteratee ByteString IO ()
chunkedRead = joinI $ convStream consChunk printLines
First argument - file name (/var/log/messages.1.gz will do)
second - size of block to consume input. with low size (10 bytes) of consumed
blocks it leaks very fast, with larger blocks (~10000) it works almost without
leaks.
So. Is it bugs within my code, or iteratee-compress should behave differently?
_______________________________________________
Haskell-Cafe mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/haskell-cafe
