[Haskell-cafe] I would like a clarification about Enumerators and Iteratees, please :)

[Alfredo Di Napoli]

Hi guys,

I've started playing with Iteratee and Enumerators: very cool and addictive
stuff.

I have wrote this simple code:

https://gist.github.com/3899017

In a nutshell, it gives back the number of occurences for a single char in
case the argument passed from the command line is a single char,
or the number of lines of the entire file.

I've tried it first on a small file (~100MB) and then on a huge one (~3GB).
As far as I understood Iteratee IO should be a smart way to do IO,
avoiding to keep the entire file in memory; what I observe in the second
case, instead, is a sort of memory leak. The memory grows and grows
until the entire machine hangs.

If I split the huge file in small chunks, memory comsumption is still
high but the computation terminates and is fast.

So my two questions:

a) What am I missing? Should be memory used be constant? And how can I
achieve this purpose?
b) I'm using the package enumerator because as far as I can see is the most
used. How does it compares with iteratee? Which of the two are more
performant?


Bye and thanks,
A.
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] I would like a clarification about Enumerators and Iteratees, please :)

[Gregory Collins]

You have a space leak in countCharBS. Put a bang pattern on your
accumulator.

On Tue, Oct 16, 2012 at 2:46 PM, Alfredo Di Napoli 
alfredo.dinap...@gmail.com wrote:

 Hi guys,

 I've started playing with Iteratee and Enumerators: very cool and
 addictive stuff.

 I have wrote this simple code:

 https://gist.github.com/3899017

 In a nutshell, it gives back the number of occurences for a single char in
 case the argument passed from the command line is a single char,
 or the number of lines of the entire file.

 I've tried it first on a small file (~100MB) and then on a huge one
 (~3GB). As far as I understood Iteratee IO should be a smart way to do IO,
 avoiding to keep the entire file in memory; what I observe in the second
 case, instead, is a sort of memory leak. The memory grows and grows
 until the entire machine hangs.

 If I split the huge file in small chunks, memory comsumption is still
 high but the computation terminates and is fast.

 So my two questions:

 a) What am I missing? Should be memory used be constant? And how can I
 achieve this purpose?
 b) I'm using the package enumerator because as far as I can see is the
 most used. How does it compares with iteratee? Which of the two are more
 performant?


 Bye and thanks,
 A.

 ___
 Haskell-Cafe mailing list
 Haskell-Cafe@haskell.org
 http://www.haskell.org/mailman/listinfo/haskell-cafe




-- 
Gregory Collins g...@gregorycollins.net
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe