Re: [Haskell-cafe] Either Monad and Laziness
On 12 Sep 2012, at 16:04, Eric Velten de Melo wrote: The behaviour I want to achieve is like this: I want the program when compiled to read from a file, parsing the PGM and at the same time apply transformations to the entries as they are read and write them back to another PGM file. Such problems are the main motivation for iteratees, conduits, pipes, etc. Every such library contains procedures for doing exactly what you want. It would be really awesome, though, if it were possible to use a parser written in Parsec with this, in the spirit of avoiding code rewriting and enhancing expressivity and abstraction. The polyparse library on Hackage is another parser combinator framework that allows lazy incremental parsing. http://hackage.haskell.org/package/polyparse A PDF paper/tutorial is here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.118.1754rep=rep1type=pdf Regards, Malcolm ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
Subject: Re: [Haskell-cafe] Either Monad and Laziness On 9/14/12 5:16 PM, Eric Velten de Melo wrote: But now I'm kinda lost. Is there an easy way to explain the difference between: -iteratee -conduit -enumerator I tend to group them into three families. 'iteratee' and 'enumerator' are fairly directly drawn from Oleg's code, with mostly implementation differences (at least when compared to the other families). They've tended to keep Oleg's original names (iteratee, enumerator, enumeratee). The biggest user-visible difference between iteratee and enumerator is the level of datastream abstraction. iteratee abstracts over the stream, and enumerator abstracts over elements of the stream. The stream is explicitly a list of elements. This exposes some of the details of data chunking to the user, which has both advantages and disadvantages (iteratee exposes this also, but it's not necessary as is the case for enumerator). The second family (chronologically) includes conduit and (maybe) iterIO. I've written a little about this group at http://johnlato.blogspot.sg/2012/06/understandings-of-iteratees.html Although they serve the same goals in spirit, the implementation may or may not necessarily be an iteratee/enumerator arrangement (e.g. conduit). This is a technical observation, not a criticism, depending on exactly what you consider to define the style in the first place. This group has usually renamed functions. I discuss some of the other differences on my blog. The third familiy is all the pipes-* stuff. This group tends towards emphasizing the relationship between iteratee/enumerator pairs and coroutines, and also emphasizing (to use Oleg terminology) composition of enumeratees. I've been meaning to write more about this group, but thus far have been unable to do so. I'd rather not hijack by evangelizing, but obviously I think iteratee provides several important advantages over the other options. John L. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
On 9/14/12 5:16 PM, Eric Velten de Melo wrote: But now I'm kinda lost. Is there an easy way to explain the difference between: -iteratee -conduit -enumerator John Lato's iteratee library is the original one based on Oleg Kiselyov's work. I've used it a fair deal and am quite fond of it. Some folks didn't like it so much though; whence enumerator, conduit, pipes, pipes-core,... I've followed the discussions back and forth over those libraries, but I've not used them nor sat down to compare them head-to-head. -- Live well, ~wren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
On 13 September 2012 20:29, wren ng thornton w...@freegeek.org wrote: On 9/12/12 5:37 PM, Francesco Mazzoli wrote: At Wed, 12 Sep 2012 12:04:31 -0300, Eric Velten de Melo wrote: It would be really awesome, though, if it were possible to use a parser written in Parsec with this, in the spirit of avoiding code rewriting and enhancing expressivity and abstraction. There is http://hackage.haskell.org/package/attoparsec-conduit and http://hackage.haskell.org/package/attoparsec-enumerator, which turn attoparsec parsers into enumerators/conduits, and http://hackage.haskell.org/package/attoparsec-parsec, which is a compatibility layer between attoaparsec and parsec. Good luck :). Not to mention attoparsec-iteratee, for the iteratee minded folks: http://hackage.haskell.org/package/attoparsec-iteratee Hm... I guess I'm spoiled for choice then. :) But now I'm kinda lost. Is there an easy way to explain the difference between: -iteratee -conduit -enumerator I'm very curious about everything concerning Haskell and new interesting abstractions and ways of doing things, but I might not have the time to delve deeper into that. -- Live well, ~wren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
On 9/12/12 5:37 PM, Francesco Mazzoli wrote: At Wed, 12 Sep 2012 12:04:31 -0300, Eric Velten de Melo wrote: It would be really awesome, though, if it were possible to use a parser written in Parsec with this, in the spirit of avoiding code rewriting and enhancing expressivity and abstraction. There is http://hackage.haskell.org/package/attoparsec-conduit and http://hackage.haskell.org/package/attoparsec-enumerator, which turn attoparsec parsers into enumerators/conduits, and http://hackage.haskell.org/package/attoparsec-parsec, which is a compatibility layer between attoaparsec and parsec. Good luck :). Not to mention attoparsec-iteratee, for the iteratee minded folks: http://hackage.haskell.org/package/attoparsec-iteratee -- Live well, ~wren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Either Monad and Laziness
I am currently trying to rewrite the Graphics.Pgm library from hackage to parse the PGM to a lazy array. Laziness and IO really do not mix. The problem is that even using a lazy array structure, because the parser returns an Either structure it is only possible to know if the parser was successful or not after the whole file is read, That is one of the problems. Unexpected memory blowups could be another problem. The drawbacks of lazy IO are well documented by now. The behaviour I want to achieve is like this: I want the program when compiled to read from a file, parsing the PGM and at the same time apply transformations to the entries as they are read and write them back to another PGM file. Such problems are the main motivation for iteratees, conduits, pipes, etc. Every such library contains procedures for doing exactly what you want. Please check Hackage. John Lato's iteratee library, for example, has procedure for handling sound (AIFF) files -- which may be very big. IterateeM has the TIFF decoder -- which is incremental and strict. TIFF is much harder to parse than PGM. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
Thanks for all the tips! The iteratees seem worth checking out. I'll see what I can do and will report back if I come up with something. Eric On 12 September 2012 03:03, o...@okmij.org wrote: I am currently trying to rewrite the Graphics.Pgm library from hackage to parse the PGM to a lazy array. Laziness and IO really do not mix. The problem is that even using a lazy array structure, because the parser returns an Either structure it is only possible to know if the parser was successful or not after the whole file is read, That is one of the problems. Unexpected memory blowups could be another problem. The drawbacks of lazy IO are well documented by now. The behaviour I want to achieve is like this: I want the program when compiled to read from a file, parsing the PGM and at the same time apply transformations to the entries as they are read and write them back to another PGM file. Such problems are the main motivation for iteratees, conduits, pipes, etc. Every such library contains procedures for doing exactly what you want. Please check Hackage. John Lato's iteratee library, for example, has procedure for handling sound (AIFF) files -- which may be very big. IterateeM has the TIFF decoder -- which is incremental and strict. TIFF is much harder to parse than PGM. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
On 12 September 2012 11:46, Eric Velten de Melo ericvm...@gmail.com wrote: Thanks for all the tips! The iteratees seem worth checking out. I'll see what I can do and will report back if I come up with something. Eric On 12 September 2012 03:03, o...@okmij.org wrote: I am currently trying to rewrite the Graphics.Pgm library from hackage to parse the PGM to a lazy array. Laziness and IO really do not mix. The problem is that even using a lazy array structure, because the parser returns an Either structure it is only possible to know if the parser was successful or not after the whole file is read, That is one of the problems. Unexpected memory blowups could be another problem. The drawbacks of lazy IO are well documented by now. The behaviour I want to achieve is like this: I want the program when compiled to read from a file, parsing the PGM and at the same time apply transformations to the entries as they are read and write them back to another PGM file. Such problems are the main motivation for iteratees, conduits, pipes, etc. Every such library contains procedures for doing exactly what you want. Please check Hackage. John Lato's iteratee library, for example, has procedure for handling sound (AIFF) files -- which may be very big. IterateeM has the TIFF decoder -- which is incremental and strict. TIFF is much harder to parse than PGM. It would be really awesome, though, if it were possible to use a parser written in Parsec with this, in the spirit of avoiding code rewriting and enhancing expressivity and abstraction. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
At Wed, 12 Sep 2012 12:04:31 -0300, Eric Velten de Melo wrote: It would be really awesome, though, if it were possible to use a parser written in Parsec with this, in the spirit of avoiding code rewriting and enhancing expressivity and abstraction. There is http://hackage.haskell.org/package/attoparsec-conduit and http://hackage.haskell.org/package/attoparsec-enumerator, which turn attoparsec parsers into enumerators/conduits, and http://hackage.haskell.org/package/attoparsec-parsec, which is a compatibility layer between attoaparsec and parsec. Good luck :). -- Francesco * Often in error, never in doubt ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Either Monad and Laziness
Hello, I am currently trying to rewrite the Graphics.Pgm library from hackage to parse the PGM to a lazy array. The current implementation parses it straight to UArray, which is strict. The behaviour I want to achieve is like this: I want the program when compiled to read from a file, parsing the PGM and at the same time apply transformations to the entries as they are read and write them back to another PGM file. The problem is that even using a lazy array structure, because the parser returns an Either structure it is only possible to know if the parser was successful or not after the whole file is read, therefore requiring it to read the entire file before applying the transformations, ruining the property of laziness. Is there some way to keep this from happening? Should I even want to make it like this? Not really a real life situation, but imagine I want to read a really large PGM file which does not fit into RAM memory and I don't want to be forced to have the whole array in the memory at once. One alternative I thought was parsing only the PGM header and then read the rest of the input without using Parsec and the Either Monad. In the event the data is corrupted, though, I would not know how to recover from it. Any thoughts? Hopefully I'm not saying anything really stupid. Eric ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
On Tue, Sep 11, 2012 at 9:36 AM, Eric Velten de Melo ericvm...@gmail.com wrote: Any thoughts? Hopefully I'm not saying anything really stupid. You can intersperse decoding errors in the output, e.g. output is [Either Error DecodedChunk]. Then all the processors have to deal with it, but if you just want to pass the error through then just 'map . fmap' instead of 'map'. This means processors can also inject their own errors or logs into the output, which may be very useful. Or you could use runtime exceptions, i.e. the decoder is lazy but can call error. This is bad for reliability but if you know you always want to crash on a bad parse it keeps the return value simple. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Either Monad and Laziness
Use a tuple: (Result,Maybe Error) rather than an Either. Do everything lazily, and in the case of an error, undo the result. -- Původní zpráva -- Od: Eric Velten de Melo ericvm...@gmail.com Datum: 11. 9. 2012 Předmět: [Haskell-cafe] Either Monad and Laziness Hello, I am currently trying to rewrite the Graphics.Pgm library from hackage to parse the PGM to a lazy array. The current implementation parses it straight to UArray, which is strict. The behaviour I want to achieve is like this: I want the program when compiled to read from a file, parsing the PGM and at the same time apply transformations to the entries as they are read and write them back to another PGM file. The problem is that even using a lazy array structure, because the parser returns an Either structure it is only possible to know if the parser was successful or not after the whole file is read, therefore requiring it to read the entire file before applying the transformations, ruining the property of laziness. Is there some way to keep this from happening? Should I even want to make it like this? Not really a real life situation, but imagine I want to read a really large PGM file which does not fit into RAM memory and I don't want to be forced to have the whole array in the memory at once. One alternative I thought was parsing only the PGM header and then read the rest of the input without using Parsec and the Either Monad. In the event the data is corrupted, though, I would not know how to recover from it. Any thoughts? Hopefully I'm not saying anything really stupid. Eric ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe (http://www.haskell.org/mailman/listinfo/haskell-cafe)___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
