that's too bad, i used lazy deserialization for an external sort thing i did aeons ago.
http://www.haskell.org/pipermail/haskell-cafe/2007-July/029156.html that was an amusing exercise in lazy IO. these days it's probably better off doing something with pipes et al instead of unsafeInterleaveIO. b On Mar 13, 2013, at 2:54 PM, Scott Lawrence wrote: > I tried it, but it still goes and reads the whole list. Looking at the > `binary` package source code it seems that strict evaluation is hard-coded in > a few places, presumably for performance reasons. It also seems to > necessarily read the bytestring sequentially, so complex tree-like data > structures would presumably encounter problems even if it worked for a list. > > Ah well. As long as I'm not duplicating someone else's work, I'm more than > happy to go at this from scratch. > > On Wed, 13 Mar 2013, Jeff Shaw wrote: > >> On 3/13/2013 12:15 AM, Scott Lawrence wrote: >>> Hey all, >>> All the object serialization/deserialization libraries I could find (pretty >>> much just binary and cereal) seem to be strict with respect to the actual >>> data being serialized. In particular, if I've serialized a large [Int] to a >>> file, and I want to get the first element, it seems I have no choice but to >>> deserialize the entire data structure. This is obviously an issue for large >>> data sets. >>> There are obvious workarounds (explicitly fetch elements from the >>> "database" instead of relying on unsafeInterleaveIO to deal with it all >>> magically), but it seems like it should be possible to build a cereal-like >>> library that allows proper lazy deserialization. Does it exist, and I've >>> just missed it? >>> Thanks, >> I haven't tested this, but I suspect something like this could give you lazy >> binary serialization and deserialization. It's not tail recursive, though. >> >> newtype LazyBinaryList a = LazyBinaryList [a] >> >> instance Binary a => LazyBinaryList a where >> put (LazyBinaryList []) = putWord8 0 >> put (LazyBinaryList (x:xs)) = putWord8 1 >> put x >> put (LazyBinaryList >> xs) >> get = do >> t <- getWord8 >> case t of >> 0 -> return (LazyBinaryList []) >> 1 -> do >> x <- get >> (LazyBinaryList xs) <- get >> return $ LazyBinaryList (x:xs) >> > > -- > Scott Lawrence > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
