-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/6/10 13:22 , Antoine Latter wrote: > On Mon, Dec 6, 2010 at 12:03 PM, Tyler Pirtle <tee...@gmail.com> wrote: >> On Sun, Dec 5, 2010 at 9:46 PM, Antoine Latter <aslat...@gmail.com> wrote: >>> On Sun, Dec 5, 2010 at 10:45 PM, Tyler Pirtle <tee...@gmail.com> wrote: >>>> Hi cafe, >>>> >>>> I'm just getting into Foreign.Storable and friends and I'm confused >>>> about the class storable. For GHC, there are instances of storable for >>>> all kinds of basic types (bool, int, etc) - but I can't find the >>>> actual declaration of those instances. >>>> >>>> I'm confused that it seems that all Storable instances operate on a >>>> Ptr, yet none of these types allow access to an underlying Ptr. I >>>> noticed that it's possible via Foreign.Marshal.Utils to call 'new' and >>>> get a datatype wrapped by a Ptr, but this isn't memory managed - I'd >>>> have to explicitly free it? Is that my only choice? >>> >>> The Storable class defines how to copy a particular Haskell type to or >>> from a raw memory buffer - specifically represented by the Ptr type. >>> It is most commonly used when interacting with non-Haskell (or >>> 'Foreign') code, which is why a lot of the tools look like they >>> require manual memory management (because foreign-owned resources must >>> often be managed separately anyway). >>> >>> Not all of the means of creating a Ptr type require manual memory >>> management - the 'alloca' family of Haskell functions allocate a >>> buffer and then free it automatically when outside the scope of the >>> passed-in callback (although 'continuation' or 'action' would be the >>> more Haskell-y way to refer to the idea): >>> >>> alloca :: Storable a => (Ptr a -> IO b) -> IO b >>> >>> This can be used to call into C code expecting pointer input or output >>> types to great effect: >>> >>> wrapperAroundForeignCode :: InputType -> IO OutputType >>> wrapperAroundForeignCode in = >>> alloca $ \inPtr -> >>> alloca $ outPtr -> do >>> poke inPtr in >>> c_call inPtr outPtr >>> peek outPtr >>> >>> The functions 'peek' and 'poke' are from the Storable class, and I >>> used the 'alloca' function to allocate temporary storage for the >>> pointers I pass into C-land. >>> >>> Is there a particular problem you're trying to solve? We might be able >>> to offer more specific advice. The Storable and Foreign operations may >>> not even be the best solution to what you're trying to do. >>> >> >> >> Hey Antoine, >> >> Thanks for the clarity, it's very helpful. There is in fact a particular >> problem I'm trying to solve - persisting data structures. I'm a huge >> fan of Data.Vector.Storable.MMap, and I'm interested in other things >> like it - but i realize that the whole thing is built up/on/around >> storables, and building vectors with storables (read == peek, write == >> poke, etc), because i'm trying to write the raw structures themselves >> to disk (via mmap). >> >> I am aware of Data.Binary, but I feel that this kind of serialization >> for the application I'm building would be too cumbersome considering the >> number of objects I'm dealing with (on the order of hundreds-of-millions >> to billions), especially considering that the application I'm building >> has some very nice pure-ish semantics (an append-only list). I'd >> like the application to able to simply load a file and interact with >> that memory - not have to load the file and then deserialize everything. >> >> If you have any suggestions here, or if anyone has any general feelings >> about the design or implementation of Data.Vector.Storable.MMap I'd be >> very interested in hearing them. Or about any ideas involving persisting >> native data structures in an append-only fashion, too. ;) >> > > If you took the approach of Data.Vector.Storable.MMap, every time you > read an element out of the array you would be un-marshalling the > object from a pointer into a Haskell type - in effect, making a copy. > There are probably ways to do this for ByteStrings to make this copy > free, but that's about it.
IIRC bytestring-mmap uses pinned bytestrings; might be easier/faster to use that directly if the vector package is troublesome. You'd want to use the bytestring internals module for the equivalent of peek/poke. - -- brandon s. allbery [linux,solaris,freebsd,perl] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkz9Mm0ACgkQIn7hlCsL25UgKgCgqV/BIXRDm5BVEPBzNllpVVD9 QsYAoJMU7kvHWxoAmb2eYV9b5tll9U0d =p1GN -----END PGP SIGNATURE----- _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe