Re: [Haskell-cafe] Data.Binary and little endian encoding
Duncan Coutts writes: >> FWIW, I've used Data.Binary extensively and have found it a joy to >> work with. I've used it to serialize/deserialize ethernet packets in >> real time for a VPN implementation and have never had a problem. It's >> quite fast and robust. > Is that code available? We could do with something serious for > benchmarking the binary package, especially if we go for any major > re-engineering. I'm using Data.Binary in the bioinformatics library to encode/decode SFF files. These contain gene sequences produced with the new pyrosequencing technology from Roche/454, have sizes (with the latest incarnation of the sequencing) of about 1-2GB. Plenty of test data at http://www.ncbi.nlm.nih.gov/sites/entrez?db=sra A program using the library is described here: http://blog.malde.org/index.php/flower/ All available with (L)GPL licenses. -k -- If I haven't seen further, it is by standing in the footprints of giants ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, May 28, 2009 at 6:34 PM, Duncan Coutts wrote: > If it's not available publicly perhaps you might share it privately. Don > and I have discussed a few times writing a paper on the design and > implementation of a binary library. > I would definitely like to read such a paper. Except for Don's blog entries there's precious little written on the design of high performance Haskell libraries. -- Johan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, 2009-05-28 at 12:08 -0400, John Van Enk wrote: > > I'm trying to implement the protocol, so that I can implement > other things on top of that. > > I'm also trying to figure out how bad/good Haskell Binary IO > really is that it's been addressed a few times differently :-) > > > FWIW, I've used Data.Binary extensively and have found it a joy to > work with. I've used it to serialize/deserialize ethernet packets in > real time for a VPN implementation and have never had a problem. It's > quite fast and robust. Is that code available? We could do with something serious for benchmarking the binary package, especially if we go for any major re-engineering. If it's not available publicly perhaps you might share it privately. Don and I have discussed a few times writing a paper on the design and implementation of a binary library. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
> > > I'm trying to implement the protocol, so that I can implement other things > on top of that. > > I'm also trying to figure out how bad/good Haskell Binary IO really is that > it's been addressed a few times differently :-) > > FWIW, I've used Data.Binary extensively and have found it a joy to work with. I've used it to serialize/deserialize ethernet packets in real time for a VPN implementation and have never had a problem. It's quite fast and robust. /jve ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
leimy2k: > I'm also trying to figure out how bad/good Haskell Binary IO really is that > it's been addressed a few times differently :-) FWIW Binary IO as implemented in Data.Binary is widely used in our production systems at Galois. I'd be fairly confident in it. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, May 28, 2009 at 5:42 AM, Khudyakov Alexey wrote: > On Thursday 28 of May 2009 07:52:56 David Leimbach wrote: > > Sorry took so long to get back... Thank you for the response. Been > really > > busy lately :-) > > > > There are also a lot of 9P implementations in many languages that you can > > interoperate with: > > > > http://9p.cat-v.org/implementations > > > Thank you for that link. I didn't find it earlier. But actually I asked > what > are you trying to write? I'm trying to implement the protocol, so that I can implement other things on top of that. I'm also trying to figure out how bad/good Haskell Binary IO really is that it's been addressed a few times differently :-) > > -- > Khudyakov Alexey > ___ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thursday 28 of May 2009 07:52:56 David Leimbach wrote: > Sorry took so long to get back... Thank you for the response. Been really > busy lately :-) > > There are also a lot of 9P implementations in many languages that you can > interoperate with: > > http://9p.cat-v.org/implementations > Thank you for that link. I didn't find it earlier. But actually I asked what are you trying to write? -- Khudyakov Alexey ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
Sorry took so long to get back... Thank you for the response. Been really
busy lately :-)
On Sat, May 16, 2009 at 3:46 AM, Khudyakov Alexey wrote:
> On Friday 15 May 2009 06:52:29 David Leimbach wrote:
> > I actually need little endian encoding... wondering if anyone else hit
> this
> > with Data.Binary. (because I'm working with Bell Lab's 9P protocol which
> > does encode things on the network in little-endian order).
> > Anyone got some "tricks" for this?
> >
> > Dave
>
> You could just define data type and Binary instance for 9P messages.
> Something
> like this:
>
> P9Message = Tversion { tag :: Word16, msize :: Word32, version :: String }
>| ...
>
> instance Binary P9Message where
> put (Tverstion t m v) = putWord16le t >> putWord32le m >> put v
> -- and so on...
>
> get = do
>length <- getWord32le
>id <- getWord16le
>case is of
> p9TMessage -> do ...
>
> There are a lot of boilerplate code thought...
Thank you, this still looks like a useful way to proceed, combined with the
BinaryLE approach perhaps, to avoid a lot of boilerplate.
>
>
> BTW could you say what do you want to do with 9P? I tried to play with it
> using libixp library but without any success. It was mainly to understand
> how
> does it works and how can it be used.
>From a services point of view, 9P gives you a way to host them, and even
devices, on a network share that can be "mounted" into the filesystem's
namespace. The net result is you've plugged into the standard unix
utilities that do open, read, write etc, and can do a lot of interesting
things with mere shell scripts.
Operating systems that can be clients of a 9P service include Linux,
Inferno, Plan 9, and anything else that runs FUSE 9P (several BSDs).
>From a client perspective, having a 9P implementation gives you a more
fine-grained programatic interface to accessing other 9P services.
There are also a lot of 9P implementations in many languages that you can
interoperate with:
http://9p.cat-v.org/implementations
>
>
> ___
> Haskell-Cafe mailing list
> Haskell-Cafe@haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
Am Sonntag, 17. Mai 2009 15:08:29 schrieb Don Stewart:
> Sven.Panne:
> > [...]
> > I think most problems can be fixed in a rather pragmatic way by adding a
> > few functions to the binary package:
> [...]
> Patches are welcome.
Attached. A few remarks:
* This is only a quick and mildly tested implementation of the IEEE
functions, especially NaNs, infinities and denormalized numbers are untested.
These problems could totally be avoided if we can coerce representations
directly, changing only their interpretation.
* The *host functions assume an IEEE platform, but this can easily be changed
(see comments).
* Perhaps one can use unsafeCoerce for word32ToFloat and friends, but I
haven't checked this.
* I've seen a few "{- INLINE -}" comments. Is this really wanted or only a
typo?
* A comment about using peek/poke for the *le/*be functions is wrong, because
this would introduce alignment constraints on some platforms.
I think the main point is to provide a nice and efficient API, hiding all the
dirty stuff in the implementation.
> > One final remarks: I think the low level functions of the binary package
> > should really keep the notions of "endianess" and "alignment constraints"
> > separate, something which isn't done currently: The *host functions have
> > alignment restrictions, the *be/*le functions don't. There is no good
> > reason for this non-orthogonality.
>
> That seems reasonable.
There are various ways to achieve this, but the most obvious way leads to a
combinatorial explosion of functions:
* 3 (LE/BE/host) * 2 (aligned/unaligned)
Furthermore, it would be good to split the binary package into the 2 layers
already discussed first, then it is perhaps a bit clearer what a nice API
would look like. I think it would be best to shift this API design discussion
to the libraries list.
Cheers,
S.
Only in binary-0.5.0.1: dist
diff -r -u binary-0.5.0.1.orig/src/Data/Binary/Builder.hs binary-0.5.0.1/src/Data/Binary/Builder.hs
--- binary-0.5.0.1.orig/src/Data/Binary/Builder.hs Sat Mar 7 23:59:44 2009
+++ binary-0.5.0.1/src/Data/Binary/Builder.hs Mon May 18 17:36:22 2009
@@ -41,20 +41,27 @@
, putWord16be -- :: Word16 -> Builder
, putWord32be -- :: Word32 -> Builder
, putWord64be -- :: Word64 -> Builder
+, putFloatIEEEbe-- :: Float -> Builder
+, putDoubleIEEEbe -- :: Double -> Builder
-- ** Little-endian writes
, putWord16le -- :: Word16 -> Builder
, putWord32le -- :: Word32 -> Builder
, putWord64le -- :: Word64 -> Builder
+, putFloatIEEEle-- :: Float -> Builder
+, putDoubleIEEEle -- :: Double -> Builder
-- ** Host-endian, unaligned writes
, putWordhost -- :: Word -> Builder
, putWord16host -- :: Word16 -> Builder
, putWord32host -- :: Word32 -> Builder
, putWord64host -- :: Word64 -> Builder
+, putFloatIEEEhost -- :: Float -> Builder
+, putDoubleIEEEhost -- :: Double -> Builder
) where
+import Prelude hiding (significand, exponent)
import Foreign
import Data.Monoid
import Data.Word
@@ -360,6 +367,60 @@
-- on a little endian machine:
-- putWord64le w64 = writeN 8 (\p -> poke (castPtr p) w64)
+-- | Write a Float in IEEE big endian format
+putFloatIEEEbe :: Float -> Builder
+putFloatIEEEbe = putWord32be . floatToWord32
+{-# INLINE putFloatIEEEbe #-}
+
+-- | Write a Double in IEEE big endian format
+putDoubleIEEEbe :: Double -> Builder
+putDoubleIEEEbe = putWord64be . doubleToWord64
+{-# INLINE putDoubleIEEEbe #-}
+
+-- | Write a Float in IEEE little endian format
+putFloatIEEEle :: Float -> Builder
+putFloatIEEEle = putWord32le . floatToWord32
+{-# INLINE putFloatIEEEle #-}
+
+-- | Write a Double in IEEE little endian format
+putDoubleIEEEle :: Double -> Builder
+putDoubleIEEEle = putWord64le . doubleToWord64
+{-# INLINE putDoubleIEEEle #-}
+
+floatToWord32 :: Float -> Word32
+-- floatToWord32 = unsafeReinterpret
+floatToWord32 = encodeIEEE 8 23
+
+doubleToWord64 :: Double -> Word64
+-- doubleToWord64 = unsafeReinterpret
+doubleToWord64 = encodeIEEE 11 52
+
+-- TODO: Check if this works for denormalized numbers, NaNs and infinities.
+encodeIEEE :: (RealFloat a, Bits b, Integral b) => Int -> Int -> a -> b
+encodeIEEE exponentBits significandBits f =
+ (signBit `shiftL` (exponentBits + significandBits)) .|.
+ (exponentField `shiftL` significandBits) .|.
+ significandField
+ where (significand, exponent) = decodeFloat f
+
+ signBit | significand < 0 = 1
+ | otherwise = 0
+ exponentField | significand == 0 && exponent == 0 = 0
+ | otherwise = fromIntegral exponent + exponentBias + fromIntegral significandBits
+ significandField = fromIntegral (abs significand) .&. significandMask
+
+ exponentBias = bit (exponentBits - 1) - 1
+ significandMask = bit significa
Re: [Haskell-cafe] Data.Binary and little endian encoding
Sven.Panne: > Am Freitag, 15. Mai 2009 06:37:22 schrieb Don Stewart: > > timd: > > > On a related matter, I am using Data.Binary to serialise data from > > > haskell for use from other languages. [...] > > [...] > > Yep, it's possible, just not portably so. Google for Data.Binary IEEE > > discussions. > > I think this topic pops up over and over again, and the proposed "solutions" > are no solutions at all, neither from a performance point of view, nor from > an > ease of use point of view. Proposing insane bit fiddling by hand when all one > technically needs is often a "peek" or "poke" amounts to simply ignoring an > API problem. ;-) > > I think most problems can be fixed in a rather pragmatic way by adding a few > functions to the binary package: > > Add to Data.Binary.Builder: > >putFloatIEEEbe :: Float -> Builder >putDoubleIEEEbe :: Double -> Builder >putFloatIEEEle :: Float -> Builder >putDoubleIEEEle :: Double -> Builder >putFloatIEEEhost :: Float -> Builder >putDoubleIEEEhost :: Double -> Builder > > Add to Data.Binary.Get: > >getFloatIEEEbe :: Get Float >getDoubleIEEEbe :: Get Double >getFloatIEEEle :: Get Float >getDoubleIEEEle :: Get Double >getFloatIEEEhost :: Get Float >getDoubleIEEEhost :: Get Double > > Add to Data.Binary.Put: > >putFloatIEEEbe :: Float -> Put >putDoubleIEEEbe :: Double -> Put >putFloatIEEEle :: Float -> Put >putDoubleIEEEle :: Double -> Put >putFloatIEEEhost :: Float -> Put >putDoubleIEEEhost :: Double -> Put > > The *host functions are basically peek/poke for most platforms. The *le/*be > functions can use peek/poke if the endianess matches (compile time decision) > *and* the alignment is OK for the given platform (runtime decision). Non-IEEE > platforms always have to do the bit fiddling internally, but all this is > hidden behind the above API. > > IIRC I have proposed something similar 1-2 years ago, but I can't remember > any > reason why this hasn't been implemented. Any comments on the above functions? Patches are welcome. > One final remarks: I think the low level functions of the binary package > should really keep the notions of "endianess" and "alignment constraints" > separate, something which isn't done currently: The *host functions have > alignment restrictions, the *be/*le functions don't. There is no good reason > for this non-orthogonality. That seems reasonable. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
Am Freitag, 15. Mai 2009 06:37:22 schrieb Don Stewart: > timd: > > On a related matter, I am using Data.Binary to serialise data from > > haskell for use from other languages. [...] > [...] > Yep, it's possible, just not portably so. Google for Data.Binary IEEE > discussions. I think this topic pops up over and over again, and the proposed "solutions" are no solutions at all, neither from a performance point of view, nor from an ease of use point of view. Proposing insane bit fiddling by hand when all one technically needs is often a "peek" or "poke" amounts to simply ignoring an API problem. ;-) I think most problems can be fixed in a rather pragmatic way by adding a few functions to the binary package: Add to Data.Binary.Builder: putFloatIEEEbe :: Float -> Builder putDoubleIEEEbe :: Double -> Builder putFloatIEEEle :: Float -> Builder putDoubleIEEEle :: Double -> Builder putFloatIEEEhost :: Float -> Builder putDoubleIEEEhost :: Double -> Builder Add to Data.Binary.Get: getFloatIEEEbe :: Get Float getDoubleIEEEbe :: Get Double getFloatIEEEle :: Get Float getDoubleIEEEle :: Get Double getFloatIEEEhost :: Get Float getDoubleIEEEhost :: Get Double Add to Data.Binary.Put: putFloatIEEEbe :: Float -> Put putDoubleIEEEbe :: Double -> Put putFloatIEEEle :: Float -> Put putDoubleIEEEle :: Double -> Put putFloatIEEEhost :: Float -> Put putDoubleIEEEhost :: Double -> Put The *host functions are basically peek/poke for most platforms. The *le/*be functions can use peek/poke if the endianess matches (compile time decision) *and* the alignment is OK for the given platform (runtime decision). Non-IEEE platforms always have to do the bit fiddling internally, but all this is hidden behind the above API. IIRC I have proposed something similar 1-2 years ago, but I can't remember any reason why this hasn't been implemented. Any comments on the above functions? One final remarks: I think the low level functions of the binary package should really keep the notions of "endianess" and "alignment constraints" separate, something which isn't done currently: The *host functions have alignment restrictions, the *be/*le functions don't. There is no good reason for this non-orthogonality. Cheers, S. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Friday 15 May 2009 06:52:29 David Leimbach wrote:
> I actually need little endian encoding... wondering if anyone else hit this
> with Data.Binary. (because I'm working with Bell Lab's 9P protocol which
> does encode things on the network in little-endian order).
> Anyone got some "tricks" for this?
>
> Dave
You could just define data type and Binary instance for 9P messages. Something
like this:
P9Message = Tversion { tag :: Word16, msize :: Word32, version :: String }
| ...
instance Binary P9Message where
put (Tverstion t m v) = putWord16le t >> putWord32le m >> put v
-- and so on...
get = do
length <- getWord32le
id <- getWord16le
case is of
p9TMessage -> do ...
There are a lot of boilerplate code thought...
BTW could you say what do you want to do with 9P? I tried to play with it
using libixp library but without any success. It was mainly to understand how
does it works and how can it be used.
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, 2009-05-14 at 20:46 -0700, David Leimbach wrote: > > > On Thu, May 14, 2009 at 8:40 PM, Don Stewart wrote: > leimy2k: > > > I actually need little endian encoding... wondering if > anyone else hit this > > with Data.Binary. (because I'm working with Bell Lab's 9P > protocol which does > > encode things on the network in little-endian order). > > > > Anyone got some "tricks" for this? > > > Yes! > There are big, little and host-endian primitives in the > Get/Put monads. > > > > http://hackage.haskell.org/packages/archive/binary/0.5.0.1/doc/html/Data-Binary-Put.html#v%3AputWord16le > > You can use these to build encoders directly. > > > Cool... I just have to write my own encoder and decoder now. > > > As a request could we get encodeLe decodeLe for a later version of > this library? :-) That'd be totally awesome. The thing you're missing (and which admittedly is not clear) is that the binary package has two parts. One is a layer where you get full control over the binary representation. The other is a portable serialisation layer for pickling and unpickling Haskell values. That pickling layer (ie the Binary class) is not for working with externally-defined binary formats. It might seem like we could co-opt the Binary class for this purpose eg by parametrising by a dozen things like endian, padding, etc etc but I don't think it scales or is sufficiently flexible (and it'd be slow). What is missing in the binary package is a nice set of combinators for using the low level layer to easily construct parsers for externally-defined formats. That's what you'd want for your P9 protocol. To reduce confusion we should also split the Haskell picking layer from the lower layer. This has been on our TODO list for some time. It needs to be done pretty carefully however and we've not really had the time. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
timd: > On a related matter, I am using Data.Binary to serialise data from > haskell > for use from other languages. The Data.Binary encoding of a Double is a > long > integer for the mantissa, and an int for the exponent. This doesn't > work too well for interacting with other languages as I'd need to have > an arbitrary precision int type there to decode/encode. The CORBA CDR > standard encodes doubles in a big ended fashion like this (excuse my > possibly incorrect ascii art): > > > | byte | msb lsb | > |--+---| > |0 | S E6 E0 | > |1 | E10 E9 E8 E7 F3 F2 F1 F0 | > |2 | F11F4 | > |3 | F19 F12 | > |4 | F27 F20 | > |5 | F35 F28 | > |6 | F43 F36 | > |7 | F51 F44 | > > Up until now, my code is pure haskell. Is it possible to get at the > internal bits of a Double/CDouble in ghc? Or Should I use the FFI and > write C to encode something like the above? Yep, it's possible, just not portably so. Google for Data.Binary IEEE discussions. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
RE: [Haskell-cafe] Data.Binary and little endian encoding
On a related matter, I am using Data.Binary to serialise data from haskell for use from other languages. The Data.Binary encoding of a Double is a long integer for the mantissa, and an int for the exponent. This doesn't work too well for interacting with other languages as I'd need to have an arbitrary precision int type there to decode/encode. The CORBA CDR standard encodes doubles in a big ended fashion like this (excuse my possibly incorrect ascii art): | byte | msb lsb | |--+---| |0 | S E6 E0 | |1 | E10 E9 E8 E7 F3 F2 F1 F0 | |2 | F11F4 | |3 | F19 F12 | |4 | F27 F20 | |5 | F35 F28 | |6 | F43 F36 | |7 | F51 F44 | Up until now, my code is pure haskell. Is it possible to get at the internal bits of a Double/CDouble in ghc? Or Should I use the FFI and write C to encode something like the above? Tim From: haskell-cafe-boun...@haskell.org [mailto:haskell-cafe-boun...@haskell.org] On Behalf Of David Leimbach Sent: Friday, 15 May 2009 1:58 PM To: Don Stewart Cc: Haskell Cafe Subject: Re: [Haskell-cafe] Data.Binary and little endian encoding On Thu, May 14, 2009 at 8:54 PM, Don Stewart wrote: > I'm speaking specifically of the encode/decode functions. I have no idea how > they're implemented. > > Are you saying that encode is doing something really simple and the default > encodings for things just happen to be big endian? If so, then I understand > the pain but it still means I have to roll my own :-) I guess if one must > choose, big endian kind of makes sense, except that the whole world is little > endian now, except for networks :-) (No one *really* cares about anything but > x86 anyway these days right?) Oh, 'encode' has type: encode :: Binary a => a -> ByteString it just encodes with the default instances, which are all network order: http://en.wikipedia.org/wiki/Endianness#Endianness_in_networking Yeah I understand that Big Endian == Network Byte Order... which would be true, if I wasn't talking about Plan 9's 9P protocol which specifies little endian bytes on the wire (as far as I can tell anyway from the man page). Dave -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, May 14, 2009 at 9:10 PM, David Leimbach wrote: > > > On Thu, May 14, 2009 at 8:57 PM, David Leimbach wrote: > >> >> >> On Thu, May 14, 2009 at 8:54 PM, Don Stewart wrote: >> >>> > I'm speaking specifically of the encode/decode functions. I have no >>> idea how >>> > they're implemented. >>> > >>> > Are you saying that encode is doing something really simple and the >>> default >>> > encodings for things just happen to be big endian? If so, then I >>> understand >>> > the pain but it still means I have to roll my own :-) I guess if >>> one must >>> > choose, big endian kind of makes sense, except that the whole world is >>> little >>> > endian now, except for networks :-) (No one *really* cares about >>> anything but >>> > x86 anyway these days right?) >>> >>> Oh, 'encode' has type: >>> >>>encode :: Binary a => a -> ByteString >>> >>> it just encodes with the default instances, which are all network order: >>> >>>http://en.wikipedia.org/wiki/Endianness#Endianness_in_networking >>> >> >> Yeah I understand that Big Endian == Network Byte Order... which would be >> true, if I wasn't talking about Plan 9's 9P protocol which specifies little >> endian bytes on the wire (as far as I can tell anyway from the man page). >> >> Dave >> > > FYI here's what I've ended up trying to write to negotiate the "version" of > a 9p server: > > main = withSocketsDo $ >do > ainfo <- getAddrInfo Nothing (Just "127.0.0.1") (Just "6872") -- > hardcoded for now, it's an IRC filesystem server > let a = head ainfo > sock <- socket AF_INET Stream defaultProtocol > connect sock (addrAddress a) > sendAll sock $ (toLazyByteString (putWord32le (fromIntegral (16 > ::Int32 > sendAll sock $ (encode (100 ::Int8)) > sendAll sock $ (toLazyByteString (putWord32le (fromIntegral (1024 > ::Int32 > sendAll sock $ (encode (C.pack "9P2000")) > > I totally forgot the tag part of 9p ugh. I think I should probably just go to bed now. > > I feel like I should use wireshark or something to watch the bytes :-) I'm > not feeling very sure about this. > > >> >>> >>> -- Don >>> >> >> > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, May 14, 2009 at 8:57 PM, David Leimbach wrote: > > > On Thu, May 14, 2009 at 8:54 PM, Don Stewart wrote: > >> > I'm speaking specifically of the encode/decode functions. I have no >> idea how >> > they're implemented. >> > >> > Are you saying that encode is doing something really simple and the >> default >> > encodings for things just happen to be big endian? If so, then I >> understand >> > the pain but it still means I have to roll my own :-) I guess if >> one must >> > choose, big endian kind of makes sense, except that the whole world is >> little >> > endian now, except for networks :-) (No one *really* cares about >> anything but >> > x86 anyway these days right?) >> >> Oh, 'encode' has type: >> >>encode :: Binary a => a -> ByteString >> >> it just encodes with the default instances, which are all network order: >> >>http://en.wikipedia.org/wiki/Endianness#Endianness_in_networking >> > > Yeah I understand that Big Endian == Network Byte Order... which would be > true, if I wasn't talking about Plan 9's 9P protocol which specifies little > endian bytes on the wire (as far as I can tell anyway from the man page). > > Dave > FYI here's what I've ended up trying to write to negotiate the "version" of a 9p server: main = withSocketsDo $ do ainfo <- getAddrInfo Nothing (Just "127.0.0.1") (Just "6872") -- hardcoded for now, it's an IRC filesystem server let a = head ainfo sock <- socket AF_INET Stream defaultProtocol connect sock (addrAddress a) sendAll sock $ (toLazyByteString (putWord32le (fromIntegral (16 ::Int32 sendAll sock $ (encode (100 ::Int8)) sendAll sock $ (toLazyByteString (putWord32le (fromIntegral (1024 ::Int32 sendAll sock $ (encode (C.pack "9P2000")) I feel like I should use wireshark or something to watch the bytes :-) I'm not feeling very sure about this. > >> >> -- Don >> > > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, May 14, 2009 at 8:54 PM, Don Stewart wrote: > > I'm speaking specifically of the encode/decode functions. I have no idea > how > > they're implemented. > > > > Are you saying that encode is doing something really simple and the > default > > encodings for things just happen to be big endian? If so, then I > understand > > the pain but it still means I have to roll my own :-) I guess if one > must > > choose, big endian kind of makes sense, except that the whole world is > little > > endian now, except for networks :-) (No one *really* cares about > anything but > > x86 anyway these days right?) > > Oh, 'encode' has type: > >encode :: Binary a => a -> ByteString > > it just encodes with the default instances, which are all network order: > >http://en.wikipedia.org/wiki/Endianness#Endianness_in_networking > Yeah I understand that Big Endian == Network Byte Order... which would be true, if I wasn't talking about Plan 9's 9P protocol which specifies little endian bytes on the wire (as far as I can tell anyway from the man page). Dave > > -- Don > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
> I'm speaking specifically of the encode/decode functions. I have no idea how > they're implemented. > > Are you saying that encode is doing something really simple and the default > encodings for things just happen to be big endian? If so, then I understand > the pain but it still means I have to roll my own :-) I guess if one must > choose, big endian kind of makes sense, except that the whole world is little > endian now, except for networks :-) (No one *really* cares about anything but > x86 anyway these days right?) Oh, 'encode' has type: encode :: Binary a => a -> ByteString it just encodes with the default instances, which are all network order: http://en.wikipedia.org/wiki/Endianness#Endianness_in_networking -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, May 14, 2009 at 8:46 PM, Don Stewart wrote: > leimy2k: > > > > > > On Thu, May 14, 2009 at 8:40 PM, Don Stewart wrote: > > > > leimy2k: > > > I actually need little endian encoding... wondering if anyone else > hit > > this > > > with Data.Binary. (because I'm working with Bell Lab's 9P protocol > which > > does > > > encode things on the network in little-endian order). > > > > > > Anyone got some "tricks" for this? > > > > Yes! > > There are big, little and host-endian primitives in the Get/Put > monads. > > > > > http://hackage.haskell.org/packages/archive/binary/0.5.0.1/doc/html/ > > Data-Binary-Put.html#v%3AputWord16le > > > > You can use these to build encoders directly. > > > > > > Cool... I just have to write my own encoder and decoder now. > > > > As a request could we get encodeLe decodeLe for a later version of this > > library? :-) That'd be totally awesome. > > Oh, you mean entirely different instances for all the current ones, that > use LE encodings? > Well the library is leaning towards "Network Byte Order" in that it has encode/decode that only encode/decode for Big Endian. Us folks who have to do little endian all now have to write our own encoding/decoding :-) I'm speaking specifically of the encode/decode functions. I have no idea how they're implemented. Are you saying that encode is doing something really simple and the default encodings for things just happen to be big endian? If so, then I understand the pain but it still means I have to roll my own :-) I guess if one must choose, big endian kind of makes sense, except that the whole world is little endian now, except for networks :-) (No one *really* cares about anything but x86 anyway these days right?) I'm only half-kidding. > > -- Don > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
leimy2k: > > > On Thu, May 14, 2009 at 8:40 PM, Don Stewart wrote: > > leimy2k: > > I actually need little endian encoding... wondering if anyone else hit > this > > with Data.Binary. (because I'm working with Bell Lab's 9P protocol which > does > > encode things on the network in little-endian order). > > > > Anyone got some "tricks" for this? > > Yes! > There are big, little and host-endian primitives in the Get/Put monads. > >http://hackage.haskell.org/packages/archive/binary/0.5.0.1/doc/html/ > Data-Binary-Put.html#v%3AputWord16le > > You can use these to build encoders directly. > > > Cool... I just have to write my own encoder and decoder now. > > As a request could we get encodeLe decodeLe for a later version of this > library? :-) That'd be totally awesome. Oh, you mean entirely different instances for all the current ones, that use LE encodings? -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
On Thu, May 14, 2009 at 8:40 PM, Don Stewart wrote: > leimy2k: > > I actually need little endian encoding... wondering if anyone else hit > this > > with Data.Binary. (because I'm working with Bell Lab's 9P protocol which > does > > encode things on the network in little-endian order). > > > > Anyone got some "tricks" for this? > > Yes! > There are big, little and host-endian primitives in the Get/Put monads. > > > http://hackage.haskell.org/packages/archive/binary/0.5.0.1/doc/html/Data-Binary-Put.html#v%3AputWord16le > > You can use these to build encoders directly. Cool... I just have to write my own encoder and decoder now. As a request could we get encodeLe decodeLe for a later version of this library? :-) That'd be totally awesome. > > > Cheers, > Don > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and little endian encoding
leimy2k: > I actually need little endian encoding... wondering if anyone else hit this > with Data.Binary. (because I'm working with Bell Lab's 9P protocol which does > encode things on the network in little-endian order). > > Anyone got some "tricks" for this? Yes! There are big, little and host-endian primitives in the Get/Put monads. http://hackage.haskell.org/packages/archive/binary/0.5.0.1/doc/html/Data-Binary-Put.html#v%3AputWord16le You can use these to build encoders directly. Cheers, Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Data.Binary and little endian encoding
I actually need little endian encoding... wondering if anyone else hit this with Data.Binary. (because I'm working with Bell Lab's 9P protocol which does encode things on the network in little-endian order). Anyone got some "tricks" for this? Dave ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
