Re[3]: [Haskell-cafe] Preferred way to get data from a socket
Hello Bulat, Wednesday, June 27, 2007, 3:34:02 PM, you wrote: >> What machine did you do the IO benchmarks on? Since we get well over 10x >> that speed word writing in Data.Binary now, for example, on a fast >> machine. (Duncan, what's the max throughput we've seen?) > test box was Duron 1.2 GHz with 230 mb/s raw memory copy throughput. oh, i've said about ByteString i/o speed because it was what Chad asked. but actually he will probably need just to (de)serialize data and best speeds here i've seen is 40-50 mb/s on my box. when going to (de)serialization my lib reads/writes each byte individually and it's the reason why it should be slower than Data.Binary -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] Preferred way to get data from a socket
Hello Donald, Wednesday, June 27, 2007, 11:40:28 AM, you wrote: >> using my library should allow 30-50 mb/s i/o speed but its >> installation may be tricky since it was not updated over a year > That's interesting, Bulat. Two points I'd like to ask about the streams > library: > What machine did you do the IO benchmarks on? Since we get well over 10x > that speed word writing in Data.Binary now, for example, on a fast > machine. (Duncan, what's the max throughput we've seen?) test box was Duron 1.2 GHz with 230 mb/s raw memory copy throughput. reading 20-byte lines: 50 mb/s reading 80-byte lines: 70 mb/s > And secondly, will the streams stuff be updated? I had trouble > benchmarking against it back in January, and am wondering if it will be > maintained? afaik, problem is ArrayRef library which is used in Streams but not compatible with ghc 6.6. i will look into it if it will be really required for someone -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Preferred way to get data from a socket
bulat.ziganshin: > Hello Donald, > > Wednesday, June 27, 2007, 6:37:07 AM, you wrote: > > >> I also know Bulat Ziganshin had put together a nice-looking Streams > >> library (http://unix.freshmeat.net/projects/streams/) based on John > >> Goerzen's previous HVIO work, but I wasn't sure if the ByteString > >> stuff matches the speed and encapsulates all of the functionality of > >> that anyway. Or can/should they be used together somehow? > > > Should be similar in speed, and most high-perf stuff seems to use > > ByteStrings now. ByteStrings also have some nice high level > > optimisations not available to lower level libraries. > > i recommend you to try ByteString own i/o capabilities at first. if it > will not satisfy you, you can try Streams 0.2 which supports > ByteString i/o: > > http://www.haskell.org/library/StreamsBeta.tar.gz > > using my library should allow 30-50 mb/s i/o speed but its > installation may be tricky since it was not updated over a year That's interesting, Bulat. Two points I'd like to ask about the streams library: What machine did you do the IO benchmarks on? Since we get well over 10x that speed word writing in Data.Binary now, for example, on a fast machine. (Duncan, what's the max throughput we've seen?) And secondly, will the streams stuff be updated? I had trouble benchmarking against it back in January, and am wondering if it will be maintained? -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] Preferred way to get data from a socket
Hello Donald, Wednesday, June 27, 2007, 6:37:07 AM, you wrote: >> I also know Bulat Ziganshin had put together a nice-looking Streams >> library (http://unix.freshmeat.net/projects/streams/) based on John >> Goerzen's previous HVIO work, but I wasn't sure if the ByteString >> stuff matches the speed and encapsulates all of the functionality of >> that anyway. Or can/should they be used together somehow? > Should be similar in speed, and most high-perf stuff seems to use > ByteStrings now. ByteStrings also have some nice high level > optimisations not available to lower level libraries. i recommend you to try ByteString own i/o capabilities at first. if it will not satisfy you, you can try Streams 0.2 which supports ByteString i/o: http://www.haskell.org/library/StreamsBeta.tar.gz using my library should allow 30-50 mb/s i/o speed but its installation may be tricky since it was not updated over a year -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Preferred way to get data from a socket
chad.scherrer: > Ok, cool. FWIW, the current documentation for Network says: > > "For really fast I/O, it might be worth looking at the hGetBuf and > hPutBuf family of functions in System.IO." > > But this looked pretty low-level to me, and I figured it might be outdated. > > I also know Bulat Ziganshin had put together a nice-looking Streams > library (http://unix.freshmeat.net/projects/streams/) based on John > Goerzen's previous HVIO work, but I wasn't sure if the ByteString > stuff matches the speed and encapsulates all of the functionality of > that anyway. Or can/should they be used together somehow? Should be similar in speed, and most high-perf stuff seems to use ByteStrings now. ByteStrings also have some nice high level optimisations not available to lower level libraries. > > Chad > > > >> --8<--- > >> > >> import Network > >> import qualified Data.ByteString.Lazy as B > >> > >> hostName = "myComputer" > >> portID = PortNumber 54321 > >> > >> theData :: IO B.ByteString > >> theData = connectTo hostName portID >>= B.hGetContents > >> ___ > > > >Looks like the obvious, right way to me. > > > >-- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Preferred way to get data from a socket
Ok, cool. FWIW, the current documentation for Network says: "For really fast I/O, it might be worth looking at the hGetBuf and hPutBuf family of functions in System.IO." But this looked pretty low-level to me, and I figured it might be outdated. I also know Bulat Ziganshin had put together a nice-looking Streams library (http://unix.freshmeat.net/projects/streams/) based on John Goerzen's previous HVIO work, but I wasn't sure if the ByteString stuff matches the speed and encapsulates all of the functionality of that anyway. Or can/should they be used together somehow? Chad > --8<--- > > import Network > import qualified Data.ByteString.Lazy as B > > hostName = "myComputer" > portID = PortNumber 54321 > > theData :: IO B.ByteString > theData = connectTo hostName portID >>= B.hGetContents > ___ Looks like the obvious, right way to me. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Preferred way to get data from a socket
chad.scherrer: > I've never used sockets before, but I need to now, and I need to be > able to get a lot of data quickly. I was thinking about doing > something like this (below), but I'm wondering if there's a way that > would be faster. Is the obvious way of doing this "the right way"? I'm > happy to install outside libraries if that would help, as long as they > work on Linux and MS. Thanks! > > -Chad > > --8<--- > > import Network > import qualified Data.ByteString.Lazy as B > > hostName = "myComputer" > portID = PortNumber 54321 > > theData :: IO B.ByteString > theData = connectTo hostName portID >>= B.hGetContents > ___ Looks like the obvious, right way to me. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Preferred way to get data from a socket
I've never used sockets before, but I need to now, and I need to be able to get a lot of data quickly. I was thinking about doing something like this (below), but I'm wondering if there's a way that would be faster. Is the obvious way of doing this "the right way"? I'm happy to install outside libraries if that would help, as long as they work on Linux and MS. Thanks! -Chad --8<--- import Network import qualified Data.ByteString.Lazy as B hostName = "myComputer" portID = PortNumber 54321 theData :: IO B.ByteString theData = connectTo hostName portID >>= B.hGetContents ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
