RE: I/O overhead in opening and writing files

2012-08-28 Thread J Baptist
Even with Text.Show.ByteString, which is supposed to avoid unnecessary string conversions, performance does not improve. Date: Mon, 27 Aug 2012 20:05:19 -0700 Subject: Re: I/O overhead in opening and writing files From: b...@serpentine.com To: arc38...@hotmail.com CC: johan.tib...@gmail.com

Re: I/O overhead in opening and writing files

2012-08-27 Thread wren ng thornton
On 8/27/12 6:25 PM, J Baptist wrote: Using ByteStrings and the C calls does indeed speed things up a bit, but not much. real0m6.053suser0m1.480ssys 0m4.550s For your interest:The original version (with Strings and openFile): http://hpaste.org/73803Faster (with Strings and c_open):

Re: I/O overhead in opening and writing files

2012-08-27 Thread wren ng thornton
On 8/27/12 5:33 PM, Donn Cave wrote: It does, but it can also read directly to Ptr Word8 (fdReadBuf), which you'd think would be closer to hardware speed - but then you might lose the advantage trying to peek the data out of the buffer. In principle you ought to be able to stuff that pointer rig

Re: I/O overhead in opening and writing files

2012-08-27 Thread Bryan O'Sullivan
On Mon, Aug 27, 2012 at 3:25 PM, J Baptist wrote: > The problem may be that even with ByteStrings, we are stuck using show, > and thus Strings, at some point. > Wait, what are you actually trying to do? If you have a benchmark that's half sane and half bonkers (cf. use of show), then yeah, it's

RE: I/O overhead in opening and writing files

2012-08-27 Thread J Baptist
> From: felipe.le...@gmail.com > Do these timings include RTS startup? Yes, this is the result of the time command on the whole executable. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org htt

Re: I/O overhead in opening and writing files

2012-08-27 Thread Felipe Almeida Lessa
On Mon, Aug 27, 2012 at 7:25 PM, J Baptist wrote: > real 0m6.053s > user 0m1.480s > sys 0m4.550s Do these timings include RTS startup? -- Felipe. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/

Re: I/O overhead in opening and writing files

2012-08-27 Thread Don Stewart
73801 > > The problem may be that even with ByteStrings, we are stuck using show, > and thus Strings, at some point. > > Ideas? > > > > From: johan.tib...@gmail.com 'johan.tib...@gmail.com');> > > Date: Mon, 27 Aug 2012 13:48:27 -0700 > > S

RE: I/O overhead in opening and writing files

2012-08-27 Thread J Baptist
(with ByteStrings and c_open): http://hpaste.org/73801 The problem may be that even with ByteStrings, we are stuck using show, and thus Strings, at some point. Ideas? > From: johan.tib...@gmail.com > Date: Mon, 27 Aug 2012 13:48:27 -0700 > Subject: Re: I/O overhead in opening and writing f

Re: I/O overhead in opening and writing files

2012-08-27 Thread Donn Cave
Quoth Johan Tibell , ... > Could you try using the Data.ByteString API. I don't have the code in > front of me so I don't know if the System.Posix API uses Strings. If > it does, that's most likely the issue. It does, but it can also read directly to Ptr Word8 (fdReadBuf), which you'd think would

Re: I/O overhead in opening and writing files

2012-08-27 Thread Austin Seipp
In this vein, you may be interested in trying out the unix-bytestring package (it contains ByteString based bindings for POSIX I/O - but you'll still need the unix package to get at the underlying file descriptor.) http://hackage.haskell.org/packages/archive/unix-bytestring/0.3.5.4/doc/html/System

Re: I/O overhead in opening and writing files

2012-08-27 Thread Johan Tibell
On Mon, Aug 27, 2012 at 1:43 PM, J Baptist wrote: > I'm looking into high-performance I/O, particularly on a tmpfs (in-memory) > filesystem. This involves creating lots of little files. Unfortunately, it > seems that Haskell's performance in this area is not comparable to that of > C. I assume tha

I/O overhead in opening and writing files

2012-08-27 Thread J Baptist
I'm looking into high-performance I/O, particularly on a tmpfs (in-memory) filesystem. This involves creating lots of little files. Unfortunately, it seems that Haskell's performance in this area is not comparable to that of C. I assume that this is because of the overhead involved in opening a