On Mon, 17 Apr 2000, Stephen C. Tweedie wrote:
> Hi,
>
> On Mon, Apr 17, 2000 at 07:21:31PM +0200, Benno Senoner wrote:
>
> > > The only way you can get much better is to do non-writeback IO
> > > asynchronously. Use O_SYNC for writes, and submit the IOs from multiple
> > > threads, to let
>> 2) Why am I not having any of these problems ? Unlike Benno's code, I
>>have a working application that runs just fine. I get smooth
>>throughput from the disk subsystem too.
>
>What do you mean exactly with "unlike Benno's code" ?
>My code just tries to simulate the operation of a bus
Hi,
On Tue, Apr 18, 2000 at 10:57:25AM -0400, Paul Barton-Davis wrote:
> >> 1) pre-allocation takes a *long* time. Allocating 24 203MB files on a
> >>clean ext2 partition of 18GB takes many, many minutes, for example.
> >>Presumably, the same overhead is being incurred when block
> >>
>> 1) pre-allocation takes a *long* time. Allocating 24 203MB files on a
>>clean ext2 partition of 18GB takes many, many minutes, for example.
>>Presumably, the same overhead is being incurred when block
>>allocation happens "on the fly".
>
>It is not the allocation which is taking age
Hi,
On Mon, Apr 17, 2000 at 07:10:43PM +0200, Martin Schenk wrote:
> If you are interested in a more efficient fsync (and a real fdatasync),
> I have some patches that provide better performance for very large
> files (where fsync is mostly busy scanning the page cache for changes),
> and a fdat
Hi,
On Mon, Apr 17, 2000 at 01:05:12PM -0400, Paul Barton-Davis wrote:
>
> Acknowledging your much greater wisdom in this are than me, I don't
> understand the above given that, in my experience:
>
> 1) pre-allocation takes a *long* time. Allocating 24 203MB files on a
>clean ext2 partition
O_DIRECT on Linux XFS is still a work in progress, we only have
direct reads so far. A very basic implementation was made available
this weekend.
We also have a preallocation interface available via an ioctl call, it should be
fast as XFS is an extent based filesystem, but before direct write is
On Mon, 17 Apr 2000, Paul Barton-Davis wrote:
> Acknowledging your much greater wisdom in this are than me, I don't
> understand the above given that, in my experience:
>
> 1) pre-allocation takes a *long* time. Allocating 24 203MB files on a
>clean ext2 partition of 18GB takes many, many m
Hi,
On Mon, Apr 17, 2000 at 01:45:15PM -0400, Paul Barton-Davis wrote:
> >> 2) Why am I not having any of these problems ? Unlike Benno's code, I
> >>Seagate 4.5GB Cheetah U2W 10K rpm IBM 9GB UltraStar U2W 10K rpm
> >>Quantum 4.5GB Viking U2W 7.5K rpm 3 x IBM 18G
Hi,
On Mon, Apr 17, 2000 at 07:21:31PM +0200, Benno Senoner wrote:
> > The only way you can get much better is to do non-writeback IO
> > asynchronously. Use O_SYNC for writes, and submit the IOs from multiple
> > threads, to let the kernel schedule the multiple IOs. Use large block
> > sizes
>> 2) Why am I not having any of these problems ? Unlike Benno's code, I
>>Seagate 4.5GB Cheetah U2W 10K rpmIBM 9GB UltraStar U2W 10K rpm
>>Quantum 4.5GB Viking U2W 7.5K rpm3 x IBM 18GB UltraStar
>
>Ahh --- SCSI. The request queuing for SCSI is very different to
>that for
>It depends. Using more threads can lead to more conflicting IO seeks
>unless you can schedule enough IOs at once to give the scheduler a good
>chance to sort the IOs into decent-sized blocks.
>
>The objective should probably be to make sure that you have a few hundred
>KB of outstanding IO reque
On Mon, 17 Apr 2000, Stephen C. Tweedie wrote:
>
> Of course not. You are doing sync IO in each case, and are doing it
> from a single thread.
>
> > Again I think O_DIRECT is the only way which will allow us to use smaller
> > buffers, but we will gain almost nothing in terms of total delivere
Hi,
On Mon, Apr 17, 2000 at 01:10:41PM -0400, Paul Barton-Davis wrote:
>
> I had a question about this. Doug Gilbert told me that he heard using
> multiple threads to schedule I/O requests could be a win, and that was
> also my intuition. Other people have claimed that its often not a win,
> and
Hi,
On Mon, Apr 17, 2000 at 01:05:12PM -0400, Paul Barton-Davis wrote:
>
> 2) Why am I not having any of these problems ? Unlike Benno's code, I
>Seagate 4.5GB Cheetah U2W 10K rpm IBM 9GB UltraStar U2W 10K rpm
>Quantum 4.5GB Viking U2W 7.5K rpm 3 x IBM 18GB UltraStar
Ahh
>The only way you can get much better is to do non-writeback IO
>asynchronously. Use O_SYNC for writes, and submit the IOs from multiple
>threads, to let the kernel schedule the multiple IOs. Use large block
>sizes for each IO to prevent massive amounts of disk seeking. O_DIRECT
>in this case
"Stephen C. Tweedie" wrote:
>
> Try opening the file with open(O_SYNC), and write the data in 128k chunks
> or so. Alternatively call fsync() every so often to force the data to
> disk (though fsync is not particularly efficient on large files with
> 2.2).
>
If you are interested in a more eff
>> From: Paul Barton-Davis <[EMAIL PROTECTED]>
>> i mentioned in some remarks to benno how important i thought it was to
>> preallocate the files used for hard disk recording under linux.
>
>Preallocation will make little difference. The real issue is that the
>buffer cache is doing write-behind,
>From: "Stephen C. Tweedie" <[EMAIL PROTECTED]>
>
>Try opening the file with open(O_SYNC), and write the data in 128k chunks
>or so. Alternatively call fsync() every so often to force the data to
>disk (though fsync is not particularly efficient on large files with
>2.2).
With fsync() I coul
Hi,
On Mon, Apr 17, 2000 at 04:50:05PM +0200, Benno Senoner wrote:
>
> Stephen, I tried all possible combinations , in my hdrbench code.
...
> I tried:
> -fsync() on all write descriptors at regular intervals ranging from 1sec to
> 10sec
> - fdatasync() on all write descriptors , same as above
On Mon, 17 Apr 2000, Stephen C. Tweedie wrote:
>
> Preallocation will make little difference. The real issue is that the
> buffer cache is doing write-behind, ie. it is batching up the writes into
> big chunks which get blasted to disk once every five seconds or so,
> causing large IO request qu
Hi,
On Fri, Apr 14, 2000 at 06:15:09PM +1000, Andrew Clausen wrote:
>
> Any comments?
Yes!
> Date: Fri, 14 Apr 2000 08:10:10 -0400
> Message-Id: <[EMAIL PROTECTED]>
> From: Paul Barton-Davis <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Subject: [linux-audio-dev] info point on linux hdr
> Send
Hi,
On Sat, Apr 15, 2000 at 06:50:48PM +0200, Benno Senoner wrote:
>
> Anyway does anyone know if implementing O_DIRECT would be a big amount
> of work in kernel 2.3.x ?
I'll be doing it, and it should be fairly straightforward. There are
one or two infrastructure changes required, however, so
On Sat, 15 Apr 2000, Stephen C. Tweedie wrote:
> Hi,
>
> On Sat, Apr 15, 2000 at 12:32:50PM +0200, Benno Senoner wrote:
> > On Sat, 15 Apr 2000, Paul Barton-Davis wrote:
>
> > > >PS: Does anyone know how to make a RAW I/O device on a spare disk partition,
> > > >and then put an ext2 over it (run
Hi,
On Sat, Apr 15, 2000 at 12:32:50PM +0200, Benno Senoner wrote:
> On Sat, 15 Apr 2000, Paul Barton-Davis wrote:
> > >PS: Does anyone know how to make a RAW I/O device on a spare disk partition,
> > >and then put an ext2 over it (running the whole partition in RAWIO mode) ?
> > >Is that possib
>It would be interesting to compare filesystem latencies in the HDrecording
>case. As said it's amazing how long the disk thread can get blocked during a
>large buffer flush/metadata update, on a PII I saw watched the disk thread
>blocking for several seconds (up 8 secs in the worst case).
>That m
On Sat, 15 Apr 2000, Paul Barton-Davis wrote:
> >PS: Does anyone know how to make a RAW I/O device on a spare disk partition,
> >and then put an ext2 over it (running the whole partition in RAWIO mode) ?
> >Is that possible ?
>
> I don't think so. The point of raw i/o is to provide character
> de
Date:Sat, 15 Apr 2000 01:35:10 +0200
From: Andi Kleen <[EMAIL PROTECTED]>
On Sat, Apr 15, 2000 at 12:24:16AM +0200, Andrew Clausen wrote:
> i mentioned in some remarks to benno how important i thought it was to
> preallocate the files used for hard disk recording under linu
>PS: Does anyone know how to make a RAW I/O device on a spare disk partition,
>and then put an ext2 over it (running the whole partition in RAWIO mode) ?
>Is that possible ?
I don't think so. The point of raw i/o is to provide character
device-like access to a device that would otherwise be a blo
>Unfortunately efficient preallocation is rather hard with the current
>ext2. To do it efficiently you just want to allocate the blocks in the
>bitmaps without writing into the actual allocated blocks (otherwise
>it would be as slow as the manual write-every-block-from-userspace trick)
yes, its s
Andrew Clausen <[EMAIL PROTECTED]> writes:
> Hi all,
>
> Any comments?From: Paul Barton-Davis <[EMAIL PROTECTED]>
> Subject: [linux-audio-dev] info point on linux hdr
> To: [EMAIL PROTECTED]
> Date: Fri Apr 14 07:10:10 2000 -0500
>
> i mentioned in some remarks to benno how important i thought
On Sat, Apr 15, 2000 at 12:24:16AM +0200, Andrew Clausen wrote:
> i mentioned in some remarks to benno how important i thought it was to
> preallocate the files used for hard disk recording under linux.
[...]
Unfortunately efficient preallocation is rather hard with the current
ext2. To do it ef
I can only say that I am not using any form of preallocation and still get
acceptable performance, even if I am forced to use quite big buffers to
overcome to filesystem latencies.
I tried to compare ext2 to FAT, but I do not get visible differences in terms of
buffer usage spikes (= disk thread b
Hi all,
Any comments?
i mentioned in some remarks to benno how important i thought it was to
preallocate the files used for hard disk recording under linux.
i was doing more work on ardour yesterday, and had the occasion to
create some new "tapes", of lengths from 2 to 40 minutes.
the simple
34 matches
Mail list logo
