On Wed, 7 Mar 2001, Stephen C. Tweedie wrote:
> Hi,
>
> On Tue, Mar 06, 2001 at 10:44:34AM -0800, Linus Torvalds wrote:
>
> > On Tue, 6 Mar 2001, Alan Cox wrote:
> > > You want a write barrier. Write buffering (at least for short intervals) in
> > > the drive is very sensible. The kernel needs
Hi,
Jens Axboe:
> > But most disks these days support IDE-SCSI, and SCSI does have ordered
> > tags, so...
>
> Any proof to back this up? To my knowledge, only some WDC ATA disks
> can be ATAPI driven.
>
Ummm, no, but that was my impression. If that's wrong, I apologize and
will state the oppos
On Fri, Mar 09 2001, Matthias Urlichs wrote:
> Matthias Urlichs:
> > On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> > > SCSI certainly lets us do both of these operations independently. IDE
> > > has the sync/flush command afaik, but I'm not sure whether the IDE
> > > tagged command stuff has t
>> It's pretty clear that the IDE drive(r) is *not* waiting for the physical
>> write to take place before returning control to the user program, whereas
>> the SCSI drive(r) is.
>
>This would not be unexpected.
>
>IDE drives generally always do write buffering. I don't even know if you
>_can_ tur
Hi,
Matthias Urlichs:
> On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> > SCSI certainly lets us do both of these operations independently. IDE
> > has the sync/flush command afaik, but I'm not sure whether the IDE
> > tagged command stuff has the equivalent of SCSI's ordered tag bits.
> > Andr
On Wednesday, March 07, 2001 08:56:59 PM + "Stephen C. Tweedie"
<[EMAIL PROTECTED]> wrote:
> Hi,
>
> On Wed, Mar 07, 2001 at 09:15:36PM +0100, Jens Axboe wrote:
>> On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
>> >
>> > For most fs'es, that's not an issue. The fs won't start writeback o
Hi,
On Wed, Mar 07, 2001 at 10:36:38AM -0800, Linus Torvalds wrote:
> On Wed, 7 Mar 2001, Jeremy Hansen wrote:
> >
> > So in the meantime as this gets worked out on a lower level, we've decided
> > to take the fsync() out of berkeley db for mysql transaction logs and
> > mount the filesystem -o
Hi!
> If not, then the drive could by all means optimise the access pattern
> provided it acked the data or provided the results in the same order as the
> instructions were given. This would probably shorten the time for a new
> pathological set (distributed evenly across the disk surface, but
On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> On Wed, Mar 07, 2001 at 09:15:36PM +0100, Jens Axboe wrote:
> > On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> > >
> > > For most fs'es, that's not an issue. The fs won't start writeback on
> > > the primary disk at all until the journal commit
Hi,
On Wed, Mar 07, 2001 at 09:15:36PM +0100, Jens Axboe wrote:
> On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> >
> > For most fs'es, that's not an issue. The fs won't start writeback on
> > the primary disk at all until the journal commit has been acknowledged
> > as firm on disk.
>
> But
On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> On Wed, Mar 07, 2001 at 07:51:52PM +0100, Jens Axboe wrote:
> > On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> >
> > My bigger concern is when the journalled fs has a log on a different
> > queue.
>
> For most fs'es, that's not an issue. The fs
Hi,
On Wed, Mar 07, 2001 at 07:51:52PM +0100, Jens Axboe wrote:
> On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
>
> My bigger concern is when the journalled fs has a log on a different
> queue.
For most fs'es, that's not an issue. The fs won't start writeback on
the primary disk at all until
On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> > Yep, it's much harder than it seems. Especially because for the barrier
> > to be really useful, having inter-request dependencies becomes a
> > requirement. So you can say something like 'flush X and Y, but don't
> > flush Y before X is done'.
>
Hi,
On Wed, Mar 07, 2001 at 03:12:41PM +0100, Jens Axboe wrote:
>
> Yep, it's much harder than it seems. Especially because for the barrier
> to be really useful, having inter-request dependencies becomes a
> requirement. So you can say something like 'flush X and Y, but don't
> flush Y before X
On Wed, 7 Mar 2001, Jeremy Hansen wrote:
>
> So in the meantime as this gets worked out on a lower level, we've decided
> to take the fsync() out of berkeley db for mysql transaction logs and
> mount the filesystem -o sync.
>
> Can anyone perhaps tell me why this may be a bad idea?
Two reason
age
> > FAX 321-676-2355
> > ----- Original Message -
> > From: "Andre Hedrick" <[EMAIL PROTECTED]>
> > To: "Linus Torvalds" <[EMAIL PROTECTED]>
> > Cc: "Douglas Gilbert" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]
On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> SCSI certainly lets us do both of these operations independently. IDE
> has the sync/flush command afaik, but I'm not sure whether the IDE
> tagged command stuff has the equivalent of SCSI's ordered tag bits.
> Andre?
IDE has no concept of ordered
On Wed, Mar 07 2001, Stephen C. Tweedie wrote:
> > SCSI has ordered tag, which fit the model Alan described quite nicely.
> > I've been meaning to implement this for some time, it would be handy
> > for journalled fs to use such a barrier. Since ATA doesn't do queueing
> > (at least not in current
Hi,
On Tue, Mar 06, 2001 at 09:37:20PM +0100, Jens Axboe wrote:
>
> SCSI has ordered tag, which fit the model Alan described quite nicely.
> I've been meaning to implement this for some time, it would be handy
> for journalled fs to use such a barrier. Since ATA doesn't do queueing
> (at least n
Hi,
On Tue, Mar 06, 2001 at 10:44:34AM -0800, Linus Torvalds wrote:
> On Tue, 6 Mar 2001, Alan Cox wrote:
> > You want a write barrier. Write buffering (at least for short intervals) in
> > the drive is very sensible. The kernel needs to able to send drivers a write
> > barrier which will not be
Andre Hedrick ([EMAIL PROTECTED]) wrote on Wed Mar 07 2001 - 01:58:44 EST :
> On Wed, 7 Mar 2001, Jonathan Morton wrote:
[ snip ]
> > >Since all OSes that enable WC at init will flush
> > >it at shutdown and do a periodic purge with in-activity.
> >
> > But Linux doesn't, as has been po
On Wed, 7 Mar 2001, Jonathan Morton wrote:
> Still doesn't make a difference - there is one revolution between writes,
> no matter where on disk it is.
Oh it does, because you are hitting the same sector with the same data.
Rotate your buffer and then you will see the difference.
> >Because of
>I am not going to bite on your flame bate, and are free to waste you money.
I don't flamebait. I was trying to clear up some confusion...
>No, SCSI does with queuing.
>I am saying that the ata/ide driver rips the heart out of the
>io_request_lock what to darn long. This means that upon execut
> itself is a bad thing, particularly given the amount of CPU overhead that
> IDE drives demand while attached to the controller (orders of magnitude
> higher than a good SCSI controller) - the more overhead we can hand off to
I know this is just a troll by a scsi-believer, but I'm biting anyway.
On Tue, Mar 06 2001, David Balazic wrote:
> > > Wrong model
> > >
> > > You want a write barrier. Write buffering (at least for short intervals)
> > > in the drive is very sensible. The kernel needs to able to send
> > > drivers a write barrier which will not be completed with outstanding
> > >
Linus Torvalds himself wrote :
> On Tue, 6 Mar 2001, Alan Cox wrote:
> >
> > > > I don't know if there is any way to turn of a write buffer on an IDE disk.
> > > You want a forced set of commands to kill caching at init?
> >
> > Wrong model
> >
> > You want a write barrier. Write buffering
Jonathan,
I am not going to bite on your flame bate, and are free to waste you money.
On Tue, 6 Mar 2001, Jonathan Morton wrote:
> >> It's pretty clear that the IDE drive(r) is *not* waiting for the physical
> >> write to take place before returning control to the user program, whereas
> >> th
On Tue, 6 Mar 2001, Alan Cox wrote:
>
> > > I don't know if there is any way to turn of a write buffer on an IDE disk.
> > You want a forced set of commands to kill caching at init?
>
> Wrong model
>
> You want a write barrier. Write buffering (at least for short intervals) in
> the drive is v
>Jonathan Morton ([EMAIL PROTECTED]) wrote :
>
>> The OS needs to know the physical act of writing data has finished
>>before
>> it tells the m/board to cut the power - period. Pathological data sets
>> included - they are the worst case which every engineer must take into
>> account. Out of inter
On Tue, Mar 06, 2001 at 06:14:15PM +0100, David Balazic wrote:
[snip]
> Hardware Level caching is only good for OSes which have broken
> drivers and broken caching (like plain old DOS).
>
> Linux does a good job in caching and cache control at software
> level.
Read caching, yes. But for writes,
(( please CC me , not subscribed , [EMAIL PROTECTED] )
Jonathan Morton ([EMAIL PROTECTED]) wrote :
> The OS needs to know the physical act of writing data has finished before
> it tells the m/board to cut the power - period. Pathological data sets
> included - they are the worst case
>On Tue, 6 Mar 2001, Mike Black wrote:
>
>> Write caching is the culprit for the performance diff:
Indeed, and my during-the-boring-lecture benchmark on my 18Gb IBM
TravelStar bears this out. I was confused earlier by the fact that one of
my Seagate drives blatently ignores the no-write-caching
ience Innovations
> http://www.csihq.com/~mike My home page
> FAX 321-676-2355
> - Original Message -
> From: "Andre Hedrick" <[EMAIL PROTECTED]>
> To: "Linus Torvalds" <[EMAIL PROTECTED]>
> Cc: "Douglas Gilbert" <[EMAIL PROT
>> Pathological shutdown pattern: assuming scatter-gather is not allowed (for
>> IDE), and a 20ms full-stroke seek, write sectors at alternately opposite
>> ends of the disk, working inwards until the buffer is full. 512-byte
>> sectors, 2MB of them, is 4000 writes * 20ms = around 80 seconds (no
ot; <[EMAIL PROTECTED]>
Cc: "Douglas Gilbert" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Sent: Tuesday, March 06, 2001 2:12 AM
Subject: Re: scsi vs ide performance on fsync's
On Mon, 5 Mar 2001, Linus Torvalds wrote:
> Well, it's fairly hard for the kernel t
>> i assume you meant to time the xlog.c program? (or did i miss another
>> program on the thread?)
Yes.
>> i've an IBM-DJSA-210 (travelstar 10GB, 5411rpm) which appears to do
>> *something* with the write cache flag -- it gets 0.10s elapsed real time
>> in default config; and gets 2.91s if i d
On Tue, 6 Mar 2001, Jonathan Morton wrote:
> Pathological shutdown pattern: assuming scatter-gather is not allowed (for
> IDE), and a 20ms full-stroke seek, write sectors at alternately opposite
> ends of the disk, working inwards until the buffer is full. 512-byte
> sectors, 2MB of them, is 40
On Tue, 6 Mar 2001, dean gaudet wrote:
> i assume you meant to time the xlog.c program? (or did i miss another
> program on the thread?)
>
> i've an IBM-DJSA-210 (travelstar 10GB, 5411rpm) which appears to do
> *something* with the write cache flag -- it gets 0.10s elapsed real time
> in default
On Tue, 6 Mar 2001, Jonathan Morton wrote:
> Pathological shutdown pattern: assuming scatter-gather is not allowed
> (for IDE), and a 20ms full-stroke seek, write sectors at alternately
> opposite ends of the disk, working inwards until the buffer is full.
> 512-byte sectors, 2MB of them, is 400
> > I don't know if there is any way to turn of a write buffer on an IDE disk.
> You want a forced set of commands to kill caching at init?
Wrong model
You want a write barrier. Write buffering (at least for short intervals) in
the drive is very sensible. The kernel needs to able to send drivers
>> It's pretty clear that the IDE drive(r) is *not* waiting for the physical
>> write to take place before returning control to the user program, whereas
>> the SCSI drive(r) is. Both devices appear to be performing the write
>
>Wrong, IDE does not unplug thus the request is almost, I hate to adm
On Mon, 5 Mar 2001, Linus Torvalds wrote:
> Well, it's fairly hard for the kernel to do much about that - it's almost
> certainly just IDE doing write buffering on the disk itself. No OS
> involved.
I am pushing for WC to be defaulted in the off state, but as you know I
have a bigger fight than
On Tue, 6 Mar 2001, Jonathan Morton wrote:
> It's pretty clear that the IDE drive(r) is *not* waiting for the physical
> write to take place before returning control to the user program, whereas
> the SCSI drive(r) is. Both devices appear to be performing the write
Wrong, IDE does not unplug th
>I don't know if there is any way to turn of a write buffer on an IDE disk.
hdparm has an option of this nature, but it makes no difference (as I
reported). It's worth noting that even turning off UDMA to the disk on my
machine doesn't help the situation - although it does slow things down a
lit
On Tue, 6 Mar 2001, Douglas Gilbert wrote:
>
> > On the other hand, it's also entirely possible that IDE is just a lot
> > better than what the SCSI-bigots tend to claim. It's not all that
> > surprising, considering that the PC industry has pushed untold billions of
> > dollars into improving
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Linus Torvalds wrote:
> Well, it's entirely possible that the mid-level SCSI layer is doing
> something horribly stupid.
Well it's in good company as FreeBSD 4.2 on the same hardware
returns the same result (inc
On Tue, 6 Mar 2001, Jonathan Morton wrote:
>
> It's pretty clear that the IDE drive(r) is *not* waiting for the physical
> write to take place before returning control to the user program, whereas
> the SCSI drive(r) is.
This would not be unexpected.
IDE drives generally always do write buffe
I've run the test on my own system and noted something interesting about
the results:
When the write() call extended the file (rather than just overwriting a
section of a file already long enough), the performance drop was seen, and
it was slower on SCSI than IDE - this is independent of whether
On Mon, 5 Mar 2001, Jeremy Hansen wrote:
>
> Right now I'm running 2.4.2-ac11 on both machines and getting the same
> results:
>
> SCSI:
>
> [root@orville /root]# time /root/xlog file.out fsync
>
> real0m21.266s
> user0m0.000s
> sys 0m0.310s
>
> IDE:
>
> [root@kahlbi /root]# ti
On 2 Mar 2001, Linus Torvalds wrote:
> In article <[EMAIL PROTECTED]>,
> Jeremy Hansen <[EMAIL PROTECTED]> wrote:
> >
> >The SCSI adapter on the raid array is an Adaptec 39160, the raid
> >controller is a CMD-7040. Kernel 2.4.0 using XFS for the filesystem on
> >the raid array, kernel 2.2.18 on
Since the intention of fsync and fdatasync seems to be
to write dirty fs buffers to persistent storage (i.e.
the "oxide") then the best time is not necessarily
the objective. Given the IDE times that people have
been reporting, it is very unlikely that any of those
IDE disks were really doing 200
Chris Mason <[EMAIL PROTECTED]> writes:
> filemap_fdatawait, filemap_fdatasync, and fsync_inode_buffers all restrict
> their scans to a list of dirty buffers for that specific file. Only
> file_fsync goes through all the dirty buffers on the device, and the ext2
> fsync path never calls file_fs
Douglas Gilbert wrote:
> There is definitely something strange going on here.
> As the bonnie test below shows, the SCSI disk used
> for my tests should vastly outperform the old IDE one:
First thank you and others with my clueless investigation about
the module loading under Debian GNU/Linux. (I
TECTED] 321-676-2923,x203
> http://www.csihq.com Computer Science Innovations
> http://www.csihq.com/~mike My home page
> FAX 321-676-2355
> - Original Message -
> From: "Jeremy Hansen" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Cc: <[EMA
On Fri, 2 Mar 2001, Chris Mason wrote:
> For why ide is beating scsi in this benchmark...make sure tagged queueing
> is on (or increase the queue length?). For the xlog.c test posted, I would
> expect scsi to get faster than ide as the size of the write increases.
I have seen that many drives ei
In article <[EMAIL PROTECTED]>,
Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>
>The SCSI adapter on the raid array is an Adaptec 39160, the raid
>controller is a CMD-7040. Kernel 2.4.0 using XFS for the filesystem on
>the raid array, kernel 2.2.18 on ext2 on the IDE drive. The filesystem is
>not th
>
>
> On Friday, March 02, 2001 01:25:25 PM -0600 Steve Lord <[EMAIL PROTECTED]> wrote:
>
> >> For why ide is beating scsi in this benchmark...make sure tagged queueing
> >> is on (or increase the queue length?). For the xlog.c test posted, I
> >> would expect scsi to get faster than ide as th
On Friday, March 02, 2001 01:25:25 PM -0600 Steve Lord <[EMAIL PROTECTED]> wrote:
>> For why ide is beating scsi in this benchmark...make sure tagged queueing
>> is on (or increase the queue length?). For the xlog.c test posted, I
>> would expect scsi to get faster than ide as the size of the
Okay I now have to create TCQ for ATA becasue I am not going to lose again
now that I am winning ;-)
On Fri, 2 Mar 2001, Chris Mason wrote:
>
>
> On Friday, March 02, 2001 12:39:01 PM -0600 Steve Lord <[EMAIL PROTECTED]> wrote:
>
> [ file_fsync syncs all dirty buffers on the FS ]
> >
> > So
On Fri, 2 Mar 2001, Steve Lord wrote:
> >
> >
> > On Friday, March 02, 2001 12:39:01 PM -0600 Steve Lord <[EMAIL PROTECTED]> wrote:
> >
> > [ file_fsync syncs all dirty buffers on the FS ]
> > >
> > > So it looks like fsync is going to cost more for bigger devices. Given the
> > > O_SYNC changes
>
>
> On Friday, March 02, 2001 12:39:01 PM -0600 Steve Lord <[EMAIL PROTECTED]> wrote:
>
> [ file_fsync syncs all dirty buffers on the FS ]
> >
> > So it looks like fsync is going to cost more for bigger devices. Given the
> > O_SYNC changes Stephen Tweedie did, couldnt fsync look more like t
On Friday, March 02, 2001 12:39:01 PM -0600 Steve Lord <[EMAIL PROTECTED]> wrote:
[ file_fsync syncs all dirty buffers on the FS ]
>
> So it looks like fsync is going to cost more for bigger devices. Given the
> O_SYNC changes Stephen Tweedie did, couldnt fsync look more like this:
>
>
>
>
> We're doing some mysql benchmarking. For some reason it seems that ide
> drives are currently beating a scsi raid array and it seems to be related
> to fsync's. Bonnie stats show the scsi array to blow away ide as
> expected, but mysql tests still have the idea beating on plain insert
>
We're doing some mysql benchmarking. For some reason it seems that ide
drives are currently beating a scsi raid array and it seems to be related
to fsync's. Bonnie stats show the scsi array to blow away ide as
expected, but mysql tests still have the idea beating on plain insert
speeds. Can an
64 matches
Mail list logo
