On Fri, 04 May 2007 11:39:22 +0400 Alex Tomas <[EMAIL PROTECTED]> wrote:
> Andrew Morton wrote:
> > I'm still not understanding. The terms you're using are a bit ambiguous.
> >
> > What does "find some dirty unallocated blocks" mean? Find a page which is
> > dirty and which does not have a disk
Andrew Morton wrote:
I'm still not understanding. The terms you're using are a bit ambiguous.
What does "find some dirty unallocated blocks" mean? Find a page which is
dirty and which does not have a disk mapping?
Normally the above operation would be implemented via
ext4_writeback_writepage(
On Fri, 04 May 2007 10:57:12 +0400 Alex Tomas <[EMAIL PROTECTED]> wrote:
> Andrew Morton wrote:
> > On Fri, 04 May 2007 10:18:12 +0400 Alex Tomas <[EMAIL PROTECTED]> wrote:
> >
> >> Andrew Morton wrote:
> >>> Yes, there can be issues with needing to allocate journal space within the
> >>> context
Andrew Morton wrote:
On Fri, 04 May 2007 10:18:12 +0400 Alex Tomas <[EMAIL PROTECTED]> wrote:
Andrew Morton wrote:
Yes, there can be issues with needing to allocate journal space within the
context of a commit. But
no-no, this isn't required. we only need to mark pages/blocks within
transact
On Fri, 04 May 2007 10:18:12 +0400 Alex Tomas <[EMAIL PROTECTED]> wrote:
> Andrew Morton wrote:
> > Yes, there can be issues with needing to allocate journal space within the
> > context of a commit. But
>
> no-no, this isn't required. we only need to mark pages/blocks within
> transaction, othe
Andrew Morton wrote:
Yes, there can be issues with needing to allocate journal space within the
context of a commit. But
no-no, this isn't required. we only need to mark pages/blocks within
transaction, otherwise race is possible when we allocate blocks in transaction,
then transacton starts t
On Thu, 03 May 2007 21:38:10 +0400
Alex Tomas <[EMAIL PROTECTED]> wrote:
> Andrew Morton wrote:
> > We can make great improvements here, and I've (twice) previously decribed
> > how: hoist the entire ordered-mode data handling out of ext3, and out of
> > the buffer_head layer and move it up into t
Andrew Morton wrote:
We can make great improvements here, and I've (twice) previously decribed
how: hoist the entire ordered-mode data handling out of ext3, and out of
the buffer_head layer and move it up into the VFS pagecache layer.
Basically, do ordered-data with a commit-time inode walk, cal
On Wed, 2007-05-02 at 08:53 +0200, Jens Axboe wrote:
> On Fri, Apr 27 2007, Linus Torvalds wrote:
> > So I do believe that we could probably do something about the IO
> > scheduling _too_:
> >
> > - break up large write requests (yeah, it will make for worse IO
> >throughput, but if make it
On Fri, Apr 27 2007, Linus Torvalds wrote:
> So I do believe that we could probably do something about the IO
> scheduling _too_:
>
> - break up large write requests (yeah, it will make for worse IO
>throughput, but if make it configurable, and especially with
>controllers that don't h
On Sat, Apr 28 2007, Linus Torvalds wrote:
> > The main problem is that if the user extracts tar archive, tar eventually
> > blocks on writeback I/O --- O.K. But if bash attempts to write one page to
> > .bash_history file at the same time, it blocks too --- bad, the user is
> > annoyed.
>
> Right
On Sat, Apr 28 2007, Mikulas Patocka wrote:
> >So perhaps if there's any privileged reads going on then we should limit
> >writes to a depth of 2 at most, with some timeout mechanism that would
>
> SCSI has a "high priority" bit in the command block, so you can just set
> it --- but I am not sure
On Sat, 28 Apr 2007, Lee Revell wrote:
On 4/28/07, Mikulas Patocka <[EMAIL PROTECTED]> wrote:
I most wonder, why vim fsyncs its swapfile regularly (blocking typing
during that) and doesn't fsync the resulting file on :w :-/
Never seen this. Why would fsync block typing unless vim was doing
Lee Revell wrote:
On 4/28/07, Mikulas Patocka <[EMAIL PROTECTED]> wrote:
I most wonder, why vim fsyncs its swapfile regularly (blocking typing
during that) and doesn't fsync the resulting file on :w :-/
Never seen this. Why would fsync block typing unless vim was doing
disk IO for every keys
On Sat, Apr 28, 2007 at 07:37:17AM +0200, Mikulas Patocka wrote:
SpadFS doesn't write to unallocated parts like log filesystems (LFS) or
phase tree filesystems (TUX2); it writes inside normal used structures,
but it marks each structure with generation tags --- when it updates
global table of tag
On Sat, Apr 28, 2007 at 07:37:17AM +0200, Mikulas Patocka wrote:
> SpadFS doesn't write to unallocated parts like log filesystems (LFS) or
> phase tree filesystems (TUX2); it writes inside normal used structures,
> but it marks each structure with generation tags --- when it updates
> global tab
On 4/28/07, Mikulas Patocka <[EMAIL PROTECTED]> wrote:
I most wonder, why vim fsyncs its swapfile regularly (blocking typing
during that) and doesn't fsync the resulting file on :w :-/
Never seen this. Why would fsync block typing unless vim was doing
disk IO for every keystroke?
Lee
-
To un
hm, fsync.
Aside: why the heck do applications think that their data is so important
that they need to fsync it all the time. I used to run a kernel on my
laptop which had "return 0;" at the top of fsync() and fdatasync(). Most
pleasurable.
But wedging for 20 minutes is probably excessive puni
On Sat, 28 Apr 2007, Linus Torvalds wrote:
The main problem is that if the user extracts tar archive, tar eventually
blocks on writeback I/O --- O.K. But if bash attempts to write one page to
.bash_history file at the same time, it blocks too --- bad, the user is
annoyed.
Right, but it's act
So perhaps if there's any privileged reads going on then we should limit
writes to a depth of 2 at most, with some timeout mechanism that would
SCSI has a "high priority" bit in the command block, so you can just set
it --- but I am not sure how well do disks support it.
Mikulas
gradually a
On Sat, 28 Apr 2007 09:30:06 -0700 (PDT)
Linus Torvalds <[EMAIL PROTECTED]> wrote:
> There are worse examples. Try connecting some flash disk over USB-1, and
> untar to it. Ugh.
>
> I'd love to have some per-device dirty limit, but it's harder than it
> should be.
this one should help:
Patch:
* Linus Torvalds <[EMAIL PROTECTED]> wrote:
> Even with a good software IO scheduler, when you have disks that do
> tagged queueing, if you fill up the disk queue with a few dozen
> (depends on the disk what the queue limit is) huge write requests, it
> doesn't really matter if the _software_
On Sat, 28 Apr 2007, Matthias Andree wrote:
>
> Another thing that is rather unpleasant (haven't yet tried fiddling with
> the dirty limits) is UDF to DVD-RAM - try rsyncing /home to a DVD-RAM,
> that's going to leave you with tons of dirty buffers that clear slowly
> -- "watch -n 1 grep -i dirt
On Sat, 28 Apr 2007, Mikulas Patocka wrote:
> >
> > Especially with lots of memory, allowing 40% of that memory to be dirty is
> > just insane (even if we limit it to "just" 40% of the normal memory zone.
> > That can be gigabytes. And no amount of IO scheduling will make it
> > pleasant to try
On Sat, 28 Apr 2007 10:51:48 +0200 Matthias Andree <[EMAIL PROTECTED]> wrote:
> On Fri, 27 Apr 2007, Andrew Morton wrote:
>
> > But none of this explains a 20-minute hang, unless a *lot* of fsyncs are
> > being performed, perhaps.
>
> Another thing that is rather unpleasant (haven't yet tried fi
On Fri, 27 Apr 2007, Andrew Morton wrote:
> But none of this explains a 20-minute hang, unless a *lot* of fsyncs are
> being performed, perhaps.
Another thing that is rather unpleasant (haven't yet tried fiddling with
the dirty limits) is UDF to DVD-RAM - try rsyncing /home to a DVD-RAM,
that's g
On Fri, 27 Apr 2007, Linus Torvalds wrote:
> Oh, well.. Journalling sucks.
>
> I was actually _really_ hoping that somebody would come along and tell
> everybody that this whole journal-logging is stupid, and that it's just
> better to not ever re-write blocks on disk, but instead write to new
On Fri, 27 Apr 2007, Linus Torvalds wrote:
>
>
> On Fri, 27 Apr 2007, Marat Buharov wrote:
> >
> > On 4/27/07, Andrew Morton <[EMAIL PROTECTED]> wrote:
> > > Aside: why the heck do applications think that their data is so important
> > > that they need to fsync it all the time. I used to run a
On Sat, 2007-04-28 at 00:01 -0700, Andrew Morton wrote:
> On Sat, 28 Apr 2007 08:32:32 +0200 Mike Galbraith <[EMAIL PROTECTED]> wrote:
>
> > On Sat, 2007-04-28 at 06:25 +0200, Mike Galbraith wrote:
> > As promised, I tested with a kernel that I know for fact that I have
> > tested heavy IO on pre
On Sat, 28 Apr 2007 08:32:32 +0200 Mike Galbraith <[EMAIL PROTECTED]> wrote:
> On Sat, 2007-04-28 at 06:25 +0200, Mike Galbraith wrote:
> > On Fri, 2007-04-27 at 08:18 -0700, Linus Torvalds wrote:
> >
> > > Actually, you don't need to apply the patch - just do
> > >
> > > echo 5 > /proc/sys/vm
On Sat, 2007-04-28 at 06:25 +0200, Mike Galbraith wrote:
> On Fri, 2007-04-27 at 08:18 -0700, Linus Torvalds wrote:
>
> > Actually, you don't need to apply the patch - just do
> >
> > echo 5 > /proc/sys/vm/dirty_background_ratio
> > echo 10 > /proc/sys/vm/dirty_ratio
>
> That seems to ha
On Fri, 27 Apr 2007, Linus Torvalds wrote:
On Fri, 27 Apr 2007, Mike Galbraith wrote:
As subject states, my GUI is going away for extended periods of time
when my very full and likely highly fragmented (how to find out)
filesystem is under heavy write load. While write is under way, if
am
On Fri, 27 Apr 2007, Bill Huey wrote:
On Fri, Apr 27, 2007 at 12:50:34PM -0700, Linus Torvalds wrote:
Oh, well.. Journalling sucks.
I was actually _really_ hoping that somebody would come along and tell
everybody that this whole journal-logging is stupid, and that it's just
better to not ever
On Sat, 28 Apr 2007, Mikulas Patocka wrote:
On Fri, 27 Apr 2007, Bill Huey wrote:
Hi
SpadFS doesn't write to unallocated parts like log filesystems (LFS) or
phase tree filesystems (TUX2);
--- BTW, I don't think that writing to unallocated parts of disk is good
idea. These filesystems have
On Fri, 2007-04-27 at 08:18 -0700, Linus Torvalds wrote:
> Actually, you don't need to apply the patch - just do
>
> echo 5 > /proc/sys/vm/dirty_background_ratio
> echo 10 > /proc/sys/vm/dirty_ratio
That seems to have done the trick. Amarok and GUI aren't exactly speed
demons while
On Fri, 27 Apr 2007 13:31:30 -0600
Andreas Dilger <[EMAIL PROTECTED]> wrote:
> On Apr 27, 2007 08:30 -0700, Linus Torvalds wrote:
> > On a good filesystem, when you do "fsync()" on a file, nothing at all
> > happens to any other files. On ext3, it seems to sync the global journal,
> > which mea
On Fri, 27 Apr 2007 13:09:06 -0600
Zan Lynx <[EMAIL PROTECTED]> wrote:
> On Fri, 2007-04-27 at 11:31 -0700, Andrew Morton wrote:
> [snip]
> > ext3's problem here is that a single fsync() requires that ext3 sync the
> > whole filesystem. Because
> >
> > - a journal commit can contain metadata fro
On Fri, 27 Apr 2007, Jan Engelhardt wrote:
>
> Interesting. For my laptop, I have configured like 90 for
> dirty_background_ratio and 95 for dirty_ratio. Makes for a nice
> delayed write, but I do not do workloads bigger than extracing kernel
> tarballs (~250 MB) and coding away on that machine
Linus Torvalds wrote:
On Fri, 27 Apr 2007, Andreas Dilger wrote:
It's true that this is a "feature" of ext3 with data=ordered (the default),
but I suspect the same thing is now true in reiserfs too.
Oh, well.. Journalling sucks.
Go back to ext2? ;)
I was actually _really_ hoping that someb
Linus Torvalds wrote:
On Fri, 27 Apr 2007, Andreas Dilger wrote:
It's true that this is a "feature" of ext3 with data=ordered (the default),
but I suspect the same thing is now true in reiserfs too.
Oh, well.. Journalling sucks.
I was actually _really_ hoping that somebody would come
Linus Torvalds wrote:
There was even somebody who did something like that for a PhD thesis, I
forget the details (and it apparently died when the thesis was presumably
accepted ;).
You mean SpadFS[1] right ?
Linus
Gabriel
[1] http://artax.karlin.mff.cuni.c
In article <[EMAIL PROTECTED]> you write:
>I was actually _really_ hoping that somebody would come along and tell
>everybody that this whole journal-logging is stupid, and that it's just
>better to not ever re-write blocks on disk, but instead write to new
>blocks with version numbers (and not r
On Fri, Apr 27, 2007 at 12:50:34PM -0700, Linus Torvalds wrote:
> Oh, well.. Journalling sucks.
>
> I was actually _really_ hoping that somebody would come along and tell
> everybody that this whole journal-logging is stupid, and that it's just
> better to not ever re-write blocks on disk, but i
On Apr 27 2007 08:18, Linus Torvalds wrote:
>
>Actually, you don't need to apply the patch - just do
>
> echo 5 > /proc/sys/vm/dirty_background_ratio
> echo 10 > /proc/sys/vm/dirty_ratio
>
>and say if it seems to improve things. I think those are much saner
>defaults especially for a
ay, April 27, 2007 12:51 PM
> To: Andreas Dilger
> Cc: Marat Buharov; Andrew Morton; Mike Galbraith; LKML; Jens Axboe;
> [EMAIL PROTECTED]; Alex Tomas
> Subject: Re: [ext3][kernels >= 2.6.20.7 at least] KDE going comatose
> when FS is under heavy write load (massive starvation)
On Fri, 27 Apr 2007, Andreas Dilger wrote:
>
> It's true that this is a "feature" of ext3 with data=ordered (the default),
> but I suspect the same thing is now true in reiserfs too.
Oh, well.. Journalling sucks.
I was actually _really_ hoping that somebody would come along and tell
everybody
On Fri, 2007-04-27 at 13:31 -0600, Andreas Dilger wrote:
> I believe
> Alex has a patch to have it checkpoint much smaller chunks to the fs.
I wouldn't be averse to test driving such a patch (understatement). You
have a pointer?
-Mike
-
To unsubscribe from this list: send the line "unsub
Linus Torvalds wrote:
On Fri, 27 Apr 2007, John Anthony Kazos Jr. wrote:
Could[/should] this stuff be changed from ratios to amounts? Or a quick
boot-time test to use a ratio if the memory is small and an amount (like
tax brackets, I would expect) if it's great?
Yes, the "percentage" thing w
On Apr 27, 2007 08:30 -0700, Linus Torvalds wrote:
> On a good filesystem, when you do "fsync()" on a file, nothing at all
> happens to any other files. On ext3, it seems to sync the global journal,
> which means that just about *everything* that writes even a single byte
> (well, at least anyt
On Fri, 2007-04-27 at 11:31 -0700, Andrew Morton wrote:
> But none of this explains a 20-minute hang, unless a *lot* of fsyncs are
> being performed, perhaps.
Yes. I need to do a lot more testing. All I see is one, and it's game
over. Bizarre.
-Mike
-
To unsubscribe from this list: s
On Fri, 2007-04-27 at 08:18 -0700, Linus Torvalds wrote:
> Actually, you don't need to apply the patch - just do
>
> echo 5 > /proc/sys/vm/dirty_background_ratio
> echo 10 > /proc/sys/vm/dirty_ratio
>
I'll try this, and do some testing with other kernels as well.
-Mike
-
T
On Fri, 2007-04-27 at 11:31 -0700, Andrew Morton wrote:
[snip]
> ext3's problem here is that a single fsync() requires that ext3 sync the
> whole filesystem. Because
>
> - a journal commit can contain metadata from multiple files, and if we
> want to journal one file's metadata via fsync(), we
On Fri, 27 Apr 2007 08:18:34 -0700 (PDT) Linus Torvalds <[EMAIL PROTECTED]>
wrote:
> echo 5 > /proc/sys/vm/dirty_background_ratio
> echo 10 > /proc/sys/vm/dirty_ratio
That'll help a lot.
ext3's problem here is that a single fsync() requires that ext3 sync the
whole filesystem. Beca
Linus Torvalds wrote:
>
> On Fri, 27 Apr 2007, John Anthony Kazos Jr. wrote:
>> Could[/should] this stuff be changed from ratios to amounts? Or a quick
>> boot-time test to use a ratio if the memory is small and an amount (like
>> tax brackets, I would expect) if it's great?
>
> Yes, the "perce
On Fri, 27 Apr 2007, John Anthony Kazos Jr. wrote:
>
> Could[/should] this stuff be changed from ratios to amounts? Or a quick
> boot-time test to use a ratio if the memory is small and an amount (like
> tax brackets, I would expect) if it's great?
Yes, the "percentage" thing was likely wrong
> One thing to try out (and dammit, I should make it the default now in
> 2.6.21) is to just make the dirty limits much lower. We've been talking
> about this for ages, I think this might be the right time to do it.
Could[/should] this stuff be changed from ratios to amounts? Or a quick
boot-ti
On Fri, 27 Apr 2007, Marat Buharov wrote:
>
> On 4/27/07, Andrew Morton <[EMAIL PROTECTED]> wrote:
> > Aside: why the heck do applications think that their data is so important
> > that they need to fsync it all the time. I used to run a kernel on my
> > laptop which had "return 0;" at the top o
On Fri, 27 Apr 2007, Mike Galbraith wrote:
>
> As subject states, my GUI is going away for extended periods of time
> when my very full and likely highly fragmented (how to find out)
> filesystem is under heavy write load. While write is under way, if
> amarok (mp3 player) is running, no song c
Peter Zijlstra wrote:
No way is globally disabling fsync() a good thing. I guess Andrew just
is a sucker for punishment :-)
Mmm... perhaps another nice thing to include in laptop-mode operation?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message
Marat Buharov wrote:
On 4/27/07, Andrew Morton <[EMAIL PROTECTED]> wrote:
Aside: why the heck do applications think that their data is so important
that they need to fsync it all the time. I used to run a kernel on my
laptop which had "return 0;" at the top of fsync() and fdatasync(). Most
ple
On Fri, 2007-04-27 at 15:59 +0400, Marat Buharov wrote:
> On 4/27/07, Andrew Morton <[EMAIL PROTECTED]> wrote:
> > Aside: why the heck do applications think that their data is so important
> > that they need to fsync it all the time. I used to run a kernel on my
> > laptop which had "return 0;" at
On 4/27/07, Andrew Morton <[EMAIL PROTECTED]> wrote:
Aside: why the heck do applications think that their data is so important
that they need to fsync it all the time. I used to run a kernel on my
laptop which had "return 0;" at the top of fsync() and fdatasync(). Most
pleasurable.
So, if hav
On Fri, 2007-04-27 at 01:33 -0700, Andrew Morton wrote:
> Another livelock possibility is that bonnie is redirtying pages faster than
> commit can write them out, so commit got livelocked:
>
> When I was doing the original port-from-2.2 I found that an application
> which does
>
> for ( ;
On Fri, 2007-04-27 at 01:33 -0700, Andrew Morton wrote:
> On Fri, 27 Apr 2007 09:59:27 +0200 Mike Galbraith <[EMAIL PROTECTED]> wrote:
>
> > Greetings,
> >
> > As subject states, my GUI is going away for extended periods of time
> > when my very full and likely highly fragmented (how to find out)
On Fri, 27 Apr 2007 09:59:27 +0200 Mike Galbraith <[EMAIL PROTECTED]> wrote:
> Greetings,
>
> As subject states, my GUI is going away for extended periods of time
> when my very full and likely highly fragmented (how to find out)
> filesystem is under heavy write load. While write is under way,
Greetings,
As subject states, my GUI is going away for extended periods of time
when my very full and likely highly fragmented (how to find out)
filesystem is under heavy write load. While write is under way, if
amarok (mp3 player) is running, no song change will occur until write is
finished, an
66 matches
Mail list logo
