Re: Ingo's RAID patch for 2.2.18 final?

[Andrea Arcangeli]

On Fri, Feb 16, 2001 at 10:53:51AM -0500, David Mansfield wrote:
> This may be a bit OT, but when you say O_DIRECT, that implies that you
> can pass that flag to open(2) and it will bypass the page cache, and

yes.

> read directly into user-space buffers (zero-copy IO)?  Does this also

yes.

> bypass the read-ahead mechanisms in the kernel?  Does it imply O_SYNC?

yes.

It's rawio through the fs. It's not included into 2.4-latest yet.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[David Mansfield]

Andrea Arcangeli wrote:
> 
> On Wed, Jan 24, 2001 at 04:05:12PM +0100, Sasi Peter wrote:
> > > This isn't obvious. Your working may not fit in cache and so the kernel
> > > understand it's worthless to swapout stuff to make space to a polluted
> > > cache.
> >
> > But your understanding agrees on that the larger chunks for each stream
> > we read into cache, the more efficient for this kind of RAID disk
> > structure the read is, thus basically the more cache we have, the more
> > bandwidth we can serve. (disks give more data in the same time with
> > fewer long reads than with several shorter ones)
> 
> The size of the I/O requests doesn't grow linearly with with the size of the
> cache, as far as you have some mbyte of cache you will also be able to sumbit
> full sized requests to disk (512K per req on 2.4). In your workload you just
> had enough memory for the readahead.
> 
> In general if your working set doesn't fit in cache, the size of the cache is
> unrelated to the bandwith you get out of your RAID, infact if your working set
> doesn't fit in cache you should not pass through the cache at all to get the
> best performance and to save CPU cycles and L1 dcache and L2 cache (O_DIRECT).
> 

This may be a bit OT, but when you say O_DIRECT, that implies that you
can pass that flag to open(2) and it will bypass the page cache, and
read directly into user-space buffers (zero-copy IO)?  Does this also
bypass the read-ahead mechanisms in the kernel?  Does it imply O_SYNC?

Lots of questions... no answers.  Sigh.

David

-- 
David Mansfield   (718) 963-2020
[EMAIL PROTECTED]
Ultramaster Group, LLC   www.ultramaster.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Andrea Arcangeli]

On Wed, Jan 24, 2001 at 04:05:12PM +0100, Sasi Peter wrote:
> > This isn't obvious. Your working may not fit in cache and so the kernel
> > understand it's worthless to swapout stuff to make space to a polluted
> > cache.
> 
> But your understanding agrees on that the larger chunks for each stream 
> we read into cache, the more efficient for this kind of RAID disk 
> structure the read is, thus basically the more cache we have, the more 
> bandwidth we can serve. (disks give more data in the same time with 
> fewer long reads than with several shorter ones)

The size of the I/O requests doesn't grow linearly with with the size of the
cache, as far as you have some mbyte of cache you will also be able to sumbit
full sized requests to disk (512K per req on 2.4). In your workload you just
had enough memory for the readahead.

In general if your working set doesn't fit in cache, the size of the cache is
unrelated to the bandwith you get out of your RAID, infact if your working set
doesn't fit in cache you should not pass through the cache at all to get the
best performance and to save CPU cycles and L1 dcache and L2 cache (O_DIRECT).

> So might it have been an accidental behaviour of the previous kernels 
> to swap out pages in favor of caching under high I/O pressure, but it 
> was certainly a benefical behaviour.

I trust you this is the case for your workload, but make sure to not assume
that because there's less cache and no swap you're running slower, you may be
running _much_ faster instead ;).

> What should I test with? (2.4.0/1pre?)

latest pre patch is ok.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Sasi Peter]

> > (30+ high speed streams from 4 disks does really need some caching).
> This isn't obvious. Your working may not fit in cache and so the 
kernel
> understand it's worthless to swapout stuff to make space to a 
polluted cache.

But your understanding agrees on that the larger chunks for each stream 
we read into cache, the more efficient for this kind of RAID disk 
structure the read is, thus basically the more cache we have, the more 
bandwidth we can serve. (disks give more data in the same time with 
fewer long reads than with several shorter ones)

So might it have been an accidental behaviour of the previous kernels 
to swap out pages in favor of caching under high I/O pressure, but it 
was certainly a benefical behaviour.

> > Can't say, of these many daemons nothing can be swapped out (and 
should
> > under I/O load)!
> If you run `cp /dev/zero .` on a smart VM nothing must be swapped out 
even if
> it generated nearly the maximal I/O flood possible. It's worthless to 
let a
> polluted cache to grow. It won't help anyways and you'll run slower 
the next
> time you'll have to pagein from swap.
> It _enterely_ depends on the I/O load pattern if it worth to swapout 
or not
> to make space for filesystem cache.

Ok (possibily incorrect, but simple) definition of I/O pressure of mine 
is when the _real_ _physical_ disks are working all the time, pushing 
data out of the box (in this case through the network).

"cp /dev/zero ." is a somewhat different from my case:
- mine is an IRL case (don't know how often a pattern like the "cp" 
case show up IRL)
- Mine is about _reading_ from disks
 
> > Be this, if this is the price for stability.
> As said we can add bits of page aging (that can't destabilize 
anything and it
> will only affect performance behaviour), but I'd prefer to be sure 
you really
> get a slowdown due the new VM behaviour (because more aging if done 
without
> multiqueue O(1) approch can introduces waste of CPU and cachelines in 
kernel
> space), so could you try to kill notes and squid and the other unused 
stuff and
> to see if you return to deliver performance as with the older 
kernels? I still

It might and it should, but actually I gotta have these started in case 
someone drops in for using them. As I understand the only thing this is 
worth trying out for is that maybe even with more cache I will have 
less performance, than before, because in this case to or not to swap 
out dows not really matter. Is this correct? Beacuse I will have to 
have these running anyways...

> miss this important information (last thing you said me was that with 
100mbyte
> in cache it swapouts, and without knowing the details of the I/O 
pattern it

like when decreasing constantly, at reaching that only 100MB cache we 
have left do we start swapping to have more cache, or at least have the 
100MB not less.

> looked sane). After that I'd also like to know what happens with 
2.4.0 that
> uses multiqueue and that is also able to detect pollution and to 
avoid swapping
> out in such case.

What should I test with? (2.4.0/1pre?)

-- SaPE / Sasi Péter / mailto: [EMAIL PROTECTED] / http://sape.iq.rulez.org/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Andrea Arcangeli]

On Wed, Jan 24, 2001 at 01:43:26AM +0100, Sasi Peter wrote:
> (30+ high speed streams from 4 disks does really need some caching).

This isn't obvious. Your working may not fit in cache and so the kernel
understand it's worthless to swapout stuff to make space to a polluted cache.

> Can't say, of these many daemons nothing can be swapped out (and should
> under I/O load)!

If you run `cp /dev/zero .` on a smart VM nothing must be swapped out even if
it generated nearly the maximal I/O flood possible. It's worthless to let a
polluted cache to grow. It won't help anyways and you'll run slower the next
time you'll have to pagein from swap.

It _enterely_ depends on the I/O load pattern if it worth to swapout or not
to make space for filesystem cache.

> The performance just used to be better, but the whole more instable.
> 
> Be this, if this is the price for stability.

As said we can add bits of page aging (that can't destabilize anything and it
will only affect performance behaviour), but I'd prefer to be sure you really
get a slowdown due the new VM behaviour (because more aging if done without
multiqueue O(1) approch can introduces waste of CPU and cachelines in kernel
space), so could you try to kill notes and squid and the other unused stuff and
to see if you return to deliver performance as with the older kernels? I still
miss this important information (last thing you said me was that with 100mbyte
in cache it swapouts, and without knowing the details of the I/O pattern it
looked sane). After that I'd also like to know what happens with 2.4.0 that
uses multiqueue and that is also able to detect pollution and to avoid swapping
out in such case.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Sasi Peter]

On Wed, 24 Jan 2001, Andrea Arcangeli wrote:

> > 2.2.19preXaaX Virtually disabled I/O cache extention-by-swapout, working
> > on previous (semi)stock kernels (raid+ide patched) :(
> Can you measure a performance degradation because of that? Previous kernels was
> certainly not a good example because they was swapping out stuff even with
> `cp /dev/zero .`.

Still produced an overall average extra 30+% more serviced bandwidth for
fileserving. (large files, we can call them high BW streaming...)

> You said me your machine start to swapout when the filesystem cache reaches
> 100mbytes (on your 384Mbyte box). That seems sane behaviour on a misc load. We

Nope.

> could add some additional bit of page aging to swapout more when it worth
> indeed, but current balance looks just quite sane.

Ok let's see: the box does a lot of this fileserving. 3MB/s+

Sometimes I use it interactivelly (pine, X, netscape).

Sometimes others log in remotely, pine, etc.

Sometimes they read their mail (apache, neomail)

bind runs for one (1) zone.

Sshd is listening.

Sendmail is listening.

Infrequently friends use squid proxy on it.

Notes runs, currently is being tested, buteven more infrequently.

=> So, basically, a lot of stuff is runing, but they are rarely used.
Compared to the gain which could result swapping out the more or less
inactive processes' pages, freeing ram for the all time fileservings cache
(30+ high speed streams from 4 disks does really need some caching).

I do not say the case is "nothing is active please store everything on
swap, get back only what is needed", but the actual situation is on the
contrary: 0 (zero) bytes swapped out!

Can't say, of these many daemons nothing can be swapped out (and should
under I/O load)!

And even if I don't think this zero swapping is ok, I do use it, because I
do not get a single "VM: do_try_to_free_pages failed for ..." no more...
The performance just used to be better, but the whole more instable.

Be this, if this is the price for stability.

-- 
SaPE - Peter, Sasi - mailto:[EMAIL PROTECTED] - http://sape.iq.rulez.org/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Andrea Arcangeli]

On Wed, Jan 24, 2001 at 12:52:57AM +0100, Sasi Peter wrote:
> On Sun, 14 Jan 2001, Godfrey Livingstone wrote:
> 
> > You MUST apply this patch before the two raid patches. The VM patch stablises
> > the 2.2.18 virtual memory system and if you don't apply my two repackaged
> > patches will fail. The above VM patch has been accepted into 2.2.19pre3 and
> > many people are using it so is not untested.
> 
> 2.2.19preXaaX Virtually disabled I/O cache extention-by-swapout, working
> on previous (semi)stock kernels (raid+ide patched) :(

Can you measure a performance degradation because of that? Previous kernels was
certainly not a good example because they was swapping out stuff even with
`cp /dev/zero .`.

> Thus I wouldn't advise VM global till it gets somewhatbalanced to
> non-swapless configs...

You said me your machine start to swapout when the filesystem cache reaches
100mbytes (on your 384Mbyte box). That seems sane behaviour on a misc load. We
could add some additional bit of page aging to swapout more when it worth
indeed, but current balance looks just quite sane.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Sasi Peter]

On Sun, 14 Jan 2001, Godfrey Livingstone wrote:

> You MUST apply this patch before the two raid patches. The VM patch stablises
> the 2.2.18 virtual memory system and if you don't apply my two repackaged
> patches will fail. The above VM patch has been accepted into 2.2.19pre3 and
> many people are using it so is not untested.

2.2.19preXaaX Virtually disabled I/O cache extention-by-swapout, working
on previous (semi)stock kernels (raid+ide patched) :(

Thus I wouldn't advise VM global till it gets somewhatbalanced to
non-swapless configs...

-- 
SaPE - Peter, Sasi - mailto:[EMAIL PROTECTED] - http://sape.iq.rulez.org/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Ingo Molnar]

On 13 Jan 2001 [EMAIL PROTECTED] wrote:

> What is at 
> look official enough to me...
>
> raid-2.2.18-B0  12-Jan-2001 10:18   392k

yep, it is the 'official' 2.2.18 RAID patch.

Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[junio]

> "GL" == Godfrey Livingstone <[EMAIL PROTECTED]> writes:

GL> There is as yet no official patch for raid 0.90 for 2.2.18

GL> This question would be better asked on  linux raid list
GL> [EMAIL PROTECTED]

What is at 
look official enough to me...

raid-2.2.18-B0  12-Jan-2001 10:18   392k

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Godfrey Livingstone]

There is as yet no official patch for raid 0.90 for 2.2.18

This question would be better asked on  linux raid list
[EMAIL PROTECTED]

You can apply the patches below.

If you apply the following you get the raid patched kernel.

 
http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.18pre25/VM-global-2.2.18pre25-7.bz2

You MUST apply this patch before the two raid patches. The VM patch stablises
the 2.2.18 virtual memory system and if you don't apply my two repackaged
patches will fail. The above VM patch has been accepted into 2.2.19pre3 and
many people are using it so is not untested.

Raid patches.

 http://www.hattaway-associates.com/raidpatches/raid-2.2.18-A2.bz2

For faster raid 1

 http://www.hattaway-associates.com/raidpatches/raid1readbalance-2.2.18.bz2

Hope this helps

Godfrey Livingstone



Jens Petersohn wrote:

> My appologies if this has been asked before. I'm looking for
> Ingo Molnar's RAID patch for 2.2.18-final. I tried applying A2, but
> it has a number of conflicts in raid1.c which I cannot resolve in
> my meager spare time.
>
> Thanks in advance,
>
> --Jens Petersohn
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> Please read the FAQ at http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Oliver Teuber]

hi

use the 2.2.18aa2 patch from andrea  ... raid 0.9 is included!

cu, oli

ftp://ftp.de.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.2/2.2.18aa2.bz2

On Thu, Jan 11, 2001 at 11:42:33PM +0100, Takacs Sandor wrote:
> On Thu, 11 Jan 2001, Alan Cox wrote:
> 
> > > I tried to apply it. If I finish it I will send the patch to mingo :)
> > 
> > try http://www.linuxraid.org/
> 
> If I see it there is no raid patch for 2.2.18 final, only
> 2.2.18pre13. This patch (raid-2.2.18-A2) rejects some diffs. I will apply
> it by hand :)
> 
> -- 
> Takika
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Andrea Arcangeli]

On Thu, Jan 11, 2001 at 03:36:13PM -0600, Jens Petersohn wrote:
> My appologies if this has been asked before. I'm looking for
> Ingo Molnar's RAID patch for 2.2.18-final. I tried applying A2, but
> it has a number of conflicts in raid1.c which I cannot resolve in
> my meager spare time.

I had to fix those things myself too. However I stay on top of 2.2.19pre7 not
on top of 2.2.18. Raid 0.90 is included into 2.2.19pre7aa1:


ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.2/2.2.19pre7aa1.bz2

The single raid patch can be downloaded from the the below link but it's going
to reject too if applied on top of 2.2.19pre7 because of lvm that is applied
first (it is equivalent to the raid-2.2.18-A2).


ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.2/2.2.19pre7aa1/93_raid-2.2.18-A2_2.2.19pre7aa1-5.bz2

Then later on the generic-map patch changes both lvm and raid to make them
stackable (also for GFS).

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Matthias Kilian]

On Thu, 11 Jan 2001, Takacs Sandor wrote:

> If I see it there is no raid patch for 2.2.18 final, only
> 2.2.18pre13. This patch (raid-2.2.18-A2) rejects some diffs. I will apply
> it by hand :)

I've allready done it:
http://www.escape.de/users/outback/linux/raid-2.2.18.bz2

But it's untested, since I'm now using 2.4.0 :-)

Bye,
Kili

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Takacs Sandor]

On Thu, 11 Jan 2001, Jens Petersohn wrote:

> > > try http://www.linuxraid.org/
> > 
> Check lower on the page. There is a patch for the patch to bring it up
> to A3. This resulting patch will apply semi-cleanly (with fuzz) to the
> final kernel.

Oh, I see it. So, mingo why not create an A3 patch for 2.2.18? :)

-- 
Takika

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Jens Petersohn]

> 
> On Thu, 11 Jan 2001, Alan Cox wrote:
> 
> > > I tried to apply it. If I finish it I will send the patch to mingo :)
> > 
> > try http://www.linuxraid.org/
> 
> If I see it there is no raid patch for 2.2.18 final, only
> 2.2.18pre13. This patch (raid-2.2.18-A2) rejects some diffs. I will apply
> it by hand :)
> 
> -- 
> Takika

Check lower on the page. There is a patch for the patch to bring it up
to A3. This resulting patch will apply semi-cleanly (with fuzz) to the
final kernel.

--Jens Petersohn
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Takacs Sandor]

On Thu, 11 Jan 2001, Alan Cox wrote:

> > I tried to apply it. If I finish it I will send the patch to mingo :)
> 
> try http://www.linuxraid.org/

If I see it there is no raid patch for 2.2.18 final, only
2.2.18pre13. This patch (raid-2.2.18-A2) rejects some diffs. I will apply
it by hand :)

-- 
Takika

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Alan Cox]

> I tried to apply it. If I finish it I will send the patch to mingo :)

try http://www.linuxraid.org/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: Ingo's RAID patch for 2.2.18 final?

[Takacs Sandor]

On Thu, 11 Jan 2001, Jens Petersohn wrote:

> My appologies if this has been asked before. I'm looking for
> Ingo Molnar's RAID patch for 2.2.18-final. I tried applying A2, but
> it has a number of conflicts in raid1.c which I cannot resolve in
> my meager spare time.

I tried to apply it. If I finish it I will send the patch to mingo :)

-- 
Takika

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/