Re: howto boost write(2) performance?

2007-10-09 Thread Nick Piggin
On Tuesday 09 October 2007 23:50, Michael Stiller wrote: > Hi list, > > i'm developing an application (in C) which needs to write about > 1Gbit/s (125Mb/s) to a disk array attached via U320 SCSI. > It runs on Dual Core 2 Xeons @2Ghz utilizing kernel 2.6.22.7. > > I buffer the data in (currently 4)

Re: remove zero_page (was Re: -mm merge plans for 2.6.24)

2007-10-09 Thread Nick Piggin
On Wednesday 10 October 2007 12:22, Linus Torvalds wrote: > On Tue, 9 Oct 2007, Nick Piggin wrote: > > Where do you suggest I go from here? Is there any way I can > > convince you to try it? Make it a config option? (just kidding) > > No, I'll take the damn patch, but q

Re: [13/18] x86_64: Allow fallback for the stack

2007-10-09 Thread Nick Piggin
On Wednesday 10 October 2007 11:26, Christoph Lameter wrote: > On Tue, 9 Oct 2007, Nick Piggin wrote: > > > We already use 32k stacks on IA64. So the memory argument fail there. > > > > I'm talking about generic code. > > The stack size is set in arch code not

Re: remove zero_page (was Re: -mm merge plans for 2.6.24)

2007-10-09 Thread Nick Piggin
On Wednesday 10 October 2007 00:52, Linus Torvalds wrote: > On Tue, 9 Oct 2007, Nick Piggin wrote: > > I have done some tests which indicate a couple of very basic common tools > > don't do much zero-page activity (ie. kbuild). And also combined with > > some logical arg

Re: [13/18] x86_64: Allow fallback for the stack

2007-10-09 Thread Nick Piggin
On Wednesday 10 October 2007 04:39, Christoph Lameter wrote: > On Mon, 8 Oct 2007, Nick Piggin wrote: > > The tight memory restrictions on stack usage do not come about because > > of the difficulty in increasing the stack size :) It is because we want > > to k

Re: [PATCH -mm -v4 1/3] i386/x86_64 boot: setup data

2007-10-09 Thread Nick Piggin
On Tuesday 09 October 2007 18:55, Huang, Ying wrote: > On Tue, 2007-10-09 at 02:06 +1000, Nick Piggin wrote: > > I'm just wondering whether you really need to access highmem in > > boot code... > > Because the zero page (boot_parameters) of i386 boot protocol has 4k &

Re: [PATCH -mm -v4 1/3] i386/x86_64 boot: setup data

2007-10-09 Thread Nick Piggin
On Tuesday 09 October 2007 18:22, Huang, Ying wrote: > On Tue, 2007-10-09 at 01:25 +1000, Nick Piggin wrote: > > On Tuesday 09 October 2007 16:40, Huang, Ying wrote: > > > +unsigned long copy_from_phys(void *to, unsigned long from_phys, > > > + unsi

Re: [PATCH -mm -v4 1/3] i386/x86_64 boot: setup data

2007-10-09 Thread Nick Piggin
On Tuesday 09 October 2007 16:40, Huang, Ying wrote: > +unsigned long copy_from_phys(void *to, unsigned long from_phys, > + unsigned long n) > +{ > + struct page *page; > + void *from; > + unsigned long remain = n, offset, trunck; > + > + while (remain) { >

Re: remove zero_page (was Re: -mm merge plans for 2.6.24)

2007-10-09 Thread Nick Piggin
On Thursday 04 October 2007 01:21, Linus Torvalds wrote: > On Wed, 3 Oct 2007, Nick Piggin wrote: > > I don't know if Linus actually disliked the patch itself, or disliked > > my (maybe confusingly worded) rationale? > > Yes. I'd happily accept the patch, but

Re: [PATCH] mm: set_page_dirty_balance() vs ->page_mkwrite()

2007-10-09 Thread Nick Piggin
On Tuesday 09 October 2007 12:12, Mark Fasheh wrote: > On Mon, Oct 08, 2007 at 05:47:52PM +1000, Nick Piggin wrote: > > > block_page_mkwrite() is just using generic interfaces to do this, > > > same as pretty much any write() system call. The idea was to make it > >

Re: [13/18] x86_64: Allow fallback for the stack

2007-10-08 Thread Nick Piggin
On Tuesday 09 October 2007 03:36, Christoph Lameter wrote: > On Sun, 7 Oct 2007, Nick Piggin wrote: > > > The problem can become non-rare on special low memory machines doing > > > wild swapping things though. > > > > But only your huge systems will be using hu

Re: [PATCH] mm: set_page_dirty_balance() vs ->page_mkwrite()

2007-10-08 Thread Nick Piggin
On Tuesday 09 October 2007 09:36, David Chinner wrote: > On Mon, Oct 08, 2007 at 04:37:00PM +1000, Nick Piggin wrote: > > On Tuesday 09 October 2007 02:54, Peter Zijlstra wrote: > > > Force a balance call if ->page_mkwrite() was successful. > > > > Would it b

Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Nick Piggin
On Tuesday 09 October 2007 03:51, Andrew Morton wrote: > On Mon, 8 Oct 2007 10:28:43 -0700 > > I'll now add remap_file_pages soon. > > Maybe those other 2 tests aren't strong enough (?). > > Or maybe they don't return a non-0 exit status even when they fail... > > (I'll check.) > > Perhaps Yan Zhe

Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Nick Piggin
On Tuesday 09 October 2007 03:04, Andrew Morton wrote: > On Mon, 8 Oct 2007 19:45:08 +0800 "Yan Zheng" <[EMAIL PROTECTED]> wrote: > > Hi all > > > > The test for VM_CAN_NONLINEAR always fails > > > > Signed-off-by: Yan Zheng<[EMAIL PROTECTED]> > > > > diff -ur linux-2.6.23-rc9/mm/fremap.c linu

Re: [PATCH]fix VM_CAN_NONLINEAR check in sys_remap_file_pages

2007-10-08 Thread Nick Piggin
instead of failing. I doubt anybody will be using nonlinear mappings on anything but regular files for the time being, but as a trivial fix, I think this probably should go into 2.6.23. Thanks for spotting this problem Acked-by: Nick Piggin <[EMAIL PROTECTED]> > I hope Nick or Miklos

Re: [PATCH] mm: set_page_dirty_balance() vs ->page_mkwrite()

2007-10-08 Thread Nick Piggin
On Tuesday 09 October 2007 02:54, Peter Zijlstra wrote: > It seems that with the recent usage of ->page_mkwrite() a little detail > was overlooked. > > .22-rc1 merged OCFS2 usage of this hook > .23-rc1 merged XFS usage > .24-rc1 will most likely merge NFS usage > > Please consider this for .23 fina

[patch] fs: restore nobh

2007-10-07 Thread Nick Piggin
Hi, This is overdue, sorry. Got a little complicated, and I've been away from my filesystem test setup so I didn't want ot send it (lucky, coz I found a bug after more substantial testing). Anyway, RFC? --- Implement nobh in new aops. This is a bit tricky. FWIW, nobh_truncate is now implemented

Re: race with page_referenced_one->ptep_test_and_clear_young and pagetable setup/pulldown

2007-10-07 Thread Nick Piggin
On Friday 05 October 2007 12:44, Andrew Morton wrote: > On Thu, 04 Oct 2007 18:43:32 -0700 Jeremy Fitzhardinge <[EMAIL PROTECTED]> wrote: > > David's change 10a8d6ae4b3182d6588a5809a8366343bc295c20, "i386: add > > ptep_test_and_clear_{dirty,young}" has introduced an SMP race which > > affects the

Re: [DEBUG PATCH] demonstration of hang in sched domain partitioning code

2007-10-07 Thread Nick Piggin
On Friday 05 October 2007 19:24, Paul Jackson wrote: > Nick, > > I'm running into a problem trying to use your suggested > partition_sched_domains(cpumask_t *partition) routine the way > I thought I was supposed to be able to use it. > > If I ask to set up the same partition twice in a row having >

Re: [13/18] x86_64: Allow fallback for the stack

2007-10-07 Thread Nick Piggin
On Friday 05 October 2007 07:20, Christoph Lameter wrote: > On Thu, 4 Oct 2007, Rik van Riel wrote: > > > Well we can now address the rarity. That is the whole point of the > > > patchset. > > > > Introducing complexity to fight a very rare problem with a good > > fallback (refusing to fork more ta

Re: [BUG] kernel BUG at arch/i386/mm/highmem.c:15! on 2.6.23-rc8/rc9

2007-10-04 Thread Nick Piggin
On Thursday 04 October 2007 00:53, Nick Piggin wrote: > On Thursday 04 October 2007 16:37, gurudas pai wrote: > > Hi, > > > > While running Oracle database test on x86/6GB RAM machine panics with > > following messages. > > Hi, > > Hmm, seems like somet

Re: [BUG] kernel BUG at arch/i386/mm/highmem.c:15! on 2.6.23-rc8/rc9

2007-10-04 Thread Nick Piggin
On Thursday 04 October 2007 16:37, gurudas pai wrote: > Hi, > > While running Oracle database test on x86/6GB RAM machine panics with > following messages. Hi, Hmm, seems like something in sys_remap_file_pages might have broken. It's a bit hard to work out from the backtrace, though. Is it possi

[rfc][patch 2/3] x86: fix IO write barriers

2007-10-03 Thread Nick Piggin
wmb() on x86 must always include a barrier, because stores can go out of order in many cases when dealing with devices (eg. WC memory). Signed-off-by: Nick Piggin <[EMAIL PROTECTED]> Index: linux-2.6/include/asm-i386/sy

[rfc][patch 3/3] x86: optimise barriers

2007-10-03 Thread Nick Piggin
cture. smp_rmb on buggy pentium pros remains a locked op, which is apparently required. Signed-off-by: Nick Piggin <[EMAIL PROTECTED]> --- Index: linux-2.6/include/asm-i386/system.h === --- linux-2.6.orig/include/asm-i386/system.

[rfc][patch 1/3] x86_64: fence nontemproal stores

2007-10-03 Thread Nick Piggin
o maybe we need sfences _before_ movnt* everywhere too? ] Signed-off-by: Nick Piggin <[EMAIL PROTECTED]> Index: linux-2.6/arch/x86_64/lib/copy_user_nocache.S === --- linux-2.6.orig/arch/x86_64/lib/copy_user_nocache.S +++ linu

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 22:17, Paul Jackson wrote: > Nick wrote: > > OK, so I don't exactly understand you either. To make it simple, can > > you give a concrete example of a cpuset hierarchy that wouldn't > > work? > > It's more a matter of knowing how my third party batch scheduler > coders

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 22:41, Paul Jackson wrote: > > pdflush > > is not pinned at all and can be dynamically created and destroyed. Ditto > > for kjournald, as well as many others. > > Whatever is not pinned is moved out of the top cpuset, on the kind of > systems I'm most familiar with. Th

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 22:14, Paul Jackson wrote: > > These are what I'm worried about, and things like kswapd, pdflush, > > could definitely use a huge amount of CPU. > > > > If you are interested in hard partitioning the system, you most > > definitely want these things to be balanced acros

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 21:38, Paul Jackson wrote: > > OK, so to really do anything different (from a non-partitioned setup), > > you would need to set sched_load_balance=0 for the root cpuset? > > Suppose you do that to hard partition the machine, what happens to > > newly created tasks like

Re: pgd_none_or_clear_bad strangeness?

2007-10-03 Thread Nick Piggin
On Tue, Oct 02, 2007 at 05:20:03PM -0500, Matt Mackall wrote: > In lib/pagewalk.c, I've been using the various forms of > {pgd,pud,pmd}_none_or_clear_bad while walking page tables as that > seemed the canonical way to do things. Lately (eg with -rc7-mm1), > these have been triggering messages like

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 19:55, Paul Jackson wrote: > > > Yeah -- cpusets are hierarchical. And some of the use cases for > > > which cpusets are designed are hierarchical. > > > > But partitioning isn't. > > Yup. We've got a square peg and a round hole. An impedance mismatch. > That's the r

remove zero_page (was Re: -mm merge plans for 2.6.24)

2007-10-03 Thread Nick Piggin
On Tuesday 02 October 2007 07:22, Andrew Morton wrote: > remove-zero_page.patch > > Linus dislikes it. Probably drop it. I don't know if Linus actually disliked the patch itself, or disliked my (maybe confusingly worded) rationale? To clarify: it is not zero_page that fundamentally causes a p

Re: [patch] sched: fix sched-domains partitioning by cpusets

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 19:39, Paul Jackson wrote: > > in any case i'd like to see the externally visible API get in foremost - > > and there now seems to be agreement about that. (yay!) Any internal > > shaping of APIs can be done flexibly between cpusets and the scheduler. > > Yup - though N

Re: [patch] sched: fix sched-domains partitioning by cpusets

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 19:21, Paul Jackson wrote: > Nick wrote: > > Sorry for the confusion: I only meant the sched.c part of that > > patch, not the full thing. > > Ah - ok. We're getting closer then. Good. > > Let me be sure I've got this right then. > > You prefer the interface from your

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 17:25, Paul Jackson wrote: > Nick wrote: > > BTW. as far as the sched.c changes in your patch go, I much prefer > > the partition_sched_domains API: http://lkml.org/lkml/2006/10/19/85 > > > > The caller should manage everything itself, rather than > > partition_sched_do

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 16:58, Paul Jackson wrote: > > > Yup - it's asking for load balancing over that set. That is why it is > > > called that. There's no idea here of better or worse load balancing, > > > that's an internal kernel scheduler subtlety -- it's just a request > > > that load

Re: [PATCH] mark read_crX() asm code as volatile

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 16:18, H. Peter Anvin wrote: > Nick Piggin wrote: > >> This should work because the result gets used before reading again: > >> > >> read_cr3(a); > >> write_cr3(a | 1); > >> read_cr3(a); > >> > >>

Re: [patch] sched: fix sched-domains partitioning by cpusets

2007-10-03 Thread Nick Piggin
On Wednesday 03 October 2007 16:56, Paul Jackson wrote: > I must NAQ this patch, and I'm surprised to see Nick propose it > again, as I thought he had already agreed that it didn't suffice. Sorry for the confusion: I only meant the sched.c part of that patch, not the full thing. - To unsubscribe

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-02 Thread Nick Piggin
On Tuesday 02 October 2007 04:15, Paul Jackson wrote: > Nick wrote: > > which you could equally achieve by adding > > a second set of sched domains (and the global domains could keep > > globally balancing). > > Hmmm ... this could be the key to this discussion. > > Nick - can two sched domains ove

Re: wibbling over the cpuset shed domain connnection

2007-10-02 Thread Nick Piggin
On Wednesday 03 October 2007 15:21, Paul Jackson wrote: > > In the meantime, that patch should be merged though, shouldn't it? > > Which patch do you refer to: > 1) the year old patch to disconnect cpusets and sched domains: > cpuset-remove-sched-domain-hooks-from-cpusets.patch > 2) my patc

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-10-02 Thread Nick Piggin
On Monday 01 October 2007 13:42, Paul Jackson wrote: > Nick wrote: > > Moreover, sched_load_balance doesn't really sound like a good name > > for asking for a partition. > > Yup - it's not a good name for asking for a partition. > > That's because it isn't asking for a partition. > > It's asking fo

Re: wibbling over the cpuset shed domain connnection

2007-10-02 Thread Nick Piggin
On Tuesday 02 October 2007 07:34, Paul Jackson wrote: > In -mm merge plans for 2.6.24, Andrew wrote: > > cpuset-remove-sched-domain-hooks-from-cpusets.patch > > > > Paul continues to wibble over this. Hold, I guess. > > Oh dear ... after looking at the following to figure out what > a wibble is,

Re: [PATCH] mark read_crX() asm code as volatile

2007-10-02 Thread Nick Piggin
On Wednesday 03 October 2007 04:27, Chuck Ebbert wrote: > On 10/02/2007 11:28 AM, Arjan van de Ven wrote: > > On Tue, 02 Oct 2007 18:08:32 +0400 > > > > Kirill Korotaev <[EMAIL PROTECTED]> wrote: > >> Some gcc versions (I checked at least 4.1.1 from RHEL5 & 4.1.2 from > >> gentoo) can generate inco

Re: per BDI dirty limit (was Re: -mm merge plans for 2.6.24)

2007-10-02 Thread Nick Piggin
On Tuesday 02 October 2007 21:40, Peter Zijlstra wrote: > On Tue, 2007-10-02 at 13:21 +0200, Kay Sievers wrote: > > How about adding this information to the tree then, instead of > > creating a new top-level hack, just because something that you think > > you need doesn't exist. > > So you suggest

Re: kswapd min order, slub max order [was Re: -mm merge plans for 2.6.24]

2007-10-02 Thread Nick Piggin
On Wednesday 03 October 2007 02:06, Hugh Dickins wrote: > On Mon, 1 Oct 2007, Andrew Morton wrote: > > # > > # slub && antifrag > > # > > have-kswapd-keep-a-minimum-order-free-other-than-order-0.patch > > only-check-absolute-watermarks-for-alloc_high-and-alloc_harder-allocation > >s.patch slub-expl

Re: [PATCH 05/12] mm: trylock_page

2007-10-02 Thread Nick Piggin
On Sunday 30 September 2007 01:01, Peter Zijlstra wrote: > On Fri, 2007-09-28 at 13:11 +1000, Nick Piggin wrote: > > On Friday 28 September 2007 17:42, Peter Zijlstra wrote: > > > Replace raw TestSetPageLocked() usage with trylock_page() > > > > I have such a thing q

Re: [PATCH] Add ability to dump mapped pages with /proc/sys/vm/drop_caches

2007-10-02 Thread Nick Piggin
On Monday 01 October 2007 04:03, Soeren Sandmann wrote: > This patch adds the ability to drop mapped pages with > /proc/sys/vm/drop_caches. This is useful to get repeatable > measurements of startup time for applications. > > Without it, pages that are mapped in already-running applications will >

Re: [15/17] SLUB: Support virtual fallback via SLAB_VFALLBACK

2007-10-02 Thread Nick Piggin
On Tuesday 02 October 2007 06:50, Christoph Lameter wrote: > On Fri, 28 Sep 2007, Nick Piggin wrote: > > I thought it was slower. Have you fixed the performance regression? > > (OK, I read further down that you are still working on it but not > > confirmed yet...) > > Th

Re: [15/17] SLUB: Support virtual fallback via SLAB_VFALLBACK

2007-10-02 Thread Nick Piggin
On Tuesday 02 October 2007 07:01, Christoph Lameter wrote: > On Sat, 29 Sep 2007, Peter Zijlstra wrote: > > On Fri, 2007-09-28 at 11:20 -0700, Christoph Lameter wrote: > > > Really? That means we can no longer even allocate stacks for forking. > > > > I think I'm running with 4k stacks... > > 4k st

Re: 2.6.21 -> 2.6.22 & 2.6.23-rc8 performance regression

2007-09-30 Thread Nick Piggin
Hi Denys, thanks for reporting (btw. please reply-to-all when replying on lkml). You say that SLAB is better than SLUB on an otherwise identical kernel, but I didn't see if you quantified the actual numbers? It sounds like there is still a regression with SLAB? On Monday 01 October 2007 03:48,

Re: [15/17] SLUB: Support virtual fallback via SLAB_VFALLBACK

2007-09-30 Thread Nick Piggin
On Monday 01 October 2007 06:12, Andrew Morton wrote: > On Sun, 30 Sep 2007 05:09:28 +1000 Nick Piggin <[EMAIL PROTECTED]> wrote: > > On Sunday 30 September 2007 05:20, Andrew Morton wrote: > > > We can't "run out of unfragmented memory" for an orde

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-09-30 Thread Nick Piggin
On Monday 01 October 2007 04:07, Paul Jackson wrote: > Nick wrote: > > The user should just be able to specify exactly the partitioning of > > tasks required, and cpusets should ask the scheduler to do the best > > job of load balancing possible. > > If the cpusets which have 'sched_load_balance' e

Re: [patch] x86: improved memory barrier implementation

2007-09-30 Thread Nick Piggin
On Sat, Sep 29, 2007 at 09:07:30AM -0700, Linus Torvalds wrote: > > > On Sat, 29 Sep 2007, Nick Piggin wrote: > > > > > > The non-temporal stores should be basically considered to be "IO", not > > > any > > > normal memory operatio

Re: [rfc][patch] i386: remove comment about barriers

2007-09-30 Thread Nick Piggin
On Sat, Sep 29, 2007 at 12:12:52PM -0700, Davide Libenzi wrote: > On Sat, 29 Sep 2007, Nick Piggin wrote: > > > [ This is true for x86's sfence/lfence, but raises a question about Linux's > > memory barriers. Does anybody expect that a sequence of smp_wmb and smp_rmb &

Re: [rfc][patch] i386: remove comment about barriers

2007-09-30 Thread Nick Piggin
On Sat, Sep 29, 2007 at 08:16:47PM -0700, Paul E. McKenney wrote: > On Sat, Sep 29, 2007 at 03:28:48PM +0200, Nick Piggin wrote: > > Acked-by: Paul E. McKenney <[EMAIL PROTECTED]> > Thanks v much for confirming, everyone. > > Signed-off-by: Nick

Re: [PATCH] cpuset and sched domains: sched_load_balance flag

2007-09-30 Thread Nick Piggin
On Sunday 30 September 2007 20:44, Paul Jackson wrote: > From: Paul Jackson <[EMAIL PROTECTED]> > > Add a new per-cpuset flag called 'sched_load_balance'. > > When enabled in a cpuset (the default value) it tells the kernel > scheduler that the scheduler should provide the normal load > balancing o

Re: [15/17] SLUB: Support virtual fallback via SLAB_VFALLBACK

2007-09-30 Thread Nick Piggin
On Sunday 30 September 2007 05:20, Andrew Morton wrote: > On Sat, 29 Sep 2007 06:19:33 +1000 Nick Piggin <[EMAIL PROTECTED]> wrote: > > On Saturday 29 September 2007 19:27, Andrew Morton wrote: > > > On Sat, 29 Sep 2007 11:14:02 +0200 Peter Zijlstra > > >

[rfc][patch] i386: remove comment about barriers

2007-09-29 Thread Nick Piggin
smp_wmb and smp_rmb would ever provide a full smp_mb barrier? I've always assumed no, but I don't know if it is actually documented? ] Signed-off-by: Nick Piggin <[EMAIL PROTECTED]> --- Index: linux-2.6/include/asm-i386/system.h

Re: [patch] x86: improved memory barrier implementation

2007-09-29 Thread Nick Piggin
On Fri, Sep 28, 2007 at 06:18:31PM +0100, Alan Cox wrote: > > on the broken ppro stores config option if you just tell me what should > > be there (again, remember that my patch isn't actually changing anything > > already there except for smp_rmb side). > > The PPro needs rmb to ensure a store do

Re: [patch] x86: improved memory barrier implementation

2007-09-29 Thread Nick Piggin
On Fri, Sep 28, 2007 at 09:15:06AM -0700, Linus Torvalds wrote: > > > On Fri, 28 Sep 2007, Alan Cox wrote: > > > > However > > - You've not shown the patch has any performance gain > > It would be nice to see this. Actually, in a userspace test I have (which actually does enough work to trigg

Re: [PATCH] writeback: remove unnecessary wait in throttle_vm_writeout()

2007-09-29 Thread Nick Piggin
On Saturday 29 September 2007 03:57, Mathieu Desnoyers wrote: > Isn't the actual instrumentation present in the VM subsystem consisting > mostly of event counters ? This kind of profiling provides limited help in > following specific delays in the kernel. Martin Bligh's paper "Linux > Kernel Debug

Re: [15/17] SLUB: Support virtual fallback via SLAB_VFALLBACK

2007-09-29 Thread Nick Piggin
On Saturday 29 September 2007 04:41, Christoph Lameter wrote: > On Fri, 28 Sep 2007, Peter Zijlstra wrote: > > memory got massively fragemented, as anti-frag gets easily defeated. > > setting min_free_kbytes to 12M does seem to solve it - it forces 2 max > > order blocks to stay available, so we do

Re: [15/17] SLUB: Support virtual fallback via SLAB_VFALLBACK

2007-09-29 Thread Nick Piggin
On Saturday 29 September 2007 19:27, Andrew Morton wrote: > On Sat, 29 Sep 2007 11:14:02 +0200 Peter Zijlstra <[EMAIL PROTECTED]> wrote: > > > oom-killings, or page allocation failures? The latter, one hopes. > > > > Linux version 2.6.23-rc4-mm1-dirty ([EMAIL PROTECTED]) (gcc version 4.1.2 > > (

Re: [RFC][PATCH] mm: couple rcu and memory reclaim

2007-09-29 Thread Nick Piggin
On Monday 24 September 2007 18:45, Peter Zijlstra wrote: > Just an idea I had, it seems like a good idea to wait for RCU callbacks > in reclaim so that we won't get all of memory stuck there. I think it would be much too aggressive (_especially_ with preemptible RCU, I would have thought?). And no

Re: Strange system hangs

2007-09-29 Thread Nick Piggin
On Friday 28 September 2007 18:42, Krzysztof Oledzki wrote: > Hello, > > I am experiencing weird system hangs. Once about 2-5 weeks system freezes > and stops accepting remote connections, so it is no longer possible to > connect to most important services: smtp (postfix), www (squid) or even > ssh

Re: Network slowdown due to CFS

2007-09-29 Thread Nick Piggin
On Friday 28 September 2007 00:42, Jarek Poplawski wrote: > On Thu, Sep 27, 2007 at 03:31:23PM +0200, Ingo Molnar wrote: > > * Jarek Poplawski <[EMAIL PROTECTED]> wrote: > > ... > > > > OK, but let's forget about fixing iperf. Probably I got this wrong, > > > but I've thought this "bad" iperf patch

Re: General slowness with using 64Gb HIGHMEM option on 32-bit kernel

2007-09-28 Thread Nick Piggin
On Saturday 29 September 2007 03:15, Sergey Popov wrote: > Short description: after recompiling 32-bit kernel with 64Gb highmem > support and rebooting into it, OS started working very much slower > then before. > > Specifications: Intel Core 2 Duo [EMAIL PROTECTED],4Ghz on Intel i965Q-based > moth

Re: [15/17] SLUB: Support virtual fallback via SLAB_VFALLBACK

2007-09-28 Thread Nick Piggin
On Saturday 29 September 2007 03:33, Christoph Lameter wrote: > On Fri, 28 Sep 2007, Nick Piggin wrote: > > On Wednesday 19 September 2007 13:36, Christoph Lameter wrote: > > > SLAB_VFALLBACK can be specified for selected slab caches. If fallback > > > is available the

Re: [PATCH 05/12] mm: trylock_page

2007-09-28 Thread Nick Piggin
On Friday 28 September 2007 17:42, Peter Zijlstra wrote: > Replace raw TestSetPageLocked() usage with trylock_page() I have such a thing queued too, for the lock bitops patches for when 2.6.24 opens, Andrew promises me :). I guess they should be identical, except I don't like doing trylock_page i

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-28 Thread Nick Piggin
On Thursday 20 September 2007 11:38, David Chinner wrote: > On Wed, Sep 19, 2007 at 04:04:30PM +0200, Andrea Arcangeli wrote: > > Plus of course you don't like fsblock because it requires work to > > adapt a fs to it, I can't argue about that. > > No, I don't like fsblock because it is inherently

Re: [patch] x86: improved memory barrier implementation

2007-09-28 Thread Nick Piggin
On Fri, Sep 28, 2007 at 05:07:19PM +0100, Alan Cox wrote: > > The only alternative is to assume a weak memory model, and add the required > > barriers to spin_unlock -- something that has been explicitly avoided, but > > We have the barriers in spin_unlock already for Pentium Pro and IDT > Winchip

Re: [PATCH] writeback: remove unnecessary wait in throttle_vm_writeout()

2007-09-28 Thread Nick Piggin
On Saturday 29 September 2007 02:23, Mathieu Desnoyers wrote: > * Ingo Molnar ([EMAIL PROTECTED]) wrote: > > * Andrew Morton <[EMAIL PROTECTED]> wrote: > > > This is a pretty major bugfix. > > > > > > GFP_NOIO and GFP_NOFS callers should have been spending really large > > > amounts of time stuck i

Re: [PATCH] 9p: fix compile error if !CONFIG_SYSCTL

2007-09-28 Thread Nick Piggin
On Saturday 29 September 2007 00:34, Linus Torvalds wrote: > On Fri, 28 Sep 2007, Nick Piggin wrote: > > > God I hate select. > > > > IMO a better implementation would result in a notification / confirmation > > of turning on new items, and the ability to desele

[patch] x86: improved memory barrier implementation

2007-09-28 Thread Nick Piggin
f these CPUs with ooostores do SMP? If not, then smp_wmb can also be a simple barrier on i386 too. Signed-off-by: Nick Piggin <[EMAIL PROTECTED]> --- Index: linux-2.6/include/asm-i386/system.h === --- linux-2.6.orig/include/asm

Re: [PATCH] writeback: remove unnecessary wait in throttle_vm_writeout()

2007-09-28 Thread Nick Piggin
On Friday 28 September 2007 18:02, Ingo Molnar wrote: > * Andrew Morton <[EMAIL PROTECTED]> wrote: > > This is a pretty major bugfix. > > > > GFP_NOIO and GFP_NOFS callers should have been spending really large > > amounts of time stuck in that sleep. > > > > I wonder why nobody noticed this happen

Re: [PATCH] writeback: remove unnecessary wait in throttle_vm_writeout()

2007-09-28 Thread Nick Piggin
On Thursday 27 September 2007 11:50, Fengguang Wu wrote: > We don't want to introduce pointless delays in throttle_vm_writeout() > when the writeback limits are not yet exceeded, do we? I don't think so (ie. I agree with you). IIRC, Marcelo initially did the throttle_vm_write

Re: [PATCH] TASK_KILLABLE version 2

2007-09-28 Thread Nick Piggin
On Wednesday 26 September 2007 21:57, Ric Wheeler wrote: > Bob Bell wrote: > > On Sat, Sep 01, 2007 at 08:43:49PM -0600, Matthew Wilcox wrote: > >> Here's the second version of TASK_KILLABLE. A few changes since > >> version 1: > > > > > > > >> I obviously haven't covered every place that can res

Re: [15/17] SLUB: Support virtual fallback via SLAB_VFALLBACK

2007-09-28 Thread Nick Piggin
On Wednesday 19 September 2007 13:36, Christoph Lameter wrote: > SLAB_VFALLBACK can be specified for selected slab caches. If fallback is > available then the conservative settings for higher order allocations are > overridden. We then request an order that can accomodate at mininum > 100 objects.

Re: XFS Fails Quality Assurance Tests on ARM

2007-09-28 Thread Nick Piggin
On Sunday 02 September 2007 08:14, Andi Kleen wrote: > David Miller <[EMAIL PROTECTED]> writes: > > From: Byron Bradley <[EMAIL PROTECTED]> > > Date: Fri, 31 Aug 2007 03:12:46 + (UTC) > > > > > Anybody got any ideas of how we fix this? > > > > I don't know how much testing XFS gets on ARM, but

Re: [PATCH] 9p: fix compile error if !CONFIG_SYSCTL

2007-09-28 Thread Nick Piggin
On Thursday 20 September 2007 16:31, Andrew Morton wrote: > I struggled for five minutes trying to work out how to make CONFIG_SYSCTL > go away and gave up in disgust. > > God I hate select. IMO a better implementation would result in a notification / confirmation of turning on new items, and the

Re: 2.6.22.3 out of memory crash

2007-09-28 Thread Nick Piggin
On Thursday 20 September 2007 16:56, Dag Nygren wrote: > Hi, > > I have the kview zoom function mapped to the scrollwheel > with ctrl. > Now my wife found out and watching our photos wanted > to zoom out a bit. Then she just rotated the scrollwheel a lot > and expected the zoom to happen. What did

Re: [13/17] Virtual compound page freeing in interrupt context

2007-09-20 Thread Nick Piggin
On Wednesday 19 September 2007 13:36, Christoph Lameter wrote: > If we are in an interrupt context then simply defer the free via a > workqueue. > > In an interrupt context it is not possible to use vmalloc_addr() to > determine the vmalloc address. So add a variant that does that too. > > Removing

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-19 Thread Nick Piggin
On Wednesday 19 September 2007 04:30, Linus Torvalds wrote: > On Tue, 18 Sep 2007, Nick Piggin wrote: > > ROFL! Yeah of course, how could I have forgotten about our trusty OOM > > killer as the solution to the fragmentation problem? It would only have > > been funnier if y

Re: [PATCH] mm: use pagevec to rotate reclaimable page

2007-09-18 Thread Nick Piggin
On Wednesday 19 September 2007 03:44, Andrew Morton wrote: > On Tue, 18 Sep 2007 11:29:50 +1000 Nick Piggin <[EMAIL PROTECTED]> wrote: > > It would be interesting to test -mm kernels. They have a patch which > > reduces zone lock contention quite a lot. > > They d

Re: [PATCH] mm: use pagevec to rotate reclaimable page

2007-09-18 Thread Nick Piggin
On Tuesday 18 September 2007 20:41, Hisashi Hifumi wrote: > I modified my patch based on your comment. > > At 11:37 07/09/14, Andrew Morton wrote: > >So I do think that for safety and sanity's sake, we should be taking a > > ref on the pages when they are in a pagevec. That's going to hurt your

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-18 Thread Nick Piggin
On Tuesday 18 September 2007 08:21, Christoph Lameter wrote: > On Sun, 16 Sep 2007, Nick Piggin wrote: > > > > So if you argue that vmap is a downside, then please tell me how you > > > > consider the -ENOMEM of your approach to be better? > > > > > >

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-18 Thread Nick Piggin
On Tuesday 18 September 2007 08:00, Christoph Lameter wrote: > On Sun, 16 Sep 2007, Nick Piggin wrote: > > I don't know how it would prevent fragmentation from building up > > anyway. It's commonly the case that potentially unmovable objects > > are allowed to fill

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-18 Thread Nick Piggin
On Tuesday 18 September 2007 08:05, Christoph Lameter wrote: > On Sun, 16 Sep 2007, Nick Piggin wrote: > > > > fsblock doesn't need any of those hacks, of course. > > > > > > Nor does mine for the low orders that we are considering. For order > > >

Re: [PATCH 001/104] KVM: Fix *nopage() in kvm_main.c

2007-09-18 Thread Nick Piggin
On Tuesday 18 September 2007 04:19, Avi Kivity wrote: > Nick Piggin wrote: > >> I should mention that the converting to use ->fault() is a 15-minute > >> change; the tricky part is adding backwards compatibility for the > >> external module package. > > >

Re: VM/VFS bug with large amount of memory and file systems?

2007-09-18 Thread Nick Piggin
On Tuesday 18 September 2007 03:04, Rik van Riel wrote: > Nick Piggin wrote: > > (Rik has a patch sitting in -mm I believe which would make this problem > > even worse, by doing even less highmem scanning in response to lowmem > > allocations). > > My patch should n

Re: VM/VFS bug with large amount of memory and file systems?

2007-09-17 Thread Nick Piggin
On Tuesday 18 September 2007 00:09, Anton Altaparmakov wrote: > On 17 Sep 2007, at 15:04, Anton Altaparmakov wrote: > > On 15 Sep 2007, at 11:52, Andrew Morton wrote: > >> On Sat, 15 Sep 2007 12:08:17 +0200 Peter Zijlstra > >> > >> <[EMAIL PROTECTED]> wrote: > >>> Anyway, looks like all of zone_nor

Re: [PATCH 001/104] KVM: Fix *nopage() in kvm_main.c

2007-09-17 Thread Nick Piggin
On Monday 17 September 2007 19:18, Avi Kivity wrote: > Avi Kivity wrote: > > Christoph Hellwig wrote: > >> On Mon, Sep 17, 2007 at 10:30:43AM +0200, Avi Kivity wrote: > >>> From: Nguyen Anh Quynh <[EMAIL PROTECTED]> > >>> > >>> *nopage() in kvm_main.c should only store the type of mmap() fault if >

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-17 Thread Nick Piggin
On Monday 17 September 2007 14:07, David Chinner wrote: > On Fri, Sep 14, 2007 at 06:48:55AM +1000, Nick Piggin wrote: > > OK, the vunmap batching code wipes your TLB flushing and IPIs off > > the table. Diffstat below, but the TLB portions are here (besides that > > _ev

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-17 Thread Nick Piggin
On Saturday 15 September 2007 04:08, Christoph Lameter wrote: > On Fri, 14 Sep 2007, Nick Piggin wrote: > > However fsblock can do everything that higher order pagecache can > > do in terms of avoiding vmap and giving contiguous memory to block > > devices by opportunistica

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-17 Thread Nick Piggin
On Monday 17 September 2007 04:13, Mel Gorman wrote: > On (15/09/07 14:14), Goswin von Brederlow didst pronounce: > > I keep coming back to the fact that movable objects should be moved > > out of the way for unmovable ones. Anything else just allows > > fragmentation to build up. > > This is easi

Re: 2.6.22.6: kernel BUG at fs/locks.c:171

2007-09-17 Thread Nick Piggin
On Saturday 15 September 2007 20:22, Soeren Sonnenburg wrote: > On Sat, 2007-09-15 at 09:47 +, Soeren Sonnenburg wrote: > > Memtest did not find anything after 16 passes so I finally stopped it > > applied your patch and used > > > > CONFIG_DEBUG_SLAB=y > > CONFIG_DEBUG_SLAB_LEAK=y > > > > and

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-17 Thread Nick Piggin
On Saturday 15 September 2007 03:52, Christoph Lameter wrote: > On Fri, 14 Sep 2007, Nick Piggin wrote: > > > > [*] ok, this isn't quite true because if you can actually put a hard > > > > limit on unmovable allocations then anti-frag will fundamentally help > &

Re: 2.6.22.6: kernel BUG at fs/locks.c:171

2007-09-14 Thread Nick Piggin
On Friday 14 September 2007 16:02, Soeren Sonnenburg wrote: > On Thu, 2007-09-13 at 09:51 +1000, Nick Piggin wrote: > > On Thursday 13 September 2007 19:20, Soeren Sonnenburg wrote: > > > Dear all, > > > > > > I've just seen this in dmesg on a AMD K

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-14 Thread Nick Piggin
On Thursday 13 September 2007 09:17, Christoph Lameter wrote: > On Wed, 12 Sep 2007, Nick Piggin wrote: > > I will still argue that my approach is the better technical solution for > > large block support than yours, I don't think we made progress on that. > > And I

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-14 Thread Nick Piggin
On Thursday 13 September 2007 09:06, Christoph Lameter wrote: > On Wed, 12 Sep 2007, Nick Piggin wrote: > > So lumpy reclaim does not change my formula nor significantly help > > against a fragmentation attack. AFAIKS. > > Lumpy reclaim improves the situation sign

<    1   2   3   4   5   6   7   8   9   10   >