Re: tunings for many httpds...

2002-06-26 Thread Peter Wemm
Patrick Thomas wrote: Incidently, looking at the PV entry angle for a moment. Suppose you create a 1GB sysvshm (pageable) segment. That's 262144 pages. Mapping thi s once means you consume 262144 PV entries. At 28 bytes each, that is about 7.3MB of KVM. Now, fork this process

Re: tunings for many httpds...

2002-06-26 Thread Terry Lambert
Peter Wemm wrote: Even more importantly it would be nice if we could share compatible pmap pages, then we would have no need for 4MB pages... 50 mappings of the same shared memory segment would wind up using the same pmap pages as if only one mapping had been made. Such a

Re: tunings for many httpds...

2002-06-26 Thread Matthew Dillon
Hmm. I'm fairly sure that Linux does not quite do it that way. I believe the 2-level page tables are copy-on-write, but that only gives you shareability across a fork() and then only for a little while. I'm fairly certain that Linux cannot share page tables for post-fork

Re: tunings for many httpds...

2002-06-26 Thread Terry Lambert
Patrick Thomas wrote: Uhh, Terry, neither form of SHM uses KVA. Either pageable or physically backed. The memory is only mapped into processes and is NOT mapped into KVA anywhere. (*) You and Alfred are right. It looks like this was corrected by John Dyson, shortly after he

Re: tunings for many httpds...

2002-06-26 Thread Terry Lambert
Patrick Thomas wrote: Ok, I'm confused now - so I understood you to originally say that SHM does not eat into KVA regardless of whether I set the kern.ipc.shm_use_phys to '1' or not. Don't be confused. It takes 1/1024th of the KVA space I thought it did, times the number of processes atached

Re: tunings for many httpds...

2002-06-26 Thread Terry Lambert
Matthew Dillon wrote: Hmm. I'm fairly sure that Linux does not quite do it that way. I believe the 2-level page tables are copy-on-write, but that only gives you shareability across a fork() and then only for a little while. I'm fairly certain that Linux cannot share page

Re: tunings for many httpds...

2002-06-26 Thread Matthew Dillon
: http://old.lwn.net/2002/0124/kernel.php3 : What Rik van Riel is up to. : :FWIW: In the original Article: : : http://lwn.net/Articles/3327/ : :they say they've gon to a 3 level page table scheme for the Hammer :port. : :-- Terry They are still using a 2 level page table

Re: tunings for many httpds...

2002-06-26 Thread Matthew Dillon
:[commenting live from ottawa] : :On Wed, Jun 26, 2002 at 10:11:54AM -0700, Matthew Dillon wrote: : They are still using a 2 level page table scheme in the linux : kernel. That is, they didn't try to integrate the 3-level hardware : in the hammer with the 2 level kernel

Re: tunings for many httpds...

2002-06-26 Thread Matthew Dillon
:[commenting live from ottawa] Pictures! We want pictures! -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe

Re: tunings for many httpds...

2002-06-26 Thread Bosko Milekic
On Wed, Jun 26, 2002 at 11:24:47AM -0700, Matthew Dillon wrote: :[commenting live from ottawa] Pictures! We want pictures! It's pretty cool that the Linux camp has decided to do the Summit stuff too (I'm assuming that this is a relatively new phenomenon).

Re: tunings for many httpds...

2002-06-25 Thread Alfred Perlstein
* Terry Lambert [EMAIL PROTECTED] [020624 19:58] wrote: Alfred Perlstein wrote: * Terry Lambert [EMAIL PROTECTED] [020624 19:17] wrote: System V shared memory is allocated out of KVA space (annoying, but true). You keep saying this but the backing object allocated for sysvshm

Re: tunings for many httpds...

2002-06-25 Thread Terry Lambert
Alfred Perlstein wrote: You keep saying this but the backing object allocated for sysvshm is taken from either an OBJT_PHYS or OBJT_SWAP object. Uh, it's only ever an OBJT_SWAP; see line 532 of kern/sysv_shm.c. Your sources seem to be really out of date... Yes. I was referring to

Re: tunings for many httpds...

2002-06-25 Thread Alfred Perlstein
* Terry Lambert [EMAIL PROTECTED] [020625 13:08] wrote: At what point does it eat KVA that is other than for the backing data structures? It eats address space, not RAM. And even if the mappings are not active (which they usually are, because of LRU and processes accessing

Re: tunings for many httpds...

2002-06-25 Thread Matthew Dillon
: :This is also the _default_ for how solaris manages sysv segments, :although it would be nice if we could get the OBJT_PHYS stuff to :use 4meg pages (unless someone already did that?)... : :Anyhow, I'm glad we corrected your misconception and we now have :a more accurate understanding of how

Re: tunings for many httpds...

2002-06-25 Thread Patrick Thomas
Incidently, looking at the PV entry angle for a moment. Suppose you create a 1GB sysvshm (pageable) segment. That's 262144 pages. Mapping this once means you consume 262144 PV entries. At 28 bytes each, that is about 7.3MB of KVM. Now, fork this process 300 times. The numbers become

tunings for many httpds...

2002-06-24 Thread Patrick Thomas
As a splinter to the ongoing KVA/crash/memory discussion, I am wondering: - given a machine that will run 250+ httpds and another ~800 misc. processes, what system tunings would any of you suggest other than the ones I have done: In my kernel: maxusers=256 (was 512, change to 256

Re: tunings for many httpds...

2002-06-24 Thread Mike Silbersack
On Mon, 24 Jun 2002, Patrick Thomas wrote: Two things about this interested me: a) watching `top` output anytime of the day, i see several httpd processes in sbwait - granted I can only see 40 lines of processes or so in `top`, but usually at least two show sbwait. Worrisome ? sbwait is

Re: tunings for many httpds...

2002-06-24 Thread Terry Lambert
Patrick Thomas wrote: As a splinter to the ongoing KVA/crash/memory discussion, I am wondering: - given a machine that will run 250+ httpds and another ~800 misc. processes, what system tunings would any of you suggest other than the ones I have done: Can't comment. Best guess: whatever