Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
[ resuming an old thread ] "Luke Lonergan" <[EMAIL PROTECTED]> writes: > I also tried different settings (shmall set to 500K) and shmmax ended up at > -1. So, it seems that shmall is in pages. I didn't try other allowable > configs, which was a problem before. I went back and experimented some more with this issue, and I believe I've finally nailed down what the problem is: as of 10.4.something, OS X rejects attempts to set SHMMAX to anything that isn't an exact multiple of the page size (4096). Every other Unix on the planet, including earlier versions of OS X, will round off the value to whatever unit they feel like supporting ... but not Tiger. Thanks for this usability improvement, Apple. Also, the /etc/sysctl.conf file is utterly useless for setting SHM settings in 10.4.2, because whatever it sets will be overridden by the sysctl command that /etc/rc does just after reading /etc/sysctl.conf. Perhaps in prior versions, the behavior was "first set wins" ... but now it is definitely "last set wins". (SHMMAX eventually gets locked down and can't be changed anymore; I'm not sure what triggers that, but the sysctl in /etc/rc definitely gets to change what you put in /etc/sysctl.conf.) Perhaps Apple can be persuaded to change the order of these operations in /etc/rc, but until they do there is no point in suggesting that people use /etc/sysctl.conf. Bottom line: in 10.4.2, you have to edit /etc/rc to adjust SHMMAX, and you'd better be sure it is an exact multiple of 4096. I'll add a note about this to our documentation. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
Jeff, On 8/30/05 5:28 AM, "Jeff Trout" <[EMAIL PROTECTED]> wrote: > Just ran software update and (besides a couple apps) it had a > security update. I just did that to test this. > skittlebrau:~ postgres$ grep shm /etc/rc Luke-Lonergans-Computer:~ lukelonergan$ cat /etc/sysctl.conf kern.sysv.shmall=327680 kern.sysv.shmmax=335544320 #kern.sysv.shmall=268435456 #kern.sysv.shmmax=268435456 > skittlebrau:~ postgres$ sysctl -a | grep shm Luke-Lonergans-Computer:~ lukelonergan$ sysctl -a | grep shm kern.sysv.shmmax: 335544320 kern.sysv.shmmin: 1 kern.sysv.shmmni: 32 kern.sysv.shmseg: 8 kern.sysv.shmall: 327680 Thx for the tip. I also tried different settings (shmall set to 500K) and shmmax ended up at -1. So, it seems that shmall is in pages. I didn't try other allowable configs, which was a problem before. - Luke ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
On Aug 30, 2005, at 12:37 AM, Tom Lane wrote: Did that. Set shmall first, shmall second, both together in one sysctl command; no joy anywhere. Are you trying this on fully up-to-date Tiger? Just ran software update and (besides a couple apps) it had a security update. skittlebrau:~ postgres$ grep shm /etc/rc # sysctl -w kern.sysv.shmmax=4194305 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=1024 sysctl -w kern.sysv.shmmax=335544320 sysctl -w kern.sysv.shmmin=1 sysctl -w kern.sysv.shmmni=32 sysctl -w kern.sysv.shmseg=16 sysctl -w kern.sysv.shmall=327680 skittlebrau:~ postgres$ sysctl -a | grep shm kern.sysv.shmmax: 335544320 kern.sysv.shmmin: 1 kern.sysv.shmmni: 32 kern.sysv.shmseg: 16 kern.sysv.shmall: 327680 good luck. -- Jeff Trout <[EMAIL PROTECTED]> http://www.jefftrout.com/ http://www.stuarthamm.net/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
This is from my Powerbook in /etc/rc sysctl -w kern.sysv.shmmax=512 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=1024 I also had to up maxprocperuid to 200 to get buildfarm to run I'm pretty sure shmall had to be increased to allow shmmax to be increased. Dave On 30-Aug-05, at 12:37 AM, Tom Lane wrote: Jeff - <[EMAIL PROTECTED]> writes: On Aug 29, 2005, at 8:18 PM, Tom Lane wrote: Has anyone been able to set kern.sysv.shmmax above 4MB at all in latest OS X? yeah, you need to set shmmax and shmall. Did that. Set shmall first, shmall second, both together in one sysctl command; no joy anywhere. Are you trying this on fully up-to-date Tiger? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
Tom, On 8/29/05 9:37 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > Did that. Set shmall first, shmall second, both together in one sysctl > command; no joy anywhere. Are you trying this on fully up-to-date > Tiger? Did you try the values I sent earlier? If you set them both in /etc/sysctl.conf and reboot, it should work. We get 100% repeatability on 10.4.2 platforms. - Luke ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
Jeff - <[EMAIL PROTECTED]> writes: > On Aug 29, 2005, at 8:18 PM, Tom Lane wrote: >> Has anyone been able to set kern.sysv.shmmax above 4MB at all in >> latest OS X? > yeah, you need to set shmmax and shmall. Did that. Set shmall first, shmall second, both together in one sysctl command; no joy anywhere. Are you trying this on fully up-to-date Tiger? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
On Aug 29, 2005, at 10:37 PM, Tom Lane wrote: "Thomas F. O'Connell" <[EMAIL PROTECTED]> writes: After restarting, I have: # sysctl -a | grep shm kern.sysv.shmmax: 134217728 kern.sysv.shmmin: 1 kern.sysv.shmmni: 32 kern.sysv.shmseg: 8 kern.sysv.shmall: 1024 Of course, this still doesn't seem to be enough to let postgres play nicely with the iSight... Don't you need to increase shmall too? Also, which OS X release is this? 10.4 seems to have regressed compared to 10.3 :-( regards, tom lane This is 10.4.2. Everything worked as advertised. I didn't need to increase shmall for the settings to stick, but maybe I ought to for them to work. -- Thomas F. O'Connell Co-Founder, Information Architect Sitening, LLC Strategic Open Source: Open Your i™ http://www.sitening.com/ 110 30th Avenue North, Suite 6 Nashville, TN 37203-6320 615-469-5150 615-469-5151 (fax) ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
"Thomas F. O'Connell" <[EMAIL PROTECTED]> writes: > After restarting, I have: > # sysctl -a | grep shm > kern.sysv.shmmax: 134217728 > kern.sysv.shmmin: 1 > kern.sysv.shmmni: 32 > kern.sysv.shmseg: 8 > kern.sysv.shmall: 1024 > Of course, this still doesn't seem to be enough to let postgres play > nicely with the iSight... Don't you need to increase shmall too? Also, which OS X release is this? 10.4 seems to have regressed compared to 10.3 :-( regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
Tom, On 8/29/05 6:41 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > Interesting. I wonder if there's some bit of code that thinks that > SHMALL is measured in bytes (contrary to OSX's general convention > that it's measured in pages). I don't know, but I agree that the behavior has changed from Panther to Tiger. I wish BSD/Apple would go the way of Solaris 10 and be dynamic. This ipc/shmem limits business has been a plague since its introduction. - Luke ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
In order to get postgres working in concert with an iSight on a PowerBook, I had to increase shmmax, and it seemed to work just fine by editing /etc/rc: sysctl -w kern.sysv.shmmax=134217728 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=1024 After restarting, I have: # sysctl -a | grep shm kern.sysv.shmmax: 134217728 kern.sysv.shmmin: 1 kern.sysv.shmmni: 32 kern.sysv.shmseg: 8 kern.sysv.shmall: 1024 Of course, this still doesn't seem to be enough to let postgres play nicely with the iSight... -- Thomas F. O'Connell Co-Founder, Information Architect Sitening, LLC Strategic Open Source: Open Your i™ http://www.sitening.com/ 110 30th Avenue North, Suite 6 Nashville, TN 37203-6320 615-469-5150 615-469-5151 (fax) On Aug 29, 2005, at 7:18 PM, Tom Lane wrote: Has anyone been able to set kern.sysv.shmmax above 4MB at all in latest OS X? I just spent a while trying what seemed every possible permutation of setting up /etc/sysctl.conf and editing /etc/rc directly, and it just fails (symptom: sysctl shows shmmax as -1, and Postgres cannot start). Grrr. Apple's been pretty lame about this from day one, but at least it's been possible to do it one way or another. (Curiously, it seems you can now change shmall after bootup in 10.4.2, but that's not much help when you can't change shmmax.) regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
On Aug 29, 2005, at 8:18 PM, Tom Lane wrote: Has anyone been able to set kern.sysv.shmmax above 4MB at all in latest OS X? I just spent a while trying what seemed every possible permutation of setting up /etc/sysctl.conf and editing /etc/rc directly, and it just fails (symptom: sysctl shows shmmax as -1, and Postgres cannot start). yeah, you need to set shmmax and shmall. shmall seems to be measured in pages. Here's the settings I've been using: skittlebrau:/etc root# grep shm rc # sysctl -w kern.sysv.shmmax=4194305 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=1024 sysctl -w kern.sysv.shmmax=335544320 sysctl -w kern.sysv.shmmin=1 sysctl -w kern.sysv.shmmni=32 sysctl -w kern.sysv.shmseg=16 sysctl -w kern.sysv.shmall=327680 and just to verify: skittlebrau:/etc root# sysctl -a | grep shm kern.sysv.shmmax: 335544320 kern.sysv.shmmin: 1 kern.sysv.shmmni: 32 kern.sysv.shmseg: 16 kern.sysv.shmall: 327680 I've been running PG ith 10k shared bufs for a while. The pain is sometimes /ec/rc gets rewritten :( -- Jeff Trout <[EMAIL PROTECTED]> http://www.jefftrout.com/ http://www.stuarthamm.net/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
"Luke Lonergan" <[EMAIL PROTECTED]> writes: > Yes - it's very strange, I've had the same experience though I finally found > that setting SHMMAX and SHMALL to the same values, namely 268435456, seems > to work out fine. Interesting. I wonder if there's some bit of code that thinks that SHMALL is measured in bytes (contrary to OSX's general convention that it's measured in pages). regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
Tom Lane wrote: Has anyone been able to set kern.sysv.shmmax above 4MB at all in latest OS X? I just spent a while trying what seemed every possible permutation of setting up /etc/sysctl.conf and editing /etc/rc directly, and it just fails (symptom: sysctl shows shmmax as -1, and Postgres cannot start).' Is this perhaps one of the artificial limitations that they put in place to make you buy Apple OSX Server? Sincerely, Joshua D. Drake Grrr. Apple's been pretty lame about this from day one, but at least it's been possible to do it one way or another. (Curiously, it seems you can now change shmall after bootup in 10.4.2, but that's not much help when you can't change shmmax.) regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
Tom, On 8/29/05 5:18 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > Has anyone been able to set kern.sysv.shmmax above 4MB at all in latest > OS X? I just spent a while trying what seemed every possible > permutation of setting up /etc/sysctl.conf and editing /etc/rc directly, > and it just fails (symptom: sysctl shows shmmax as -1, and Postgres > cannot start). > > Grrr. Apple's been pretty lame about this from day one, but at least > it's been possible to do it one way or another. > > (Curiously, it seems you can now change shmall after bootup in 10.4.2, > but that's not much help when you can't change shmmax.) Yes - it's very strange, I've had the same experience though I finally found that setting SHMMAX and SHMALL to the same values, namely 268435456, seems to work out fine. I don't have a hypothesis as to why that value works though. I suspect that equal values of all sizes would work, though I stopped trying when I found this pair and it worked on multiple machines. - Luke ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] SHMMAX seems entirely broken in OS X 10.4.2
Has anyone been able to set kern.sysv.shmmax above 4MB at all in latest OS X? I just spent a while trying what seemed every possible permutation of setting up /etc/sysctl.conf and editing /etc/rc directly, and it just fails (symptom: sysctl shows shmmax as -1, and Postgres cannot start). Grrr. Apple's been pretty lame about this from day one, but at least it's been possible to do it one way or another. (Curiously, it seems you can now change shmall after bootup in 10.4.2, but that's not much help when you can't change shmmax.) regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq