Re: [HACKERS] write_pipe_chunks patch messes up early error message output
Tom Lane wrote: I think we probably need a flag variable separate from the GUC variable to tell when to send using the chunk protocol. Is there any reason we can't just use a check on whether SysLoggerPID is not 0? It should only be set if the syslogger has in fact started. cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Straightforward changes for increased SMP scalability
David Strong presented some excellent results of his SMP scalability testing at Ottawa in May. http://www.pgcon.org/2007/schedule/events/16.en.html There are some easy things we can do to take advantage of those results, especially the ones that were hardware independent. The hardware independent results were these two: - Avoid contention on WALInsertLock (+28% gain) - Increase NUM_BUFFER_PARTITIONS (+7.7% gain) Scalability begins to slow down at 8 CPUs on 8.2.4 and David was able to show good gains even at 8 CPUs with these changes. Proposals 1. For the first result, I suggest that we introduce some padding into the shmem structure XLogCtlData to alleviate false sharing that may exist between holders of WALInsertLock, WALWriteLock and info_lck. The cost of this will be at most about 200 bytes of shmem, with a low risk change. The benefits are hard to quantify, but we know this is an area of high contention and we should do all we can to reduce that. This hasn't been discussed previously, though we have seen good benefit from avoiding false sharing in other cases, e.g. LWLOCK padding. 2. Increase NUM_BUFFER_PARTITIONS from 16 to 256 (or higher). This has been discussed previously: http://archives.postgresql.org/pgsql-hackers/2006-09/msg00967.php Both of these changes are simple enough to consider for 8.3 Comments? -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] stored procedure stats in collector
Neil Conway wrote: (schemaname, procname, nargs) is still ambiguous in the face of function overloading. Although the presence of procid uniquely identifies each function anyway, if you're going to include the name and argument information, it might be worth including the argument types as well (as an array of regtype, perhaps). This is true. I was being a bit selfish here - in our environment we don't use overloading that much - schema, name and nargs are usually sufficient. It is also convinient to be able to do select * from pg... and have the output not to wrap around. From looking quickly at the patch, I don't think the current coding handles set-returning functions (ExecMakeTableFunctionResult). Hmm, should be handled. The number of tuples returned is not counted though, perhaps this is another thing to add ... Regards, Martin ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] compiler warnings on the buildfarm
Do any of the build farm machines not support 64-bit integers? I just added a --enable-bigint flag to configure.in and tested building without it and got an error at xlog.c: xlog.c: In function 'ValidXLOGHeader': xlog.c:3240: error: 'UINT64_FORMAT' undeclared (first use in this function) xlog.c:3240: error: (Each undeclared identifier is reported only once xlog.c:3240: error: for each function it appears in.) snprintf(fhdrident_str, sizeof(fhdrident_str), UINT64_FORMAT, longhdr-xlp_sysid); snprintf(sysident_str, sizeof(sysident_str), UINT64_FORMAT, ControlFile-system_identifier); It's possible I've done the autoconf hackery wrong though. Should UINT64_FORMAT still be defined if there's no int64? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] compiler warnings on the buildfarm
Gregory Stark [EMAIL PROTECTED] writes: It's possible I've done the autoconf hackery wrong though. Should UINT64_FORMAT still be defined if there's no int64? Yes. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] write_pipe_chunks patch messes up early error message output
Andrew Dunstan [EMAIL PROTECTED] writes: Is there any reason we can't just use a check on whether SysLoggerPID is not 0? (a) that really shouldn't be exported out of postmaster.c, and (b) it is not readily available to child backends is it? regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] write_pipe_chunks patch messes up early error message output
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Is there any reason we can't just use a check on whether SysLoggerPID is not 0? (a) that really shouldn't be exported out of postmaster.c, and (b) it is not readily available to child backends is it? It's already used in elog.c in Win32 code: if ((!Redirect_stderr || am_syslogger || (!IsUnderPostmaster SysLoggerPID==0)) pgwin32_is_service()) write_eventlog(edata-elevel, buf.data); Child backends might have an out of date version if we restart the Syslogger, but would that matter in this case? For current purposes all we need is to know that the syslogger has in fact started, ISTM. If that makes you puke we can do something more elegant, but I suspect it will amount to the same thing. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] write_pipe_chunks patch messes up early error message output
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: (a) that really shouldn't be exported out of postmaster.c, and (b) it is not readily available to child backends is it? It's already used in elog.c in Win32 code: if ((!Redirect_stderr || am_syslogger || (!IsUnderPostmaster SysLoggerPID==0)) pgwin32_is_service()) write_eventlog(edata-elevel, buf.data); Child backends might have an out of date version if we restart the Syslogger, but would that matter in this case? This code is already too ugly to live :-(. But aside from esthetics, there is a functional reason to have a separate flag variable. Consider the transient state where the syslogger has failed and we are trying to start a new one. If the postmaster wishes to elog anything (like, say, the log entry about the syslogger having failed) in this interval, then it *should* use the chunk protocol, because we expect that the data will eventually be eaten by the new syslogger. I think offhand that the correct semantics of the flag are we have redirected our original stderr into a pipe for syslogger, and in fact that we should transition the output format exactly at the instant where we do that; the starting of the child process happens at a slightly different time, and restarting of the child (if needed) is yet a different issue. Another thing that tracking such a flag would help us clean up is the syslogger's own elogging behavior. IIRC the original syslogger is launched with its stderr pointing to the original stderr, and so it's useful for any messages generated by syslogger itself to be copied onto that stderr. After a relaunch, though, this is no longer possible and it'd probably be best if syslogger doesn't even try writing to its stderr. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] write_pipe_chunks patch messes up early error message output
Is there any reason we can't just use a check on whether SysLoggerPID is not 0? (a) that really shouldn't be exported out of postmaster.c, and (b) it is not readily available to child backends is it? Should there be child backends when the logger did not start ? I'd think startup would be aborted if that happed ? Andreas ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] write_pipe_chunks patch messes up early error message output
Tom Lane wrote: I think offhand that the correct semantics of the flag are we have redirected our original stderr into a pipe for syslogger, and in fact that we should transition the output format exactly at the instant where we do that; the starting of the child process happens at a slightly different time, and restarting of the child (if needed) is yet a different issue. We could expose syslogger's redirection_done flag, which I think has the semantics you want. AFAICS it is never unset once set. (I should note that the distance between the point where this is set and where SysLoggerPID is set on return from SysLogger_Start is negligible, and no logging statements are called there, but I take your point about continuing to use chunking during a syslogger restart when SysLoggerPID might be 0.) cheers andrew ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Dealing with dangling index pointers
While looking at the HOT patch, I noticed that if there's an index tuple pointing to a non-existing heap tuple, we just silently ignore it. Such dangling index entries of course means that your database is corrupt, but we ought to handle that better. In the worst case, the heap slot is inserted to in the future, and then the bogus index entry points to a wrong tuple. ISTM we should print a warning suggesting a REINDEX, and kill the index tuple. Killing tuples in the face of corruption is dangerous, but in this case I think it's the right thing to do. We could also just emit the warning, but that could fill the logs quickly if the index tuple is accessed frequently. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] write_pipe_chunks patch messes up early error message output
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: I think offhand that the correct semantics of the flag are we have redirected our original stderr into a pipe for syslogger, We could expose syslogger's redirection_done flag, which I think has the semantics you want. Yeah, that would work. You'd have to get rid of the current ad-hoc method by which it is propagated to the syslogger child process (EXEC_BACKEND case), because now it will have to be propagated to all children; so postmaster.c should handle it in BackendParameters. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Dealing with dangling index pointers
Heikki Linnakangas [EMAIL PROTECTED] writes: While looking at the HOT patch, I noticed that if there's an index tuple pointing to a non-existing heap tuple, we just silently ignore it. This is intentional --- consider case where VACUUM has removed both index and heap entries while some other (amazingly slow...) process is in flight from the index to the heap. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Dealing with dangling index pointers
Ühel kenal päeval, E, 2007-07-16 kell 15:23, kirjutas Heikki Linnakangas: While looking at the HOT patch, I noticed that if there's an index tuple pointing to a non-existing heap tuple, we just silently ignore it. Such dangling index entries of course means that your database is corrupt, but we ought to handle that better. In the worst case, the heap slot is inserted to in the future, and then the bogus index entry points to a wrong tuple. ISTM we should print a warning suggesting a REINDEX, and kill the index tuple. Killing tuples in the face of corruption is dangerous, but in this case I think it's the right thing to do. We could also just emit the warning, but that could fill the logs quickly if the index tuple is accessed frequently. maybe issue a warning and set the DELETED index bit ? marking the invalid pointer as deleted should make it effectively disappear from use, without adding too much complexity - Hannu ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Dealing with dangling index pointers
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: While looking at the HOT patch, I noticed that if there's an index tuple pointing to a non-existing heap tuple, we just silently ignore it. This is intentional --- consider case where VACUUM has removed both index and heap entries while some other (amazingly slow...) process is in flight from the index to the heap. Hmm. In b-tree we keep the index page pinned while we do the heap fetch to avoid that, but apparently we don't have that interlock in other indexams. Ok, never mind. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Straightforward changes for increased SMP scalability
Simon Riggs wrote: Proposals 1. For the first result, I suggest that we introduce some padding into the shmem structure XLogCtlData to alleviate false sharing that may exist between holders of WALInsertLock, WALWriteLock and info_lck. The cost of this will be at most about 200 bytes of shmem, with a low risk change. The benefits are hard to quantify, but we know this is an area of high contention and we should do all we can to reduce that. This hasn't been discussed previously, though we have seen good benefit from avoiding false sharing in other cases, e.g. LWLOCK padding. 2. Increase NUM_BUFFER_PARTITIONS from 16 to 256 (or higher). This has been discussed previously: http://archives.postgresql.org/pgsql-hackers/2006-09/msg00967.php Both of these changes are simple enough to consider for 8.3 Comments? +1 on the idea (I can speak to the technical side). What I can say is that it is pretty much known that after 8 cores we slow down. Although 8.2 is better than any other release in this regard. Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Dealing with dangling index pointers
Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: This is intentional --- consider case where VACUUM has removed both index and heap entries while some other (amazingly slow...) process is in flight from the index to the heap. Hmm. In b-tree we keep the index page pinned while we do the heap fetch to avoid that, but apparently we don't have that interlock in other indexams. Right. This is actually connected to the fact that only btrees are used as system catalog indexes, and so only btrees need to be safe for use with SnapshotNow semantics. If VACUUM has managed to remove the target tuple while we are in flight, then it's further possible that someone else has inserted something new into that same tuple slot, and maybe even committed by the time we get there. Under SnapshotNow rules we would take the new tuple as a valid search result, though it (probably) doesn't actually satisfy the index search condition. With any MVCC-safe snapshot we will reject the new tuple as not meeting the snapshot. (BTW, this answers Teodor's question awhile back about whether he could use a GIN index in a system catalog. Nope, not without more work on index interlocking.) regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Straightforward changes for increased SMP scalability
Joshua D. Drake [EMAIL PROTECTED] writes: +1 on the idea (I can speak to the technical side). What I can say is that it is pretty much known that after 8 cores we slow down. Although 8.2 is better than any other release in this regard. Wait, what benchmarks have you seen where we slow down? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Straightforward changes for increased SMP scalability
Gregory Stark wrote: Joshua D. Drake [EMAIL PROTECTED] writes: +1 on the idea (I can speak to the technical side). What I can say is that it is pretty much known that after 8 cores we slow down. Although 8.2 is better than any other release in this regard. Wait, what benchmarks have you seen where we slow down? The production type. :) Hmm maybe that is a bad way to put it. I am not saying we slow down like we move slower than before. I mean per processor performance goes down. If I have 4 Cores things rock and roll. If I have 8 cores (and obvious sufficient workload) things rock and roll louder than 4 cores. If I have 16 cores, things are still really loud but I start to not be able to tell the difference. The percentage of improvement is much lower. E.g, 16 cores works and PostgreSQL work great, but it is not nearly as fantastic with 16 cores as 8 cores (in terms percentage gain). Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] minor compiler warning on OpenBSD
On Wed, Jul 11, 2007 at 07:18:17PM -0400, Tom Lane wrote: Now if we could only get rid of those flex-induced warnings in ecpg... Don't you get the same in the backend's parser code? I surely do. It seems these are only missing prototypes. How about adding an include file with those prototypes? Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Straightforward changes for increased SMP scalability
Simon Riggs wrote: Proposals 1. For the first result, I suggest that we introduce some padding into the shmem structure XLogCtlData to alleviate false sharing that may exist between holders of WALInsertLock, WALWriteLock and info_lck. The cost of this will be at most about 200 bytes of shmem, with a low risk change. The benefits are hard to quantify, but we know this is an area of high contention and we should do all we can to reduce that. This hasn't been discussed previously, though we have seen good benefit from avoiding false sharing in other cases, e.g. LWLOCK padding. 2. Increase NUM_BUFFER_PARTITIONS from 16 to 256 (or higher). This has been discussed previously: http://archives.postgresql.org/pgsql-hackers/2006-09/msg00967.php Both of these changes are simple enough to consider for 8.3 Comments? +1 on the idea (I can speak to the technical side). What I can say is that it is pretty much known that after 8 cores we slow down. Although 8.2 is better than any other release in this regard. Joshua D. Drake Here's a quick update. We're working on moving the patches we made against Postgres 8.2.4 to 8.3 to see what is still valid. So far, the base 8.3 shows ~7% improvement at 8 cores over 8.2.4. The NUM_BUFFER_PARTITIONS patch is fairly simple. We've noticed gains with NUM_BUFFER_PARTITIONS set between 256 and 2048, but little to no gain after 2048, although this might depend on the benchmark and platform being used. We've measured ~3% gain from the 8.3 base with NUM_BUFFER_PARTITIONS set to 2048. This might be the way this patch behaves with 8.3 or we might find that the NUM_BUFFER_PARTITIONS patch complements patch X as the 7.7% number reported for NUM_BUFFER_PARTITIONS in our presentation had a number of other patches enabled. This was also running at a 20 cores. We plan to start releasing patches this week for your consideration, along with their current gains. David -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Straightforward changes for increased SMP scalability
Joshua D. Drake wrote: Gregory Stark wrote: Joshua D. Drake [EMAIL PROTECTED] writes: +1 on the idea (I can speak to the technical side). What I can say is that it is pretty much known that after 8 cores we slow down. Although 8.2 is better than any other release in this regard. Wait, what benchmarks have you seen where we slow down? The production type. :) Hmm maybe that is a bad way to put it. I am not saying we slow down like we move slower than before. I mean per processor performance goes down. If I have 4 Cores things rock and roll. If I have 8 cores (and obvious sufficient workload) things rock and roll louder than 4 cores. If I have 16 cores, things are still really loud but I start to not be able to tell the difference. The percentage of improvement is much lower. E.g, 16 cores works and PostgreSQL work great, but it is not nearly as fantastic with 16 cores as 8 cores (in terms percentage gain). That's not the same thing as slowing down, it just means that scaling isn't always linear, which isn't surprising. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] minor compiler warning on OpenBSD
Michael Meskes [EMAIL PROTECTED] writes: On Wed, Jul 11, 2007 at 07:18:17PM -0400, Tom Lane wrote: Now if we could only get rid of those flex-induced warnings in ecpg... Don't you get the same in the backend's parser code? I surely do. No, ecpg is the only one producing warnings for me. What flex version do you use? It seems these are only missing prototypes. How about adding an include file with those prototypes? What I get with flex 2.5.4 is pgc.c: In function `base_yylex': pgc.c:1564: warning: label `find_rule' defined but not used preproc.y: At top level: pgc.c:3818: warning: `yy_flex_realloc' defined but not used neither of which seem fixable that way. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Straightforward changes for increased SMP scalability
Andrew Dunstan wrote: Joshua D. Drake wrote: Gregory Stark wrote: Joshua D. Drake [EMAIL PROTECTED] writes: If I have 16 cores, things are still really loud but I start to not be able to tell the difference. The percentage of improvement is much lower. E.g, 16 cores works and PostgreSQL work great, but it is not nearly as fantastic with 16 cores as 8 cores (in terms percentage gain). That's not the same thing as slowing down, it just means that scaling isn't always linear, which isn't surprising. Right. Which is why I reposted, but it also makes what Simon proposes that much more attractive *because* it helps the linear problem (in theory). Joshua D. Drake cheers andrew -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] minor compiler warning on OpenBSD
Tom Lane wrote: Michael Meskes [EMAIL PROTECTED] writes: On Wed, Jul 11, 2007 at 07:18:17PM -0400, Tom Lane wrote: Now if we could only get rid of those flex-induced warnings in ecpg... Don't you get the same in the backend's parser code? I surely do. No, ecpg is the only one producing warnings for me. What flex version do you use? It seems these are only missing prototypes. How about adding an include file with those prototypes? What I get with flex 2.5.4 is pgc.c: In function `base_yylex': pgc.c:1564: warning: label `find_rule' defined but not used preproc.y: At top level: pgc.c:3818: warning: `yy_flex_realloc' defined but not used neither of which seem fixable that way. I think Michael is refering to: In file included from bootparse.y:380: bootscanner.c:1855: warning: no previous prototype for ‘boot_yyget_lineno’ bootscanner.c:1864: warning: no previous prototype for ‘boot_yyget_in’ bootscanner.c:1872: warning: no previous prototype for ‘boot_yyget_out’ bootscanner.c:1880: warning: no previous prototype for ‘boot_yyget_leng’ bootscanner.c:1889: warning: no previous prototype for ‘boot_yyget_text’ bootscanner.c:1898: warning: no previous prototype for ‘boot_yyset_lineno’ bootscanner.c:1910: warning: no previous prototype for ‘boot_yyset_in’ bootscanner.c:1915: warning: no previous prototype for ‘boot_yyset_out’ bootscanner.c:1920: warning: no previous prototype for ‘boot_yyget_debug’ bootscanner.c:1925: warning: no previous prototype for ‘boot_yyset_debug’ bootscanner.c:1959: warning: no previous prototype for ‘boot_yylex_destroy’ In file included from gram.y:9663: scan.c:7050: warning: no previous prototype for ‘base_yyget_lineno’ scan.c:7059: warning: no previous prototype for ‘base_yyget_in’ scan.c:7067: warning: no previous prototype for ‘base_yyget_out’ scan.c:7075: warning: no previous prototype for ‘base_yyget_leng’ scan.c:7084: warning: no previous prototype for ‘base_yyget_text’ scan.c:7093: warning: no previous prototype for ‘base_yyset_lineno’ scan.c:7105: warning: no previous prototype for ‘base_yyset_in’ scan.c:7110: warning: no previous prototype for ‘base_yyset_out’ scan.c:7115: warning: no previous prototype for ‘base_yyget_debug’ scan.c:7120: warning: no previous prototype for ‘base_yyset_debug’ scan.c:7154: warning: no previous prototype for ‘base_yylex_destroy’ ... http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=Shaddt=2007-07-16%20053004stg=make Stefan ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Straightforward changes for increased SMP scalability
Simon Riggs [EMAIL PROTECTED] writes: 2. Increase NUM_BUFFER_PARTITIONS from 16 to 256 (or higher). Do you have any evidence to back up such a large increase? This change is not free; at the very least it will break contrib/pg_buffercache, which wants to lock all the partitions at once. lwlock.c was designed on the assumption that only a pretty small number of LWLocks would ever be held concurrently, and it will fall over. I don't think fixing this would be as simple as increasing MAX_SIMUL_LWLOCKS, because some of the algorithms are O(N^2). I'd like to see numbers proving that there is useful incremental gain from going above 32 or 64 partitions, before we start hacking to make this work. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] stored procedure stats in collector
Tom Lane wrote: I really dislike that approach to deciding which functions to count. The main problem with it is that it will try to count C-language functions that are added after initdb, such as contrib stuff and third-party add-ons like postgis. The percentage overhead for a typical short C function will be large, and I'm not sure anything much is to be gained by counting these. Agreed, it is a bit ugly. In the initial version there was no such exclusion. This came only after I accidentally activated the collection on a busy server. Checked cpu usage some half an hour later and immediately disabled it. System mode consumption was unacceptably high, but the results indicated that something funny was going on inside application as well - some internal functions such as texteq() and now() had unreasonably high call counts. I'm still investigating those issues, but maybe it might be useful to have stats for internal and C language functions as well? Perhaps it is possible to make the collection configurable per language. So that for instance, default is to only collect pl languages, with the option to add C or internal. Not sure how to approach this though. I think a more reasonable approach would be to count PL-language functions; which in turn suggests that the instrumentation hooks should be in the PL call handlers, not in ExecMakeFunctionResult and friends where they will drag down performance of all functions. It would be great if all the procedural languages would go through a single entry point. Right now the more exotic PL-s would need separate patching. But indeed, this approach would be less intrusive. BTW, I dunno if you've thought about the implications of inlining of SQL functions ... it's hard to see how to count those reasonably. Yes, this creates some inconsistencies in what is collected and what not. Unless, of course, only PL functions are counted :) Regards, Martin ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] minor compiler warning on OpenBSD
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Tom Lane wrote: What I get with flex 2.5.4 is pgc.c: In function `base_yylex': pgc.c:1564: warning: label `find_rule' defined but not used preproc.y: At top level: pgc.c:3818: warning: `yy_flex_realloc' defined but not used neither of which seem fixable that way. I think Michael is refering to: In file included from bootparse.y:380: bootscanner.c:1855: warning: no previous prototype for=20 =E2=80=98boot_yyget_lineno=E2=80=99 bootscanner.c:1864: warning: no previous prototype for =E2=80=98boot_yyge= t_in=E2=80=99 bootscanner.c:1872: warning: no previous prototype for =E2=80=98boot_yyge= t_out=E2=80=99 bootscanner.c:1880: warning: no previous prototype for =E2=80=98boot_yyge= t_leng=E2=80=99 bootscanner.c:1889: warning: no previous prototype for =E2=80=98boot_yyge= t_text=E2=80=99 [ shrug... ] Those are flex bugs. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Straightforward changes for increased SMP scalability
Tom, I'm happy to run some benchmarks to show the improvements with various NUM_BUFFER_PARTITIONS settings. However, I want to make sure that this is going to be useful. I can run 16 (base), 32, 64, 128 etc. type increments, but I'm more concerned about the number of cores to use. Do you have a suggestion for that? I can run with 1 to 32 cores. I had planned to run a number of tests at 8 cores, but I can adjust to what makes sense for the community. David -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom Lane Sent: Monday, July 16, 2007 9:10 AM To: Simon Riggs Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Straightforward changes for increased SMP scalability Simon Riggs [EMAIL PROTECTED] writes: 2. Increase NUM_BUFFER_PARTITIONS from 16 to 256 (or higher). Do you have any evidence to back up such a large increase? This change is not free; at the very least it will break contrib/pg_buffercache, which wants to lock all the partitions at once. lwlock.c was designed on the assumption that only a pretty small number of LWLocks would ever be held concurrently, and it will fall over. I don't think fixing this would be as simple as increasing MAX_SIMUL_LWLOCKS, because some of the algorithms are O(N^2). I'd like to see numbers proving that there is useful incremental gain from going above 32 or 64 partitions, before we start hacking to make this work. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Straightforward changes for increased SMP scalability
Strong, David [EMAIL PROTECTED] writes: I'm happy to run some benchmarks to show the improvements with various NUM_BUFFER_PARTITIONS settings. However, I want to make sure that this is going to be useful. I can run 16 (base), 32, 64, 128 etc. type increments, but I'm more concerned about the number of cores to use. Do you have a suggestion for that? I can run with 1 to 32 cores. I had planned to run a number of tests at 8 cores, but I can adjust to what makes sense for the community. Presumably the answers will be different. I'd sort of like to see several different curves for different numbers of processors, so we can evaluate reasonably fairly. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Straightforward changes for increased SMP scalability
I'm happy to run some benchmarks to show the improvements with various NUM_BUFFER_PARTITIONS settings. However, I want to make sure that this is going to be useful. I can run 16 (base), 32, 64, 128 etc. type increments, but I'm more concerned about the number of cores to use. Do you have a suggestion for that? I can run with 1 to 32 cores. I had planned to run a number of tests at 8 cores, but I can adjust to what ? makes sense for the community. Presumably the answers will be different. I'd sort of like to see several different curves for different numbers of processors, so we can evaluate reasonably fairly. regards, tom lane Tom, Correct. This is a scalability patch rather than a performance patch, although each aspect is related. I would expect the gain to be better as more cores and users are added. I can run some tests along the following lines: 1. NUM_BUFFER_PARITIONS sizes for 16, 32, 64, 128, 256, 512, 1024, 2048. 2. Cores set at 1, 2, 4, 8, 16, 24 and 32. Does anyone have any comments or suggestions? David ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Straightforward changes for increased SMP scalability
On Mon, Jul 16, 2007 at 01:23:46PM +0100, Simon Riggs wrote: Both of these changes are simple enough to consider for 8.3 I'm in favour of scalability, of course, but are they really simple enough to put in for 8.3? I was under the impression that there was a push on to get the thing shipped, and adding incremental changes near the end of the cycle strikes me as a possible source of significant additional surprises (and therefore delays). I am no code expert, though; I just wanted to be sure there's consensus on the simplicity of the changes. A -- Andrew Sullivan | [EMAIL PROTECTED] This work was visionary and imaginative, and goes to show that visionary and imaginative work need not end up well. --Dennis Ritchie ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] compiler warnings on the buildfarm
Stefan Kaltenbrunner wrote: Zdenek Kotala wrote: Stefan Kaltenbrunner wrote: Zdenek Kotala wrote: For sun studio -erroff=E_STATEMENT_NOT_REACHED is useful there. If you want to determine warning tags for each warning add -errtags. Is that supported on all versions of sun studio(Sun WorkShop 6, Sun Studio 8,11) we have on the farm ? Yes. Also on SS12. hmm - sure about that ? I was about to submit a patch to disable some compiler warnings but then I noted the following discussion thread: http://forum.java.sun.com/thread.jspa?threadID=5163903messageID=9637391 which seems to indicate that at least the compiler installed on kudu: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=kududt=2007-07-15%2003:30:01 does NOT support turning of specific warnings. I tested it on cc version 5.3 and it works. See -bash-3.00$ SUNWspro/SC6.2/bin/cc -V cc: Sun WorkShop 6 update 2 C 5.3 Patch 111680-09 2003/05/18 usage: cc [ options] files. Use 'cc -flags' for details -bash-3.00$ /ws/onnv-tools/SUNWspro/SC6.2/bin/cc pokus.c pokus.c, line 5: warning: statement not reached -bash-3.00$ /ws/onnv-tools/SUNWspro/SC6.2/bin/cc -errtags pokus.c pokus.c, line 5: warning: statement not reached (E_STATEMENT_NOT_REACHED) -bash-3.00$ /ws/onnv-tools/SUNWspro/SC6.2/bin/cc -erroff=E_STATEMENT_NOT_REACHED pokus.c -bash-3.00$ It works since Sun Workshop 4.2 (cc: WorkShop Compilers 4.2 26 Jun 1997 C 4.2 patch 105062-01). I tested it also on SunWorkshop 2.0.1 and it does not work there, but I belive that nobody uses ten years old compiler :-). Please, can you send me a cc -V output ( I think It should be added in log). Zdenek ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] bit string functions
I have been working extensively with the bit string data type. I have a number of useful c-language functions to set/clear a bit, count number of bits set, inquire if a bit is set/clear, etc. I don't see functions like these as part of any SQL standard, (although I think they ought to be). I would like to make these a part of postgresql for others to use. Is it more appropriate for these to be in contrib code or part of the postgresql proper? How can I contribute these? TJ -- TJ O'Donnell, Ph.D. President, gNova Inc. [EMAIL PROTECTED] http://www.gnova.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] SSPI authentication
A quick status update on the SSPI authentication part of the GSSAPI project. I have libpq SSPI working now, with a few hardcoded things still in there to be fixed. But it means that I can connect to a linux server using kerberos/GSSAPI *without* the need to set up MIR Kerberos libraries and settings on the client. This is great :-) The code is fairly trivial. I've set it up as a different way of doing GSSAPI authentication. This means that if you can't have both SSPI and MIT KRB GSSAPI in the same installation. I don't see a problem with this - 99.9% of windows users will just want the SSPI version anyway. But I figured I'd throw it out here to see if there are any objections to this? I'd like to make this enabled by default on Win32, since all supported windows platforms have support for it. Then we can add a configure option to turn it *off* if we want to. Comments? Do we even need such an option? Right now, the SSPI path is hardcoded to just support Kerberos. Once we have both client and server with SSPI support I see no reason to keep this restriction. Anybody against that? (Not saying that'll happen for 8.3, because it certainly needs a bunch of extra testing, but eventually) //Magnus ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] SSPI authentication
* Magnus Hagander ([EMAIL PROTECTED]) wrote: I've set it up as a different way of doing GSSAPI authentication. This means that if you can't have both SSPI and MIT KRB GSSAPI in the same installation. I don't see a problem with this - 99.9% of windows users will just want the SSPI version anyway. But I figured I'd throw it out here to see if there are any objections to this? I'm not quite sure if that would affect what we do but it sounds like it might. The main thing we use on the clients wrt Postgres is the ODBC driver but I've used psql once or twice and have been trying to get people to learn it. We've got SSPI which is used for the Windows domain (and only the windows resources) and then MIT Krb5 GSSAPI for the Unix resources. While cross-realm is a nice idea it's less than easy to get going, especially with even a half-way secure key (I'm not exactly a big fan of arc/rc4...). So, we have seperate key caches on each client that needs access to both resources and that allows us to manage things much more easily and seperately from the corporate folks running the Windows domain. Additionally, it seems likely to me that there will be cases when people running Windows don't *want* to set up an Active Directory for their Windows machines but want to use Kerberos to auth to certain resources (perhaps a campus environment where student systems aren't joined to an AD domain?). Would that be possible with this? I havn't done much w/ SSPI so I'm not sure how deeply that's tied into things like that. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] SSPI authentication
Stephen Frost wrote: * Magnus Hagander ([EMAIL PROTECTED]) wrote: I've set it up as a different way of doing GSSAPI authentication. This means that if you can't have both SSPI and MIT KRB GSSAPI in the same installation. I don't see a problem with this - 99.9% of windows users will just want the SSPI version anyway. But I figured I'd throw it out here to see if there are any objections to this? I'm not quite sure if that would affect what we do but it sounds like it might. The main thing we use on the clients wrt Postgres is the ODBC driver but I've used psql once or twice and have been trying to get people to learn it. ODBC driver should work with it - I don't know exactly how they plug into libpqs auth, but IIRC they do some stuff to make that work. Note that I'm only talking about being mutually exclusiv ewith MIT KRB GSSAPI, not with MIT KRB in krb5 mode. Though I very much want to deprecate the native kerberos auth in favor of GSSAPI as soon as possible for several reasons, so I'd suggest you don't use that once you go to 8.3 ;-) We've got SSPI which is used for the Windows domain (and only the windows resources) and then MIT Krb5 GSSAPI for the Unix resources. While cross-realm is a nice idea it's less than easy to get going, especially with even a half-way secure key (I'm not exactly a big fan of arc/rc4...). I have my Unix machines in the Active Directory, so there's no cross realm. It works fine. And if you don't trust the key, put it over SSL? ;-) If you use SSL, GSSAPI packets actually go through the SSL tunnel, unlike krb5 auth. Additionally, it seems likely to me that there will be cases when people running Windows don't *want* to set up an Active Directory for their Windows machines but want to use Kerberos to auth to certain resources (perhaps a campus environment where student systems aren't joined to an AD domain?). Would that be possible with this? I havn't done much w/ SSPI so I'm not sure how deeply that's tied into things like that. Yes, there's still support for doing GSSAPI with MIT KRB5. It's just that you have to use it *instead* of SSPI. So a rebuild is necessary. But - IIRC, you can just join your windows machine to your unix kerberos realm and be done with it - SSPI APIs should work fine in that case. //Magnus ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] bit string functions
On Mon, Jul 16, 2007 at 09:40:18AM -0700, TJ O'Donnell wrote: I would like to make these a part of postgresql for others to use. Is it more appropriate for these to be in contrib code or part of the postgresql proper? How can I contribute these? I would say just set up a project on pgfoundry. A -- Andrew Sullivan | [EMAIL PROTECTED] The very definition of news is something that hardly ever happens. --Bruce Schneier ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] SSPI authentication
* Magnus Hagander ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: I'm not quite sure if that would affect what we do but it sounds like it might. The main thing we use on the clients wrt Postgres is the ODBC driver but I've used psql once or twice and have been trying to get people to learn it. ODBC driver should work with it - I don't know exactly how they plug into libpqs auth, but IIRC they do some stuff to make that work. I wouldn't be so sure... I'm not exactly a fan of how ODBC does that anyway, it essentially uses libpq for the auth, *sometimes*, and then hijacks the connection away. Note that I'm only talking about being mutually exclusiv ewith MIT KRB GSSAPI, not with MIT KRB in krb5 mode. Though I very much want to deprecate the native kerberos auth in favor of GSSAPI as soon as possible for several reasons, so I'd suggest you don't use that once you go to 8.3 ;-) The KfW stuff from MIT provides both GSSAPI and 'native' kerberos, I believe, and most things use the GSSAPI side of it, actually. We've got SSPI which is used for the Windows domain (and only the windows resources) and then MIT Krb5 GSSAPI for the Unix resources. While cross-realm is a nice idea it's less than easy to get going, especially with even a half-way secure key (I'm not exactly a big fan of arc/rc4...). I have my Unix machines in the Active Directory, so there's no cross realm. It works fine. Yeah, that requires quite a bit more involvement between us and the corporate folks, and means that we're dependent on them to do things before we can do things. That tends to end badly. And if you don't trust the key, put it over SSL? ;-) If you use SSL, GSSAPI packets actually go through the SSL tunnel, unlike krb5 auth. Uhh, the client and the KDC don't generally use SSL to talk to each other, last I checked, and the problem is with the cross-realm key (you know, the one that you could use to fake anyone from the trusted realm) having to be least-common-denominator between Windows and Unix since it has to exist in both KDCs. That wouldn't be too much trouble if that least-common-denominator was AES256 but at the moment it's not. Additionally, it seems likely to me that there will be cases when people running Windows don't *want* to set up an Active Directory for their Windows machines but want to use Kerberos to auth to certain resources (perhaps a campus environment where student systems aren't joined to an AD domain?). Would that be possible with this? I havn't done much w/ SSPI so I'm not sure how deeply that's tied into things like that. Yes, there's still support for doing GSSAPI with MIT KRB5. It's just that you have to use it *instead* of SSPI. So a rebuild is necessary. The way this is handled in a number of other applications (putty being the one that comes to mind easily) is that two DLLs are built- one for SSPI and one for GSSAPI and you can easily switch between them on the client. That'd work fine for us. I don't like the idea of having to rebuild things under Windows, honestly.. Not that I like to build anything these days... If it's not enabled by default in some way I expect that it'd get 'forgotten'. But - IIRC, you can just join your windows machine to your unix kerberos realm and be done with it - SSPI APIs should work fine in that case. I don't think that's generally an option, again, in a university-type setting, even if you had a unix box. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] bit string functions
Andrew Sullivan [EMAIL PROTECTED] writes: On Mon, Jul 16, 2007 at 09:40:18AM -0700, TJ O'Donnell wrote: I would like to make these a part of postgresql for others to use. Is it more appropriate for these to be in contrib code or part of the postgresql proper? How can I contribute these? I would say just set up a project on pgfoundry. I agree, though I think in the long term we do need a more complete set of operators and functions in core. But we need consensus on which set people find necessary and pgfoundry is a good place to do that. I think the main guiding force will be which sets of operators and functions become necessary to have operator classes for indexes. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] bit string functions
Gregory Stark [EMAIL PROTECTED] writes: Andrew Sullivan [EMAIL PROTECTED] writes: I would say just set up a project on pgfoundry. I agree, though I think in the long term we do need a more complete set of operators and functions in core. Considering that BIT and BIT VARYING have been removed entirely from SQL:2003, it seems unlikely to me that we should expend our limited resources in that particular direction. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] SSPI authentication
Stephen Frost wrote: * Magnus Hagander ([EMAIL PROTECTED]) wrote: Stephen Frost wrote: I'm not quite sure if that would affect what we do but it sounds like it might. The main thing we use on the clients wrt Postgres is the ODBC driver but I've used psql once or twice and have been trying to get people to learn it. ODBC driver should work with it - I don't know exactly how they plug into libpqs auth, but IIRC they do some stuff to make that work. I wouldn't be so sure... I'm not exactly a fan of how ODBC does that anyway, it essentially uses libpq for the auth, *sometimes*, and then hijacks the connection away. Yeah, I'm not a fan of that either, but as long as it works.. Which leads me to the question - does it work for GSSAPI? Anybody from the ODBC crowd who can comment on that? Note that I'm only talking about being mutually exclusiv ewith MIT KRB GSSAPI, not with MIT KRB in krb5 mode. Though I very much want to deprecate the native kerberos auth in favor of GSSAPI as soon as possible for several reasons, so I'd suggest you don't use that once you go to 8.3 ;-) The KfW stuff from MIT provides both GSSAPI and 'native' kerberos, I believe, and most things use the GSSAPI side of it, actually. We're pretty much the *only* major player who use native kerberos, AFAIK. Back two years ago (I think it was) I found a really trivial security hole in it (in the MIT code) that was exposed by a trivial user input error. If anybody else was using it, they'd have found that one long ago ;-) GSSAPI really is the way to go. We've got SSPI which is used for the Windows domain (and only the windows resources) and then MIT Krb5 GSSAPI for the Unix resources. While cross-realm is a nice idea it's less than easy to get going, especially with even a half-way secure key (I'm not exactly a big fan of arc/rc4...). I have my Unix machines in the Active Directory, so there's no cross realm. It works fine. Yeah, that requires quite a bit more involvement between us and the corporate folks, and means that we're dependent on them to do things before we can do things. That tends to end badly. Heh. And if you don't trust the key, put it over SSL? ;-) If you use SSL, GSSAPI packets actually go through the SSL tunnel, unlike krb5 auth. Uhh, the client and the KDC don't generally use SSL to talk to each other, last I checked, and the problem is with the cross-realm key (you know, the one that you could use to fake anyone from the trusted realm) having to be least-common-denominator between Windows and Unix since it has to exist in both KDCs. That wouldn't be too much trouble if that least-common-denominator was AES256 but at the moment it's not. Hm, Ok, thought you meant client-server. Anyway, then use ipsec :-) Additionally, it seems likely to me that there will be cases when people running Windows don't *want* to set up an Active Directory for their Windows machines but want to use Kerberos to auth to certain resources (perhaps a campus environment where student systems aren't joined to an AD domain?). Would that be possible with this? I havn't done much w/ SSPI so I'm not sure how deeply that's tied into things like that. Yes, there's still support for doing GSSAPI with MIT KRB5. It's just that you have to use it *instead* of SSPI. So a rebuild is necessary. The way this is handled in a number of other applications (putty being the one that comes to mind easily) is that two DLLs are built- one for SSPI and one for GSSAPI and you can easily switch between them on the client. That'd work fine for us. Well, that you can do - you just need one libpq with sspi and one with gssapi. I don't like the idea of having to rebuild things under Windows, honestly.. Not that I like to build anything these days... If it's not enabled by default in some way I expect that it'd get 'forgotten'. Ok, so looking at it from the other direction, say we wanted to support both. Then we need to invent a new way for the client to tell libpq which one to use. I think that's sensible if it's a common thing, but I still see it as a *very* narrow use-case that needs both in the same DLL. Or do you have a better idea on how to solve that? //Magnus ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] SSPI authentication
* Magnus Hagander ([EMAIL PROTECTED]) wrote: The way this is handled in a number of other applications (putty being the one that comes to mind easily) is that two DLLs are built- one for SSPI and one for GSSAPI and you can easily switch between them on the client. That'd work fine for us. Well, that you can do - you just need one libpq with sspi and one with gssapi. If both are made available then I think that'd work fine for us. I'm concerned that the windows builds wouldn't include a version of libpq w/ GSSAPI... If I was confident that we could easily build it ourselves then I wouldn't care as much but, since I've never had to build libpq on Windows before, I'm not sure what effort is involved or what tools are required. I'm also not thrilled by the prospect. :) I don't like the idea of having to rebuild things under Windows, honestly.. Not that I like to build anything these days... If it's not enabled by default in some way I expect that it'd get 'forgotten'. Ok, so looking at it from the other direction, say we wanted to support both. Then we need to invent a new way for the client to tell libpq which one to use. I think that's sensible if it's a common thing, but I still see it as a *very* narrow use-case that needs both in the same DLL. Or do you have a better idea on how to solve that? Supporting both is actually exactly what Mozilla does... Check out the 'network.auth.use-sspi' flag in about:config. It's also what KfW does Include Windows LSA cache and Import windows credentials, which has the interesting option of only when principals match. I'm not sure if there's a sane way to test at run-time if KfW exists but its existance could be used as a factor. I have to admit that this does kind of make me wish a bit for a 'libpq config file' even though I'm generally against such things. Having the same easy switch as we do w/ Mozilla would be really nice. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] bit string functions
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Andrew Sullivan [EMAIL PROTECTED] writes: I would say just set up a project on pgfoundry. I agree, though I think in the long term we do need a more complete set of operators and functions in core. Considering that BIT and BIT VARYING have been removed entirely from SQL:2003, it seems unlikely to me that we should expend our limited resources in that particular direction. Hm, just thinking aloud here but, in our type system I wonder how hard it would be to write a special data type to use for _boolean. Offhand anyarray and anyelement might do funny things but if it supplies *all* the array operators and functions perhaps it would just work. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] compiler warnings on the buildfarm
Zdenek Kotala wrote: Stefan Kaltenbrunner wrote: Zdenek Kotala wrote: Stefan Kaltenbrunner wrote: Zdenek Kotala wrote: For sun studio -erroff=E_STATEMENT_NOT_REACHED is useful there. If you want to determine warning tags for each warning add -errtags. Is that supported on all versions of sun studio(Sun WorkShop 6, Sun Studio 8,11) we have on the farm ? Yes. Also on SS12. hmm - sure about that ? I was about to submit a patch to disable some compiler warnings but then I noted the following discussion thread: http://forum.java.sun.com/thread.jspa?threadID=5163903messageID=9637391 which seems to indicate that at least the compiler installed on kudu: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=kududt=2007-07-15%2003:30:01 does NOT support turning of specific warnings. I tested it on cc version 5.3 and it works. See ah cool - thanks for testing! so on my box we would need to add -erroff=E_EMPTY_TRANSLATION_UNIT,E_STATEMENT_NOT_REACHED,E_END_OF_LOOP_CODE_NOT_REACHED,E_FUNC_HAS_NO_RETURN_STMT,E_LOOP_NOT_ENTERED_AT_TOP to CFLAGS to get down to the following 2 warnings: pgstat.c, line 652: warning: const object should have initializer: all_zeroes (E_CONST_OBJ_SHOULD_HAVE_INITIZR) pgstat.c, line 2118: warning: const object should have initializer: all_zeroes (E_CONST_OBJ_SHOULD_HAVE_INITIZR) the open question is if that is what want or if we would be simply adding (unnecessary) complexity (or confusion). comments ? Stefan ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] compiler warnings on the buildfarm
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: pgstat.c, line 652: warning: const object should have initializer: all_zeroes (E_CONST_OBJ_SHOULD_HAVE_INITIZR) pgstat.c, line 2118: warning: const object should have initializer: all_zeroes (E_CONST_OBJ_SHOULD_HAVE_INITIZR) Man, even these are bogus. And that would be an interesting warning too if they made it not fire when it's bogus. bleagh. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] What is the maximum encoding-conversion growth rate, anyway?
Where are we on this? --- Tom Lane wrote: I just rearranged the code in mbutils.c a little bit to make it more robust if conversion of an over-length string is attempted, and noted this comment: /* * When converting strings between different encodings, we assume that space * for converted result is 4-to-1 growth in the worst case. The rate for * currently supported encoding pairs are within 3 (SJIS JIS X0201 half width * kanna - UTF8 is the worst case). So 4 should be enough for the moment. * * Note that this is not the same as the maximum character width in any * particular encoding. */ #define MAX_CONVERSION_GROWTH 4 It strikes me that this is overly pessimistic, since we do not support 5- or 6-byte UTF8 characters, and AFAICS there are no 1-byte characters in any supported encoding that require 4 bytes in another. Could we reduce the multiplier to 3? Or even 2? This has a direct impact on the longest COPY lines we can support, so I'd like it not to be larger than necessary. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian [EMAIL PROTECTED] http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Re: [COMMITTERS] pgsql: Create hooks to let a loadable plugin monitor (or even replace)
Gurjeet, do you have a patch to be applied for this?
---
Gurjeet Singh wrote:
On 5/30/07, Tom Lane [EMAIL PROTECTED] wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
Gurjeet Singh wrote:
But I did not understand the haste to commit the patch within almost
half an
hour of proposing the second version of the patch!!!
It happens some times when a patch applier has gotten as far as they can
go with a patch and wants to move on, with the willingness to return to
the patch if there is any additional feedback.
Er, it was quite a bit more than half an hour; about 17 hours in fact:
http://archives.postgresql.org/pgsql-patches/2007-05/msg00421.php
http://archives.postgresql.org/pgsql-committers/2007-05/msg00315.php
I was referring to these two:
http://archives.postgresql.org/pgsql-patches/2007-05/msg00431.php and
http://archives.postgresql.org/pgsql-committers/2007-05/msg00315.php
In any case this patch was just a working-out of ideas I'd proposed more
than a month previously, so I didn't expect it to be controversial.
But as Bruce says, nothing is set in stone at this point. If you have
suggestions for improvements, we can tweak the hooks pretty much any
time up till 8.3 final.
This being a community effort, we would expect that.
I also wished to propose to allow the plugin to completely replace (or
augment) the plan produced by the planner (by passing in a double-pointer of
the plan to the plugin); but I was wary that the idea might get rejected,
for being too radical an idea.
In the last version of the planner plugin patch, the plugins were maintained
as a list, hence allowing for multiple post-planner-plugins to work one
after the other (the variable PPPList); much like the layered I/O driver
architecture of Windows' NTFS sans the guarantee of ordering between the
plugins. To this we may add the ability to pass on the result plan of one
plugin to the next, letting them improve the plan incrementally. Next, we
can add string identifiers like I/O drivers to guarantee the order in which
the plugins will be executed. But again, maybe we don't need multiple
planners working simultaneously ATM.
As for the current patch,I had only a few cosmetic changes in mind:
The comment above planner.c:planner() says '...hook variable that lets a
plugin get control before and after the standard planning ...'; but if we
look at the code, we are just replacing the call to standard_planner(); we
are not calling the plugin before and after standard_planner().
Also, another cosmetic change like reducing an 'if' as follows:
Change:
PlannedStmt *
planner(Query *parse, int cursorOptions, ParamListInfo boundParams)
{
PlannedStmt *result;
if (planner_hook)
result = (*planner_hook) (parse, cursorOptions, boundParams);
else
result = standard_planner(parse, cursorOptions, boundParams);
return result;
}
To:
PlannedStmt *
planner(Query *parse, int cursorOptions, ParamListInfo boundParams)
{
planner_hook_type planner_func = planner_hook ? planner_hook :
standard_planner;
return (*planner_func) (parse, cursorOptions, boundParams);
}
The extra IFs only disorient a normal flow of logic. These two statements
aren't too complicated for readability.
Best regards,
PS: We can make the code more compact (at the cost of readability) like so:
return (*(planner_hook ? planner_hook : standard_planner))(parse,
cursorOptions, boundParams);
--
[EMAIL PROTECTED]
[EMAIL PROTECTED] gmail | hotmail | yahoo }.com
17?29'34.37N 78?30'59.76E - Hyderabad *
18?32'57.25N 73?56'25.42E - Pune
Sent from my BlackLaptop device
--
Bruce Momjian [EMAIL PROTECTED] http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] Working with CVS documentation
I have added the CVS Wiki URL to our CVS docs section. --- Greg Smith wrote: I've now finished up initial content generation on the wiki page that covers using the CVS repository: http://developer.postgresql.org/index.php/Working_with_CVS That includes all the helpful comments suggested on this list in last month's threads on this topic along with a full dump of what was in my brain. The most complete section expands Heikki's workflow into a fleshed out example I think is good enough for new potential developers to use. I've also put some notes on the Discussion page of this article that lead to links on recent general trends in this area. Anyone who isn't familiar with distributed version control systems like Git or Mercurial should find those references one way to get up to speed on what's going on there. With high-profile projects like Apache recently committing to DVCS work these tools are really becoming mainstream. Some of that reading is both informative and periodically hilarious; my favorite quote is from Linus Torvalds, who says if you actually like using CVS, you shouldn't be here. You should be in some mental institution. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster -- Bruce Momjian [EMAIL PROTECTED] http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] IsTransactionState() is being used incorrectly
Is this done or should it be kept for 8.4? --- Tom Lane wrote: I just noticed that there are a number of places (mostly GUC assignment hooks) that use IsTransactionState() to decide if it's safe for them to do catalog lookups. This seems pretty bogus because IsTransactionState will return true in an aborted transaction. I'm not sure there's any actual bug because of other constraints on when GUC updates occur, but it sure looks like trouble waiting to happen. We could fix this either by changing the definition of IsTransactionState() or by introducing another test function with a different name. Any thoughts which is preferable? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster -- Bruce Momjian [EMAIL PROTECTED] http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] IsTransactionState() is being used incorrectly
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: We could fix this either by changing the definition of IsTransactionState() or by introducing another test function with a different name. Any thoughts which is preferable? Is this done or should it be kept for 8.4? Fixed, I thought ... yeah, here: 2007-06-07 17:45 tgl * src/backend/: access/transam/xact.c, storage/ipc/procarray.c, utils/error/elog.c: Redefine IsTransactionState() to only return true for TRANS_INPROGRESS state, which is the only state in which it's safe to initiate database queries. It turns out that all but two of the callers thought that's what it meant; and the other two were using it as a proxy for will GetTopTransactionId() return a nonzero XID? Since it was in fact an unreliable guide to that, make those two just invoke GetTopTransactionId() always, then deal with a zero result if they get one. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] msvc, build and install with cygwin in the PATH
Magnus, what is your reaction to this patch?
---
Hannes Eder wrote:
Magnus Hagander wrote:
Hannes Eder wrote:
Is it worth doing this the Perl-way and using File::Find? If so, I
can
work an a patch for that.
It's certainly cleaner that way, but I don't find it a major issue.
But I'd
rather see that fix than the other one.
Here we go. See attached patch. Your comments are welcome.
Hannes.
*** ..\pgsql-cvshead\src\tools\msvc\Install.pmMo Mai 14 16:36:10 2007
--- src\tools\msvc\Install.pm Mi Jun 6 20:39:47 2007
***
*** 10,15
--- 10,18
use Carp;
use File::Basename;
use File::Copy;
+ use File::Find;
+ use File::Glob;
+ use File::Spec;
use Exporter;
our (@ISA,@EXPORT_OK);
***
*** 99,104
--- 102,142
print \n;
}
+ sub FindFiles
+ {
+ my $spec = shift;
+ my $nonrecursive = shift;
+ my $pat = basename($spec);
+ my $dir = dirname($spec);
+
+ if ($dir eq '') { $dir = '.'; }
+
+ -d $dir || croak Could not list directory $dir: $!\n;
+
+ if ($nonrecursive)
+ {
+ return glob($spec);
+ }
+
+ # borrowed from File::DosGlob
+ # escape regex metachars but not glob chars
+ $pat =~ s:([].+^\-\${}[|]):\\$1:g;
+ # and convert DOS-style wildcards to regex
+ $pat =~ s/\*/.*/g;
+ $pat =~ s/\?/.?/g;
+
+ $pat = '^' . $pat . '\z';
+
+ my @res;
+ find(
+ {
+ wanted = sub { /$pat/s push (@res,
File::Spec-canonpath($File::Find::name)); }
+ },
+ $dir
+ );
+ return @res;
+ }
+
sub CopySetOfFiles
{
my $what = shift;
***
*** 106,126
my $target = shift;
my $silent = shift;
my $norecurse = shift;
- my $D;
- my $subdirs = $norecurse?'':'/s';
print Copying $what unless ($silent);
! open($D, dir /b $subdirs $spec |) || croak Could not list $spec\n;
! while ($D)
{
- chomp;
next if /regress/; # Skip temporary install in regression subdir
! my $tgt = $target . basename($_);
print .;
! my $src = $norecurse?(dirname($spec) . '/' . $_):$_;
! copy($src, $tgt) || croak Could not copy $src: $!\n;
}
! close($D);
print \n;
}
--- 144,161
my $target = shift;
my $silent = shift;
my $norecurse = shift;
print Copying $what unless ($silent);
!
! foreach (FindFiles($spec, $norecurse))
{
next if /regress/; # Skip temporary install in regression subdir
! my $src = $_;
! my $tgt = $target . basename($src);
print .;
! copy($src, $tgt) || croak Could not copy $src to $tgt: $!\n;
}
!
print \n;
}
***
*** 371,395
{
my $target = shift;
my $nlspath = shift;
- my $D;
print Installing NLS files...;
EnsureDirectories($target, share/locale);
! open($D,dir /b /s nls.mk|) || croak Could not list nls.mk\n;
! while ($D)
{
- chomp;
s/nls.mk/po/;
my $dir = $_;
next unless ($dir =~ /([^\\]+)\\po$/);
my $prgm = $1;
$prgm = 'postgres' if ($prgm eq 'backend');
- my $E;
- open($E,dir /b $dir\\*.po|) || croak Could not list contents of
$_\n;
! while ($E)
{
- chomp;
my $lang;
next unless /^(.*)\.po/;
$lang = $1;
--- 406,425
{
my $target = shift;
my $nlspath = shift;
print Installing NLS files...;
EnsureDirectories($target, share/locale);
!
! foreach (FindFiles(nls.mk))
{
s/nls.mk/po/;
my $dir = $_;
next unless ($dir =~ /([^\\]+)\\po$/);
my $prgm = $1;
$prgm = 'postgres' if ($prgm eq 'backend');
! foreach (FindFiles($dir\\*.po, 1))
{
my $lang;
next unless /^(.*)\.po/;
$lang = $1;
***
*** 401,409
croak(Could not run msgfmt on $dir\\$_);
print .;
}
- close($E);
}
! close($D);
print \n;
}
--- 431,438
croak(Could not run msgfmt on $dir\\$_);
print .;
}
}
!
print \n;
}
---(end of broadcast)---
TIP 6: explain analyze is your friend
--
Bruce Momjian [EMAIL PROTECTED] http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
---(end of broadcast)---
TIP
Re: [HACKERS] Altering a plan
Hi, I want to take a plan generated by the postgres optimizer and insert a constant in place of another constant in the plan. There is a function OidOutputFunctionCall( ) to get the constant. Similarly, is there any function to set the value of the constant? Also what does OidInputFunctionCall( ) do? Please reply soon, this is an emergency.. Thanks a lot, Shruthi
Re: [HACKERS] Fractions in GUC variables
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Heikki Linnakangas wrote: We have these GUC variables that define a fraction of something: #autovacuum_vacuum_scale_factor = 0.2 # fraction of rel size before # vacuum #autovacuum_analyze_scale_factor = 0.1# fraction of rel size before # analyze #bgwriter_lru_percent = 1.0 # 0-100% of LRU buffers scanned/round #bgwriter_all_percent = 0.333 # 0-100% of all buffers scanned/round Autovacuum settings use fractions, and bgwriter settings use a percentage. Fortunately these settings are not related so there's not too much potential for confusion, but it seems we should have a common way to define settings like that. A nice way would be that the base unit would be a fraction, like in the autovacuum settings, but you could add a %-sign to give it as a percent, just like you can use KB/MB etc. I'm not sure if we can do anything for those without breaking backwards-compatibility, though. Any ideas? The load distributed checkpoints patch adds one more GUC variable like. I'm inclined to follow the example of the bgwriter settings because it's more closely related to them, though I like the autovacuum style more. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian [EMAIL PROTECTED] http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Change sort order on UUIDs?
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Michael Glaesemann wrote: On Jun 14, 2007, at 19:04 , [EMAIL PROTECTED] wrote: For UUID, I would value random access before sequential performance. Why would anybody scan UUID through the index in sequential order? AIUI, to allow UUID columns to be indexed using BTREE, there needs to be some ordering defined. So regardless of what this ordering is, doesn't there need to be some order? And as a (primary?) purpose of UUIDs is to be (universally) unique, and the implementation of uniqueness constraints in PostgreSQL is based on BTREE indexes, this makes the necessity of ordering doubly so. Or have I missed something? Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate -- Bruce Momjian [EMAIL PROTECTED] http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Rethinking user-defined-typmod before it's too late
Is this something for 8.3 or 8.4?
---
Tom Lane wrote:
The current discussion about the tsearch-in-core patch has convinced me
that there are plausible use-cases for typmod values that aren't simple
integers. For instance it could be sane for a type to want a locale or
language selection as a typmod, eg tsvector('ru') or tsvector('sv').
(I'm not saying we are actually going to do that to tsvector, just that
it's now clear to me that there are use-cases for such things.)
Teodor's work a few months ago generalized things enough so that
something like this is within reach. The grammar will actually allow
darn near anything for a typmod, since the grammar production is
expr_list to avoid shift/reduce conflict with the very similar-looking
productions for function calls. The only place where we are
constraining what a typmod can be is that the defined API for
user-written typmodin functions is integer array.
At the time that patch was being worked on, I think I argued that
integer typmods were enough because you'd have to pack them into such a
small output representation anyway. The hole in that logic is that you
might have a fairly small enumerated set of possibilities, but that
doesn't mean you want to make the user use a numeric code for them.
You could even make the typmod be an integer key for a lookup table,
if the set of possibilities is not hardwired.
Since this code hasn't been released yet, the API isn't set in stone
... but as soon as we ship 8.3, it will be, or at least changing it will
be orders of magnitude more painful than it is today. So, late as this
is in the devel cycle, I think now is the time to reconsider.
I propose changing the typmodin signature to typmodin(cstring[]) returns
int4, that is, the typmods will be passed as strings not integers. This
will incur a bit of extra conversion overhead for the normal uses where
the typmods are integers, but I think the gain in flexibility is worth
it. I'm inclined to make the code in parse_type.c take either integer
constants, simple string literals, or unqualified names as input ---
so you could write either tsvector('ru') or tsvector(ru) when using a
type that wants a nonintegral typmod.
Note that the typmodout side is already OK since it is defined to return
a string.
Comments?
regards, tom lane
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
--
Bruce Momjian [EMAIL PROTECTED] http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
Re: [HACKERS] Rethinking user-defined-typmod before it's too late
Bruce Momjian [EMAIL PROTECTED] writes: Is this something for 8.3 or 8.4? My goodness, you are a bit behind on the email. We fixed that a month ago. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Updated tsearch documentation
I think the tsearch documentation is nearing completion:
http://momjian.us/expire/fulltext/HTML/textsearch.html
but I am not happy with how tsearch is enabled in a user table:
http://momjian.us/expire/fulltext/HTML/textsearch-app-tutorial.html
Aside from the fact that it needs more examples, it only illustrates an
example where someone creates a table, populates it, then adds a
tsvector column, populates that, then creates an index.
That seems quite inflexible. Is there a way to avoid having a separate
tsvector column? What happens if the table is dynamic? How is that
column updated based on table changes? Triggers? Where are the
examples? Can you create an index like this:
CREATE INDEX textsearch_id ON pgweb USING gin(to_tsvector(column));
That avoids having to have a separate column because you can just say:
WHERE to_query('XXX') @@ to_tsvector(column)
How do we make sure that the to_query is using the same text search
configuration as the 'column' or index? Perhaps we should suggest:
CREATE INDEX textsearch_idx ON pgweb USING gin(to_tsvector('english',column));
so that at least the configuration is documented in the index.
--
Bruce Momjian [EMAIL PROTECTED] http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Re: [HACKERS] plpgsql FOR loop doesn't guard against strange step values
On 7/14/07, Tom Lane [EMAIL PROTECTED] wrote: I just noticed that when the BY option was added to plpgsql FOR loops, no real error checking was done. If you specify a zero step value, you'll have an infinite loop. If you specify a negative value, the loop variable will increment in the wrong direction until integer overflow occurs. Neither of these behaviors seem desirable in the least. while i read this the day you posted it, i didn't have time to answer until now... my answer is: sorry, my bad... have to admit that the idea of preventing zero in the BY doesn't cross my mind. http://archives.postgresql.org/pgsql-hackers/2006-04/msg01100.php i remember my original proposal include that negative values shouldn't be allowed, i don't know where my way was corrupted... maybe because for statement didn't make any effort to prevent things like: FOR i IN 10..1 LOOP Another problem is that no check for overflow is done when incrementing the loop variable, which means that an infinite loop is possible if the step value is larger than the distance from the loop upper bound to INT_MAX --- the loop variable could overflow before it is seen to be greater than the upper bound, and after wrapping around to negative it's still less than the upper bound, so the loop continues to run. mmm... yeah! I suggest throwing an error for zero or negative step value, and terminating the loop if the loop variable overflows. http://archives.postgresql.org/pgsql-committers/2007-07/msg00142.php at least the part that prevents overflow and probably the one that reject zero in BY are clearly bugs and should be backpatched to 8.2, aren't they? -- regards, Jaime Casanova Programming today is a race between software engineers striving to build bigger and better idiot-proof programs and the universe trying to produce bigger and better idiots. So far, the universe is winning. Richard Cook ---(end of broadcast)--- TIP 6: explain analyze is your friend
