Re: [HACKERS][PATCHES] odd output in restore mode
Simon Riggs wrote: Well, this is a strange conclusion, leaving me slightly bemused. The discussion between Andrew and I at PGcon concluded that we would * document which other tools to use * remove the delay Now we have rejected the patch which does that, but then re-requested the exact same thing again. The patch interprets "remove the delay" as "remove the delay in a way which will not screw up existing users of pg_standby when they upgrade". Doing that requires us to have a configurable delay, which defaults to the current behaviour, but that can be set to zero (the recommended way). Which is what the patch implements. Andrew, Heikki: ISTM its time to just make the changes yourselves. This is just going round and round to no benefit. This doesn't warrant such a long discussion and review process. You ought to know by now that the length and ferocity of the discussion bears no relation at all to the importance of the subject ;-) Personally, I think it's reasonable to provide the delay as long as it's switchable, although I would have preferred zero to be the default. If we remove it altogether then we force bigger changes on people who are currently using Windows copy. But I can live with that since changing their archive_command is the better path by far anyway, either to use Gnu cp or the copy / rename trick. cheers andrew -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
On Thu, 2008-07-31 at 12:32 -0400, Tom Lane wrote: > "Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > > Martin Zaun wrote: > >> With these avenues to be explored, can the pg_standby patch on the > >> CommitFest wiki be moved to the "Returned with Feedback" section? > > > Yes, I think we can conclude that we don't want this patch as it is. > > Instead, we want a documentation patch that describes the problem, > > mentioning that GNU cp is safe, or you can use the copy+rename trick. > > Right, after which we remove the presently hacked-in delay. > > I've updated the commitfest page accordingly. Well, this is a strange conclusion, leaving me slightly bemused. The discussion between Andrew and I at PGcon concluded that we would * document which other tools to use * remove the delay Now we have rejected the patch which does that, but then re-requested the exact same thing again. The patch interprets "remove the delay" as "remove the delay in a way which will not screw up existing users of pg_standby when they upgrade". Doing that requires us to have a configurable delay, which defaults to the current behaviour, but that can be set to zero (the recommended way). Which is what the patch implements. Andrew, Heikki: ISTM its time to just make the changes yourselves. This is just going round and round to no benefit. This doesn't warrant such a long discussion and review process. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
"Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > Martin Zaun wrote: >> With these avenues to be explored, can the pg_standby patch on the >> CommitFest wiki be moved to the "Returned with Feedback" section? > Yes, I think we can conclude that we don't want this patch as it is. > Instead, we want a documentation patch that describes the problem, > mentioning that GNU cp is safe, or you can use the copy+rename trick. Right, after which we remove the presently hacked-in delay. I've updated the commitfest page accordingly. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Martin Zaun wrote: Heikki Linnakangas wrote: Andrew Dunstan wrote: Greg Smith wrote: On Wed, 23 Jul 2008, Kevin Grittner wrote: I've been working on an improved archive_command shell script that I expect to submit for comments and potential inclusion in the documentation as a better base for other people to build on. This is one of the options for how it can operate. It would be painful but not impossible to convert a subset of that script to run under Windows as well, at least enough to cover this particular issue. A Perl script using the (standard) File::Copy module along with the builtin function rename() should be moderately portable. It would to be nice not to have to maintain two scripts. It's also not very nice to require a Perl installation on Windows, just for a replacement of Copy. Would a simple .bat script work? With these avenues to be explored, can the pg_standby patch on the CommitFest wiki be moved to the "Returned with Feedback" section? Yes, I think we can conclude that we don't want this patch as it is. Instead, we want a documentation patch that describes the problem, mentioning that GNU cp is safe, or you can use the copy+rename trick. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Heikki Linnakangas wrote: Andrew Dunstan wrote: Greg Smith wrote: On Wed, 23 Jul 2008, Kevin Grittner wrote: I've been working on an improved archive_command shell script that I expect to submit for comments and potential inclusion in the documentation as a better base for other people to build on. This is one of the options for how it can operate. It would be painful but not impossible to convert a subset of that script to run under Windows as well, at least enough to cover this particular issue. A Perl script using the (standard) File::Copy module along with the builtin function rename() should be moderately portable. It would to be nice not to have to maintain two scripts. It's also not very nice to require a Perl installation on Windows, just for a replacement of Copy. Would a simple .bat script work? With these avenues to be explored, can the pg_standby patch on the CommitFest wiki be moved to the "Returned with Feedback" section? Regards, Martin -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Andrew Dunstan wrote: Greg Smith wrote: On Wed, 23 Jul 2008, Kevin Grittner wrote: In our scripts we handle this by copying to a temp directory on the same mount point as the archive directory and doing a mv to the archive location when the copy is successfully completed. I think that this even works on Windows. Could that just be documented as a strong recommendation for the archive script? This is exactly what I always do. I think the way cp is shown in the examples promotes what's really a bad practice for lots of reasons, this particular problem being just one of them. I've been working on an improved archive_command shell script that I expect to submit for comments and potential inclusion in the documentation as a better base for other people to build on. This is one of the options for how it can operate. It would be painful but not impossible to convert a subset of that script to run under Windows as well, at least enough to cover this particular issue. A Perl script using the (standard) File::Copy module along with the builtin function rename() should be moderately portable. It would to be nice not to have to maintain two scripts. It's also not very nice to require a Perl installation on Windows, just for a replacement of Copy. Would a simple .bat script work? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Greg Smith wrote: On Wed, 23 Jul 2008, Kevin Grittner wrote: In our scripts we handle this by copying to a temp directory on the same mount point as the archive directory and doing a mv to the archive location when the copy is successfully completed. I think that this even works on Windows. Could that just be documented as a strong recommendation for the archive script? This is exactly what I always do. I think the way cp is shown in the examples promotes what's really a bad practice for lots of reasons, this particular problem being just one of them. I've been working on an improved archive_command shell script that I expect to submit for comments and potential inclusion in the documentation as a better base for other people to build on. This is one of the options for how it can operate. It would be painful but not impossible to convert a subset of that script to run under Windows as well, at least enough to cover this particular issue. A Perl script using the (standard) File::Copy module along with the builtin function rename() should be moderately portable. It would to be nice not to have to maintain two scripts. cheers andrew -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
On Wed, 23 Jul 2008, Kevin Grittner wrote: In our scripts we handle this by copying to a temp directory on the same mount point as the archive directory and doing a mv to the archive location when the copy is successfully completed. I think that this even works on Windows. Could that just be documented as a strong recommendation for the archive script? This is exactly what I always do. I think the way cp is shown in the examples promotes what's really a bad practice for lots of reasons, this particular problem being just one of them. I've been working on an improved archive_command shell script that I expect to submit for comments and potential inclusion in the documentation as a better base for other people to build on. This is one of the options for how it can operate. It would be painful but not impossible to convert a subset of that script to run under Windows as well, at least enough to cover this particular issue. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Andrew Dunstan wrote: Heikki Linnakangas wrote: Andrew Dunstan wrote: - or maybe provide a .bat file or perl script that would work as na archive_command on Windows. We're not talking about archive_command. We're talking about the thing that copies files to the directory that pg_standby polls. Er, that's what the archive_command is. Look at the pg_standby docs and you'll see that that's where we're currently recommending use of windows copy. Perhaps you're confusing this with the restore_command? Oh, right. I was thinking that archive_command copies the files to an archive location, and there's yet another process copying files from there to the directory pg_standby polls. But indeed in the simple configuration, archive_command is the command that we're interested in. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Heikki Linnakangas wrote: Andrew Dunstan wrote: - or maybe provide a .bat file or perl script that would work as na archive_command on Windows. We're not talking about archive_command. We're talking about the thing that copies files to the directory that pg_standby polls. Er, that's what the archive_command is. Look at the pg_standby docs and you'll see that that's where we're currently recommending use of windows copy. Perhaps you're confusing this with the restore_command? cheers andrew -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Andrew Dunstan wrote: Kevin Grittner wrote: "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote: We really need a more reliable way of detecting that a file has been fully copied. In our scripts we handle this by copying to a temp directory on the same mount point as the archive directory and doing a mv to the archive location when the copy is successfully completed. I think that this even works on Windows. Could that just be documented as a strong recommendation for the archive script? Needs testing at least. If it does in fact work then we can just adjust the docs and be done Yeah. - or maybe provide a .bat file or perl script that would work as na archive_command on Windows. We're not talking about archive_command. We're talking about the thing that copies files to the directory that pg_standby polls. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
On Fri, 2008-07-25 at 16:58 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Fri, 2008-07-25 at 16:31 -0400, Tom Lane wrote: > >> I thought the latest conclusion was that changing the behavior of > >> pg_standby itself wouldn't address the problem anyway, and that what we > >> need is just a docs patch recommending that people use safe copying > >> methods in their scripts that copy to the archive area? > > > Plus the rest of this patch, which is really very simple. > > Why? AFAICT the patch is just a kluge that adds user-visible complexity > without providing a solution that's actually sure to work. First, I'm not the one objecting to the current behaviour. Currently, there is a wait in there that can be removed if you use a copy utility that sets size after it does a copy. So we agreed to make it optional (at PGCon). -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Simon Riggs <[EMAIL PROTECTED]> writes: > On Fri, 2008-07-25 at 16:31 -0400, Tom Lane wrote: >> I thought the latest conclusion was that changing the behavior of >> pg_standby itself wouldn't address the problem anyway, and that what we >> need is just a docs patch recommending that people use safe copying >> methods in their scripts that copy to the archive area? > Plus the rest of this patch, which is really very simple. Why? AFAICT the patch is just a kluge that adds user-visible complexity without providing a solution that's actually sure to work. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
On Fri, 2008-07-25 at 16:31 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: > >> reviewing your patch > > > Current status is this: > > * My understanding is that Dave and Andrew (and therefore Simon) think > > the approach proposed here is an acceptable one. Heikki disagrees and > > wants different approach. Perhaps I misunderstand. > > * Patch needs work to complete the proposed approach > > * I'm willing to change the patch, but not able to test it on Windows. > > I thought the latest conclusion was that changing the behavior of > pg_standby itself wouldn't address the problem anyway, and that what we > need is just a docs patch recommending that people use safe copying > methods in their scripts that copy to the archive area? Plus the rest of this patch, which is really very simple. pg_standby currently waits (on Windows) for the sleep time. We agreed that this sleep would be on by default, but optional. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Simon Riggs <[EMAIL PROTECTED]> writes: > On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: >> reviewing your patch > Current status is this: > * My understanding is that Dave and Andrew (and therefore Simon) think > the approach proposed here is an acceptable one. Heikki disagrees and > wants different approach. Perhaps I misunderstand. > * Patch needs work to complete the proposed approach > * I'm willing to change the patch, but not able to test it on Windows. I thought the latest conclusion was that changing the behavior of pg_standby itself wouldn't address the problem anyway, and that what we need is just a docs patch recommending that people use safe copying methods in their scripts that copy to the archive area? regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: > reviewing your patch Current status is this: * My understanding is that Dave and Andrew (and therefore Simon) think the approach proposed here is an acceptable one. Heikki disagrees and wants different approach. Perhaps I misunderstand. * Patch needs work to complete the proposed approach * I'm willing to change the patch, but not able to test it on Windows. Is there someone able to test the patch, if I make the changes? If not, we should just kick this out of the CommitFest queue now and be done. If nobody cares enough about this issue to test a fix, we shouldn't bother. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
On Wed, 2008-07-23 at 21:38 +0300, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: > >> 8. Unresolved question of implementing now/later a "cp" replacement > > > > The patch implements what's been agreed. > > > > I'm not rewriting "cp", for reasons already discussed. > > > > Not a comment to you Martin, but it's fairly clear that I'm not > > maintaining this correctly for Windows. I've never claimed to have > > tested this on Windows, and only included Windows related items as > > requested by others. I need to make it clear that I'm not going to > > maintain it at all, for Windows. If others wish to report Windows issues > > then they can suggest appropriate fixes and test them also. > > Hmm. I just realized that replacing the "cp" command within pg_standby > won't help at all. The problem is with the command that copies the files > *to* the archivelocation that pg_standby polls, not with the copy > pg_standby does from archivelocation to pg_xlog. And we don't have much > control over that. > > We really need a more reliable way of detecting that a file has been > fully copied. One simple improvement would be to check the xlp_magic > field of the last page, though it still wouldn't be bullet-proof. > > Do the commands that preallocate the space keep the file exclusively > locked during the copy? If they do, shouldn't we get an error in trying > to run the restore copy command, and retry after the 1s sleep in > RestoreWALFileForRecovery? Though if the archive location is a samba > mount or something, I guess we can't rely on Windows-style exclusive > locking. With respect, I need to refer you back to the my last paragraph above. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Kevin Grittner wrote: "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote: We really need a more reliable way of detecting that a file has been fully copied. In our scripts we handle this by copying to a temp directory on the same mount point as the archive directory and doing a mv to the archive location when the copy is successfully completed. I think that this even works on Windows. Could that just be documented as a strong recommendation for the archive script? Needs testing at least. If it does in fact work then we can just adjust the docs and be done - or maybe provide a .bat file or perl script that would work as na archive_command on Windows. cheers andrew -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
>>> "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote: > We really need a more reliable way of detecting that a file has been > fully copied. In our scripts we handle this by copying to a temp directory on the same mount point as the archive directory and doing a mv to the archive location when the copy is successfully completed. I think that this even works on Windows. Could that just be documented as a strong recommendation for the archive script? -Kevin -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Simon Riggs wrote: On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: 8. Unresolved question of implementing now/later a "cp" replacement The patch implements what's been agreed. I'm not rewriting "cp", for reasons already discussed. Not a comment to you Martin, but it's fairly clear that I'm not maintaining this correctly for Windows. I've never claimed to have tested this on Windows, and only included Windows related items as requested by others. I need to make it clear that I'm not going to maintain it at all, for Windows. If others wish to report Windows issues then they can suggest appropriate fixes and test them also. Hmm. I just realized that replacing the "cp" command within pg_standby won't help at all. The problem is with the command that copies the files *to* the archivelocation that pg_standby polls, not with the copy pg_standby does from archivelocation to pg_xlog. And we don't have much control over that. We really need a more reliable way of detecting that a file has been fully copied. One simple improvement would be to check the xlp_magic field of the last page, though it still wouldn't be bullet-proof. Do the commands that preallocate the space keep the file exclusively locked during the copy? If they do, shouldn't we get an error in trying to run the restore copy command, and retry after the 1s sleep in RestoreWALFileForRecovery? Though if the archive location is a samba mount or something, I guess we can't rely on Windows-style exclusive locking. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: > 1. Issues with applying the patch to CVS HEAD: For me, the patch applies cleanly to CVS HEAD. I do notice that there are two files "standby.sgml" and "pgstandby.sgml". I can't see where "standby.sgml" comes from, but I haven't created it; perhaps it is a relic of the SGML build process. I've recreated my source tree since I wrote the patch also. Weird. I'll redo the patch so it points at pgstandby.sgml, which is the one thats listed as being in the main source tree. > 2. Missing description for new command-line options in pgstandby.sgml > > - no description of the proposed new command-line options -h and -p? These are done. The patch issues have missed those hunks. > 3. No coding style issues seen > > Just one comment: the logic that selects the actual restore command to > be used has moved from CustomizableInitialize() to main() -- a matter > of personal taste, perhaps. But in my view the: > + the #ifdef WIN32/HAVE_WORKING_LINK logic has become easier to read Thanks > 4. Issue: missing break in switch, silent override of '-l' argument? > > This behaviour has been in there before Well spotted. I don't claim to test this for Windows. > 5. Minor wording issue in usage message on new '-p' option > > I was wondering if the "always" in the usage text > fprintf(stderr, " -p always uses GNU compatible 'cp' command > on all platforms\n"); > is too strong, since multiple restore command options overwrite each > other, e.g. "-p -c" applies Windows's "copy" instead of Gnu's "cp". I was assuming you don't turn the switch off again immediately afterwards. > 6. Minor code comment suggestion > > Unrelated to this patch, I wonder if the code comments on all four > time-related vars better read "seconds" instead of "amount of time": > int sleeptime = 5; /* amount of time to sleep between file > checks */ > int holdtime = 0; /* amount of time to wait once file appears > full */ > int waittime = -1; /* how long we have been waiting, -1 no wait > * yet */ > int maxwaittime = 0;/* how long are we prepared to wait for? */ As you say, unrelated to the patch. > 7. Question: benefits of separate holdtime option from sleeptime? > > Simon Riggs wrote: > > * provide "holdtime" delay, default 0 (on all platforms) > > Going back on the hackers+patches emails and parsing the code > comments, I'm sorry if I missed that, but I'm not sure I've understood > the exact tuning benefits that introducing the new holdtime option > provides over using the existing sleeptime, as it's been the case > (just on Win32 only). This is central to the patch, since the complaint was about the delay introduced by doing that previously. > 8. Unresolved question of implementing now/later a "cp" replacement The patch implements what's been agreed. I'm not rewriting "cp", for reasons already discussed. Not a comment to you Martin, but it's fairly clear that I'm not maintaining this correctly for Windows. I've never claimed to have tested this on Windows, and only included Windows related items as requested by others. I need to make it clear that I'm not going to maintain it at all, for Windows. If others wish to report Windows issues then they can suggest appropriate fixes and test them also. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: > 1. Issues with applying the patch to CVS HEAD: Sounds awful. Thanks for the review, will fix. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS][PATCHES] odd output in restore mode
Below my comments on the CommitFest patch:
pg_standby minor changes for Windows
Simon, I'm sorry you got me, a Postgres newbie, signed up for
reviewing your patch ;)
To start with, I'm not quite sure of the status of this patch
since Bruce's last comment on the -patches alias:
Bruce Momjian wrote:
> OK, based on these observations I think we need to learn more about the
> issues before making any changes to our code.
From easy to difficult:
1. Issues with applying the patch to CVS HEAD:
The second file in the patch
Index: doc/src/sgml/standby.sgml
appears to be misnamed -- the existing file in HEAD is
Index: doc/src/sgml/pgstandby.sgml
However, still had issues after fixing the file name:
[EMAIL PROTECTED]:~/pg/pgsql$ patch -c -p0 < ../pg_standby.patch
patching file contrib/pg_standby/pg_standby.c
patching file doc/src/sgml/pgstandby.sgml
Hunk #1 FAILED at 136.
Hunk #2 FAILED at 168.
Hunk #3 FAILED at 245.
Hunk #4 FAILED at 255.
4 out of 4 hunks FAILED -- saving rejects to file
doc/src/sgml/pgstandby.sgml.rej
2. Missing description for new command-line options in pgstandby.sgml
Simon Riggs wrote:
> Patch implements
> * recommendation to use GnuWin32 cp on Windows
Saw that in the changes to pgstandby.sgml, and looks ok to me, but:
- no description of the proposed new command-line options -h and -p?
3. No coding style issues seen
Just one comment: the logic that selects the actual restore command to
be used has moved from CustomizableInitialize() to main() -- a matter
of personal taste, perhaps. But in my view the:
+ the #ifdef WIN32/HAVE_WORKING_LINK logic has become easier to read
4. Issue: missing break in switch, silent override of '-l' argument?
This behaviour has been in there before and is not addresses by the
patch: The user-selected Win32 "mklink" command mode is never applied
due to a missing 'break' in CustomizableInitialize():
switch (restoreCommandType)
{
case RESTORE_COMMAND_WIN32_MKLINK:
SET_RESTORE_COMMAND("mklink", WALFilePath, xlogFilePath);
case RESTORE_COMMAND_WIN32_COPY:
SET_RESTORE_COMMAND("copy", WALFilePath, xlogFilePath);
break;
A similar behaviour on Non-Win32 platforms where the user-selected
"ln" may be silently changed to "cp" in main():
#if HAVE_WORKING_LINK
restoreCommandType = RESTORE_COMMAND_LN;
#else
restoreCommandType = RESTORE_COMMAND_CP;
#endif
If both Win32/Non-Win32 cases reflect the intended behaviour:
- I'd prefer a code comment in the above case-fall-through,
- suggest a message to the user about the ignored "ln" / "mklink",
- observe that the logic to override of the '-l' option is now in two
places: CustomizableInitialize() and main().
5. Minor wording issue in usage message on new '-p' option
I was wondering if the "always" in the usage text
fprintf(stderr, " -p always uses GNU compatible 'cp' command on all
platforms\n");
is too strong, since multiple restore command options overwrite each
other, e.g. "-p -c" applies Windows's "copy" instead of Gnu's "cp".
6. Minor code comment suggestion
Unrelated to this patch, I wonder if the code comments on all four
time-related vars better read "seconds" instead of "amount of time":
int sleeptime = 5; /* amount of time to sleep between file checks
*/
int holdtime = 0; /* amount of time to wait once file appears
full */
int waittime = -1; /* how long we have been waiting, -1 no wait
* yet */
int maxwaittime = 0;/* how long are we prepared to wait for? */
7. Question: benefits of separate holdtime option from sleeptime?
Simon Riggs wrote:
> * provide "holdtime" delay, default 0 (on all platforms)
Going back on the hackers+patches emails and parsing the code
comments, I'm sorry if I missed that, but I'm not sure I've understood
the exact tuning benefits that introducing the new holdtime option
provides over using the existing sleeptime, as it's been the case
(just on Win32 only).
8. Unresolved question of implementing now/later a "cp" replacement
Simon Riggs wrote:
> On Tue, 2008-07-01 at 13:44 +0300, Heikki Linnakangas wrote:
>> This seems pretty kludgey to me. I wouldn't want to install GnuWin32
>> utilities on a production system just for the "cp" command, and I don't
>> know how I would tune holdtime properly for using "copy". And it seems
>> risky to have defaults that are known to not work reliably.
>>
>> How about implementing a replacement function for "cp" ourselves? It
>> seems pretty trivial to do. We could use that
Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717
> On Mon, Jul 21, 2008 at 08:19:35AM +0900, Tatsuo Ishii wrote: > > > > Thus I think we should avoid this kind of ORDER BY. Probably we should > > > > avoid LIMIT/OFFSET and FOR UPDATE as well. > > > > > > What of index-optimized SELECT max(...) ? > > > > Aggregate functions in a recursive term is prohibited by the > > standard. For example, > > > > WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT max(n) FROM x) > > SELECT * FROM x; > > > > produces an error. > > On the other side of UNION ALL, it's OK, right? For example, > > WITH RECURSIVE x(n) AS ( > SELECT max(i) FROM t > UNION ALL > SELECT n+1 FROM x WHERE n < 20 > ) Yes, aggregate functions in the non-recursive term is allowed by the standard. -- Tatsuo Ishii SRA OSS, Inc. Japan -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717
On Mon, Jul 21, 2008 at 08:19:35AM +0900, Tatsuo Ishii wrote: > > > Thus I think we should avoid this kind of ORDER BY. Probably we should > > > avoid LIMIT/OFFSET and FOR UPDATE as well. > > > > What of index-optimized SELECT max(...) ? > > Aggregate functions in a recursive term is prohibited by the > standard. For example, > > WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT max(n) FROM x) > SELECT * FROM x; > > produces an error. On the other side of UNION ALL, it's OK, right? For example, WITH RECURSIVE x(n) AS ( SELECT max(i) FROM t UNION ALL SELECT n+1 FROM x WHERE n < 20 ) Cheers, David. -- David Fetter <[EMAIL PROTECTED]> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717
> > Thus I think we should avoid this kind of ORDER BY. Probably we should > > avoid LIMIT/OFFSET and FOR UPDATE as well. > > What of index-optimized SELECT max(...) ? Aggregate functions in a recursive term is prohibited by the standard. For example, WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT max(n) FROM x) SELECT * FROM x; produces an error. -- Tatsuo Ishii SRA OSS, Inc. Japan -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717
Tatsuo Ishii <[EMAIL PROTECTED]> writes: > Thus I think we should avoid this kind of ORDER BY. Probably we should > avoid LIMIT/OFFSET and FOR UPDATE as well. What of index-optimized SELECT max(...) ? regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717
> This crashes the backend:
>
> WITH RECURSIVE t(n) AS (
> VALUES (1)
> UNION ALL
> SELECT n+1 FROM t WHERE n < 5 ORDER BY 1
> )
> SELECT n FROM t;
>
> apparently because of the ORDER BY 1
Thanks for the report. I think ORDER BY in this case is useless
anyway. ORDER BY affects (VALUES (1) UNION ALL SELECT n+1 FROM t WHERE
n < 5). Since this is a recursive query, value for (VALUES (1) UNION
ALL SELECT n+1 FROM t WHERE n < 5) will not be determined until the
recursion stops. So the meaning of ORDER BY is vague. If caller wants
to get the sorted result of the recursion, he could always write:
WITH RECURSIVE t(n) AS (
VALUES (1)
UNION ALL
SELECT n+1 FROM t WHERE n < 5
)
SELECT n FROM t ORDER BY 1;
Thus I think we should avoid this kind of ORDER BY. Probably we should
avoid LIMIT/OFFSET and FOR UPDATE as well. Included patches add the
checking plus minor error messages clarifications. Also I include new
error cases sql.
> ( ORDER BY t.n will just error out )
>
> Compiled with:
>
> ./configure \
>--prefix=${install_dir} \
>--with-pgport=${pgport} \
>--quiet \
>--enable-depend \
>--enable-cassert \
>--enable-debug \
>--with-openssl
>
>
> hth
>
> Erik Rijkers
>
>
>
>
>
-- UNION
WITH RECURSIVE x(n) AS (SELECT 1 UNION SELECT n+1 FROM x)
SELECT * FROM x;
-- INTERSECT
WITH RECURSIVE x(n) AS (SELECT 1 INTERSECT SELECT n+1 FROM x)
SELECT * FROM x;
WITH RECURSIVE x(n) AS (SELECT 1 INTERSECT ALL SELECT n+1 FROM x)
SELECT * FROM x;
-- EXCEPT
WITH RECURSIVE x(n) AS (SELECT 1 EXCEPT SELECT n+1 FROM x)
SELECT * FROM x;
WITH RECURSIVE x(n) AS (SELECT 1 EXCEPT ALL SELECT n+1 FROM x)
SELECT * FROM x;
-- no non-recursive term
WITH RECURSIVE x(n) AS (SELECT n FROM x)
SELECT * FROM x;
-- recursive term in the left hand side
WITH RECURSIVE x(n) AS (SELECT n FROM x UNION ALL SELECT 1)
SELECT * FROM x;
CREATE TEMP TABLE y (a int);
INSERT INTO y SELECT generate_series(1, 10);
-- LEFT JOIN
WITH RECURSIVE x(n) AS (SELECT a FROM y WHERE a = 1 UNION ALL SELECT x.n+1 FROM
y LEFT JOIN x ON x.n = y.a where n < 10)
SELECT * FROM x;
-- RIGHT JOIN
WITH RECURSIVE x(n) AS (SELECT a FROM y WHERE a = 1 UNION ALL SELECT x.n+1 FROM
x RIGHT JOIN y ON x.n = y.a where n < 10)
SELECT * FROM x;
-- FULL JOIN
WITH RECURSIVE x(n) AS (SELECT a FROM y WHERE a = 1 UNION ALL SELECT x.n+1 FROM
x FULL JOIN y ON x.n = y.a where n < 10)
SELECT * FROM x;
-- subquery
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x
WHERE n IN (SELECT * FROM x))
SELECT * FROM x;
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x
WHERE n = 1 AND n IN (SELECT * FROM x))
SELECT * FROM x;
-- GROUP BY
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x GROUP BY n)
SELECT * FROM x;
-- HAVING
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x HAVING n < 10)
SELECT * FROM x;
-- aggregate functions
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT count(*) FROM x)
SELECT * FROM x;
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT sum(*) FROM x)
SELECT * FROM x;
-- ORDER BY
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x ORDER BY 1)
SELECT * FROM x;
-- LIMIT/OFFSET
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x LIMIT 10 OFFSET 1)
SELECT * FROM x;
-- FOR UPDATE
WITH RECURSIVE x(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM x FOR UPDATE)
SELECT * FROM x;
recursive_query.patch.gz
Description: Binary data
--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECUSIVE patches 0717
On Fri, July 18, 2008 03:41, Tatsuo Ishii wrote:
>> > Here is the lastest WITH RECURSIVE patches against CVS HEAD created by
>> > Yoshiyuki Asaba and minor corrections by Tatsuo Ishii.
>>
>> I tried this patch vs. CVS HEAD used my usual configure option with
>> only --with-prefix set, then tried to make, and got:
>>
>> make[3]: *** No rule to make target `parse_cte.o', needed by `objfiles.txt'.
>> Stop.
>> make[3]: Leaving directory `/home/shackle/pgsql/src/backend/parser'
>> make[2]: *** [parser-recursive] Error 2
>> make[2]: Leaving directory `/home/shackle/pgsql/src/backend'
>> make[1]: *** [all] Error 2
>> make[1]: Leaving directory `/home/shackle/pgsql/src'
>> make: *** [all] Error 2
>>
>> Is there something missing?
>
> Oops. I forgot to include patches against newly added files. Please
> try included patches.
>
This crashes the backend:
WITH RECURSIVE t(n) AS (
VALUES (1)
UNION ALL
SELECT n+1 FROM t WHERE n < 5 ORDER BY 1
)
SELECT n FROM t;
apparently because of the ORDER BY 1
( ORDER BY t.n will just error out )
Compiled with:
./configure \
--prefix=${install_dir} \
--with-pgport=${pgport} \
--quiet \
--enable-depend \
--enable-cassert \
--enable-debug \
--with-openssl
hth
Erik Rijkers
--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GIN improvements
Updated: http://www.sigaev.ru/misc/fast_insert_gin-0.9.gz need more review of fast_insert yet? It looked like a number of people commented on it already. I still havn't clearness of acceptability for suggested aminsertcleanup calling. -- Teodor Sigaev E-mail: [EMAIL PROTECTED] WWW: http://www.sigaev.ru/ -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GIN improvements
I've committed the multicolumn one with minor revisions (fix some poor English in docs and comments, add regression-test coverage). Do you Thank you very much. need more review of fast_insert yet? It looked like a number of people commented on it already. I should modify it to support/synchronize with multicolumn GIN - both patches touch the same pieces of code, and I didn't make a single patch to simplify review. -- Teodor Sigaev E-mail: [EMAIL PROTECTED] WWW: http://www.sigaev.ru/ -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GIN improvements
Neil, > I was tasked with reviewing this for the current commit fest, although > so far I've just been working on grokking the rest of the GIN code. But > if you'd like to review it instead, that's fine with me. I have plenty of other stuff I could assign you if you're not needed on GIN. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GIN improvements
On Tue, 2008-07-08 at 14:51 -0400, Tom Lane wrote: > I'd still like to take a look. I was tasked with reviewing this for the current commit fest, although so far I've just been working on grokking the rest of the GIN code. But if you'd like to review it instead, that's fine with me. -Neil -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GIN improvements
Teodor Sigaev <[EMAIL PROTECTED]> writes: >> I looked this over and it looks good in general. > May I think that patch passed review and commit it? I'd still like to take a look. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GIN improvements
I looked this over and it looks good in general. May I think that patch passed review and commit it? -- Teodor Sigaev E-mail: [EMAIL PROTECTED] WWW: http://www.sigaev.ru/ -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Explain XML patch v2
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Freitag, 4. Juli 2008 schrieb Tom Raney: >> Regarding the XML datum, in order to support that, will all users need >> to compile with libxml? Are there any lighter weight solutions to >> serialize other than libxml? > You can create XML without libxml. Seems to me that anyone who wants this feature will probably also want the existing libxml-based features, so they'll be building with libxml anyway. So I'd not be in favor of expending any extra code on a roll-your-own solution. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
Tom, I think this patch is plenty complicated enough without adding useless restrictive options. +1 for no additonal GUC options. --Josh Berkus -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
[ catching up on back email ] Gregory Stark <[EMAIL PROTECTED]> writes: > "Yoshiyuki Asaba" <[EMAIL PROTECTED]> writes: >> Recursive query does not consume stack. The server enters an infinite >> loop without consuming stack. Stack-depth error does not happen. > We could have a separate guc variable which limits the maximum number of > levels of recursive iterations. That might be a useful feature for DBAs that > want to limit their users from issuing an infinite query. This whole thread seems to be proposing more and more complicated solutions for what is really a non-problem given Yoshiyuki-san's point. It's trivial to construct SQL queries that will run for longer than the MTBF of your hardware --- eg, forget a few join constraints. We've gotten along fine with nothing but query cancel and statement_timeout for that, and I've seen no one proposing that we need to "fix it". We don't disallow you from writing an infinite loop in plpgsql, either. I think this patch is plenty complicated enough without adding useless restrictive options. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
Hans-Juergen Schoenig wrote: Gregory Stark wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: i don't think statement_timeout is a good idea at all. it is not deterministic. depending on the load on the server some queries will execute while others fail. a separate GUC is needed. I don't think we need to add clutter to GUC for something that exists to handle the problem at hand. If our real concern is server utilization based on user or query resources we need to look at an overall solution for that issue not a one off for a single feature. Sincerely, Joshua D. Drake -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
Gregory Stark wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: Couldn't we just have it pay attention to the existing max_stack_depth? Recursive query does not consume stack. The server enters an infinite loop without consuming stack. Stack-depth error does not happen. We could have a separate guc variable which limits the maximum number of levels of recursive iterations. That might be a useful feature for DBAs that want to limit their users from issuing an infinite query. statement_timeout :) Good point. Though it occurs to me that if you set FETCH_COUNT in psql (or do the equivalent in your code ) statement_timeout becomes much less useful. i don't think statement_timeout is a good idea at all. it is not deterministic. depending on the load on the server some queries will execute while others fail. a separate GUC is needed. best regards, hans -- Cybertec Schönig & Schönig GmbH PostgreSQL Solutions and Support Gröhrmühlgasse 26, A-2700 Wiener Neustadt Tel: +43/1/205 10 35 / 340 www.postgresql-support.de, www.postgresql-support.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
On Sat, May 24, 2008 at 05:01:11AM +0900, Yoshiyuki Asaba wrote: > Hi, > > From: David Fetter <[EMAIL PROTECTED]> > Subject: Re: [PATCHES] WITH RECURSIVE patch V0.1 > Date: Fri, 23 May 2008 11:26:30 -0700 > > > Where is the new patch? > > I will create the revised patch on June. This is a patch for this > problem. Thanks very much :) Cheers, David. -- David Fetter <[EMAIL PROTECTED]> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: >> >> Couldn't we just have it pay attention to the existing >> >> max_stack_depth? >> > >> > Recursive query does not consume stack. The server enters an infinite >> > loop without consuming stack. Stack-depth error does not happen. >> >> We could have a separate guc variable which limits the maximum number of >> levels of recursive iterations. That might be a useful feature for DBAs that >> want to limit their users from issuing an infinite query. > > statement_timeout :) Good point. Though it occurs to me that if you set FETCH_COUNT in psql (or do the equivalent in your code ) statement_timeout becomes much less useful. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production Tuning -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
> >> Couldn't we just have it pay attention to the existing > >> max_stack_depth? > > > > Recursive query does not consume stack. The server enters an infinite > > loop without consuming stack. Stack-depth error does not happen. > > We could have a separate guc variable which limits the maximum number of > levels of recursive iterations. That might be a useful feature for DBAs that > want to limit their users from issuing an infinite query. > statement_timeout :) Joshua D. Drake -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
"Yoshiyuki Asaba" <[EMAIL PROTECTED]> writes: > Hi, > > From: David Fetter <[EMAIL PROTECTED]> > Subject: Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1 > Date: Mon, 19 May 2008 04:36:30 -0700 > >> > > I think it's the other way around. The server should not emit >> > > infinite number of records. >> > >> > How about adding new GUC parameter "max_recursive_call"? >> >> Couldn't we just have it pay attention to the existing >> max_stack_depth? > > Recursive query does not consume stack. The server enters an infinite > loop without consuming stack. Stack-depth error does not happen. We could have a separate guc variable which limits the maximum number of levels of recursive iterations. That might be a useful feature for DBAs that want to limit their users from issuing an infinite query. Note that users can always construct their query to limit the number of recursive iterations. So this would only be useful for DBAs that don't trust their users and want to impose a limit. It doesn't add any actual expressive power that SQL doesn't have already. The recursive query syntax in the spec actually does include the ability to assign an output column to show what level of recursive iteration you're on. So alternately we could have a GUC variable which just allows the DBA to prohibit any recursive query without such a column and a fiter imposing a maximum value on it. That's probably the most appropriate option. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support! -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
Hi, From: David Fetter <[EMAIL PROTECTED]> Subject: Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1 Date: Mon, 19 May 2008 04:36:30 -0700 > > > I think it's the other way around. The server should not emit > > > infinite number of records. > > > > How about adding new GUC parameter "max_recursive_call"? > > Couldn't we just have it pay attention to the existing > max_stack_depth? Recursive query does not consume stack. The server enters an infinite loop without consuming stack. Stack-depth error does not happen. Regards, -- Yoshiyuki Asaba [EMAIL PROTECTED] -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] WITH RECURSIVE patch V0.1
Merlin Moncure wrote: On Sun, May 18, 2008 at 5:22 PM, Zoltan Boszormenyi <[EMAIL PROTECTED]> wrote: Can we get the rows in tree order, please? I.e. something like this: Is ordering by tree order defined in the standard when no explicit order is given? If not, it probably returns them in the order they are pulled up, which might be the fastest way +1 for the fastest way, which I expect to often be "find all level 1 matches", "find all level 2 matches", ... If ORDER BY is important, it should be specified (although it may be difficult or impossible to properly represent ORDER BY for a tree? not sure?) I think most uses of recursive require extra client side code to deal with anyways, so only relative order is important (order within a particular branch). There are things I'd like to use this for right now. Currently I use plpgsql procedures to implement my own recursion. :-) Cheers, mark -- Mark Mielke <[EMAIL PROTECTED]>
Re: [HACKERS] [PATCHES] [NOVICE] encoding problems
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Well, 8.3 is already different from 8.2, and a lot of people will see >> this particular aspect of it as a regression. I'm okay with >> backpatching to 8.3 ... though the patch needed rather more testing >> than you gave it. > OK, so Alvaro and Tom want this backpatched. However, it isn't going to > match 8.2 behavior --- is that OK? Huh? 8.3 is already hugely different from 8.2 because of the newline formatting changes. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] [NOVICE] encoding problems
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Guillaume Smet wrote: > >> I understand your point of view but I really think it's more a > >> regression fix than a behavior change. > > > If I can get other hackers to say we should backpatch we can consider > > it. > > Well, 8.3 is already different from 8.2, and a lot of people will see > this particular aspect of it as a regression. I'm okay with > backpatching to 8.3 ... though the patch needed rather more testing > than you gave it. OK, so Alvaro and Tom want this backpatched. However, it isn't going to match 8.2 behavior --- is that OK? -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] [NOVICE] encoding problems
Bruce Momjian <[EMAIL PROTECTED]> writes: > Guillaume Smet wrote: >> I understand your point of view but I really think it's more a >> regression fix than a behavior change. > If I can get other hackers to say we should backpatch we can consider > it. Well, 8.3 is already different from 8.2, and a lot of people will see this particular aspect of it as a regression. I'm okay with backpatching to 8.3 ... though the patch needed rather more testing than you gave it. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction
"Hell, Robert" <[EMAIL PROTECTED]> writes: > You're right - that's just a typo in the subject of the post. > It's called cursor_tuple_fraction in the submitted patch. Ah, I hadn't actually read the patch yet ;-). As penance for the noise, I will do so now. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction
On Fri, 2008-05-02 at 16:36 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > OK, if that's the view then the patch is ready for commit, AFAICS. > > Use of the plural in the name seems a bit odd to me. Anyone have a > problem with calling it "cursor_tuple_fraction" instead? Agreed. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction
You're right - that's just a typo in the subject of the post. It's called cursor_tuple_fraction in the submitted patch. Regards, Robert -Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Freitag, 02. Mai 2008 22:36 To: Simon Riggs Cc: Heikki Linnakangas; Hell, Robert; pgsql-patches@postgresql.org; [EMAIL PROTECTED] Subject: Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction Simon Riggs <[EMAIL PROTECTED]> writes: > OK, if that's the view then the patch is ready for commit, AFAICS. Use of the plural in the name seems a bit odd to me. Anyone have a problem with calling it "cursor_tuple_fraction" instead? regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction
Simon Riggs <[EMAIL PROTECTED]> writes: > OK, if that's the view then the patch is ready for commit, AFAICS. Use of the plural in the name seems a bit odd to me. Anyone have a problem with calling it "cursor_tuple_fraction" instead? regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction
On Fri, 2008-05-02 at 12:01 -0400, Tom Lane wrote: > I see this as being basically another cost parameter, and as such > I don't think it needs more documentation than any of those have. > (Now admittedly you could argue that they could all use a ton more > documentation than they now have, but it's not reasonable to insist > on just this one meeting a different standard.) OK, if that's the view then the patch is ready for commit, AFAICS. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction
On Fri, 2008-05-02 at 12:01 -0400, Tom Lane wrote:
> "Heikki Linnakangas" <[EMAIL PROTECTED]> writes:
> > Simon Riggs wrote:
> >> * We've said here http://www.postgresql.org/docs/faqs.TODO.html that we
> >> "Don't want hints". If that's what we really think, then this patch must
> >> surely be rejected because its a hint... That isn't my view. I *now*
> >> think we do need hints of various kinds.
>
> > cursors_tuple_fraction or OPTIMIZE FOR xxx ROWS isn't the kind of hints
> > we've said "no" to in the past.
>
> More to the point, I think what we've generally meant by "hints" is
> nonstandard decoration on individual SQL commands (either explicit
> syntax or one of those interpret-some-comments kluges).
Yes, that is definitely an Oracle compatibility thought.
> Simon is
> reading the policy in such a way that it would forbid all the planner
> cost parameters, which is surely not what is intended.
So we're allowed to influence the behaviour of the planner, but just not
by touching the individual statements. OK.
Can we allow a statement like
SET index_weighting = '{{my_index, 0.1},{another_index, 0.5}}'
That would allow us to tell a specific SQL statement that it should use
a cost weighting of 0.1 * normal cost for the "my_index" index (etc).
SET enable_seqscan = off; is a blunt instrument that can sometimes
achieve the same thing, but insufficiently exact to be really useful.
Many people use that (Sun, in their first published PostgreSQL
benchmark...)
We/I want to make the planner even better, but the above is roughly what
people want while they're waiting for us to get the planner right.
> I see this as being basically another cost parameter, and as such
> I don't think it needs more documentation than any of those have.
> (Now admittedly you could argue that they could all use a ton more
> documentation than they now have, but it's not reasonable to insist
> on just this one meeting a different standard.)
OK, seems fair.
--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com
--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] GUC parameter cursors_tuple_fraction
"Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > Simon Riggs wrote: >> * We've said here http://www.postgresql.org/docs/faqs.TODO.html that we >> "Don't want hints". If that's what we really think, then this patch must >> surely be rejected because its a hint... That isn't my view. I *now* >> think we do need hints of various kinds. > cursors_tuple_fraction or OPTIMIZE FOR xxx ROWS isn't the kind of hints > we've said "no" to in the past. More to the point, I think what we've generally meant by "hints" is nonstandard decoration on individual SQL commands (either explicit syntax or one of those interpret-some-comments kluges). Simon is reading the policy in such a way that it would forbid all the planner cost parameters, which is surely not what is intended. I see this as being basically another cost parameter, and as such I don't think it needs more documentation than any of those have. (Now admittedly you could argue that they could all use a ton more documentation than they now have, but it's not reasonable to insist on just this one meeting a different standard.) regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Hmm, I'm now wondering if the location should be added to Value as well, > so that it can be passed down to Const? Just for the record, we don't want it in Const. Parse locations are only useful in the "raw grammar" output, mainly because they aren't helpful unless you still have the original query string laying about. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])
Tom Lane escribió: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > I came up with the attached patch. > > I wasn't envisioning anything anywhere near this invasive. We only > need locations on constants in a few contexts, I think. Aha. OK, I'll commit the original patch and let you deal with the rest of it :-) > BTW, you broke _equalAConst() ... it was a bad idea anyway to recast > it on the assumption that there would never again be more than one > field. Oops, sorry, I had intended to revert that part and forgot. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])
Alvaro Herrera <[EMAIL PROTECTED]> writes: > I came up with the attached patch. I wasn't envisioning anything anywhere near this invasive. We only need locations on constants in a few contexts, I think. BTW, you broke _equalAConst() ... it was a bad idea anyway to recast it on the assumption that there would never again be more than one field. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])
Alvaro Herrera escribió: > Tom Lane escribió: > > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > > Tom Lane escribió: > > >> They're logically different things, and after I get done putting a parse > > >> location field into A_Const, they'll still be physically different too. > > > > > Aha. Are you working from Brendan's patch? I was going to commit it. > > > > Sure, go ahead. I was going to add the parse location at the same time, > > but it can perfectly well be done as a separate patch. > > I came up with the attached patch. I added the location bits (although > I am unsure if I got the locations right in the parser), but they are > unused -- figuring out how to use them would take me longer than I can > to spend on this. Hmm, I'm now wondering if the location should be added to Value as well, so that it can be passed down to Const? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])
Tom Lane escribió:
> Alvaro Herrera <[EMAIL PROTECTED]> writes:
> > Tom Lane escribió:
> >> They're logically different things, and after I get done putting a parse
> >> location field into A_Const, they'll still be physically different too.
>
> > Aha. Are you working from Brendan's patch? I was going to commit it.
>
> Sure, go ahead. I was going to add the parse location at the same time,
> but it can perfectly well be done as a separate patch.
I came up with the attached patch. I added the location bits (although
I am unsure if I got the locations right in the parser), but they are
unused -- figuring out how to use them would take me longer than I can
to spend on this.
--
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Index: src/backend/nodes/copyfuncs.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/nodes/copyfuncs.c,v
retrieving revision 1.392
diff -c -p -r1.392 copyfuncs.c
*** src/backend/nodes/copyfuncs.c 14 Apr 2008 17:05:33 - 1.392
--- src/backend/nodes/copyfuncs.c 28 Apr 2008 20:20:44 -
*** _copyAConst(A_Const *from)
*** 1639,1645
break;
}
! COPY_NODE_FIELD(typename);
return newnode;
}
--- 1639,1645
break;
}
! COPY_SCALAR_FIELD(location);
return newnode;
}
Index: src/backend/nodes/equalfuncs.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/nodes/equalfuncs.c,v
retrieving revision 1.321
diff -c -p -r1.321 equalfuncs.c
*** src/backend/nodes/equalfuncs.c 14 Apr 2008 17:05:33 - 1.321
--- src/backend/nodes/equalfuncs.c 28 Apr 2008 16:17:02 -
*** _equalParamRef(ParamRef *a, ParamRef *b)
*** 1691,1701
static bool
_equalAConst(A_Const *a, A_Const *b)
{
! if (!equal(&a->val, &b->val)) /* hack for in-line Value field */
! return false;
! COMPARE_NODE_FIELD(typename);
!
! return true;
}
static bool
--- 1691,1697
static bool
_equalAConst(A_Const *a, A_Const *b)
{
! return equal(&a->val, &b->val); /* hack for in-line Value field */
}
static bool
Index: src/backend/nodes/outfuncs.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/nodes/outfuncs.c,v
retrieving revision 1.325
diff -c -p -r1.325 outfuncs.c
*** src/backend/nodes/outfuncs.c 13 Apr 2008 20:51:20 - 1.325
--- src/backend/nodes/outfuncs.c 28 Apr 2008 17:24:10 -
*** _outAConst(StringInfo str, A_Const *node
*** 1945,1951
appendStringInfo(str, " :val ");
_outValue(str, &(node->val));
! WRITE_NODE_FIELD(typename);
}
static void
--- 1945,1951
appendStringInfo(str, " :val ");
_outValue(str, &(node->val));
! WRITE_INT_FIELD(location);
}
static void
Index: src/backend/parser/gram.y
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/parser/gram.y,v
retrieving revision 2.612
diff -c -p -r2.612 gram.y
*** src/backend/parser/gram.y 14 Apr 2008 17:05:33 - 2.612
--- src/backend/parser/gram.y 28 Apr 2008 19:48:16 -
*** static bool QueryIsRule = FALSE;
*** 91,101
static Node *makeColumnRef(char *relname, List *indirection, int location);
static Node *makeTypeCast(Node *arg, TypeName *typename);
! static Node *makeStringConst(char *str, TypeName *typename);
! static Node *makeIntConst(int val);
! static Node *makeFloatConst(char *str);
! static Node *makeAConst(Value *v);
! static A_Const *makeBoolAConst(bool state);
static FuncCall *makeOverlaps(List *largs, List *rargs, int location);
static void check_qualified_name(List *names);
static List *check_func_name(List *names);
--- 91,102
static Node *makeColumnRef(char *relname, List *indirection, int location);
static Node *makeTypeCast(Node *arg, TypeName *typename);
! static Node *makeStringConst(char *str, int location);
! static Node *makeStringConstCast(char *str, TypeName *typename, int location);
! static Node *makeIntConst(int val, int location);
! static Node *makeFloatConst(char *str, int location);
! static Node *makeAConst(Value *v, int location);
! static Node *makeBoolAConst(bool state, int location);
static FuncCall *makeOverlaps(List *largs, List *rargs, int location);
static void check_qualified_name(List *names);
static List *check_func_name(List *names);
*** set_rest: /* Generic SET syntaxes: */
*** 1112,1118
n->kind = VAR_SET_VALUE;
n->name = "client_encoding";
if ($2 != NULL)
! n->args = list_make1(makeStringConst($2, NULL));
else
n->kind = VAR_SET_DEFAULT;
$$ = n;
--- 1113,1119
n->kind = VAR_SET_VALUE;
n->name = "client_encoding";
if ($2 != NULL)
! n->args = list_make
Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Tom Lane escribió: >> They're logically different things, and after I get done putting a parse >> location field into A_Const, they'll still be physically different too. > Aha. Are you working from Brendan's patch? I was going to commit it. Sure, go ahead. I was going to add the parse location at the same time, but it can perfectly well be done as a separate patch. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Removing typename from A_Const (was: Empty arrays with ARRAY[])
Tom Lane escribió: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Brendan Jurd escribi�: > >> Here's my attempt to remove the typename field from A_Const. There > >> were a few places (notably flatten_set_variable_args() in guc.c, and > >> typenameTypeMod() in parse_type.c) where the code expected to see an > >> A_Const with a typename, and I had to adjust for an A_Const within a > >> TypeCast. Nonetheless, there was an overall net reduction of 34 lines > >> of code, so I think this was a win. > > > Do say ... why don't we do away with A_Const altogether and just replace > > it with Value? After this patch, I don't see what's the difference. > > They're logically different things, and after I get done putting a parse > location field into A_Const, they'll still be physically different too. Aha. Are you working from Brendan's patch? I was going to commit it. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Avahi support for Postgresql
On Tue, 2008-04-01 at 15:34 +, [EMAIL PROTECTED] wrote: > I would really prefer a more loosely coupled system. The functionality will be much the same. The implementation would be more difficult and obscure and there would be more points of failure and more things to configure, but it wouldn't remove much risk, I think. Anyway, this feature is already in Postgres when it's built for MacOS X. So this decision seems to have been made already, at least for that platform. -- [EMAIL PROTECTED] www.murrayc.com www.openismus.com -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Avahi support for Postgresql
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, Apr 01, 2008 at 05:07:31PM +0200, Mathias Hasselmann wrote: [...] > > Personally, I'be rather scared than delighted ;-) > > So in data centers you don't even trust the machines in your broadcast > domain? Kind of. Put it another way: never have services running you don't use. [...] > > Isn't there a less-intrusive option to linking a lib into each and every > > possible server, like a config file in which to put what is to be announced? > > You could directly talk to the D-Bus interface of Avahi. libavahi-client > just is a convenience wrapper. Well, but this route will be much more > cumbersome. So this goes through the D-Bus. Makes kind of sense. Thanks for the enlightenment. > One other route is calling avahi-publish-service on startup and killing > it on shutdown, but: avahi-publish-service really only exists for > demonstration purposes and doesn't handle service name collisions for > instance. I don't believe that a high-profile application like > Postgresql should rely on low-quality hacks, like invoking educational > demo programs. Unelegant as it might seem -- this solution still affords a lot more when it comes to "separation of concerns". I'm still a bit wary at the prospect that each and every daemon evolves into a huge fuzzball linked to all conceivable service-lets with a multitude of funny side-effects (remember tcpwrappers?). Of course, "you can always disable this at compile time", but let's face it: with the predominance of binary distribs, the path of least resistance will be to put up with whatever strange side-effects. I would really prefer a more loosely coupled system. Regards - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFH8lYJBcgs9XrR2kYRAmDJAJ4jWKYkhUfKEAIaZVnIbAAEqJF2AwCfS/6D 4rH9OoY7wjia7h1cuk5CjZI= =AF1W -END PGP SIGNATURE- -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Avahi support for Postgresql
Am Dienstag, den 01.04.2008, 12:02 + schrieb [EMAIL PROTECTED]: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On Tue, Apr 01, 2008 at 09:35:56AM +0200, Mathias Hasselmann wrote: > > Am Samstag, den 29.03.2008, 12:25 + schrieb [EMAIL PROTECTED]: > > [...] > > > Sorry for a dumb question, but I couldn't figure that out from your > > > references [1]..[4]: does that mean that the PostgreSQL server would > > > "advertise itself" on the local net? Or what is the purpose of liking-in > > > libavahi into the postmaster? > > > > Yes, that's the purpose. > > > > > Surely one wouldn't want this in a data center? > > > > Yes, this feature definitely targets small-office use, personal use, DB > > developers [...] > > Still you can tell Avahi to explicitly announce at a certain, non-local > > domain, but this feature is not implemented by the patch. Maybe database > > developers in large network environments could make use of such > > announcements. It would be trivial to add. > > Personally, I'be rather scared than delighted ;-) So in data centers you don't even trust the machines in your broadcast domain? > > > Is there a possiblity to disable that at run time? > > > > The feature is disabled by default. As long as you do not specify a > > zeroconf_name in your configuration file, nothing happens. This is the > > same behavior as established by the Bonjour code. > > Thanks, good to know. > > Isn't there a less-intrusive option to linking a lib into each and every > possible server, like a config file in which to put what is to be announced? You could directly talk to the D-Bus interface of Avahi. libavahi-client just is a convenience wrapper. Well, but this route will be much more cumbersome. One other route is calling avahi-publish-service on startup and killing it on shutdown, but: avahi-publish-service really only exists for demonstration purposes and doesn't handle service name collisions for instance. I don't believe that a high-profile application like Postgresql should rely on low-quality hacks, like invoking educational demo programs. Ciao, Mathias -- Mathias Hasselmann <[EMAIL PROTECTED]> http://www.openismus.com/ - We can get it done. -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Avahi support for Postgresql
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, Apr 01, 2008 at 09:35:56AM +0200, Mathias Hasselmann wrote: > Am Samstag, den 29.03.2008, 12:25 + schrieb [EMAIL PROTECTED]: > [...] > > Sorry for a dumb question, but I couldn't figure that out from your > > references [1]..[4]: does that mean that the PostgreSQL server would > > "advertise itself" on the local net? Or what is the purpose of liking-in > > libavahi into the postmaster? > > Yes, that's the purpose. > > > Surely one wouldn't want this in a data center? > > Yes, this feature definitely targets small-office use, personal use, DB > developers [...] > Still you can tell Avahi to explicitly announce at a certain, non-local > domain, but this feature is not implemented by the patch. Maybe database > developers in large network environments could make use of such > announcements. It would be trivial to add. Personally, I'be rather scared than delighted ;-) > > Is there a possiblity to disable that at run time? > > The feature is disabled by default. As long as you do not specify a > zeroconf_name in your configuration file, nothing happens. This is the > same behavior as established by the Bonjour code. Thanks, good to know. Isn't there a less-intrusive option to linking a lib into each and every possible server, like a config file in which to put what is to be announced? Regards - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFH8iRFBcgs9XrR2kYRAmJ0AJkB7MkxfYI0nVa4RqHVEV1HYjz41gCdEgWz YQ2T4Y/xfoLRF4D6hMLbpEk= =Goho -END PGP SIGNATURE- -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Avahi support for Postgresql
Am Samstag, den 29.03.2008, 12:25 + schrieb [EMAIL PROTECTED]: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On Sat, Feb 23, 2008 at 01:13:38PM +0100, Mathias Hasselmann wrote: > > [...] > > > Avahi/Bonjour/DNS-SD support[1] is very important, for integrating > > Postgresql with modern desktop environments like OSX, GNOME, KDE: It's > > very convenient to choose active DBMS servers in your local network from > > a list, instead of memorizing "cryptic" connection parameters. > > [...] > > > People not wanting DNS-SD support for their server can easily control > > that feature via the "--with-avahi" configure scripts. > > Sorry for a dumb question, but I couldn't figure that out from your > references [1]..[4]: does that mean that the PostgreSQL server would > "advertise itself" on the local net? Or what is the purpose of liking-in > libavahi into the postmaster? Yes, that's the purpose. > Surely one wouldn't want this in a data center? Yes, this feature definitely targets small-office use, personal use, DB developers. Don't know enough about data centers to judge the impact there, but since Avahi - as used in the patch - announces to the local network only, the impact sould be small. Still you can tell Avahi to explicitly announce at a certain, non-local domain, but this feature is not implemented by the patch. Maybe database developers in large network environments could make use of such announcements. It would be trivial to add. > Is there a possiblity to disable that at run time? The feature is disabled by default. As long as you do not specify a zeroconf_name in your configuration file, nothing happens. This is the same behavior as established by the Bonjour code. Ciao, Mathias -- Mathias Hasselmann <[EMAIL PROTECTED]> http://www.openismus.com/ - We can get it done. -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Avahi support for Postgresql
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, Feb 23, 2008 at 01:13:38PM +0100, Mathias Hasselmann wrote: [...] > Avahi/Bonjour/DNS-SD support[1] is very important, for integrating > Postgresql with modern desktop environments like OSX, GNOME, KDE: It's > very convenient to choose active DBMS servers in your local network from > a list, instead of memorizing "cryptic" connection parameters. [...] > People not wanting DNS-SD support for their server can easily control > that feature via the "--with-avahi" configure scripts. Sorry for a dumb question, but I couldn't figure that out from your references [1]..[4]: does that mean that the PostgreSQL server would "advertise itself" on the local net? Or what is the purpose of liking-in libavahi into the postmaster? Surely one wouldn't want this in a data center? Is there a possiblity to disable that at run time? Thanks for any insights - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFH7jUxBcgs9XrR2kYRAsfOAJ0Ulfydo3G1kzQo3HTOdjtswA1A2gCfYYyL VHprJ63unId85/Iht4ha2RE= =get0 -END PGP SIGNATURE- -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Peter Eisentraut napsal(a): Zdenek Kotala wrote: But how it was mentioned in this thread maybe somethink like this "CREATE TABLESPACE name LOCATION '/my/location' SEGMENTS 10GB" should good solution. If segments is not mentioned then default value is used. I think you would need a tool to resegmentize a table or tablespace offline, usable for example when recovering a backup. Do you mean something like strip(1) command? I don't see any usecase for terrabytes data. You usually have a problem to find place where you can backup. Also, tablespace configuration information is of course also stored in a table. pg_tablespace probably won't become large, but it would probably still need to be special-cased, along with other system catalogs perhaps. It is true and unfortunately singularity. Same as database list which is in a table as well, but it is stored also as a text file for startup purpose. I more incline to use non table configuration file for tablespaces, because I don't see any advantage to have it under MVCC control and it allow also to define storage for pg_global and pg_default. An then, how to coordindate offline resegmenting and online tablespace operations in a crash-safe way? Another factor I just thought of is that tar, commonly used as part of a backup procedure, can on some systems only handle files up to 8 GB in size. There are supposed to be newer formats that can avoid that restriction, but it's not clear how widely available these are and what the incantation is to get at them. Of course we don't use tar directly, but if we ever make large segments the default, we ought to provide some clear advice for the user on how to make their backups. I think tar is OK - minimal on Solaris. See man largefile. Default segment size still should be 1GB. If DBA makes a decision to increase this to higher value, then it is his responsibility to find way how to process this big files. Zdenek -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
On Wed, Mar 19, 2008 at 10:51:12AM +0100, Martijn van Oosterhout wrote: > On Wed, Mar 19, 2008 at 09:38:12AM +0100, Peter Eisentraut wrote: > > Another factor I just thought of is that tar, commonly used as part of a > > backup procedure, can on some systems only handle files up to 8 GB in size. > > > > There are supposed to be newer formats that can avoid that restriction, but > > it's not clear how widely available these are and what the incantation is > > to > > get at them. Of course we don't use tar directly, but if we ever make > > large > > segments the default, we ought to provide some clear advice for the user on > > how to make their backups. > > By my reading, GNU tar handles larger files and no-one else (not even a > POSIX standard tar) can... > > Have a nice day, > -- > Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > > Please line up in a tree and maintain the heap invariant while > > boarding. Thank you for flying nlogn airlines. The star program written by Joerg Schilling is a very well written POSIX compatible tar program that can easily handle files larger than 8GB. It is another backup option. Cheers, Ken -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
On Wed, Mar 19, 2008 at 09:38:12AM +0100, Peter Eisentraut wrote: > Another factor I just thought of is that tar, commonly used as part of a > backup procedure, can on some systems only handle files up to 8 GB in size. > There are supposed to be newer formats that can avoid that restriction, but > it's not clear how widely available these are and what the incantation is to > get at them. Of course we don't use tar directly, but if we ever make large > segments the default, we ought to provide some clear advice for the user on > how to make their backups. By my reading, GNU tar handles larger files and no-one else (not even a POSIX standard tar) can... Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ > Please line up in a tree and maintain the heap invariant while > boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Zdenek Kotala wrote: > But how it was mentioned in this thread maybe > somethink like this "CREATE TABLESPACE name LOCATION '/my/location' > SEGMENTS 10GB" should good solution. If segments is not mentioned then > default value is used. I think you would need a tool to resegmentize a table or tablespace offline, usable for example when recovering a backup. Also, tablespace configuration information is of course also stored in a table. pg_tablespace probably won't become large, but it would probably still need to be special-cased, along with other system catalogs perhaps. An then, how to coordindate offline resegmenting and online tablespace operations in a crash-safe way? Another factor I just thought of is that tar, commonly used as part of a backup procedure, can on some systems only handle files up to 8 GB in size. There are supposed to be newer formats that can avoid that restriction, but it's not clear how widely available these are and what the incantation is to get at them. Of course we don't use tar directly, but if we ever make large segments the default, we ought to provide some clear advice for the user on how to make their backups. -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Zdenek Kotala wrote: >> Maybe we can put additional test into configure and collect appropriate >> data from buildfarm. > It might be good to just check first if it returns realistic values for the > example cases that have been mentioned. Yeah, please just make up a ten-line C program that prints the numbers you want, and post it on -hackers for people to try. If manual testing says that it's printing useful numbers, then it would be time enough to think about how to get it into the buildfarm. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Zdenek Kotala wrote: > Yes, agree. It seems only ZFS is OK at this moment and if somebody sets > 32TB he gets nonsegment mode anyway. Surely if you set the segment size to INT64_MAX, you will get nonsegmented behavior anyway, so two code paths might not be necessary at all. > I looked into posix standard and > there is useful function which can be used. See > > http://www.opengroup.org/onlinepubs/009695399/functions/pathconf.html > > Maybe we can put additional test into configure and collect appropriate > data from buildfarm. It might be good to just check first if it returns realistic values for the example cases that have been mentioned. -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Zdenek Kotala <[EMAIL PROTECTED]> writes: > I think current patch could stay in CVS and I will rip out non segment > code path in a new patch. Sure, I feel no need to revert what's applied. Have at it. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: Tom Lane napsal(a): These examples suggest that maybe what we want is not so much a "no segments ever" mode as a segment size larger than 1GB. PS: ZFS is happy with 2^64bit size and UFS has 1TB file size limit (depends on solaris version) So even on Solaris, "no segments ever" is actually a pretty awful idea. As it stands, the code would fail on tables > 1TB. I'm thinking we need to reconsider this patch. Rather than disabling segmentation altogether, we should see it as allowing use of segments larger than 1GB. I suggest that we ought to just flat rip out the "non segmenting" code paths in md.c, and instead look into what segment sizes are appropriate on different platforms. Yes, agree. It seems only ZFS is OK at this moment and if somebody sets 32TB he gets nonsegment mode anyway. I looked into posix standard and there is useful function which can be used. See http://www.opengroup.org/onlinepubs/009695399/functions/pathconf.html Maybe we can put additional test into configure and collect appropriate data from buildfarm. I think current patch could stay in CVS and I will rip out non segment code path in a new patch. Zdenek -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Tom Lane wrote: > Zdenek Kotala <[EMAIL PROTECTED]> writes: > > Tom Lane napsal(a): > >> These examples suggest that maybe what we want is not so much a "no > >> segments ever" mode as a segment size larger than 1GB. > > > PS: ZFS is happy with 2^64bit size and UFS has 1TB file size limit > > (depends on solaris version) > > So even on Solaris, "no segments ever" is actually a pretty awful idea. > As it stands, the code would fail on tables > 1TB. > > I'm thinking we need to reconsider this patch. Rather than disabling > segmentation altogether, we should see it as allowing use of segments > larger than 1GB. I suggest that we ought to just flat rip out the "non > segmenting" code paths in md.c, and instead look into what segment sizes > are appropriate on different platforms. Agreed. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Zdenek Kotala <[EMAIL PROTECTED]> writes: > Tom Lane napsal(a): >> These examples suggest that maybe what we want is not so much a "no >> segments ever" mode as a segment size larger than 1GB. > PS: ZFS is happy with 2^64bit size and UFS has 1TB file size limit > (depends on solaris version) So even on Solaris, "no segments ever" is actually a pretty awful idea. As it stands, the code would fail on tables > 1TB. I'm thinking we need to reconsider this patch. Rather than disabling segmentation altogether, we should see it as allowing use of segments larger than 1GB. I suggest that we ought to just flat rip out the "non segmenting" code paths in md.c, and instead look into what segment sizes are appropriate on different platforms. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Tom Lane napsal(a): "Zeugswetter Andreas OSB SD" <[EMAIL PROTECTED]> writes: Exactly, e.g. AIX is one of those. jfs (not the newer jfs2) has an option to enable large files, which is not the default and cannot be changed post crfs. And even if it is enabled, jfs has a 64 Gb filesize limit ! Anybody know others that support large but not huge files ? Yeah, HPUX 10 is similar --- 128GB hard maximum. It does say you can convert an existing filesystem to largefile support, but it has to be unmounted. These examples suggest that maybe what we want is not so much a "no segments ever" mode as a segment size larger than 1GB. Patch allows to use bigger than 2/4GB segment files and it is possible changed it in source file. But how it was mentioned in this thread maybe somethink like this "CREATE TABLESPACE name LOCATION '/my/location' SEGMENTS 10GB" should good solution. If segments is not mentioned then default value is used. Zdenek PS: ZFS is happy with 2^64bit size and UFS has 1TB file size limit (depends on solaris version) -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
"Zeugswetter Andreas OSB SD" <[EMAIL PROTECTED]> writes: > Exactly, e.g. AIX is one of those. jfs (not the newer jfs2) has an > option > to enable large files, which is not the default and cannot be changed > post crfs. > And even if it is enabled, jfs has a 64 Gb filesize limit ! > Anybody know others that support large but not huge files ? Yeah, HPUX 10 is similar --- 128GB hard maximum. It does say you can convert an existing filesystem to largefile support, but it has to be unmounted. These examples suggest that maybe what we want is not so much a "no segments ever" mode as a segment size larger than 1GB. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I think this needs to be treated as experimental until it's got a few >> more than zero miles under its belt. > OK, then maybe we should document that. Agreed, but at this point we don't even know what hazards we need to document. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
On Mon, 10 Mar 2008, Tom Lane wrote: Peter Eisentraut <[EMAIL PROTECTED]> writes: Tom Lane wrote: Applied with minor corrections. Why is this not the default when supported? Fear. Maybe eventually, but right now I think it's too risky. One point that I already found out the hard way is that sizeof(off_t) = 8 does not guarantee the availability of largefile support; there can also be filesystem-level constraints, and perhaps other things we know not of at this point. Just to note an additional filesystem that will need special action... The VxFS filesystem has a largefiles option, per filesystem. At least that was the case on SCO UnixWare (No, I no longer run it). LER regards, tom lane -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: [EMAIL PROTECTED] US Mail: 430 Valona Loop, Round Rock, TX 78681-3893 -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
> > Why is this not the default when supported? I am wondering both from the > > point of view of the user, and in terms of development direction. > > Also it would get more buildfarm coverage if it were default. If it > breaks something we'll notice earlier. No we don't, because the buildfarm does not test huge files. Andreas -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
> > Why is this not the default when supported? > > Fear. > > Maybe eventually, but right now I think it's too risky. > > One point that I already found out the hard way is that sizeof(off_t) = 8 > does not guarantee the availability of largefile support; there can also > be filesystem-level constraints, and perhaps other things we know not of > at this point. Exactly, e.g. AIX is one of those. jfs (not the newer jfs2) has an option to enable large files, which is not the default and cannot be changed post crfs. And even if it is enabled, jfs has a 64 Gb filesize limit ! Anybody know others that support large but not huge files ? Andreas -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Tom Lane wrote: > I think this needs to be treated as experimental until it's got a few > more than zero miles under its belt. OK, then maybe we should document that. > I wouldn't be too surprised to > find that we have to implement it as a run-time switch instead of > compile-time, in order to not fail miserably when somebody sticks a > tablespace on an archaic filesystem. Yes, that sounds quite useful. Let's wait and see what happens. -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Also it would get more buildfarm coverage if it were default. If it > breaks something we'll notice earlier. Since nothing the regression tests do even approach 1GB, the odds that the buildfarm will notice problems are approximately zero. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Peter Eisentraut wrote: > Tom Lane wrote: > > Zdenek Kotala <[EMAIL PROTECTED]> writes: > > > There is latest version of nonsegment support patch. I changed > > > LET_OS_MANAGE_FILESIZE to USE_SEGMENTED_FILES and I added > > > -disable-segmented-files switch to configure. I kept tuplestore behavior > > > and it still split file in both mode. > > > > Applied with minor corrections. > > Why is this not the default when supported? I am wondering both from the > point of view of the user, and in terms of development direction. Also it would get more buildfarm coverage if it were default. If it breaks something we'll notice earlier. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Applied with minor corrections. > Why is this not the default when supported? Fear. Maybe eventually, but right now I think it's too risky. One point that I already found out the hard way is that sizeof(off_t) = 8 does not guarantee the availability of largefile support; there can also be filesystem-level constraints, and perhaps other things we know not of at this point. I think this needs to be treated as experimental until it's got a few more than zero miles under its belt. I wouldn't be too surprised to find that we have to implement it as a run-time switch instead of compile-time, in order to not fail miserably when somebody sticks a tablespace on an archaic filesystem. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Tom Lane wrote: > Zdenek Kotala <[EMAIL PROTECTED]> writes: > > There is latest version of nonsegment support patch. I changed > > LET_OS_MANAGE_FILESIZE to USE_SEGMENTED_FILES and I added > > -disable-segmented-files switch to configure. I kept tuplestore behavior > > and it still split file in both mode. > > Applied with minor corrections. Why is this not the default when supported? I am wondering both from the point of view of the user, and in terms of development direction. -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] Fix for large file support (nonsegment mode support)
Zdenek Kotala <[EMAIL PROTECTED]> writes: > There is latest version of nonsegment support patch. I changed > LET_OS_MANAGE_FILESIZE to USE_SEGMENTED_FILES and I added > -disable-segmented-files switch to configure. I kept tuplestore behavior > and it still split file in both mode. Applied with minor corrections. regards, tom lane -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches
Re: [HACKERS] [PATCHES] 2WRS [WIP]
Referring to tuplesort.c andtuplestore.c BACKGROUND: Starting from dumptuples() [ tuplesort.c ] write functions move the tuple from a buffer to another in order to finally write it in a logical tape. Is there a way (even the most inefficient way) to use current read/write functions provided by PostgreSQL in order to retrieve the first tuple of a certain run while performing External Sorting? NOTE: I need the first tuple in order to manipulate the whole corresponding run, tuple by tuple since they are written sequentially in a run. Thanks for your attention. Regards, Manolo. -- From: <[EMAIL PROTECTED]> Sent: Tuesday, February 26, 2008 4:10 PM To: "Jaime Casanova" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> Cc: "Decibel!" <[EMAIL PROTECTED]>; "David Fetter" <[EMAIL PROTECTED]>; ; <[EMAIL PROTECTED]> Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP] For the joy of all of you: that's the correct WIP patch. At the moment it only tries to create runs uding two heaps. Hope you can help me with writing those runs on tapes. I'd be very pleased to give you more details. Thenks for your time. Regards, Manolo. -- From: "Jaime Casanova" <[EMAIL PROTECTED]> Sent: Friday, February 22, 2008 5:30 AM To: <[EMAIL PROTECTED]> Cc: "Decibel!" <[EMAIL PROTECTED]>; "Manolo _" <[EMAIL PROTECTED]>; "David Fetter" <[EMAIL PROTECTED]>; ; <[EMAIL PROTECTED]> Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP] On Thu, Feb 21, 2008 at 6:44 AM, <[EMAIL PROTECTED]> wrote: Hi. That's the last release and refers to 8.3.0 and not to 8.2.5 as before. Hope you can tell me if I created it correctly please. no, it doesn't... ! /* GUC variables */ #ifdef TRACE_SORT bool trace_sort = false; #endif - #ifdef DEBUG_BOUNDED_SORT - bool optimize_bounded_sort = true; - #endif it's seems you're removing something added in 8.3 -- regards, Jaime Casanova "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs and the universe trying to produce bigger and better idiots. So far, the universe is winning." Richard Cook ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] 2WRS [WIP]
For the joy of all of you: that's the correct WIP patch. At the moment it only tries to create runs uding two heaps. Hope you can help me with writing those runs on tapes. I'd be very pleased to give you more details. Thenks for your time. Regards, Manolo. -- From: "Jaime Casanova" <[EMAIL PROTECTED]> Sent: Friday, February 22, 2008 5:30 AM To: <[EMAIL PROTECTED]> Cc: "Decibel!" <[EMAIL PROTECTED]>; "Manolo _" <[EMAIL PROTECTED]>; "David Fetter" <[EMAIL PROTECTED]>; ; <[EMAIL PROTECTED]> Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP] On Thu, Feb 21, 2008 at 6:44 AM, <[EMAIL PROTECTED]> wrote: Hi. That's the last release and refers to 8.3.0 and not to 8.2.5 as before. Hope you can tell me if I created it correctly please. no, it doesn't... ! /* GUC variables */ #ifdef TRACE_SORT bool trace_sort = false; #endif - #ifdef DEBUG_BOUNDED_SORT - bool optimize_bounded_sort = true; - #endif it's seems you're removing something added in 8.3 -- regards, Jaime Casanova "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs and the universe trying to produce bigger and better idiots. So far, the universe is winning." Richard Cook ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match tuplesort.patch Description: Binary data ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] 2WRS [WIP]
For the joy of all of you: that's the correct WIP patch. At the moment it only tries to create runs uding two heaps. Hope you can help me with writing those runs on tapes. I'd be very pleased to give you more details. Thenks for your time. Regards, Manolo. -- From: "Jaime Casanova" <[EMAIL PROTECTED]> Sent: Friday, February 22, 2008 5:30 AM To: <[EMAIL PROTECTED]> Cc: "Decibel!" <[EMAIL PROTECTED]>; "Manolo _" <[EMAIL PROTECTED]>; "David Fetter" <[EMAIL PROTECTED]>; ; <[EMAIL PROTECTED]> Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP] On Thu, Feb 21, 2008 at 6:44 AM, <[EMAIL PROTECTED]> wrote: Hi. That's the last release and refers to 8.3.0 and not to 8.2.5 as before. Hope you can tell me if I created it correctly please. no, it doesn't... ! /* GUC variables */ #ifdef TRACE_SORT bool trace_sort = false; #endif - #ifdef DEBUG_BOUNDED_SORT - bool optimize_bounded_sort = true; - #endif it's seems you're removing something added in 8.3 -- regards, Jaime Casanova "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs and the universe trying to produce bigger and better idiots. So far, the universe is winning." Richard Cook ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match tuplesort.patch Description: Binary data ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] Avahi support for Postgresql
Mathias Hasselmann wrote: > The patches were in my initial mail, but now I've also uploaded them to > my personal site for convenience: > > http://taschenorakel.de/files/pgsql-avahi-support/ Hmm, a quick look at the third patch reveals that it is using the "threaded" Avahi client. That's a showstopper. Please consider using some other approach -- writing our own handlers for AvahiPoll would seem apropos. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] Fixes for MONEY type using locale
Tom Lane wrote: > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: > > Well if we are going to continue to support money (which I am against) > > we should support the casting to numeric as that is by far a more > > common implementation of money and we will have mixed environments. > > So, you don't use MONEY, and you don't want to, but nonetheless you > know better than the people who do use MONEY what they need. > > Aside from the semantic-gap issue, there is the point that providing > a cast might actually mask application errors. I can well imagine > cases where one of the reasons for using MONEY is *exactly* that it's > not a plain number or easily convertible to one. Right. I am not thinking of an auto-cast but rather give people _some_ way to cast to/from MONEY, which is what the TODO says: * Allow MONEY to be cast to/from other numeric data types Even when we have multiple currency designations I would think people would need a way to cast. I am trying to anticpate how MONEY will be used. You are right we don't have any field requests yet, but I am expecting them. I have added documentation on the issues of casting to/from MONEY; patch attached and applied. -- Bruce Momjian <[EMAIL PROTECTED]>http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + Index: doc/src/sgml/datatype.sgml === RCS file: /cvsroot/pgsql/doc/src/sgml/datatype.sgml,v retrieving revision 1.217 diff -c -c -r1.217 datatype.sgml *** doc/src/sgml/datatype.sgml 21 Nov 2007 04:01:37 - 1.217 --- doc/src/sgml/datatype.sgml 27 Nov 2007 05:45:13 - *** *** 842,847 --- 842,855 floating-point literals, as well as typical currency formatting, such as '$1,000.00'. Output is generally in the latter form but depends on the locale. + Non-quoted numeric values can be converted to money by + casting the numeric value to text and then + money: + + SELECT 1234::text::money; + + There is no simple way of doing the reverse; casting a money value to a + numeric type. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit
Alvaro Herrera <[EMAIL PROTECTED]> writes: >> Matthew T. O'Connor wrote: >>> Can you make 0 and -1 both valid disabled values? That way it will be >>> compatible with previous releases. >> >> Heh, sure, we can do that too and it doesn't seem like anybody would >> object. I will patch the documentation so that that the "disabled" >> value is zero, and still allow -1. That way it doesn't seem like there >> should be any objection. > Patch attached. It seems like documenting vac_cost_limit as being different from the others will just create perceived complexity/confusion, with no real benefit. I'd suggest leaving the documentation and the default value alone, and applying just the part of the patch that causes 0 to be silently treated as if it were -1. A comment at the spot where this is done would be a good idea, but I don't think we need to say anything in the SGML docs. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit
Alvaro Herrera wrote:
> Matthew T. O'Connor wrote:
> > Can you make 0 and -1 both valid disabled values? That way it will be
> > compatible with previous releases.
>
> Heh, sure, we can do that too and it doesn't seem like anybody would
> object. I will patch the documentation so that that the "disabled"
> value is zero, and still allow -1. That way it doesn't seem like there
> should be any objection.
Patch attached.
--
Alvaro Herrera http://www.amazon.com/gp/registry/5ZYLFMCVHXC
"Saca el libro que tu religión considere como el indicado para encontrar la
oración que traiga paz a tu alma. Luego rebootea el computador
y ve si funciona" (Carlos Duclós)
Index: doc/src/sgml/catalogs.sgml
===
RCS file: /home/alvherre/Code/cvs/pgsql/doc/src/sgml/catalogs.sgml,v
retrieving revision 2.153
diff -c -p -r2.153 catalogs.sgml
*** doc/src/sgml/catalogs.sgml 5 Jun 2007 21:31:03 - 2.153
--- doc/src/sgml/catalogs.sgml 8 Jun 2007 14:40:10 -
***
*** 1339,1347
be used for this particular value. Observe that the
vac_cost_delay variable inherits its default value from the
configuration parameter,
!or from vacuum_cost_delay if the former is set to a negative
!value. The same applies to vac_cost_limit.
!Also, autovacuum will ignore attempts to set a per-table
freeze_max_age larger than the system-wide setting (it can only be set
smaller), and the freeze_min_age value will be limited to half the
system-wide setting.
--- 1339,1350
be used for this particular value. Observe that the
vac_cost_delay variable inherits its default value from the
configuration parameter,
!or from if the former is set to a negative
!value. vac_cost_limit is an exception to this rule, because
!the value 0 is used to indicate that the
! configuration parameter
!should be used, or if the former is set to a
!zero or negative value. Note that autovacuum will ignore attempts to set a per-table
freeze_max_age larger than the system-wide setting (it can only be set
smaller), and the freeze_min_age value will be limited to half the
system-wide setting.
Index: doc/src/sgml/config.sgml
===
RCS file: /home/alvherre/Code/cvs/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.126
diff -c -p -r1.126 config.sgml
*** doc/src/sgml/config.sgml 7 Jun 2007 19:19:56 - 1.126
--- doc/src/sgml/config.sgml 8 Jun 2007 14:15:33 -
*** SELECT * FROM parent WHERE key = 2400;
*** 3356,3362
Specifies the cost limit value that will be used in automatic
! VACUUM operations. If -1 is specified (which is the
default), the regular
value will be used. Note that
the value is distributed proportionally among the running autovacuum
--- 3356,3362
Specifies the cost limit value that will be used in automatic
! VACUUM operations. If 0 is specified (which is the
default), the regular
value will be used. Note that
the value is distributed proportionally among the running autovacuum
Index: src/backend/postmaster/autovacuum.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/postmaster/autovacuum.c,v
retrieving revision 1.47
diff -c -p -r1.47 autovacuum.c
*** src/backend/postmaster/autovacuum.c 30 May 2007 20:11:57 - 1.47
--- src/backend/postmaster/autovacuum.c 8 Jun 2007 14:23:35 -
*** static void
*** 1548,1554
autovac_balance_cost(void)
{
WorkerInfo worker;
! int vac_cost_limit = (autovacuum_vac_cost_limit >= 0 ?
autovacuum_vac_cost_limit : VacuumCostLimit);
int vac_cost_delay = (autovacuum_vac_cost_delay >= 0 ?
autovacuum_vac_cost_delay : VacuumCostDelay);
--- 1548,1554
autovac_balance_cost(void)
{
WorkerInfo worker;
! int vac_cost_limit = (autovacuum_vac_cost_limit > 0 ?
autovacuum_vac_cost_limit : VacuumCostLimit);
int vac_cost_delay = (autovacuum_vac_cost_delay >= 0 ?
autovacuum_vac_cost_delay : VacuumCostDelay);
*** table_recheck_autovac(Oid relid)
*** 2143,2151
*/
if (avForm != NULL)
{
! vac_cost_limit = (avForm->vac_cost_limit >= 0) ?
avForm->vac_cost_limit :
! ((autovacuum_vac_cost_limit >= 0) ?
autovacuum_vac_cost_limit : VacuumCostLimit);
vac_cost_delay = (avForm->vac_cost_delay >= 0) ?
--- 2143,2151
*/
if (avForm != NULL)
{
! vac_cost_limit = (avForm->vac_cost_limit > 0) ?
avForm->vac_cost_limit :
! ((autovacuum_vac_cost_limit > 0) ?
autovacuum_vac_cost_limit : VacuumCostLimit);
vac_cost_delay = (avForm->vac_cost_delay >= 0) ?
***
Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit
Matthew T. O'Connor wrote: > Alvaro Herrera wrote: > >Tom Lane wrote: > >>Alvaro Herrera <[EMAIL PROTECTED]> writes: > >>>But this is misleading (started postmaster with good value, then edited > >>>postgresql.conf and entered "-2"): > >>>17903 LOG: received SIGHUP, reloading configuration files > >>>17903 LOG: -2 is outside the valid range for parameter > >>>"autovacuum_vacuum_cost_limit" (-1 .. 1000) > >>>Note how it still says the range is -1 .. 1000. > >>Can we redefine things to make zero be the "disabled" value, thus > >>keeping the range of valid values contiguous? > > > >That would be another solution ... though it would be different from the > >valid value for autovacuum_vacuum_cost_delay (on which 0 is a valid > >value). Also it would be a different value from previous versions. > > > >I don't think either of these is a showstopper, so let's go for that if > >nobody objects. > > Can you make 0 and -1 both valid disabled values? That way it will be > compatible with previous releases. Heh, sure, we can do that too and it doesn't seem like anybody would object. I will patch the documentation so that that the "disabled" value is zero, and still allow -1. That way it doesn't seem like there should be any objection. -- Alvaro Herrera http://www.flickr.com/photos/alvherre/ "Escucha y olvidarás; ve y recordarás; haz y entenderás" (Confucio) ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML namespaces
XML is now more stabilized in the backend than when the patch appeared,
and it doesn't make sense to add features to a /contrib interface that
is to be used only for backward compatability.
Patch rejected. You can put the patch on pgfoundry if you are worried
others might need this functionality.
---
Mike Rylander wrote:
> On 3/6/07, Mike Rylander <[EMAIL PROTECTED]> wrote:
> > On 3/6/07, Peter Eisentraut <[EMAIL PROTECTED]> wrote:
> > > Mike Rylander wrote:
> > > > The patch adds support for default XML namespaces in xml2 by providing
> > > > a mechanism for supplying a prefix to a named namespace URI.
> > >
> > > How does it support multiple namespaces in one document?
> >
> > It supports one default (unprefixed) namespace URI per document, which
> > ISTM is the overwhelmingly common case (and the itch that I must
> > scratch).
>
> I think there is some confusion about what the current xml2 contrib
> module supports and what my patch adds. The current code, as it
> stands today, supports multiple namespaces just fine. The only
> requirement is that each namespace have a prefix, or else one is
> forced to use the local-name() construct with every single node for
> those nodes in unprefixed ("default") namespaces. This patch simply
> adds support for registering a prefix for an unprefixed namespace,
> which is an extremely common case in XML and causes the use of overly
> verbose contortions when designing XPath expressions. To illustrate
> this, xml2 currently supports all of these statements:
>
> SELECT xpath_nodeset('foo','/x/y');
> SELECT xpath_nodeset('foo','/x/a:y');
> SELECT xpath_nodeset(' xmlns:a="uri:for:a">foo','/b:x/a:y');
>
> All number and manner of /prefixed/ namespaces work fine today.
> However, in order to match an element or attribute with an unprefixed
> namespace, the xpath becomes a study in overly verbose, human error
> inducing repetition. For instance, consider the extremely common case
> of an xhtml document that does not use a prefix for the xhtml
> namespace. Using the xml2 contrib module as it stands today, without
> my patch, using XPath to get the title of the document might look
> something like this:
>
> /*[local-name()="html"]/*[local-name()="head"]/*[local-name()="title"]
>
> Now just imagine the XPath needed to get a portion of the body in a
> nested div based on the existence of some other node ... the logic
> gets lost in the noise simply because of the overhead of
> namespace-qualifying the elements.
>
> Namespaces were introduced in XML to address verbosity issues (among
> other things), but as XPath was designed primarily as a language for
> use inside XSLT (where namespace support is fully integrated) it
> didn't get the treatment needed to handle unprefixed namespaces. To
> address /that/ issue, my patch allows the registration of a supplied
> prefix for a supplied URI, which solves the common "default namespace"
> problem in a completely backward compatible way. The above example
> XPath can now become:
>
> /x:html/x:head/x:title
>
> simply by supplying 2 more arguments to the _ns version of any of the
> xpath_ functions available in xml2. I challenge anyone to claim that
> the [local-name()="foo] variant is easier to read and less error prone
> than the second, namespace-prefixed variant. They are exactly
> equivalent, but the second (quite obviously) is Better(tm).
>
> I understand that XML support is planned and at least partially
> implemented for 8.3, but many production instances will be unable (or,
> in fact, unwilling) to upgrade to 8.3 for quite some time. Because
> this patch is completely backward compatible it can (theoretically) be
> included in future 8.1 and 8.2 releases, and for those of us that need
> more full XML support in the short term the upgrade of a contrib
> module is probably a very viable option -- it is for me, anyway.
>
> So, to sum up, please let me know what I can do to increase the
> chances of getting this patch included. Alternatively, if my patch is
> being vetoed, please let me know that too so that I can create a local
> maintenance plan for this.
>
> Thanks in advance. I've attached the patch again for reference.
>
> --
> Mike Rylander
> [EMAIL PROTECTED]
> GPLS -- PINES Development
> Database Developer
> http://open-ils.org
[ Attachment, skipping... ]
>
> ---(end of broadcast)---
> TIP 4: Have you searched our list archives?
>
>http://archives.postgresql.org
--
Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
