Re: [HACKERS] Managing multiple branches in git
Hi, Quoting Alvaro Herrera alvhe...@commandprompt.com: The only rant I have about the outcome is that Linus did not copy more of it. He he.. nice way of looking at it ;-) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Synchronous replication: status of standby side
Hi, On Thu, Jun 4, 2009 at 12:28 AM, Kolb, Harald (NSN - DE/Munich) harald.k...@nsn.com wrote: Hi, will there be a possibility to retrieve the standby situation in case of synchronous replication ? We would need a command or similar to report the current state and state changes (like: syncing, in-sync, replication broken, ..). Observing and scaning the logfile would not be appropriate. Are there plans for such an interface ? No, not yet. You think that this feature is a high-priority TODO item? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL Developer meeting minutes up
Hi, Quoting Greg Stark st...@enterprisedb.com: This is all completely irrelevant to the CVS import. To the CVS import it is, yes. After all, CVS has no notion of renaming files. But my example is about renaming with git *after* the conversion. Git *does* support renaming (to some extent). However, it fails as explained if you feed it with corrupt data (the corruption being the missing link between the two added files - after a rename, git simply has no chance of knowing it should be the same file). I don't think we've ever renamed files because CVS can't handle it cleanly. Yes, that applies to the past. But I think we *are* going to rename files *after* the switch, because git *can* handle it cleanly - given a correct import. If that defect would only affect historic information, I'd not be half as pestering as I am. But it's such delayed effects which might surprise you years after the cause, which make me nervous. It does sound to me like we really ought to have merge commits marking the bug fixes in old releases as merged in the equivalent commits to later branches based on Tom's commit messages. Now, I don't know how you got to that conclusion, but I absolutely agree ;-) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL Developer meeting minutes up
On 4 Jun 2009, at 09:11, Markus Wanner mar...@bluegap.ch wrote: Hi, Quoting Greg Stark st...@enterprisedb.com: This is all completely irrelevant to the CVS import. To the CVS import it is, yes. After all, CVS has no notion of renaming files. But my example is about renaming with git *after* the conversion. Git *does* support renaming (to some extent). However, it fails as explained if you feed it with corrupt data (the corruption being the missing link between the two added files - after a rename, git simply has no chance of knowing it should be the same file). Hmm. I see. I'm not sure we've ever added files to back branches either. I'm less sure of that though. I don't think we've ever renamed files because CVS can't handle it cleanly. Yes, that applies to the past. But I think we *are* going to rename files *after* the switch, because git *can* handle it cleanly - given a correct import. If that defect would only affect historic information, I'd not be half as pestering as I am. But it's such delayed effects which might surprise you years after the cause, which make me nervous. It does sound to me like we really ought to have merge commits marking the bug fixes in old releases as merged in the equivalent commits to later branches based on Tom's commit messages. Now, I don't know how you got to that conclusion, but I absolutely agree ;-) Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Fix LOCK TABLE to eliminate the race condition that could make it
On Tue, 2009-06-02 at 14:33 -0400, Bruce Momjian wrote: Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: If we're going to require cascaded permissions like this, would it make sense to make GRANT cascade down the inheritance tree also? That's been discussed before. I forget whether we decided it was a good idea or not, but in any case it looks like a new feature not a bug fix. Is this a TODO? Yes. Whatever was discussed before was prior to the changes to make TRUNCATE and LOCK TABLE cascade. The cascading will fail without appropriate permissions, so GRANT should work that way also, now. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_standby -l might destory the archived file
On Tue, 2009-06-02 at 14:54 -0400, Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Tom Lane wrote: That's a good point; don't we recover files under names like RECOVERYXLOG, not under names that could possibly conflict with regular WAL files? Yes. But we rename RECOVERYXLOG to 00010057 or similar at the end of recovery, in exitArchiveRecovery(). Thinking about this some more, I think we should've changed exitArchiveRecovery() rather than RemoveOldXlogFiles(): it would be more robust if exitArchiveRecovery() always copied the last WAL file rather than just renamed it. It doesn't seem safe to rely on the file the symlink points to to be valid after recovery is finished, and we might write to it before it's recycled, so the current fix isn't complete. Hmm. I think really the reason it's coded that way is that we assumed the recovery command would be physically copying the file from someplace else. pg_standby is violating the backend's expectations by using a symlink. And I really doubt that the technique is saving anything, since the data has to be read in from the archive location anyway. I'm leaning back to the position that pg_standby's -l option is simply a bad idea and should be removed. ISTM we didn't clearly state what the recovery_command should do either way. Even if you remove the pg_standby option that will not fix the problem for people who have written their own script, or existing users of pg_standby. The safe way is to do as Heikki suggests and copy the final file into place and I would add that we must then fsync it also. That should be back-patched possibly as far as 8.0. Documenting a change is not nearly enough. Removing -l is a separate discussion. If there was a consensus against it, I would suggest that we deprecate the option, so that it does nothing. As an aside, I would be also much more comfortable if there was an option to not recycle the WAL files at all, as a safe mode for error checking at least. The question is: why do we need to zero fill the file first anyway? We could save the 8 bytes for the prev pointer on every record if we added enough zeroes onto every WAL write to zap any pre-existing header data, causing recovery to fail. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_standby -l might destory the archived file
On Tue, 2009-06-02 at 19:43 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Mon, 2009-06-01 at 14:47 +0900, Fujii Masao wrote: pg_standby can use ln command to restore an archived file, which might destroy the archived file as follows. 1) pg_standby creates the symlink to the archived file '102' 2) '102' is applied 3) the next file '103' doesn't exist and the trigger file is created 4) '102' is re-fetched 5) at the end of recovery, the symlink to '102' is rename to '202', but it still points '102' 6) after recovery, '202' is recycled (rename to '208', which still points '102') 7) '208' is written new xlog records over -- the archived file '102' comes down! One simple solution to fix this problem... err...I don't see *any* problem at all, since pg_standby does not do step (1) in the way you say and therefore never does step (5). Any links created are explicitly deleted in all cases at the end of recovery. I don't know how you came to that conclusion, but Fujii-sans description seems accurate to me, and I can reproduce the behavior on my laptop. I accept there is a bug, though the initial description did not appear to reflect the way the code works. Thank you for not making further changes while we wait to discuss. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL Developer meeting minutes up
Hi, Quoting Greg Stark greg.st...@enterprisedb.com: Hmm. I see. I'm not sure we've ever added files to back branches either. I'm less sure of that though. We did from time to time. Every merge commit in my current conversion contains at least one such file that got added as part of a back patch. The perl file mentioned in the example upstream is one of them. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Question about FUNCDETAIL_MULTIPLE
I was wondering what the philosophy is behind letting an ambiguous function be created in the first place. Is this for backwards compatibility or perhaps for historical reasons? Could someone clarify this please? Consider the following example: ( FYI: parse_func.c-ParseFuncOrColumn-164 parse_func.c-ParseFuncOrColumn-810 parse_func.c-ParseFuncOrColumn-836 namespace.c-FuncnameGetCandidates-607 namespace.c-FuncnameGetCandidates-826 parse_func.c-ParseFuncOrColumn-224 ) - create or replace function foo1(int,int default 1,int default 2) returns int as $$ select $1+$2+$3 $$ language sql; create or replace function foo1(int,int default 1) returns int as $$ select $1+$2 $$ language sql; select * from foo1(10,10) ERROR: function foo1(integer, integer) is not unique LINE 15: select * from foo1(10,10) ^ HINT: Could not choose a best candidate function. You might need to add explicit type casts. - -- Regards, Gevik -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL Developer meeting minutes up
Hi, Quoting Tom Lane t...@sss.pgh.pa.us: BTW, Markus: you do realize thomas is not me but Tom Lockhart? Uh.. thanks, that name has fallen through the cracks, before. I've added it now, it will be included in the next sample conversion. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL Developer meeting minutes up
Hi, Quoting Marko Kreen mark...@gmail.com: I'm not sure whether we should mark the old branches getting merges down or the new branches getting merged up. I suspect I'm missing something but I don't see any reason one is better than the other. As pointed out by others, it doesn't make sense to merge (all commits since the last merge) from HEAD to the back branches. You'd have to cherry-pick only the commits which actually have to get back patched. The new branches getting merged up could work. That is, applying the fix to the oldest back-branch which requires the fix first and then merge it to all newer ones, including HEAD. However, that would require some rethinking: instead of creating bugfix-patches for HEAD, then manually adjust patches for back-branches and then group committing, you'd have to create a bugfix-patch for the oldest branch first, commit that and then merge that to the newer branches. I consider merging a cleaner and simpler operation than cherry-picking, because merging allows the VCS to keep track of what needs to be propagated, while with cherry-picking, you'd have to keep track of that manually (or with the help of other tools). An example for that is the very same unability to properly track renames when cherry-picking, just like what I explained for the CVS conversion. It seems to require noticeable development effort to get a importer to a level it can do it. Will this be a requirement for import? Or just a good thing to have? Also how to check if all such merges are sensible? If that's how you'd like to have the CVS repository represented in git (which I'd support as well), I'd give it a try. With all of the work I've done for mtn cvs_import I certainly have the necessary experience in CVS conversion and with the cvs2svn algorithm itself. And note that such effort will affect only old imported history, it will not make easier to handle back-branch fixes in the future... Hm.. depends, if you want to merge from older branches to newer ones, instead of cherry-picking, it would certainly help to get the history clean. Various scenarios with git cherry-pick and similar tools would still result in duplicate commits, so we would need a git log post-processor anyway if we want to somehow group them together for eg. weekly commit summary. And such post-processor would work on old history too. I think we should decide on either using merges or using duplicate commits we try to link somehow. But then, we should IMO use that scheme for the conversion as well as later on, so as not to get a messy history, as you put it. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
On Thu, Jun 4, 2009 at 12:40 AM, Andrew Dunstan and...@dunslane.net wrote: Well, that's not too easy to read, but anyway, I got to the bottom of it. Turns out there's a change in the Perl embedding API that nobody noticed. The attached patch worked for me on Windows. We'll need something like this on other platforms I believe. I see via Google that Debian is objecting to packages that call perl_parse without calling PERL_SYS_INIT (or PERL_SYS_INIT3). I'm even wondering if we should backpatch it. Looks good to me too - thanks! Of course, that leaves the issue of the library name to be fixed, but sufficient unto the day ... Yeah - I chatted briefly with Magnus about that a while back. I added a quick hack to Mkvcbuild.pm to allow it to use either 5.10 or 5.8, but didn't even think about the msys build. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] PANIC: corrupted item lengths
I've seen a couple of *topic issues lately. What seems strange about the various errors generated in bufpage.c is that they are marked as ERRORs, yet are executed within a critical section causing the system to PANIC. There are a number of sanity checks that are made prior to any changes taking place, so they need not generate PANICs. *topic is one such error message but there are others. There is no need to take down the whole server just because one block has a corruption. I'm not advocating that corruptions are acceptable, but the server provides no way to remove the corruption, which is a problem and server really should keep a better grip on its towel in any case. I would much prefer: * VACUUMs seeing these errors would perform as with zero_damaged_pages. * other backends seeing those errors should just ERROR out. We can do this by having a new function: boolean PageIsValid() which performs the sanity checks. This can then be executed by heap_page_prune() prior to entering the critical section. That will then be called correctly by both VACUUM and other code. VACUUM can then optionally zero out the block, as is done with PageHeaderIsValid(). Votes? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Synchronous replication: status of standby side
Hi, -Original Message- From: ext Fujii Masao [mailto:masao.fu...@gmail.com] Sent: Thursday, June 04, 2009 6:25 AM To: Kolb, Harald (NSN - DE/Munich) Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Synchronous replication: status of standby side Hi, On Thu, Jun 4, 2009 at 12:28 AM, Kolb, Harald (NSN - DE/Munich) harald.k...@nsn.com wrote: Hi, will there be a possibility to retrieve the standby situation in case of synchronous replication ? We would need a command or similar to report the current state and state changes (like: syncing, in-sync, replication broken, ..). Observing and scaning the logfile would not be appropriate. Are there plans for such an interface ? No, not yet. You think that this feature is a high-priority TODO item? We are planning to use the HotStandby feature too. That would require to have the correct state information since we have to control the DB clients when and how they can connect or reconnect. But also in general, from our platform POV it's an important feature to have the right view on the system at any point of time (e.g. in case of update avoid switchover if standby is out of sync, ...). In addition, can we expect that a replication break will be logged as an important event ? Regards, Harald. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
Dave Page wrote: On Thu, Jun 4, 2009 at 12:40 AM, Andrew Dunstan and...@dunslane.net wrote: Of course, that leaves the issue of the library name to be fixed, but sufficient unto the day ... Yeah - I chatted briefly with Magnus about that a while back. I added a quick hack to Mkvcbuild.pm to allow it to use either 5.10 or 5.8, but didn't even think about the msys build. Yeah, this is something we need to look at for 8.5. I think it's too late in the cycle to start messing with it for 8.4. -- Magnus Hagander Self: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Improving the ngettext() patch
Russian plural forms for 100, 101, 102 etc. is different, as for 0, 1, 2. True. The rule IIRC is that except for 11-14 and for collective numerals, declination follows the last digit. It would be possible to generalize declination via a language-specific message-selector function, especially if the number of numerical complements were limited to 1. How awkward would it be to re-word the style of messages to avoid declination? For example, the Russian equivalent of X rows could be something like #rows -- X. David Hudson
Re: [HACKERS] Question about FUNCDETAIL_MULTIPLE
2009/6/4 Gevik Babakhani pg...@xs4all.nl: I was wondering what the philosophy is behind letting an ambiguous function be created in the first place. Is this for backwards compatibility or perhaps for historical reasons? Could someone clarify this please? This is +/- for historical reasons. We used original algorithm for ambiguous function identification. regards Pavel Stehule Consider the following example: ( FYI: parse_func.c-ParseFuncOrColumn-164 parse_func.c-ParseFuncOrColumn-810 parse_func.c-ParseFuncOrColumn-836 namespace.c-FuncnameGetCandidates-607 namespace.c-FuncnameGetCandidates-826 parse_func.c-ParseFuncOrColumn-224 ) - create or replace function foo1(int,int default 1,int default 2) returns int as $$ select $1+$2+$3 $$ language sql; create or replace function foo1(int,int default 1) returns int as $$ select $1+$2 $$ language sql; select * from foo1(10,10) ERROR: function foo1(integer, integer) is not unique LINE 15: select * from foo1(10,10) ^ HINT: Could not choose a best candidate function. You might need to add explicit type casts. - -- Regards, Gevik -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
Magnus Hagander wrote: Dave Page wrote: On Thu, Jun 4, 2009 at 12:40 AM, Andrew Dunstan and...@dunslane.net wrote: Of course, that leaves the issue of the library name to be fixed, but sufficient unto the day ... Yeah - I chatted briefly with Magnus about that a while back. I added a quick hack to Mkvcbuild.pm to allow it to use either 5.10 or 5.8, but didn't even think about the msys build. Yeah, this is something we need to look at for 8.5. I think it's too late in the cycle to start messing with it for 8.4. I don't see why. Perl 5.10 is out there and people will want to use it. I have a simple fix for this in GNUMakefile (see below) , and putting something similar in the MSVC build stuff will be simple too. If we don't want to support Perl 5.10 why was the item on the open issues list at all (and why did I just spend hours finding a solution to the hard part)? I have cleaned up the patch from yesterday slightly, too. I'm wondering how far back we should backpatch all this. This API was actually documented in Perl (just) before 7.4 was released, so in theory the API adjustment should go all the way back. However, in practice it hasn't been a problem until now, and we aren't supporting Windows releases before 8.2, so I think 8.2 is probably the right spot. cheers andrew Index: GNUmakefile === RCS file: /cvsroot/pgsql/src/pl/plperl/GNUmakefile,v retrieving revision 1.36 diff -u -r1.36 GNUmakefile --- GNUmakefile 2 Oct 2008 08:11:11 - 1.36 +++ GNUmakefile 4 Jun 2009 12:28:31 - @@ -19,7 +19,8 @@ ifeq ($(PORTNAME), win32) perl_archlibexp := $(subst \,/,$(perl_archlibexp)) perl_privlibexp := $(subst \,/,$(perl_privlibexp)) -perl_embed_ldflags = -L$(perl_archlibexp)/CORE -lperl58 +perl_lib := $(basename $(notdir $(wildcard $(perl_archlibexp)/CORE/perl*.lib))) +perl_embed_ldflags = -L$(perl_archlibexp)/CORE -l$(perl_lib) override CPPFLAGS += -DPLPERL_HAVE_UID_GID # Perl on win32 contains /* within comment all over the header file, # so disable this warning. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC,PATCH] Avoid manual shift-and-test logic in AllocSetFreeIndex
+/*
+ * fls: find last set bit.
+ *
+ * Returns the 1-based index of the most-significant bit in x. The MSB
+ * is bit number 32, the LSB is bit number 1. If x is zero, the result is
+ * undefined.
+ */
+static inline int
+fls(unsigned int x)
...
+ idx = fls((size - 1) ALLOC_MINBITS);
If size = 8, fls((size - 1) ALLOC_MINBITS) is fls(0).
The result of fls(0) is undefined.
I think we have to never call fls(0) from AllocSetFreeIndex().
My proposal code:
if (size (1 ALLOC_MINBITS))
{
idx = fls((size - 1) ALLOC_MINBITS);
Assert(idx ALLOCSET_NUM_FREELISTS);
}
Best regards,
---
Atsushi Ogawa
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
Andrew Dunstan wrote: Magnus Hagander wrote: Dave Page wrote: On Thu, Jun 4, 2009 at 12:40 AM, Andrew Dunstan and...@dunslane.net wrote: Of course, that leaves the issue of the library name to be fixed, but sufficient unto the day ... Yeah - I chatted briefly with Magnus about that a while back. I added a quick hack to Mkvcbuild.pm to allow it to use either 5.10 or 5.8, but didn't even think about the msys build. Yeah, this is something we need to look at for 8.5. I think it's too late in the cycle to start messing with it for 8.4. I don't see why. Perl 5.10 is out there and people will want to use it. I have a simple fix for this in GNUMakefile (see below) , and putting something similar in the MSVC build stuff will be simple too. If we don't want to support Perl 5.10 why was the item on the open issues list at all (and why did I just spend hours finding a solution to the hard part)? I mostly meant a general make-it-version-independent fix. The fix that's in the MSVC build system now is really just a hack that only supports 5.8 and 5.10, not any other versions. But sure, if it's that easy, go for it :-) I have cleaned up the patch from yesterday slightly, too. I'm wondering how far back we should backpatch all this. This API was actually documented in Perl (just) before 7.4 was released, so in theory the API adjustment should go all the way back. However, in practice it hasn't been a problem until now, and we aren't supporting Windows releases before 8.2, so I think 8.2 is probably the right spot. IIRC, all the complaints we've seen about it is from people using the binary installer (which would be the vast majority of the Windows users). And we're not going to switch to 5.10 in any of the backbranches there, ever. Based on that argument, backpatching doesn't need to be done at all. For the sake of those who do build from source, going back to 8.2 seems reasonable. There's certainly no point in going past it - we have stopped backpatching much more important fixes there a long time ago. -- Magnus Hagander Self: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question about FUNCDETAIL_MULTIPLE
Gevik Babakhani pg...@xs4all.nl writes: I was wondering what the philosophy is behind letting an ambiguous function be created in the first place. Is this for backwards compatibility or perhaps for historical reasons? Neither; it's a feature, and one we quite like. For example, would you really prefer that the six different versions of abs() had to have different names? regression=# \df abs List of functions Schema | Name | Result data type | Argument data types | Type +--+--+-+ pg_catalog | abs | bigint | bigint | normal pg_catalog | abs | double precision | double precision| normal pg_catalog | abs | integer | integer | normal pg_catalog | abs | numeric | numeric | normal pg_catalog | abs | real | real| normal pg_catalog | abs | smallint | smallint| normal (6 rows) Even if you were willing to do that, what about the forty-seven distinct versions of +? Overloaded operators are not fundamentally different from overloaded functions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PANIC: corrupted item lengths
Simon Riggs wrote: I would much prefer: * VACUUMs seeing these errors would perform as with zero_damaged_pages. * other backends seeing those errors should just ERROR out. We can do this by having a new function: boolean PageIsValid() which performs the sanity checks. This can then be executed by heap_page_prune() prior to entering the critical section. That will then be called correctly by both VACUUM and other code. VACUUM can then optionally zero out the block, as is done with PageHeaderIsValid(). I tend to hate automatic zeroing of pages because there's no way to get the contents later for forensics. I would support your proposal if we had a way of saving the block elsewhere before zeroing it (say create a directory corrupted+zeroed similar to lost+found in the database dir and save it annotated with the OID of the table and the block number). The main problem I see with this is that if you don't immediately act to examine the data, some of the pg_clog files that you need to be able to read these files may be gone. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PANIC: corrupted item lengths
On Thu, Jun 4, 2009 at 2:55 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: I tend to hate automatic zeroing of pages because there's no way to get the contents later for forensics. That's why we default to zero_damaged_pages=false. Saving the pages somewhere seems like it would be a good feature to add for zero_damaged_pages in general but an orthogonal issue. Simon's suggestion is a few more cases we could catch send through the zero_damaged_pages code path rather than crashing. If his analysis of the existing code is correct -- I haven't looked but I assume so -- then that sounds like the right thing to do. I would be concerned about the overhead of checking this on all the line pointers if we're just trying to look up a single item. But from the sound of his description it would only fire in the hot pruning for which we already control the frequency. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PANIC: corrupted item lengths
Simon Riggs si...@2ndquadrant.com writes: What seems strange about the various errors generated in bufpage.c is that they are marked as ERRORs, yet are executed within a critical section causing the system to PANIC. The reason we PANIC there is to reduce the probability that bad data will be written back to disk. Of course, if the bad data was read off disk in the first place, there's no hope --- but we have checks on incoming pages for that. What seems significantly more likely if we detect a problem here is that we somehow corrupted the page while it sits in shared buffers. So, there's some hope that the corruption will not get back to disk, so long as we PANIC and thereby cause shared-memory contents to be flushed. Votes? +1 for no change. We could make the page-read-time validation checks stricter, if there's some specific pattern you're seeing that gets past those checks now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
Andrew Dunstan and...@dunslane.net writes: I have cleaned up the patch from yesterday slightly, too. I'm wondering how far back we should backpatch all this. This API was actually documented in Perl (just) before 7.4 was released, so in theory the API adjustment should go all the way back. However, in practice it hasn't been a problem until now, and we aren't supporting Windows releases before 8.2, so I think 8.2 is probably the right spot. My inclination is backpatch all the way. Now that we know the problem is we are failing to adhere to Perl's published API, I bet we could find some other problems it causes, if we poked into the right nooks and crannies. And not just on Windows. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Improving the ngettext() patch
p...@thetdh.com writes: Russian plural forms for 100, 101, 102 etc. is different, as for 0, 1, 2. True. The rule IIRC is that except for 11-14 and for collective numerals, declination follows the last digit. Wow. So how does anyone represent that in the .po files? AFAICT the notation the gettext machinery provides isn't really powerful enough for this. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Question about FUNCDETAIL_MULTIPLE
Tom Lane wrote: Gevik Babakhani pg...@xs4all.nl writes: I was wondering what the philosophy is behind letting an ambiguous function be created in the first place. Is this for backwards compatibility or perhaps for historical reasons? Neither; it's a feature, and one we quite like. For example, would you really prefer that the six different versions of abs() had to have different names? regression=# \df abs List of functions Schema | Name | Result data type | Argument data types | Type +--+--+-+ pg_catalog | abs | bigint | bigint | normal pg_catalog | abs | double precision | double precision| normal pg_catalog | abs | integer | integer | normal pg_catalog | abs | numeric | numeric | normal pg_catalog | abs | real | real| normal pg_catalog | abs | smallint | smallint| normal (6 rows) Even if you were willing to do that, what about the forty-seven distinct versions of +? Overloaded operators are not fundamentally different from overloaded functions. regards, tom lane I understand the value of this future. This basically means that one has to keep the function naming and argument types as simple as logically possible in order to avoid situations like I described in my previous example. (Sorry for bothering you with questions likes this. I am trying to understand PG) -- Regards, Gevik -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Improving the ngettext() patch
* Tom Lane t...@sss.pgh.pa.us [090604 10:22]: p...@thetdh.com writes: Russian plural forms for 100, 101, 102 etc. is different, as for 0, 1, 2. True. The rule IIRC is that except for 11-14 and for collective numerals, declination follows the last digit. Wow. So how does anyone represent that in the .po files? AFAICT the notation the gettext machinery provides isn't really powerful enough for this. Well, the C/english template one includes just the msgid, and msgid_plural string. When the russian translators get to it, they make a russion .po which contains (something like) the following in the msgid header: Plural-Forms: nplurals=3; plural=n%10==1 n%100!=11 ? 0 : n%10=2 n%10=4 (n%10010 || n%100=20) ? 1 : 2;\n And then they provide msgstr[0], msgstr[1], and msgstr[2] to fill the 3 slots that above plural-forms can use when translationg plural-form strings. It's all encapsulated in the gettext tools and libraries, and the C (non-translated) base just always uses ngetttext(single, plural, n), and ngettext will (if the compiled catalog has different plural-forms) use whatever the catalog specifies, or fall back to the simple n == 1 ? singular : plural type choice when no translated catalog is available. a. -- Aidan Van Dyk Create like a god, ai...@highrise.ca command like a king, http://www.highrise.ca/ work like a slave. signature.asc Description: Digital signature
Re: [HACKERS] Improving the ngettext() patch
Aidan Van Dyk ai...@highrise.ca writes: When the russian translators get to it, they make a russion .po which contains (something like) the following in the msgid header: Plural-Forms: nplurals=3; plural=n%10==1 n%100!=11 ? 0 : n%10=2 n%10=4 (n%10010 || n%100=20) ? 1 : 2;\n Oh, I see. I didn't realize there was a mapping mechanism available to the translator. Okay, so the bottom line there is that there is some value in pluralizing the messages about FUNC_MAX_ARGS --- I withdraw the suggestion to undo that. Anyone wish to defend the ones that are ifdef'd out? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I have cleaned up the patch from yesterday slightly, too. I'm wondering how far back we should backpatch all this. This API was actually documented in Perl (just) before 7.4 was released, so in theory the API adjustment should go all the way back. However, in practice it hasn't been a problem until now, and we aren't supporting Windows releases before 8.2, so I think 8.2 is probably the right spot. My inclination is backpatch all the way. Now that we know the problem is we are failing to adhere to Perl's published API, I bet we could find some other problems it causes, if we poked into the right nooks and crannies. And not just on Windows. Yeah, the debian bug specifically mentions hppa. I'll go with your instinct. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PANIC: corrupted item lengths
On Thu, 2009-06-04 at 10:02 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: What seems strange about the various errors generated in bufpage.c is that they are marked as ERRORs, yet are executed within a critical section causing the system to PANIC. The reason we PANIC there is to reduce the probability that bad data will be written back to disk. Of course, if the bad data was read off disk in the first place, there's no hope --- but we have checks on incoming pages for that. We don't have checks for this on incoming pages: We only ever check the header. I think there is hope if we do have corruption. We don't need to PANIC, we can do what I've suggested instead. What seems significantly more likely if we detect a problem here is that we somehow corrupted the page while it sits in shared buffers. So, there's some hope that the corruption will not get back to disk, so long as we PANIC and thereby cause shared-memory contents to be flushed. If the block is marked as dirty, then yes, I can see your point. I would prefer to PANIC than to lose data. If the block is *not* dirty, i.e. it has been trashed in some other way, then it is not likely to go to disk. Anybody re-reading the block will see the same corruption and die long before they can make the page dirty. So a corrupt, yet clean block is no reason to PANIC. If the block *is* dirty it might only be because of a hint bit change and there are various other ways to dirty a block that don't trigger full page validity checks. Votes? +1 for no change. We could make the page-read-time validation checks stricter, if there's some specific pattern you're seeing that gets past those checks now. Don't know what the pattern is because the bloody things keep PANICing. Seriously, it does look like some kind of memory corruption issue, but still not sure if hardware or software related. But this thread is not about that issue, its about how we respond in the face of such issues. Main problem is no easy way to get rid of the corrupt block. You have to select out good data somehow then truncate. I like Alvaro's suggestion for the future, but mostly we need to be able to surgically remove the data block. Yes, I can do this without backend changes via normal levels of non-user level skullduggery. It would be good to have a check_page_validity option so that before we do anything to a page we do full validation, especially for example on PageAddItem(). With that turned on, I wouldn't mind PANICing, because at least you'd have reasonable proof that the corruption has happened recently and that PANICing may actually prevent the horse from escaping. Thanks, -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] User-facing aspects of serializable transactions
Bruce Momjian br...@momjian.us wrote: Added to TODO: Consider improving serialized transaction behavior to avoid anomalies * http://archives.postgresql.org/pgsql-hackers/2009-05/msg01136.php * http://archives.postgresql.org/pgsql-hackers/2009-06/msg00035.php It might be worth adding this reference, too, since it gets down to some possible implementation techniques: http://archives.postgresql.org/pgsql-hackers/2009-05/msg00217.php I was going to try to scare up some resources to advance this if we could get to some consensus. I don't get the feeling we're there yet. Suggestions welcome. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] postmaster recovery and automatic restart suppression
Hi, in case of a serious failure of a backend or an auxiliary process the postmaster performs a crash recovery and restarts the db automatically. Is there a possibility to deactivate the restart and to force the postmaster to simply exit at the end ? The background is that we will have a watchdog process which will in this case perform a fast switchover to the standby side (in case of syncronous replication) or will restart the db by its own and in addition will perform some specific actions. Regards, Harald Kolb. Best regards / freundliche Grüße - Harald Kolb COO RTP PD SW RD Area B 1 DE Mch-M Building 5532 / Room 3045 Tel: +49 89 636 47606 mailto:harald.k...@nsn.com http://www.nokiasiemensnetworks.com/global/ Nokia Siemens Networks GmbH Co. KG Sitz der Gesellschaft: München / Registered office: Munich Registergericht: München / Commercial registry: Munich, HRA 88537 WEEE-Reg.-Nr.: DE 52984304 Persönlich haftende Gesellschafterin / General Partner: Nokia Siemens Networks Management GmbH Geschäftsleitung / Board of Directors: Lydia Sommer, Olaf Horsthemke Vorsitzender des Aufsichtsrats / Chairman of supervisory board: Lauri Kivinen Sitz der Gesellschaft: München / Registered office: Munich Registergericht: München / Commercial registry: Munich, HRB 163416
Re: [HACKERS] User-facing aspects of serializable transactions
On Thu, Jun 4, 2009 at 11:32 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: I was going to try to scare up some resources to advance this if we could get to some consensus. I don't get the feeling we're there yet. Suggestions welcome. I think I might've said this before, but I think you need to do (or get someone with knowledge of the code to do) more looking at the lock bookkeeping that's required to make the SIREAD stuff work and try to figure out if it's even feasible for PostgreSQL and what the performance costs would be (an idea of how much code complexity this would introduce would be good too). A lot of the lack of consensus at this point looks to me more like lack of being sure whether this can actually work. I don't know that we're going to get any closer to consensus without some less-handwavy answer to that question. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
Magnus Hagander wrote:
Andrew Dunstan wrote:
Magnus Hagander wrote:
Dave Page wrote:
On Thu, Jun 4, 2009 at 12:40 AM, Andrew Dunstan and...@dunslane.net
wrote:
Of course, that leaves the issue of the library name to be fixed, but
sufficient unto the day ...
Yeah - I chatted briefly with Magnus about that a while back. I added
a quick hack to Mkvcbuild.pm to allow it to use either 5.10 or 5.8,
but didn't even think about the msys build.
Yeah, this is something we need to look at for 8.5. I think it's too
late in the cycle to start messing with it for 8.4.
I don't see why. Perl 5.10 is out there and people will want to use it.
I have a simple fix for this in GNUMakefile (see below) , and putting
something similar in the MSVC build stuff will be simple too. If we
don't want to support Perl 5.10 why was the item on the open issues list
at all (and why did I just spend hours finding a solution to the hard
part)?
I mostly meant a general make-it-version-independent fix. The fix that's
in the MSVC build system now is really just a hack that only supports
5.8 and 5.10, not any other versions.
But sure, if it's that easy, go for it :-)
Yeah. The patch below should be far more future-proof (and past-proof,
for that matter).
Dave, any chance you can test it?
cheers
andrew
Index: src/tools/msvc/Mkvcbuild.pm
===
RCS file: /cvsroot/pgsql/src/tools/msvc/Mkvcbuild.pm,v
retrieving revision 1.39
diff -c -r1.39 Mkvcbuild.pm
*** src/tools/msvc/Mkvcbuild.pm 7 Apr 2009 19:35:57 - 1.39
--- src/tools/msvc/Mkvcbuild.pm 4 Jun 2009 16:19:29 -
***
*** 104,116
}
}
$plperl-AddReference($postgres);
! if (-e $solution-{options}-{perl} . '\lib\CORE\perl510.lib')
{
! $plperl-AddLibrary($solution-{options}-{perl} .
'\lib\CORE\perl510.lib');
! }
! else
! {
! $plperl-AddLibrary($solution-{options}-{perl} .
'\lib\CORE\perl58.lib');
}
}
--- 104,114
}
}
$plperl-AddReference($postgres);
! my @perl_libs =
! glob($solution-{options}-{perl} .
'\lib\CORE\perl*.lib');
! if (@perl_libs == 1)
{
! $plperl-AddLibrary($perl_libs[0]);
}
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Initialise perl library as documented in perl API.
aduns...@postgresql.org (Andrew Dunstan) writes: Log Message: --- Initialise perl library as documented in perl API. Backpatch to release 7.4. Hmm, buildfarm says this doesn't work too well on Unixware :-( regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Synchronous replication: status of standby side
Where can I find a more recent version of syncrep patch. The last one that I've here is synch_rep_0428 and I'm getting segfault with it :(. In Wiki [1] the last is syncrep_0305 [1] http://wiki.postgresql.org/wiki/NTT%27s_Development_Projects []s -- Dickson S. Guedes mail/xmpp: gue...@guedesoft.net - skype: guediz http://guedesoft.net - http://www.postgresql.org.br http://www.rnp.br/keyserver/pks/lookup?search=0x8F3E3C06D428D10A signature.asc Description: Esta é uma parte de mensagem assinada digitalmente
Re: [HACKERS] User-facing aspects of serializable transactions
Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 4, 2009 at 11:32 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: I was going to try to scare up some resources to advance this if we could get to some consensus. I don't get the feeling we're there yet. Suggestions welcome. I think I might've said this before, but I think you need to do (or get someone with knowledge of the code to do) more looking at the lock bookkeeping that's required to make the SIREAD stuff work and try to figure out if it's even feasible for PostgreSQL and what the performance costs would be (an idea of how much code complexity this would introduce would be good too). A lot of the lack of consensus at this point looks to me more like lack of being sure whether this can actually work. I don't know that we're going to get any closer to consensus without some less-handwavy answer to that question. I'd feel a lot more comfortable about trying to go that route if there weren't heavy hitters insisting that no serialization failures without a reason that can be easily explained to users is a requirement. I don't believe it will ever work that way, so I see no point moving farther. Either that requirement needs to be removed, or someone who thinks it can be made to work that way will have to take up the cause if this is to go anywhere. I do agree, wholeheartedly, that if we get consensus on functional requirements, the next step would be to identify the specific changes required. In other words, I haven't forgotten your previous suggestion, and it seems like the *next* step if we can wrap *this* one. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Initialise perl library as documented in perl API.
Tom Lane wrote: aduns...@postgresql.org (Andrew Dunstan) writes: Log Message: --- Initialise perl library as documented in perl API. Backpatch to release 7.4. Hmm, buildfarm says this doesn't work too well on Unixware :-( That's what we have a buildfarm for ;-) There's a failure on FBSD too by the look of it. I'll dig some more to see what I can find. I did test on Windows and Linux. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4b2 tsearch2 strange error
Tatsuo Ishii is...@postgresql.org wrote: http://sylpheed.sraoss.jp/tmp/searcherdb84.tar.bz2 Using that on my kubuntu machine with no configure options except prefix: h...@kgrittn-desktop:~$ /usr/local/pgsql-8.4beta2/bin/psql -U hiro sylph psql (8.4beta2) Type help for help. sylph=# SELECT msg_sid, hdr_from, hdr_to, hdr_subject, msg_date, folder_id, msgnum FROM msginfo LEFT JOIN msg_folderinfo USING (msg_sid) WHERE plainto_tsquery(E'a') @@ body_index; NOTICE: text-search query contains only stop words or doesn't contain lexemes, ignored ERROR: unrecognized operator: -26 sylph=# SELECT msg_sid, hdr_from, hdr_to, hdr_subject, msg_date, folder_id, msgnum FROM msginfo LEFT JOIN msg_folderinfo USING (msg_sid) WHERE plainto_tsquery(E'a') @@ body_index; NOTICE: text-search query contains only stop words or doesn't contain lexemes, ignored ERROR: unrecognized operator: -26 sylph=# SELECT msg_sid, hdr_from, hdr_to, hdr_subject, msg_date, folder_id, msgnum FROM msginfo LEFT JOIN msg_folderinfo USING (msg_sid) WHERE plainto_tsquery(E'tsearch') @@ body_index; ERROR: tuple offset out of range: 609 sylph=# SELECT msg_sid, hdr_from, hdr_to, hdr_subject, msg_date, folder_id, msgnum FROM msginfo LEFT JOIN msg_folderinfo USING (msg_sid) WHERE plainto_tsquery(E'tsearch') @@ body_index; ERROR: tuple offset out of range: 609 Given that I can replicate this, is there some useful next step? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Initialise perl library as documented in perl API.
Andrew Dunstan and...@dunslane.net writes: That's what we have a buildfarm for ;-) There's a failure on FBSD too by the look of it. I'll dig some more to see what I can find. I see this when building HEAD on Fedora 10: plperl.c: In function 'plperl_init_interp': plperl.c:450: warning: null argument where non-null required (argument 3) Seems like a good hint ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4b2 tsearch2 strange error
I wrote: Tatsuo Ishii is...@postgresql.org wrote: http://sylpheed.sraoss.jp/tmp/searcherdb84.tar.bz2 Using that on my kubuntu machine with no configure options except prefix: [problem confirmed] Given that I can replicate this, is there some useful next step? By the way, that's 32 bit Kubuntu 7.10. Linux kgrittn-desktop 2.6.22-16-generic #1 SMP Thu Apr 2 01:27:50 GMT 2009 i686 GNU/Linux I've done a new build with --enable-debug and confirmed the problem in that environment. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PANIC: corrupted item lengths
Hi, Le 4 juin 09 à 15:55, Alvaro Herrera a écrit : I tend to hate automatic zeroing of pages because there's no way to get the contents later for forensics. I would support your proposal if we had a way of saving the block elsewhere before zeroing it (say create a directory corrupted+zeroed similar to lost+found in the database dir and save it annotated with the OID of the table and the block number). What about creating a special purpose fork for this? It could contain some metadata plus the original (maybe corrupted) block content. The main problem I see with this is that if you don't immediately act to examine the data, some of the pg_clog files that you need to be able to read these files may be gone. The necessary clogs maybe could be part of the special fork metadata associated with each saved apart corrupted blocks? Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Initialise perl library as documented in perl API.
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: That's what we have a buildfarm for ;-) There's a failure on FBSD too by the look of it. I'll dig some more to see what I can find. I see this when building HEAD on Fedora 10: plperl.c: In function 'plperl_init_interp': plperl.c:450: warning: null argument where non-null required (argument 3) Seems like a good hint ... Yeah. I didn't get that. But the odd thing is that on 5.8 especially it shouldn't matter. perl 5.8.8's perl.h has: #ifndef PERL_SYS_INIT3 # define PERL_SYS_INIT3(argvp,argcp,envp) PERL_SYS_INIT(argvp,argcp) #endif and the only place it's defined elsewhere that I can see is for OS2 (for anyone still running it!). These two Unixware machines have 5.8.8. and the FBSD machine has 5.8.7. So surely it can't be that, unless I'm missing something badly. The unixish.h file has this on 5.8: # define PERL_SYS_INIT(c,v) MALLOC_CHECK_TAINT2(*c,*v) PERL_FPU_INIT MALLOC_INIT I'm guessing the problem is actually somewhere in there. I could construct a dummy environment to pass to perl to quiet that warning, or I could even clone the environment - I'm mildly reluctant to pass the real environment to this, as the perlembed man page blandly tells us that this macro might mangle what is passed to it (even though we know on 5.8 it doesn't). cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4b2 tsearch2 strange error
Kevin Grittner kevin.gritt...@wicourts.gov writes: By the way, that's 32 bit Kubuntu 7.10. Linux kgrittn-desktop 2.6.22-16-generic #1 SMP Thu Apr 2 01:27:50 GMT 2009 i686 GNU/Linux I've done a new build with --enable-debug and confirmed the problem in that environment. Hmm ... I'll rev up my old 32-bit-Intel machine. I suspect that the missing link here is some state change in the database above and beyond just loading the data, but since we don't know exactly what that is, we'll have to resort to working with Tatsuo's bitwise dump. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] faster version of AllocSetFreeIndex for x86 architecture
On Tue, 2009-06-02 at 23:53 +0900, Atsushi Ogawa wrote: I made a faster version of AllocSetFreeIndex for x86 architecture. Results of benchmark script: Xeon(Core architecture), RedHat EL4, gcc 3.4.6 bytes : 4 8163264 128 256 512 1024 mix original: 0.780 0.780 0.820 0.870 0.930 0.970 1.030 1.080 1.130 0.950 patched : 0.380 0.170 0.170 0.170 0.170 0.180 0.170 0.180 0.180 0.280 The effect of the patch that I measured by oprofile is: - test program: pgbench -c 1 -t 5 (fsync=off) original: CPU: P4 / Xeon with 2 hyper-threads, speed 2793.55 MHz (estimated) Counted GLOBAL_POWER_EVENTS events with a unit mask of 0x01 (mandatory) count 10 samples %symbol name 66854 6.6725 AllocSetAlloc patched: CPU: P4 / Xeon with 2 hyper-threads, speed 2793.55 MHz (estimated) Counted GLOBAL_POWER_EVENTS events with a unit mask of 0x01 (mandatory) count 10 samples %symbol name 47610 4.9333 AllocSetAlloc I think this patch improves AllocSetAlloc/AllocSetFree performance. Looks like very good work. Much appreciated. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL Developer meeting minutes up
Markus Wanner wrote: The new branches getting merged up could work. That is, applying the fix to the oldest back-branch which requires the fix first and then merge it to all newer ones, including HEAD. However, that would require some rethinking: instead of creating bugfix-patches for HEAD, then manually adjust patches for back-branches and then group committing, you'd have to create a bugfix-patch for the oldest branch first, commit that and then merge that to the newer branches. That sounds a bit dangerous too, since I imagine there are some changes in the old release branches you wouldn't want merged into the newest releases (say, code affecting sections that got redesigned). Seems you'd want to do is create a new branch as close to the point where the bug was introduced - and then merge that forward into each of the branches. This concept was mentioned in a page linked earlier in the thread[1] and seems like the way monotone recommends people use their system[2]. See that page for more reasons why they think it's good. [1]http://archives.postgresql.org/pgsql-hackers/2009-06/msg00153.php [2]http://www.monotone.ca/wiki/DaggyFixes/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [pgsql-www] [HACKERS] pull raw text of a message by message-id
Bruce Momjian escribió: Alvaro Herrera wrote: Robert Haas escribi?: Oh, this is awesome. It would be handy to include a link on the HTML version of the page for when people are trying to pull down a patch that someone has foolishly included in the body of the email rather than attaching it. There already is one :-) Look at the headers -- after the Message-Id there is a link which says text/plain. I don't see a text/plain option on this page: http://archives.postgresql.org/pgsql-hackers/2009-05/msg00466.php Why? Argh. I think this whole thing died when I fixed the bug about message-id links in the message body being mangled reported by Tom, some days ago. The Message-id links are now being generated internally by Mhonarc instead of your script ... I don't think it's difficult to fix this problem but I don't have time to do it right now :-( (Maybe it's time I bit the bullet and stopped kludging the Mhonarc install all over the place and instead spent some time on the new archives code ...) -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4b2 tsearch2 strange error
I wrote: Hmm ... I'll rev up my old 32-bit-Intel machine. I suspect that the missing link here is some state change in the database above and beyond just loading the data, but since we don't know exactly what that is, we'll have to resort to working with Tatsuo's bitwise dump. I poked around in the dump for awhile. I still can't reproduce the failure from a standing start. It looks to me like Tatsuo's database was possibly produced from separate schema and data load steps, followed by some update operations. It would be nice to have a full script for reproducing the state of the database. I did find out some interesting stuff though. One thing that is particularly striking is that the GIN index is pretty bloated. If you load the data dump file as-is into an empty DB you get an index size of 2564 pages (on a 32-bit machine). The actual size of the index in Tatsuo's filesystem dump is 5459 pages! I tried to see if I could duplicate that by doing separate schema and data load (so the data is inserted into a pre-existing index). I got an index size of 5356 pages, close but still less than the actual. This is what makes me think there were some additional update operations (well, that and the sequence counter being larger than the number of rows in the table...) The bogus TIDs are coming from index pages that are clearly corrupt. For example, the 609 case comes from page 5439. I tweaked pg_filedump to know something about printing out GIN leaf pages, and got this: Block 5439 Header - Block Offset: 0x02a7e000 Offsets: Lower 24 (0x0018) Block: Size 8192 Version4Upper8184 (0x1ff8) LSN: logid 0 recoff 0x099cc544 Special 8184 (0x1ff8) Items:0 Free Space: 8160 TLI: 0x0001 Prune XID: 0x Flags: 0x () Length (including item array): 24 : 44c59c09 0100 1800f81f D... 0010: f81f0420 ... GIN Data Section: Right Bound: 0/0 Item 1: 1/3 Item 2: 64/5 Item 3: 67/4 Item 4: 90/3 Item 5: 100/5 Item 6: 106/3 Item 7: 106/5 Item 8: 114/4 Item 9: 189/5 Item 10: 204/6 Item 11: 300/7 Item 12: 302/5 Item 13: 302/6 Item 14: 309/5 Item 15: 355/2 Item 16: 355/4 Item 17: 407/3 Item 18: 472/4 Item 19: 480/6 Item 20: 483/1 Item 21: 486/1 Item 22: 486/3 Item 23: 499/5 Item 24: 560/6 Item 25: 584/6 Item 26: 588/3 Item 27: 589/3 Item 28: 660/6 Item 29: 667/4 Item 30: 718/2 Item 31: 719/4 Item 32: 738/4 Item 33: 760/6 Item 34: 763/2 Item 35: 764/2 Item 36: 784/6 Item 37: 844/7 Item 38: 912/4 Item 39: 913/3 Item 40: 913/5 Item 41: 916/5 Item 42: 930/5 Item 43: 945/1 Item 44: 945/4 Item 45: 945/5 Item 46: 973/7 Item 47: 994/5 Item 48: 1036/3 Item 49: 1046/6 Item 50: 1048/4 Item 51: 1048/6 Item 52: 1069/1 Item 53: 1192/4 Item 54: 1205/1 Item 55: 1205/3 Item 56: 1280/4 Item 57: 1317/7 Item 58: 1347/5 Item 59: 1363/1 Item 60: 1367/2 Item 61: 1389/5 Item 62: 1390/2 Item 63: 1393/2 Item 64: 1400/4 Item 65: 1417/5 Item 66: 1418/3 Item 67: 1513/3 Item 68: 1513/4 Item 69: 1513/5 Item 70: 1514/2 Item 71: 1614/4 Item 72: 1654/5 Item 73: 1666/4 Item 74: 1674/1 Item 75: 1690/6 Item 76: 1691/5 Item 77: 1697/5 Item 78: 1741/5 Item 79: 393216/609 Item 80: 327680/660 Item 81: 393216/819 Item 82: 327680/820 Special Section - GIN Index Section: Flags: 0x0003 (DATA|LEAF) Maxoff: 82 Blocks: RightLink (-1) 1ff8: 52000300R... so the last four entries (according to the maxoff) are wrong. Even more interesting, a binary dump of the page shows there is data beyond the maxoff point: Data array begins here, ItemPointerData apiece 0020: 0100 0300 4500 4300 @.c. 0030: 0400 5a000300 6400 0500 Z.d. 0040: 6a000300 6a00 0500 72000400 j.j.r... 0050: bd00 0500 cc000600 2c01 ..,. 0060: 0700 2e010500 2e01 0600 0070: 35010500 6301 0200 63010400 5.c.c... 0080: 9701 0300 d8010400 e001 0090: 0600 e3010100 e601 0100 00a0: e6010300 f301 0500 30020600 0... 00b0: 4802 0600 4c020300 4d02 ..H.L.M. 00c0: 0300 94020600 9b02 0400 00d0: ce020200 cf02 0400 e2020400 00e0: f802 0600 fb020200 fc02 00f0: 0200 10030600 4c03 0700 ..L. 0100: 90030400 9103 0300 91030500 0110: 9403 0500 a2030500 b103 0120: 0100 b1030400 b103 0500 0130: cd030700 e203 0500 0c040300
Re: [HACKERS] It's June 1; do you know where your release is?
One of the other open items is: contrib/intarray opclass definition needs updating tgl says: done, but there's another problem; see also bug #4806 The state of play on this is: * GIN cannot find items that generate no index entries, such as empty arrays or empty text documents. * This is the Wrong Thing for certain operators, notably array inclusion: x @ array[1] ought to find empty-array values of x. * Fixing this is quite nontrivial, although it seems doable (basically, empty values need to be entered with some sort of placeholder key). Behavior for, eg, arrays containing NULL elements also needs to be thought about. * However, it's been broken for awhile (since 8.2) and nobody complained until recently. So it seems acceptable to just put the issue on TODO and leave it unfixed for 8.4. * But ... the patch I applied to fix this issue http://archives.postgresql.org/message-id/17021.1234474...@sss.pgh.pa.us means that users of contrib/intarray might be subject to this inconsistency where they were not before. The contrib/intarray operators were defined to not succeed for empty arrays, which might be mathematically wrong but it made them consistent with the index behavior. Of course, they were also *in*consistent with the identically-named core array operators. It seems we have two reasonable possibilities for 8.4: revert that patch and keep contrib/intarray's behavior the same as it was, or leave the patch in and document that there was a behavioral change. Thoughts? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] User-facing aspects of serializable transactions
Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: Added to TODO: Consider improving serialized transaction behavior to avoid anomalies * http://archives.postgresql.org/pgsql-hackers/2009-05/msg01136.php * http://archives.postgresql.org/pgsql-hackers/2009-06/msg00035.php It might be worth adding this reference, too, since it gets down to some possible implementation techniques: http://archives.postgresql.org/pgsql-hackers/2009-05/msg00217.php Done. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC,PATCH] Avoid manual shift-and-test logic in AllocSetFreeIndex
Hi Atsushi,
If size = 8, fls((size - 1) ALLOC_MINBITS) is fls(0).
The result of fls(0) is undefined.
Yep, got caught out by this because my previous fls() supported zero.
I think we have to never call fls(0) from AllocSetFreeIndex().
My proposal code:
if (size (1 ALLOC_MINBITS))
{
idx = fls((size - 1) ALLOC_MINBITS);
Assert(idx ALLOCSET_NUM_FREELISTS);
}
Looks good, I'll send an updated patch.
Also, are you still seeing the same improvement with the __builtin_clz
as your inline asm implementation?
Cheers,
Jeremy
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
On Jun 4, 2009, at 3:58 PM, Tom Lane wrote: It seems we have two reasonable possibilities for 8.4: revert that patch and keep contrib/intarray's behavior the same as it was, or leave the patch in and document that there was a behavioral change. You can't keep the patch and update intarray to behave like it did before? Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] display previous query string of idle-in-transaction
Kevin Grittner wrote: Greg Stark st...@enterprisedb.com wrote: Should this patch be on the commitfest page for 8.5? Or is there a consensus already that it's a bad idea? Personally I actually think this makes a lot of sense to do. +1 It at least gives one a reasonable chance to get a useful clue +1 # And I'm going to register this patch on the next commitfest page. -- Tatsuhito Kasahara kasahara.tatsuh...@oss.ntt.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] display previous query string of idle-in-transaction
On Thu, Jun 4, 2009 at 9:54 PM, Tatsuhito Kasaharakasahara.tatsuh...@oss.ntt.co.jp wrote: Kevin Grittner wrote: Greg Stark st...@enterprisedb.com wrote: Should this patch be on the commitfest page for 8.5? Or is there a consensus already that it's a bad idea? Personally I actually think this makes a lot of sense to do. +1 It at least gives one a reasonable chance to get a useful clue +1 # And I'm going to register this patch on the next commitfest page. I don't actually remember seeing actual code to implement this - is this a patch, or just an idea? (I guess if this gets added to the CommitFest page there will be a pointer to the code, but somehow I can't find it at the moment.) The only thing I don't like about this is that I think it's kind of a hack to shove the IDLE in transaction designation and the query string into the same database column. I've never liked having to write: select sum(1) from pg_stat_activity where current_query = 'IDLE in transaction'; ...and I like it even less if I now have to modify that query to use like. We should find some way to represent this as structured data... maybe make a separate column called idle that's a boolean, or something, and let the query column contain the most recent query (whether or not it's still executing). ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH v2] Avoid manual shift-and-test logic in AllocSetFreeIndex
Move the shift-and-test login into a separate fls() function, which
can use __builtin_clz() if it's available.
This requires a new check for __builtin_clz in the configure script.
Results in a ~2% performance increase on PowerPC.
Signed-off-by: Jeremy Kerr j...@ozlabs.org
---
v2: prevent fls(0)
---
configure.in | 13 +
src/backend/utils/mmgr/aset.c | 34 +++---
2 files changed, 40 insertions(+), 7 deletions(-)
diff --git a/configure.in b/configure.in
index b8d2685..6a317b0 100644
--- a/configure.in
+++ b/configure.in
@@ -1361,6 +1361,19 @@ case $host_os in
AC_FUNC_FSEEKO;;
esac
+# GCC builtins
+#
+# We need AC_TRY_LINK here, as the prototype generated by AC_CHECK_FUNC
+# will cause gcc to try to reference a non-builtin symbol.
+
+AC_MSG_CHECKING([for __builtin_clz])
+AC_TRY_LINK([],
+ [__builtin_clz(0);],
+ [AC_DEFINE(HAVE_BUILTIN_CLZ, 1,
+ [Define to 1 if you have __builtin_clz().])
+ AC_MSG_RESULT(yes)],
+ [AC_MSG_RESULT(no)])
+
#
# Pthreads
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index 0e2d4d5..9eb3117 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -255,6 +255,31 @@ static MemoryContextMethods AllocSetMethods = {
#define AllocAllocInfo(_cxt, _chunk)
#endif
+/*
+ * fls: find last set bit.
+ *
+ * Returns the 1-based index of the most-significant bit in x. The MSB
+ * is bit number 32, the LSB is bit number 1. If x is zero, the result is
+ * undefined.
+ */
+static inline int
+fls(unsigned int x)
+{
+#ifdef HAVE_BUILTIN_CLZ
+ return 32 - __builtin_clz(x);
+#else
+ int ls = 0;
+
+ while (x != 0)
+ {
+ ls++;
+ x = 1;
+ }
+
+ return ls;
+#endif
+}
+
/* --
* AllocSetFreeIndex -
*
@@ -268,14 +293,9 @@ AllocSetFreeIndex(Size size)
{
int idx = 0;
- if (size 0)
+ if (size (1 ALLOC_MINBITS))
{
- size = (size - 1) ALLOC_MINBITS;
- while (size != 0)
- {
- idx++;
- size = 1;
- }
+ idx = fls((size - 1) ALLOC_MINBITS);
Assert(idx ALLOCSET_NUM_FREELISTS);
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC,PATCH] Avoid manual shift-and-test logic in AllocSetFreeIndex
Hi, Also, are you still seeing the same improvement with the __builtin_clz as your inline asm implementation? In my benchmark program, it is a little different performance in fls implementation and inline asm implementation. However, the result of a pgbench is almost the same improvement. Here is the result of my benchmark. Xeon(Core architecture) bytes : 4 8163264 128 256 512 1024 mix original : 0.780 0.790 0.820 0.870 0.930 0.980 1.040 1.080 1.140 0.910 inline asm: 0.320 0.180 0.190 0.180 0.190 0.180 0.190 0.180 0.190 0.170 fls : 0.270 0.260 0.290 0.290 0.290 0.290 0.290 0.300 0.290 0.380 Xeon(P4 architecrure) bytes : 4 8163264 128 256 512 1024 mix original : 0.520 0.520 0.670 0.780 0.950 1.000 1.060 1.190 1.250 0.940 inline asm: 0.610 0.530 0.530 0.520 0.520 0.540 0.540 0.580 0.540 0.600 fls : 0.390 0.370 0.780 0.780 0.780 0.790 0.780 0.780 0.780 0.520 pgbench result (measured by oprofile) CPU: Xeon(P4 architecrure) test program: pgbench -c 1 -t 5 (fsync=off) original samples %symbol name 66854 6.6725 AllocSetAlloc 11817 1.1794 AllocSetFree inline asm samples %symbol name 47610 4.9333 AllocSetAlloc 6248 0.6474 AllocSetFree fls samples %symbol name 48779 4.9954 AllocSetAlloc 7648 0.7832 AllocSetFree Best regards, --- Atsushi Ogawa -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC,PATCH] Avoid manual shift-and-test logic in AllocSetFreeIndex
Hi Atsushi, In my benchmark program, it is a little different performance in fls implementation and inline asm implementation. However, the result of a pgbench is almost the same improvement. Here is the result of my benchmark. Excellent, thank you for getting this extra set of numbers. Cheers, Jeremy -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] It's June 1; do you know where your release is?
David E. Wheeler da...@kineticode.com writes: On Jun 4, 2009, at 3:58 PM, Tom Lane wrote: It seems we have two reasonable possibilities for 8.4: revert that patch and keep contrib/intarray's behavior the same as it was, or leave the patch in and document that there was a behavioral change. You can't keep the patch and update intarray to behave like it did before? Uh, no. The point is that intarray's operators behave(d) differently from the core operators. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
