Re: [HACKERS] Transaction-controlled robustness for replication

2008-07-23 Thread Markus Wanner
Hi, Jens-Wolfhard Schicke wrote: * Does WAL get forced to disk on primary at commit time? * Does WAL get forced across link to standby at commit time? * Does WAL get forced to disk on standby at commit time? * Does WAL get applied [and synced] to disk on standby at commit time? I think

Re: [HACKERS] [PATCH]-hash index improving

2008-07-23 Thread Simon Riggs
On Wed, 2008-07-23 at 10:57 +0800, Xiao Meng wrote: Well, I'll do it after I finish my second patch. Hash index should be more efficient than btree when N is big enough. It seems meaningful to find how big N is in an experiment way. Agreed. We should also examine the basic thinking of the

Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Simon Riggs
On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: 1. Issues with applying the patch to CVS HEAD: Sounds awful. Thanks for the review, will fix. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list

[HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Markus Wanner
Hi, As you certainly know by now, Postgres-R introduces an additional manager process. That one is forked from the postmaster, so are all backends, no matter if they are processing local or remote transactions. That led to a communication problem, which has originally (i.e. around Postgres-R for

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Alexey Klyukin
Markus Wanner wrote: Besides the communication between the replication manager and the backends, which is currently done by using these imessages, the replication manager also needs to communicate with the postmaster: it needs to be able to request new helper backends and it wants to be

Re: [HACKERS] [patch] plproxy v2

2008-07-23 Thread Marko Kreen
On 7/22/08, Marko Kreen [EMAIL PROTECTED] wrote: On 7/22/08, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: Also, plroxy does _nothing_ with passwords. That means the password for remote connection must be in postgres user's .pgpass,

Re: [HACKERS] [PATCH] \ef function in psql

2008-07-23 Thread Abhijit Menon-Sen
At 2008-07-17 18:28:19 -0400, [EMAIL PROTECTED] wrote: It wouldn't take a whole lot to convince me that a pg_get_functiondef would be useful, although I don't foresee either of those applications wanting to use it because of their backward-compatibility constraints. If the code lives in psql

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Markus Wanner
Hi Alexey, thanks for your feedback, these are interesting points. Alexey Klyukin wrote: In Replicator we avoided the need for postmaster to read/write backend's shmem data by using it as a signal forwarder. When a backend wants to inform a special process (i.e. queue monitor) about

Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Simon Riggs
On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: 1. Issues with applying the patch to CVS HEAD: For me, the patch applies cleanly to CVS HEAD. I do notice that there are two files standby.sgml and pgstandby.sgml. I can't see where standby.sgml comes from, but I haven't created it;

Re: [HACKERS] [GENERAL] Fragments in tsearch2 headline

2008-07-23 Thread Teodor Sigaev
Let me know of any other changes that are needed. Looks like ready to commit, but documentation is needed. -- Teodor Sigaev E-mail: [EMAIL PROTECTED] WWW: http://www.sigaev.ru/ -- Sent via pgsql-hackers

Re: [HACKERS] [GENERAL] Fragments in tsearch2 headline

2008-07-23 Thread Oleg Bartunov
btw, is it intentional to have '' in headline ? =# select ts_headline('1 2 3 4 5 1 2 3 1','14'::tsquery,'MaxFragments=1'); ts_headline - ... b4/b 5 b1/b Oleg On Wed, 23 Jul 2008, Teodor Sigaev wrote: Let me know of any other changes that are needed.

Re: [HACKERS] [PATCHES] GIN improvements

2008-07-23 Thread Teodor Sigaev
once, for regular VACUUM I think you really have to call it within each bulkdelete operation. Exactly what I did in last patch. There's probably no point in optimizing it away in VACUUM FULL either, since surely it'll be fast to call index_cleanup when there's nothing in the pending list?

[HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tatsuo Ishii
Hi, Here is the lastest WITH RECURSIVE patches against 2007/07/17 CVS (CVS HEAD won't compile for me). This version includes regression tests and is almost ready for commit IMO. I pulled fresh CVS HEAD and it seems the problem is gone. Here is the lastest WITH RECURSIVE patches against

Re: [HACKERS] pltcl_*mod commands are broken on Solaris 10

2008-07-23 Thread Peter Eisentraut
Am Tuesday, 22. July 2008 schrieb Tom Lane: Zdenek Kotala [EMAIL PROTECTED] writes: Yeah, I looked deeply on our solaris build script and problem is with following configure setup: ./configure --prefix=/tmp/pg --with-tcl --with-tclconfig=/usr/sfw/lib It found tclconfig, but not tclsh

Re: [HACKERS] pltcl_*mod commands are broken on Solaris 10

2008-07-23 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: By the way, these programs start with package require Pgtcl but we don't provide that library. Should that bother us? Hmm. The scripts actually depend on both pltcl and Pgtcl, so just pushing them out to the Pgtcl package wouldn't really improve

Re: [HACKERS] Do we really want to migrate plproxy and citext into PG core distribution?

2008-07-23 Thread Peter Eisentraut
Am Wednesday, 23. July 2008 schrieb Tom Lane: As soon as a Fedora release happens, I'm constrained by compatibility issues as to what I can put into that branch. RHEL releases ten times more so. I gather that Debian, for instance, is even more paranoid than Red Hat about upstream version

Re: [HACKERS] [PATCH] \ef function in psql

2008-07-23 Thread Tom Lane
Abhijit Menon-Sen [EMAIL PROTECTED] writes: At 2008-07-17 18:28:19 -0400, [EMAIL PROTECTED] wrote: It wouldn't take a whole lot to convince me that a pg_get_functiondef would be useful, although I don't foresee either of those applications wanting to use it because of their

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tom Lane
Tatsuo Ishii [EMAIL PROTECTED] writes: Reviewers, please let me know if you find problems with the patches. If none, I would like to commit this weekend. Has this patch actually been reviewed yet? The only reports I've seen are from testing; nothing from anyone actually reading the code. I

Re: [HACKERS] [PATCH] \ef function in psql

2008-07-23 Thread Marko Kreen
On 7/23/08, Tom Lane [EMAIL PROTECTED] wrote: Abhijit Menon-Sen [EMAIL PROTECTED] writes: At 2008-07-17 18:28:19 -0400, [EMAIL PROTECTED] wrote: It wouldn't take a whole lot to convince me that a pg_get_functiondef would be useful, although I don't foresee either of those applications

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Tom Lane
Alexey Klyukin [EMAIL PROTECTED] writes: Markus Wanner wrote: I'm currently doing this with imessages as well, which violates the rule that the postmaster may not to touch shared memory. I didn't look into ripping that out, yet. I'm not sure it can be done with the existing signaling of

Re: [HACKERS] [PATCHES] GIN improvements

2008-07-23 Thread Tom Lane
Teodor Sigaev [EMAIL PROTECTED] writes: So, may be we just move insertcleanup call to ginbulkdelete/ginvacuumcleanup but leave aminsertcleanup field in pg_proc for a future. I'd be inclined not to add the extra AM call if we aren't going to use it now. There's no very good reason to think

Re: [HACKERS] pltcl_*mod commands are broken on Solaris 10

2008-07-23 Thread Zdenek Kotala
Tom Lane napsal(a): Peter Eisentraut [EMAIL PROTECTED] writes: By the way, these programs start with package require Pgtcl but we don't provide that library. Should that bother us? Hmm. The scripts actually depend on both pltcl and Pgtcl, so just pushing them out to the Pgtcl package

Re: [HACKERS] [PATCH] \ef function in psql

2008-07-23 Thread Tom Lane
Marko Kreen [EMAIL PROTECTED] writes: [ re pg_get_functiondef ] Please make it use full qualified names (schema.name) for both function name and result types. Current search_path juggling the pg_dump does is major PITA. Qualifying the function name seems like a good idea, but I'd advise

Re: [HACKERS] pltcl_*mod commands are broken on Solaris 10

2008-07-23 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes: Tom Lane napsal(a): Actually it looks like it's been a very long time since these scripts got any love anyway. There's no reason anymore to split modules into multiple rows (not since TOAST...) and they're not schema-safe either. Anybody feel like

Re: [HACKERS] Do we really want to migrate plproxy and citext into PG core distribution?

2008-07-23 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Do you want Tom to a) spend a month improving the optimizer b) get him to review already working code so we can package things Actually, if the alternative is having the pieces outside of core where Tom never sees them, I'd vote for (b), as

Re: [HACKERS] [PATCH]-hash index improving

2008-07-23 Thread Kenneth Marshall
On Tue, Jul 22, 2008 at 08:36:34PM -0700, Dann Corbit wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-hackers- [EMAIL PROTECTED] On Behalf Of Xiao Meng Sent: Tuesday, July 22, 2008 7:57 PM To: Simon Riggs Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS]

Re: [HACKERS] Review: DTrace probes (merged version)

2008-07-23 Thread Peter Eisentraut
This patch looked OK to me, but the commit fest comment says that it does not include comments from the reviewdemo.postgresql.org. But I don't find any comments there. The latest version of the patch there appears to be the same as here. Zdenek, could you clarify? -- Sent via pgsql-hackers

Re: [HACKERS] [PATCHES] GIN improvements

2008-07-23 Thread Tom Lane
I wrote: Yeah, I was going to complain about that next :-). Autovacuum isn't going to trigger as a result of INSERT operations; somehow we have to teach it what to do for GIN indexes. I remember we discussed this at PGCon but I don't think we decided exactly what to do... One simple idea is

Re: [HACKERS] [GENERAL] Fragments in tsearch2 headline

2008-07-23 Thread Sushant Sinha
I guess it is more readable to add cover separator at the end of a fragment than in the front. Let me know what you think and I can update it. I think the right place for cover separator is in the structure HeadlineParsedText just like startsel and stopsel. This will enable users to specify their

Re: [HACKERS] Do we really want to migrate plproxy and citext into PG core distribution?

2008-07-23 Thread Tom Lane
Greg Sabino Mullane [EMAIL PROTECTED] writes: Code outside of core, is, in reality, less reviewed, less likely to work well with recent PG versions, and more likely to cause problems. It's also less likely to be found by people, less likely to be used by people, and less likely to be included

Re: [HACKERS] pltcl_*mod commands are broken on Solaris 10

2008-07-23 Thread Peter Eisentraut
Am Tuesday, 22. July 2008 schrieb Zdenek Kotala: By main opinion main problem is in build process which does not fail and also dependency on tclsh is hidden by exec command. Fixed. Now, configure will fail if no tcl shell is found. You can specify one with the TCLSH variable. -- Sent via

Re: [HACKERS] Do we really want to migrate plproxy and citext into PG core distribution?

2008-07-23 Thread Matthew T. O'Connor
Tom Lane wrote: Greg Sabino Mullane [EMAIL PROTECTED] writes: Code outside of core, is, in reality, less reviewed, less likely to work well with recent PG versions, and more likely to cause problems. It's also less likely to be found by people, less likely to be used by people, and less likely

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Markus Wanner
Hi, Tom Lane wrote: You should also look at the current code for communication between autovac launcher and autovac workers. That seems to be largely a similar problem, and it's been solved in a way that seems to be safe enough with respect to the postmaster vs shared memory issue. Oh yeah,

Re: [HACKERS] Do we really want to migrate plproxy and citext into PG core distribution?

2008-07-23 Thread Marko Kreen
On 7/23/08, Greg Sabino Mullane [EMAIL PROTECTED] wrote: Do you want Tom to a) spend a month improving the optimizer b) get him to review already working code so we can package things Actually, if the alternative is having the pieces outside of core where Tom never sees them, I'd vote

Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Heikki Linnakangas
Simon Riggs wrote: On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: 8. Unresolved question of implementing now/later a cp replacement The patch implements what's been agreed. I'm not rewriting cp, for reasons already discussed. Not a comment to you Martin, but it's fairly clear that

Re: [HACKERS] Review: DTrace probes (merged version)

2008-07-23 Thread Zdenek Kotala
Peter Eisentraut napsal(a): This patch looked OK to me, but the commit fest comment says that it does not include comments from the reviewdemo.postgresql.org. But I don't find any comments there. The latest version of the patch there appears to be the same as here. Zdenek, could you

Re: [HACKERS] pltcl_*mod commands are broken on Solaris 10

2008-07-23 Thread Zdenek Kotala
Peter Eisentraut napsal(a): Am Tuesday, 22. July 2008 schrieb Zdenek Kotala: By main opinion main problem is in build process which does not fail and also dependency on tclsh is hidden by exec command. Fixed. Now, configure will fail if no tcl shell is found. You can specify one with the

Re: [HACKERS] Do we really want to migrate plproxy and citext into PG core distribution?

2008-07-23 Thread Dimitri Fontaine
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, As a potential user of the solution, I'd very much like to have plproxy into -core if possible and sensible. Knowing nothing about the sensible part, I'd vote for inclusion. But whether -core vote for or against inclusion, I'd appreciate

Re: [HACKERS] pltcl_*mod commands are broken on Solaris 10

2008-07-23 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala [EMAIL PROTECTED] writes: Tom Lane napsal(a): Actually it looks like it's been a very long time since these scripts got any love anyway. There's no reason anymore to split modules into multiple rows (not since TOAST...) and they're not schema-safe either.

Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Kevin Grittner
Heikki Linnakangas [EMAIL PROTECTED] wrote: We really need a more reliable way of detecting that a file has been fully copied. In our scripts we handle this by copying to a temp directory on the same mount point as the archive directory and doing a mv to the archive location when the

Re: [HACKERS] [PATCHES] GIN improvements

2008-07-23 Thread Tom Lane
Teodor Sigaev [EMAIL PROTECTED] writes: Updated: http://www.sigaev.ru/misc/fast_insert_gin-0.9.gz Here is the GIN fast-insert patch back again. Changes: * Sync with CVS HEAD * Clean up documentation and some of the code comments * Fix up custom reloptions code * Suppress some compiler warnings

[HACKERS] Research/Implementation of Nested Loop Join optimization

2008-07-23 Thread Manoel Henrique
Hi! I`m a researcher from PUC-Rio (Brazil) and we`re studying about an Joins, and we`d like to implement an optimization on the Nested Loop Join, this optimization consists on while scanning the inner table, the iteration would go from up-down then backwards(down-up) to take advantage of the

Re: [HACKERS] pltcl_*mod commands are broken on Solaris 10

2008-07-23 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes: I understand. However I have another dumb idea/question - It seems to me that it is client code. I think that it should be integrated into psql command. That doesn't seem like a particularly appropriate thing to do ... nor do I see the argument for

Re: [HACKERS] Research/Implementation of Nested Loop Join optimization

2008-07-23 Thread Dann Corbit
When you install the source tree (e.g. in folder \postgresql-8.3.x) you will want to examine nodeMergejoin.c typically found in a path similar to this: \postgresql-8.3.x\src\backend\executor\nodeMergejoin.c Here are the comments from the version on my machine: /* * INTERFACE ROUTINES

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Markus Wanner
Hi, what follows are some comments after trying to understand how the autovacuum launcher works and thoughts on how to apply this to the replication manager in Postgres-R. The initial comments in autovacuum.c say: If the fork() call fails in the postmaster, it sets a flag in the shared

Re: [HACKERS] [GENERAL] Fragments in tsearch2 headline

2008-07-23 Thread Oleg Bartunov
On Wed, 23 Jul 2008, Sushant Sinha wrote: I guess it is more readable to add cover separator at the end of a fragment than in the front. Let me know what you think and I can update it. FragmentsDelimiter should *separate* fragments and that says all. Not very difficult algorithmic problem,

Re: [HACKERS] [PATCHES] GIN improvements

2008-07-23 Thread Alvaro Herrera
Tom Lane wrote: Teodor Sigaev [EMAIL PROTECTED] writes: I didn't get much further than that because I got discouraged after looking at the locking issues around the pending-insertions list. It's a mess: These are rather severe problems. Maybe there's a better solution, but perhaps it would

Re: [HACKERS] Research/Implementation of Nested Loop Join optimization

2008-07-23 Thread Manoel Henrique
The nodeMergejoin.c is the code for the Merge Join isn`t it? I am trying to find a way to change the Nested Loop Join, It would be more like on nodeNestloop.c when rescanning the inner plan, (second time scanning the inner plan and so on) he`d change the scan direction, If the scan direction was

Re: [HACKERS] Postgres-R: internal messaging

2008-07-23 Thread Tom Lane
Markus Wanner [EMAIL PROTECTED] writes: ... crashes are more difficult. IMO the replication manager needs to stay alive during this reinitialization, to keep the GCS connection. However, it can easily detach from shared memory temporarily (the imessages stuff is the only shmem place it

Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Andrew Dunstan
Kevin Grittner wrote: Heikki Linnakangas [EMAIL PROTECTED] wrote: We really need a more reliable way of detecting that a file has been fully copied. In our scripts we handle this by copying to a temp directory on the same mount point as the archive directory

[HACKERS] PostgreSQL extensions packaging

2008-07-23 Thread Dimitri Fontaine
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I promised to have an in-depth look at the archives before to spend time on my ideas for $subject, but failed to do so. Here are the ideas (user level design if you will) :) As a PostgreSQL extension developer (see

Re: [HACKERS] [PATCHES] GIN improvements

2008-07-23 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: It's a mess: These are rather severe problems. Maybe there's a better solution, but perhaps it would be good enough to lock out concurrent access to the index while the bulkinsert procedure is working. Ugh... The idea I was toying

Re: [HACKERS] Research/Implementation of Nested Loop Join optimization

2008-07-23 Thread Dann Corbit
From: Manoel Henrique [mailto:[EMAIL PROTECTED] Sent: Wednesday, July 23, 2008 1:47 PM To: Dann Corbit Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Research/Implementation of Nested Loop Join optimization The nodeMergejoin.c is the code for the Merge Join isn`t it? I am trying

Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-23 Thread Simon Riggs
On Wed, 2008-07-23 at 21:38 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Tue, 2008-07-22 at 17:19 -0700, Martin Zaun wrote: 8. Unresolved question of implementing now/later a cp replacement The patch implements what's been agreed. I'm not rewriting cp, for reasons

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tatsuo Ishii
Tatsuo Ishii [EMAIL PROTECTED] writes: Reviewers, please let me know if you find problems with the patches. If none, I would like to commit this weekend. Has this patch actually been reviewed yet? The only reports I've seen are from testing; nothing from anyone actually reading the

Re: [HACKERS] PostgreSQL extensions packaging

2008-07-23 Thread Tom Dunstan
Hi! On Wed, Jul 23, 2008 at 5:08 PM, Dimitri Fontaine [EMAIL PROTECTED] wrote: I promised to have an in-depth look at the archives before to spend time on my ideas for $subject, but failed to do so. I guess that means you missed both the original discussion at

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread David Fetter
On Wed, Jul 23, 2008 at 10:59:20AM -0400, Tom Lane wrote: Tatsuo Ishii [EMAIL PROTECTED] writes: Reviewers, please let me know if you find problems with the patches. If none, I would like to commit this weekend. Has this patch actually been reviewed yet? The only reports I've seen are

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tatsuo Ishii
On Wed, Jul 23, 2008 at 10:59:20AM -0400, Tom Lane wrote: Tatsuo Ishii [EMAIL PROTECTED] writes: Reviewers, please let me know if you find problems with the patches. If none, I would like to commit this weekend. Has this patch actually been reviewed yet? The only reports I've seen

Re: [HACKERS] PostgreSQL extensions packaging

2008-07-23 Thread Tom Dunstan
Oops, sent with wrong from header... -- Forwarded message -- From: Tom Dunstan [EMAIL PROTECTED] To: Dimitri Fontaine [EMAIL PROTECTED] Date: Wed, 23 Jul 2008 19:40:30 -0400 Subject: Re: [HACKERS] PostgreSQL extensions packaging Hi! On Wed, Jul 23, 2008 at 5:08 PM, Dimitri

[HACKERS] issues/experience with building postgres on Windows

2008-07-23 Thread Martin Zaun
Hi, trying to build postgres on Windows, I ran into a number of problems not covered by the FAQs or the PostgreSQL 8.3.3 documentation: http://www.postgresql.org/docs/8.3/interactive/install-win32-full.html In short, it's not been an easy experience so far: - build errors with latest Visual

Re: [HACKERS] issues/experience with building postgres on Windows

2008-07-23 Thread Andrew Dunstan
Martin Zaun wrote: Hi, trying to build postgres on Windows, I ran into a number of problems not covered by the FAQs or the PostgreSQL 8.3.3 documentation: http://www.postgresql.org/docs/8.3/interactive/install-win32-full.html In short, it's not been an easy experience so far: - build

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tatsuo Ishii
On Wed, Jul 23, 2008 at 10:59:20AM -0400, Tom Lane wrote: Tatsuo Ishii [EMAIL PROTECTED] writes: Reviewers, please let me know if you find problems with the patches. If none, I would like to commit this weekend. Has this patch actually been reviewed yet? The only reports I've seen

Re: [HACKERS] Research/Implementation of Nested Loop Join optimization

2008-07-23 Thread Tom Lane
Manoel Henrique [EMAIL PROTECTED] writes: The nodeMergejoin.c is the code for the Merge Join isn`t it? I am trying to find a way to change the Nested Loop Join, It would be more like on nodeNestloop.c when rescanning the inner plan, (second time scanning the inner plan and so on) he`d change