Re: [HACKERS] Summary: changes needed in function defaults behavior

2008-12-19 Thread Asko Oja
I would agree with making it stricter. It would force people to do less stupid things. Our main use case for default parameter will be getting rid of all the old versions of functions with shorter parameter lists by just creating new versions of old functions with additional default parameters.

[HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Heikki Linnakangas
Whenever a B-tree index scan fetches a heap tuple that turns out to be dead, the B-tree item is marked as killed by calling _bt_killitems. When the page gets full, all the killed items are removed by calling _bt_vacuum_one_page. That's a problem for hot standby. If any of the killed b-tree

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 09:43 +0900, Fujii Masao wrote: Yes, please check the call points for ForceSyncCommit. Do I think every xlog flush should be synchronous, no, I don't. That's why we have a user settable parameter for it. Umm.. I focus attention on XLogFlush() called except

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Heikki Linnakangas
Simon Riggs wrote: * corner case behaviour of backendids - bgwriter writes checkpoint WAL records. Has no backendid, but needs a slotid (possibly others) Why does bgwriter need a slotid? It doesn't run any transactions. * slotids are assigned once and never changed, so allowing them to be

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-19 Thread Heikki Linnakangas
Simon Riggs wrote: On a related but different point: We don't need an interlock between dirty buffers and WAL during recovery because the WAL has already been written. Assuming the WAL has also been fsync'd. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 11:04 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On a related but different point: We don't need an interlock between dirty buffers and WAL during recovery because the WAL has already been written. Assuming the WAL has also been fsync'd. True, so this

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 10:49 +0200, Heikki Linnakangas wrote: Whenever a B-tree index scan fetches a heap tuple that turns out to be dead, the B-tree item is marked as killed by calling _bt_killitems. When the page gets full, all the killed items are removed by calling _bt_vacuum_one_page.

Re: [HACKERS] possible bug in 8.4

2008-12-19 Thread Grzegorz Jaskiewicz
true, but as soon as I drop indices on both tables: root=# explain analyze select a.a from a where a not in (select a from b); QUERY PLAN

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Heikki Linnakangas
Simon Riggs wrote: We have infrastructure in place to make this work correctly, just need to add latestRemovedXid field to xl_btree_vacuum. So that part is easily solved. That's tricky because there's no xmin/xmax on index tuples. You could conservatively use OldestXmin as latestRemovedXid,

Re: [HACKERS] Review: B-Tree emulation for GIN

2008-12-19 Thread Teodor Sigaev
Updated patch. Ibrar Ahmed wrote: Hi Teodor Sigaev, I am getting server crash in contrib regression. May be I am doing something wrong here. Regression diff is attached. BTW these queries work fine outside the regression. -- Ibrar Ahmed EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] possible bug in 8.4

2008-12-19 Thread Grzegorz Jaskiewicz
and the plan on 8.4: gj=# explain select a.a from a where a not in (select a from b); QUERY PLAN - Seq Scan on a (cost=99035.00..257874197565.00 rows=300 width=4) Filter: (NOT

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 12:24 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: We have infrastructure in place to make this work correctly, just need to add latestRemovedXid field to xl_btree_vacuum. So that part is easily solved. That's tricky because there's no xmin/xmax on index

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 10:59 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: * corner case behaviour of backendids - bgwriter writes checkpoint WAL records. Has no backendid, but needs a slotid (possibly others) Why does bgwriter need a slotid? It doesn't run any transactions. *

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Heikki Linnakangas
Simon Riggs wrote: On Fri, 2008-12-19 at 10:59 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: * corner case behaviour of backendids - bgwriter writes checkpoint WAL records. Has no backendid, but needs a slotid (possibly others) Why does bgwriter need a slotid? It doesn't run any

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Well, to be honest, I don't much like the whole notion of tracking the slots. I think we should just rely on the XLOG_RECOVERY_END records to purge stale PGPROC entries, belonging to backends that died without writing an abort record. Sorry, I meant

Re: [HACKERS] Review: B-Tree emulation for GIN

2008-12-19 Thread Ibrar Ahmed
Thanks, On Fri, Dec 19, 2008 at 3:26 PM, Teodor Sigaev teo...@sigaev.ru wrote: Updated patch. Ibrar Ahmed wrote: Hi Teodor Sigaev, I am getting server crash in contrib regression. May be I am doing something wrong here. Regression diff is attached. BTW these queries work fine outside

Re: [HACKERS] possible bug in 8.4

2008-12-19 Thread Tom Lane
Grzegorz Jaskiewicz g...@pointblue.com.pl writes: Filter: (NOT (hashed subplan)) ^^ If 8.3 does that, and 8.4 doesn't, it's most likely because you are using different work_mem settings. regards, tom lane -- Sent via pgsql-hackers

Re: [HACKERS] Latest version of Hot Standby patch

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 14:00 +0200, Heikki Linnakangas wrote: Heikki Linnakangas wrote: Well, to be honest, I don't much like the whole notion of tracking the slots. I think we should just rely on the XLOG_RECOVERY_END records to purge stale PGPROC entries, belonging to backends that died

Re: [HACKERS] Simple postgresql.conf wizard

2008-12-19 Thread Tom Lane
Mark Wong mark...@gmail.com writes: On Mon, Dec 8, 2008 at 4:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Mark Wong mark...@gmail.com writes: On Tue, Dec 2, 2008 at 2:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: Are any of the queries complicated enough to trigger GEQO planning? Sorry for the

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Tom Lane
KaiGai Kohei kai...@ak.jp.nec.com writes: Tom Lane wrote: This seems like a pretty bad idea that will eventually bite you in an uncomfortable place. Lying about what datatype a field is is just not safe. Is it also correct for system attributes? I don't think the format on storage has to

Re: [HACKERS] stat() vs cygwin

2008-12-19 Thread Andrew Dunstan
Bruce Momjian wrote: Andrew Dunstan wrote: Alvaro Herrera wrote: Andrew Dunstan wrote: I'm confused. There is a Cygwin member of buildfarm, working quite happily. Can you point me to the exact patch in question, please? I thought we resolved the matter of stat() ages

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Greg Stark
I'm confused shouldn't read-only transactions on the slave just be hacked to not set any hint bits including lp_delete? -- Greg On 19 Dec 2008, at 03:49, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Whenever a B-tree index scan fetches a heap tuple that turns out to be

Re: [HACKERS] possible bug in 8.4

2008-12-19 Thread Grzegorz Jaskiewicz
On 2008-12-19, at 13:07, Tom Lane wrote: Grzegorz Jaskiewicz g...@pointblue.com.pl writes: Filter: (NOT (hashed subplan)) ^^ If 8.3 does that, and 8.4 doesn't, it's most likely because you are using different work_mem settings. you're right, as always :) My point

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
Tom Lane wrote: KaiGai Kohei kai...@ak.jp.nec.com writes: Tom Lane wrote: This seems like a pretty bad idea that will eventually bite you in an uncomfortable place. Lying about what datatype a field is is just not safe. Is it also correct for system attributes? I don't think the format

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 09:22 -0500, Greg Stark wrote: I'm confused shouldn't read-only transactions on the slave just be hacked to not set any hint bits including lp_delete? They could be, though I see no value in doing so. But that is not Heikki's point. He is discussing what happens on

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Gregory Stark
KaiGai Kohei kai...@kaigai.gr.jp writes: I didn't replace the previous implementation blindly, I have a few reasons that the current one is better than previous one. For example, if an input handler has side-effects, what is happen in the following query? SELECT

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
Gregory Stark wrote: KaiGai Kohei kai...@kaigai.gr.jp writes: I didn't replace the previous implementation blindly, I have a few reasons that the current one is better than previous one. For example, if an input handler has side-effects, what is happen in the following query? SELECT

Re: [HACKERS] SQL/MED compatible connection manager

2008-12-19 Thread Peter Eisentraut
I have committed this without the functions. I have some thoughts on what to do about that, but right now I'm exhausted. ;-) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Bruce Momjian
KaiGai Kohei wrote: Gregory Stark wrote: KaiGai Kohei kai...@kaigai.gr.jp writes: I didn't replace the previous implementation blindly, I have a few reasons that the current one is better than previous one. For example, if an input handler has side-effects, what is happen in the

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: If I was going to add anything to the btree page header, it would be latestRemovedLSN, only set during recovery. That way we don't have to explicitly kill queries, we can do the a wait on OldestXmin then let them ERROR out when they find a page that

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 11:54 -0600, Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: If I was going to add anything to the btree page header, it would be latestRemovedLSN, only set during recovery. That way we don't have to explicitly kill queries, we can do the a wait

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 10:52 +, Simon Riggs wrote: You could conservatively use OldestXmin as latestRemovedXid, but that could stall the WAL redo a lot more than necessary. Or you could store latestRemovedXid in the page header, but that would need to be WAL-logged to ensure

Re: [HACKERS] SQL/MED compatible connection manager

2008-12-19 Thread Martin Pihlak
Peter Eisentraut wrote: I have committed this without the functions. I have some thoughts on what to do about that, but right now I'm exhausted. ;-) Great news :) Thanks a lot for your support and contributions! regards, Martin -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: I understand the need, but we won't be using SQLSTATE = 40001. That corresponds to ERRCODE_T_R_SERIALIZATION_FAILURE, which that error would not be. Isn't it a problem with serialization of database transactions? You hit it in a different way,

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Gregory Stark
Simon Riggs si...@2ndquadrant.com writes: The error message ought to be snapshot too old, which could raise a chuckle, so I called it something else. The point you raise is a good one and I think we should publish a list of retryable error messages. I contemplated once proposing a special

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 18:59 +, Gregory Stark wrote: Simon Riggs si...@2ndquadrant.com writes: The error message ought to be snapshot too old, which could raise a chuckle, so I called it something else. The point you raise is a good one and I think we should publish a list of

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: max_standby_delay is set in recovery.conf, value 0 (forever) - 2,000,000 secs, settable in milliseconds. So think of it like a deadlock detector for recovery apply. Aha! A deadlock is a type of serialization failure. (In fact, on databases with

Re: [HACKERS] Windowing Function Patch Review - Standard Conformance

2008-12-19 Thread Tom Lane
Hitoshi Harada umi.tan...@gmail.com writes: Here's the patch, including latest function default args. Regards, The comments added to planner.c contain various claims that setrefs.c will fix up window function references, but I see no code related to that in setrefs.c. Please explain.

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 13:47 -0600, Kevin Grittner wrote: Simon Riggs si...@2ndquadrant.com wrote: max_standby_delay is set in recovery.conf, value 0 (forever) - 2,000,000 secs, settable in milliseconds. So think of it like a deadlock detector for recovery apply. Aha! A deadlock

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Gregory Stark
Simon Riggs si...@2ndquadrant.com writes: Increasing the waiting time increases the failover time and thus decreases the value of the standby as an HA system. Others value high availability higher than you and so we had agreed to provide an option to allow the max waiting time to be set.

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Gregory Stark
Kevin Grittner kevin.gritt...@wicourts.gov writes: Simon Riggs si...@2ndquadrant.com wrote: max_standby_delay is set in recovery.conf, value 0 (forever) - 2,000,000 secs, settable in milliseconds. So think of it like a deadlock detector for recovery apply. Aha! A deadlock is a type

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: The SQL Standard specifically names this error as thrown when it detects the inability to guarantee the serializability of two or more concurrent SQL-transactions. Now that really should only apply when running with SERIALIZABLE transactions, I

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Gregory Stark
Kevin Grittner kevin.gritt...@wicourts.gov writes: PostgreSQL is much less prone to serialization failures, but it is certainly understandable if hot standby replication introduces new cases of it. In this case it will be possible to get this error even if you're just running a single SELECT

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
Gregory Stark st...@enterprisedb.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: PostgreSQL is much less prone to serialization failures, but it is certainly understandable if hot standby replication introduces new cases of it. In this case it will be possible to get this

Re: [HACKERS] Windowing Function Patch Review - Standard Conformance

2008-12-19 Thread Tom Lane
I've been studying the grammar for the windowing patch a bit. It seems to me that the existing window name option for window specification got left out. I think that WindowDef needs to have two name fields, one for the name (if any) defined by the window definition, and one for the referenced

Re: [HACKERS] DTrace probes patch

2008-12-19 Thread Robert Lor
Tom Lane wrote: Fujii Masao masao.fu...@gmail.com writes: I understood your intention. But, I think that its function name is somewhat confusing. I agree. If the probe is meant to track only *some* WAL writes then it needs to be named something less generic than

[HACKERS] generic reloptions improvement

2008-12-19 Thread Alvaro Herrera
Hi, Here's a patch for improving the general reloptions mechanism. What this patch does is add a table-based option parser. This allows adding new options very easily, and stops the business of having to pass the minimum and default fillfactor each time you want the reloptions processed. (This

Re: [HACKERS] DTrace probes patch

2008-12-19 Thread Tom Lane
Robert Lor robert@sun.com writes: Tom Lane wrote: I agree. If the probe is meant to track only *some* WAL writes then it needs to be named something less generic than TRACE_POSTGRESQL_WAL_BUFFER_WRITE. How about change it to TRACE_POSTGRESQL_WAL_BUFFER_WRITE_DIRTY similar to

[HACKERS] SQL standard definition shift for SERIALIZABLE

2008-12-19 Thread Kevin Grittner
I happened across this page: http://en.wikipedia.org/wiki/Isolation_(database_systems) I was all set to be righteously indignant over the slander of PostgreSQL, but figured I should check the SQL standard to be sure of my ground. The only version of the standard I have available right now is

Re: [HACKERS] SQL standard definition shift for SERIALIZABLE

2008-12-19 Thread Kevin Grittner
Kevin Grittner kevin.gritt...@wicourts.gov wrote: Is an update in order? Never mind, it appears to have been covered already. Sorry for the noise. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Kevin Grittner
Gregory Stark st...@enterprisedb.com wrote: I think the fundamental difference is that a deadlock or serialization failure can be predicted as a potential problem when writing the code. This is something that can happen for any query any time, even plain old read-only select queries.

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
but the auto-creation of pg_security rows does make this problematic. regproc works because it only supports pre-created pg_proc rows. Yes, the security label has a different characteristic at the point. Can we support two data types, one for read-only and another for possible creation? That

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
Here is one another idea. How do you think asymmetric input/output handlers? At first, I have to introduce the current implementation. It can receives user input values as TEXT or ACLITEM[], ant these are fetched at fetchWritableSystemAttribute() on execMain.c. Please note that they have no

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Alvaro Herrera
KaiGai Kohei wrote: Can we support two data types, one for read-only and another for possible creation? That is not going to work for a query like SELECT * FROM x WHERE col != 'valid_but_new_security_label'::regseclabel; because the cast would fail if the pg_security row doesn't

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Robert Treat
On Friday 19 December 2008 05:52:42 Simon Riggs wrote: BTW, I noticed the other day that Oracle 11g only allows you to have a read only slave *or* allows you to continue replaying. You need to manually switch back and forth between those modes. They can't do *both*, as Postgres will be able to

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 20:54 +, Gregory Stark wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: PostgreSQL is much less prone to serialization failures, but it is certainly understandable if hot standby replication introduces new cases of it. In this case it will be

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Simon Riggs
On Fri, 2008-12-19 at 19:29 -0500, Robert Treat wrote: On Friday 19 December 2008 05:52:42 Simon Riggs wrote: BTW, I noticed the other day that Oracle 11g only allows you to have a read only slave *or* allows you to continue replaying. You need to manually switch back and forth between

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread Bruce Momjian
KaiGai Kohei wrote: but the auto-creation of pg_security rows does make this problematic. regproc works because it only supports pre-created pg_proc rows. Yes, the security label has a different characteristic at the point. Can we support two data types, one for read-only and another

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1324)

2008-12-19 Thread KaiGai Kohei
Bruce Momjian wrote: KaiGai Kohei wrote: but the auto-creation of pg_security rows does make this problematic. regproc works because it only supports pre-created pg_proc rows. Yes, the security label has a different characteristic at the point. Can we support two data types, one for

[HACKERS] Is Window really safe as a global typedef name?

2008-12-19 Thread Tom Lane
I'm looking at the window-functions patch and wondering just what kind of trouble we'll get into if we leave its new plan node type named just Window. I've already confirmed that this is a direct conflict against a typedef in X11/X.h, and I'd be not the least bit surprised if it's used in

Re: [HACKERS] generic reloptions improvement

2008-12-19 Thread Alvaro Herrera
A small correction to this patch: this is needed because otherwise the autovac code to parse the option becomes all tangled; this avoids having to invent special values for use the default value, and it also avoid having the default value stored elsewhere in the code than in the reloptions table.

Re: [HACKERS] [COMMITTERS] pgsql: SQL/MED catalog manipulation facilities This doesn't do any

2008-12-19 Thread Tom Lane
pet...@postgresql.org (Peter Eisentraut) writes: Log Message: --- SQL/MED catalog manipulation facilities The buildfarm says this patch has got serious portability issues. grebe, dugong, and ermine are all crashing here: --- 109,115 (3 rows) ALTER FOREIGN DATA WRAPPER foo

Re: [HACKERS] Is Window really safe as a global typedef name?

2008-12-19 Thread David Fetter
On Fri, Dec 19, 2008 at 10:36:30PM -0500, Tom Lane wrote: I'm looking at the window-functions patch and wondering just what kind of trouble we'll get into if we leave its new plan node type named just Window. I've already confirmed that this is a direct conflict against a typedef in X11/X.h,

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Heikki Linnakangas
Gregory Stark wrote: Simon Riggs si...@2ndquadrant.com writes: Increasing the waiting time increases the failover time and thus decreases the value of the standby as an HA system. Others value high availability higher than you and so we had agreed to provide an option to allow the max waiting

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-19 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Gregory Stark wrote: The question I had was whether your solution for btree pointers marked dead and later dropped from the index works when the user hasn't configured a timeout and doesn't want standby queries killed. Yes, it's not any different from vacuum WAL