Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:19 AM, Tom Lane wrote: > According to > http://archives.postgresql.org/pgsql-general/2012-04/msg00374.php > advisory locks now cause problems for prepared transactions, which > ought to ignore them.  It appears to me that this got broken by > commit 62c7bd31c8878dd45c9b9b2

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-03 Thread Jan Urbański
On 02/05/12 20:18, Peter Eisentraut wrote: This doesn't work anymore with Python 3: rv = plpy.execute(...) do_something(rv[0:1]) Apparently, they changed the C API for doing slicing, or rather made one of the two APIs for it silently do nothing. Details are difficult to find, but this email me

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:57 AM, Josh Berkus wrote: > Michael, > >> What is the use case for temporary tables on a hot standby server? >> >> Perhaps this is a noobie question, but it seems to me that a hot standby >> server's use by* applications* or *users* should be limited to transactions >> tha

Re: [HACKERS] Modeling consumed shmem sizes, and some thorns

2012-05-03 Thread Simon Riggs
On Wed, May 2, 2012 at 9:38 PM, Daniel Farina wrote: > Besides accuracy, there is a thornier problem here that has to do with > hot standby (although the use case is replication more generally) when > one has heterogeneously sized database resources. As-is, it is > required that locking-related s

Re: [HACKERS] Modeling consumed shmem sizes, and some thorns

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 2:23 AM, Simon Riggs wrote: > On Wed, May 2, 2012 at 9:38 PM, Daniel Farina wrote: > >> Besides accuracy, there is a thornier problem here that has to do with >> hot standby (although the use case is replication more generally) when >> one has heterogeneously sized database

Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:41 AM, Robert Haas wrote: > On Wed, May 2, 2012 at 7:21 PM, Tom Lane wrote: >> It is getting a bit late to be considering such changes for 9.2, but >> I'm willing to review and commit this if there's not anybody who feels >> strongly that it's too late.  Personally I thin

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Hannu Krosing
On Wed, 2012-05-02 at 14:32 -0500, Merlin Moncure wrote: > On Wed, May 2, 2012 at 12:06 PM, Peter Eisentraut wrote: > > On ons, 2012-05-02 at 13:40 +0200, Hannu Krosing wrote: > >> How hard would it be to add support for LIKE syntax, similar to table > >> def in field list declaration for generic

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Pavel Stehule
Hello > (1 row) > > This works the same indeed, just seems to be a hack, though a cool > one :) > > hannu=# insert into test > hannu-# SELECT * FROM populate_record(null::test, >                              '"id"=>"456", "data"=>"zzz"'); > INSERT 0 1 few years back I proposed "anytypename" type

[HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
I had a request from a customer asking if we could make a switch to specifically disable the "unexpected EOF" message that fills lots of peoples logs. Along the same way that we have a flag to turn off the "nonstandard use of string escapes" message that is another culprit (that's actually a much *

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:26 PM, Magnus Hagander wrote: > I had a request from a customer asking if we could make a switch to > specifically disable the "unexpected EOF" message that fills lots of > peoples logs. Along the same way that we have a flag to turn off the > "nonstandard use of string es

[HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Magnus Hagander
Is there a particular reason we don't have an ALTER DATABASE switch that controls the datallowconn, or is it just something "missed out"? --  Magnus Hagander  Me: http://www.hagander.net/  Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:31 PM, Simon Riggs wrote: > On Thu, May 3, 2012 at 1:26 PM, Magnus Hagander wrote: >> I had a request from a customer asking if we could make a switch to >> specifically disable the "unexpected EOF" message that fills lots of >> peoples logs. Along the same way that we ha

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Vik Reykja
On Thu, May 3, 2012 at 2:31 PM, Simon Riggs wrote: > > Would we consider adding such a switch (it should be easy enough to > > do), or do we want to push this off to the mythical "let's improve the > > logging subsystem" project that might eventually materialize if we're > > lucky? Meaning - woul

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:34 PM, Vik Reykja wrote: > On Thu, May 3, 2012 at 2:31 PM, Simon Riggs wrote: >> >> > Would we consider adding such a switch (it should be easy enough to >> > do), or do we want to push this off to the mythical "let's improve the >> > logging subsystem" project that might

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Kevin Grittner
Magnus Hagander wrote: > Also, AFAIK we don't *have* a "message type" at this point (one of > the things said mythical project wanted to look at), so the only > thing we could really filter on would be the whole text of the > message, no? We have SQLSTATE, but this seems to be one of those sit

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:31 PM, Magnus Hagander wrote: > Is there a particular reason we don't have an ALTER DATABASE switch > that controls the datallowconn, or is it just something "missed out"? I think it can be removed, or rather deprecated. datconnlimit can be set to 0 If we need to speci

Re: [HACKERS] Torn page hazard in ginRedoUpdateMetapage()

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 12:16 AM, Tom Lane wrote: > Having said all that, I wasn't really arguing that this was a guaranteed > safe thing for us to rely on; just pointing out that it's quite likely > that the issue hasn't been seen in the field because of this type of > consideration. Well, we do

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:48 PM, Simon Riggs wrote: > On Thu, May 3, 2012 at 1:31 PM, Magnus Hagander wrote: > >> Is there a particular reason we don't have an ALTER DATABASE switch >> that controls the datallowconn, or is it just something "missed out"? > > I think it can be removed, or rather de

[HACKERS] outdated comment in heapam.c

2012-05-03 Thread Andres Freund
Hi, It seems that when Heikki added the multi_insert code the following comment in htup.h wasn't updated: /* * We ran out of opcodes, so heapam.c now has a second RmgrId. These opcodes * are associated with RM_HEAP2_ID, but are not logically different from * the ones above associated with RM_

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 4:11 AM, Simon Riggs wrote: > which seems to me to be actually harder than just rewriting as derived > table and isn't an option on Microstrategy etc, hence my observation > that GTTs don't help HS much. What I would like to see, one day, is > for temp tables to work without

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 7:13 AM, Pavel Stehule wrote: > Hello > >> (1 row) >> >> This works the same indeed, just seems to be a hack, though a cool >> one :) Yeah -- the syntax isn't great, but IMO it's more generally usable than what you're proposing because it's a scalar returning function not a

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Pavel Stehule
2012/5/3 Merlin Moncure : > On Thu, May 3, 2012 at 7:13 AM, Pavel Stehule wrote: >> Hello >> >>> (1 row) >>> >>> This works the same indeed, just seems to be a hack, though a cool >>> one :) > > Yeah -- the syntax isn't great, but IMO it's more generally usable > than what you're proposing because

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Andrew Dunstan
On 05/03/2012 09:43 AM, Pavel Stehule wrote: 2012/5/3 Merlin Moncure: On Thu, May 3, 2012 at 7:13 AM, Pavel Stehule wrote: Hello (1 row) This works the same indeed, just seems to be a hack, though a cool one :) Yeah -- the syntax isn't great, but IMO it's more generally usable than what

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Tom Lane
Magnus Hagander writes: > Is there a particular reason we don't have an ALTER DATABASE switch > that controls the datallowconn, or is it just something "missed out"? It was never intended to be a user-accessible switch, just something to protect template0. I don't agree with Simon's proposal to

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
Simon Riggs writes: > On Thu, May 3, 2012 at 1:26 PM, Magnus Hagander wrote: >> I had a request from a customer asking if we could make a switch to >> specifically disable the "unexpected EOF" message that fills lots of >> peoples logs. > Yes, if the new parameter allows a generic filter on mult

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 9:01 AM, Andrew Dunstan wrote: > On 05/03/2012 09:43 AM, Pavel Stehule wrote: >> >> 2012/5/3 Merlin Moncure: >>> >>> On Thu, May 3, 2012 at 7:13 AM, Pavel Stehule >>>  wrote: Hello > (1 row) > > This works the same indeed, just seems to be a hack,

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 4:13 PM, Tom Lane wrote: > Magnus Hagander writes: >> Is there a particular reason we don't have an ALTER DATABASE switch >> that controls the datallowconn, or is it just something "missed out"? > > It was never intended to be a user-accessible switch, just something to > p

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Tom Lane
Magnus Hagander writes: > On Thu, May 3, 2012 at 4:13 PM, Tom Lane wrote: >> Magnus Hagander writes: >>> Is there a particular reason we don't have an ALTER DATABASE switch >>> that controls the datallowconn, or is it just something "missed out"? >> It was never intended to be a user-accessible

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 4:26 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Thu, May 3, 2012 at 4:13 PM, Tom Lane wrote: >>> Magnus Hagander writes: Is there a particular reason we don't have an ALTER DATABASE switch that controls the datallowconn, or is it just something "missed

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Tom Lane
Andrew Dunstan writes: > You guys seem to be taking the original proposal off into the weeds. I > have often wanted to be able to use LIKE in type expressions, and I'd > like to see exactly that implemented. This notion of "anytypename" is utterly unworkable anyway; there's no way for the parse

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Pavel Stehule
2012/5/3 Tom Lane : > Andrew Dunstan writes: >> You guys seem to be taking the original proposal off into the weeds. I >> have often wanted to be able to use LIKE in type expressions, and I'd >> like to see exactly that implemented. > > This notion of "anytypename" is utterly unworkable anyway; th

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Andrew Dunstan
On 05/03/2012 10:18 AM, Merlin Moncure wrote: On Thu, May 3, 2012 at 9:01 AM, Andrew Dunstan wrote: On 05/03/2012 09:43 AM, Pavel Stehule wrote: 2012/5/3 Merlin Moncure: On Thu, May 3, 2012 at 7:13 AM, Pavel Stehule wrote: Hello (1 row) This works the same indeed, just seems to be a h

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:46 PM, Kevin Grittner wrote: > Magnus Hagander  wrote: > >> Also, AFAIK we don't *have* a "message type" at this point (one of >> the things said mythical project wanted to look at), so the only >> thing we could really filter on would be the whole text of the >> message,

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 4:17 PM, Tom Lane wrote: > Simon Riggs writes: >> On Thu, May 3, 2012 at 1:26 PM, Magnus Hagander wrote: >>> I had a request from a customer asking if we could make a switch to >>> specifically disable the "unexpected EOF" message that fills lots of >>> peoples logs. > >>

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Tom Lane
Pavel Stehule writes: > 2012/5/3 Tom Lane : >> This notion of "anytypename" is utterly unworkable anyway; there's no >> way for the parser to know soon enough that a given argument position >> needs to be read as a type name rather than a normal expression. > type identifier is same identifier li

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
Magnus Hagander writes: > On Thu, May 3, 2012 at 4:17 PM, Tom Lane wrote: >> I agree with Simon --- a disable for that specific message seems like a >> kluge, and an ugly one at that. (The right solution for this customer >> is to fix their broken application.) But a generic filter capability >

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 9:44 AM, Andrew Dunstan wrote: > Why would you always need FROM? that was coming from Hannu's original example: insert into test2 select * from json_to_record(jrec json) as (like test2); how do you work it so you can call: select json_to_record(jrec json) as (like test2);

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 4:53 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Thu, May 3, 2012 at 4:17 PM, Tom Lane wrote: >>> I agree with Simon --- a disable for that specific message seems like a >>> kluge, and an ugly one at that.  (The right solution for this customer >>> is to fix their

Re: [HACKERS] remove dead ports?

2012-05-03 Thread Bruce Momjian
On Tue, May 01, 2012 at 04:39:32PM -0400, Bruce Momjian wrote: > On Tue, Apr 24, 2012 at 09:29:39PM +0300, Peter Eisentraut wrote: > > I propose that we remove support for the following OS ports from our > > source tree. They are totally dead, definitely don't work, and/or > > probably no one reme

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Pavel Stehule
2012/5/3 Tom Lane : > Pavel Stehule writes: >> 2012/5/3 Tom Lane : >>> This notion of "anytypename" is utterly unworkable anyway; there's no >>> way for the parser to know soon enough that a given argument position >>> needs to be read as a type name rather than a normal expression. > >> type iden

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
Magnus Hagander writes: > On Thu, May 3, 2012 at 4:53 PM, Tom Lane wrote: >> Magnus Hagander writes: >>> Are you thinking basically "regexp against the main text", or >>> something else, when you say "generic filter capacity"? >> In the context of yesterday's discussions, I wonder whether a fil

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Tom Lane
Pavel Stehule writes: > 2012/5/3 Tom Lane : >> No, it isn't, at least not if you have any ambition to support array >> types for instance; to say nothing of types whose standard names are >> keywords, multiple words, etc. > we can identify a position "anytypename" before raising error - it can >

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of jue may 03 10:58:12 -0400 2012: > On Thu, May 3, 2012 at 4:53 PM, Tom Lane wrote: > > In the context of yesterday's discussions, I wonder whether a filter by > > SQLSTATE would be appropriate. > > I'm worried it's not really granular enough. Yeah. > r

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 11:20 AM, Alvaro Herrera wrote: > Hey, maybe we could add a UUID to each ereport() call site ;-) I can't help but feel we're designing a $10.00 solution to a $0.25 problem. I think I'd actually support adding something like a UUID to every ereport and a filtering mechanism

Re: [HACKERS] Torn page hazard in ginRedoUpdateMetapage()

2012-05-03 Thread Tom Lane
Robert Haas writes: > Are you planning to commit Noah's patch? I wasn't intending to do so personally in the near future; I've got other things on my to-do list. I won't object if somebody else commits it though. regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
Robert Haas writes: > On Thu, May 3, 2012 at 11:20 AM, Alvaro Herrera > wrote: >> Hey, maybe we could add a UUID to each ereport() call site ;-) > I can't help but feel we're designing a $10.00 solution to a $0.25 > problem. I think I'd actually support adding something like a UUID to > every e

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 11:39 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, May 3, 2012 at 11:20 AM, Alvaro Herrera >> wrote: >>> Hey, maybe we could add a UUID to each ereport() call site ;-) > >> I can't help but feel we're designing a $10.00 solution to a $0.25 >> problem.  I think I'd

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Kevin Grittner
Alvaro Herrera wrote: > Excerpts from Magnus Hagander's message: >> Tom Lane wrote: >>> In the context of yesterday's discussions, I wonder whether a >>> filter by SQLSTATE would be appropriate. >> >> I'm worried it's not really granular enough. > > Yeah. Just to be sure we're not inventing a

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Simon Riggs writes: > On Thu, May 3, 2012 at 1:19 AM, Tom Lane wrote: >> If this patch weren't already in a released branch I would be arguing >> for reverting it.  As is, I think we're going to have to clean it up. >> I don't have time to look at it in detail right now, though. > There was an a

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 10:12 AM, Tom Lane wrote: >> Maybe some keyword can help to us. What do you think about new >> operator TYPE that can returns regtype value and can be used together >> with polymorphic functions. > > Doesn't have any more attraction for me than the proposed LIKE > extension;

Re: [HACKERS] remove dead ports?

2012-05-03 Thread Peter Eisentraut
On tor, 2012-05-03 at 10:59 -0400, Bruce Momjian wrote: > Having received no replies on "general" from bsdi users considering > upgrading to 9.2, I have removed the port. I think that was quite premature. There is no requirement that bsdi users need to read pgsql-general, especially if you give t

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 11:04 AM, Tom Lane wrote: > I'm inclined to think that a saner implementation would involve > splitting the userlock lockmethod into two, one transactional and one > not.  That gets rid of the when-to-release kluges, but instead we have > to think of a way for two different

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 5:04 PM, Tom Lane wrote: > I'm inclined to think that a saner implementation would involve > splitting the userlock lockmethod into two, one transactional and one > not. Agreed > That gets rid of the when-to-release kluges, but instead we have > to think of a way for two

Re: [HACKERS] remove dead ports?

2012-05-03 Thread Bruce Momjian
On Thu, May 03, 2012 at 07:11:47PM +0300, Peter Eisentraut wrote: > On tor, 2012-05-03 at 10:59 -0400, Bruce Momjian wrote: > > Having received no replies on "general" from bsdi users considering > > upgrading to 9.2, I have removed the port. > > I think that was quite premature. There is no requ

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Simon Riggs writes: > On Thu, May 3, 2012 at 5:04 PM, Tom Lane wrote: >> That gets rid of the when-to-release kluges, but instead we have >> to think of a way for two different lockmethods to share the same >> lock keyspace. If we don't split it then we definitely need to figure >> out someplace

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Merlin Moncure writes: > On Thu, May 3, 2012 at 11:04 AM, Tom Lane wrote: >> I'm inclined to think that a saner implementation would involve >> splitting the userlock lockmethod into two, one transactional and one >> not. > hm, would that be exposed through the pg_locks view? some users might >

[HACKERS] CLOG extension

2012-05-03 Thread Robert Haas
Currently, the following can happen: 1. A backend needs a new transaction, so it calls GetNewTransactionId(). It acquires XidGenLock and then calls ExtendCLOG(). 2. ExtendCLOG() decides that a new CLOG page is needed, so it acquires CLogControlLock and then calls ZeroCLOGPage(). 3. ZeroCLOGPage()

Re: [HACKERS] remove dead ports?

2012-05-03 Thread Peter Geoghegan
On 3 May 2012 17:21, Bruce Momjian wrote: > I think I was the only user left;  I have never heard from a BSD/OS user > in the past 5-7 years. I'm inclined to agree with Bruce. While it's not reasonable to assume that the lack of a BSD/OS user complaining on -general indicates that there are none,

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 11:46 AM, Kevin Grittner wrote: > Alvaro Herrera wrote: >> Excerpts from Magnus Hagander's message: >>> Tom Lane wrote: In the context of yesterday's discussions, I wonder whether a filter by SQLSTATE would be appropriate. >>> >>> I'm worried it's not really gran

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
Robert Haas writes: > Well, nearby Tom and I discussed demoting the message to DEBUG1 when > no transaction is in progress. Presumably the two messages would > share the same SQL state, unless we're going to create separate SQL > states for connection-closed-not-in-a-txn and > connection-closed-i

Re: [HACKERS] outdated comment in heapam.c

2012-05-03 Thread Heikki Linnakangas
On 03.05.2012 16:08, Andres Freund wrote: Hi, It seems that when Heikki added the multi_insert code the following comment in htup.h wasn't updated: /* * We ran out of opcodes, so heapam.c now has a second RmgrId. These opcodes * are associated with RM_HEAP2_ID, but are not logically differen

Re: [HACKERS] Future In-Core Replication

2012-05-03 Thread Josh Berkus
On 5/2/12 10:58 PM, Jim Nasby wrote: > On 4/29/12 6:03 AM, Simon Riggs wrote: >>> The DML-WITH-LIMIT-1 is required to do single logical updates on tables >>> > with non-unique rows. >>> > And as for any logical updates we will have huge performance problem >>> > when doing UPDATE or DELETE on la

Re: [HACKERS] CLOG extension

2012-05-03 Thread Tom Lane
Robert Haas writes: > [ CLOG extension is horrid for concurrency ] Yeah. When that code was designed, a page's worth of transactions seemed like a lot so we didn't worry too much about performance glitches when we crossed a page boundary. It's time to do something about it though. The idea of

Re: [HACKERS] Future In-Core Replication

2012-05-03 Thread Josh Berkus
> Something that a in-core method might be able to do that an external one > can't would be to support a method of uniquely identifying rows in > tables with no PK's. A gross example (that undoubtedly wouldn't work in > the real world) would be using TID's. A real-world implementation might > be b

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 5:39 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, May 3, 2012 at 11:20 AM, Alvaro Herrera >> wrote: >>> Hey, maybe we could add a UUID to each ereport() call site ;-) > >> I can't help but feel we're designing a $10.00 solution to a $0.25 >> problem.  I think I'd a

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
Magnus Hagander writes: > On Thu, May 3, 2012 at 5:39 PM, Tom Lane wrote: >> I could support that with one tweak: it's only DEBUG1 if you don't >> have an open transaction.  Dropping the connection while in a >> transaction *is* an application bug; I don't care how lazy the app >> programmer is f

Re: [HACKERS] extending relations more efficiently

2012-05-03 Thread Stephen Frost
Robert, * Stephen Frost (sfr...@snowman.net) wrote: > > In all seriousness, this is not a great test case unless you can > > provide some scripts to make it easy to run it in a reproducible > > fashion. Can you? > > Yeah, sure, I'll do that. The PostGIS folks have scripts, but they're > kind of

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Josh Berkus
> AFAICS you'd either use transactional or session level, but to use > both seems bizarre. And if you really did need both, you can put a > wrapper around the function to check whether a session level exists > before you grant the transaction level lock, or vice versa. You wouldn't want to *inten

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Kevin Grittner
Tom Lane wrote: > Robert Haas writes: >> Well, nearby Tom and I discussed demoting the message to DEBUG1 >> when no transaction is in progress. Presumably the two messages >> would share the same SQL state, unless we're going to create >> separate SQL states for connection-closed-not-in-a-txn an

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
"Kevin Grittner" writes: > Would it make sense to use 08003 (connection_does_not_exist) when a > broken connection for an idle process is discovered, and 08006 > (connection_failure) for the "in transaction" failure? What about a > failure just after COMMIT and before successfully sending that >

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 7:09 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Thu, May 3, 2012 at 5:39 PM, Tom Lane wrote: >>> I could support that with one tweak: it's only DEBUG1 if you don't >>> have an open transaction.  Dropping the connection while in a >>> transaction *is* an applicatio

Re: [HACKERS] Future In-Core Replication

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 6:03 PM, Josh Berkus wrote: > I do agree that depending on user-defined PKs raises a whole host of > issues which we'd rather just sidestep, though. What do you have in mind instead? --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 2

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 12:12 PM, Simon Riggs wrote: > AFAICS you'd either use transactional or session level, but to use > both seems bizarre. I'm a bit confused by all this, because we use both transaction and session level locks internally - on the same lock tags - so I don't know why we think

Re: [HACKERS] CLOG extension

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 5:59 PM, Tom Lane wrote: > Robert Haas writes: >> [ CLOG extension is horrid for concurrency ] > > Yeah.  When that code was designed, a page's worth of transactions > seemed like a lot so we didn't worry too much about performance glitches > when we crossed a page boundary

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Robert Haas writes: > On Thu, May 3, 2012 at 12:12 PM, Simon Riggs wrote: >> AFAICS you'd either use transactional or session level, but to use >> both seems bizarre. > I'm a bit confused by all this, because we use both transaction and > session level locks internally - on the same lock tags -

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 7:21 PM, Magnus Hagander wrote: > On Thu, May 3, 2012 at 7:09 PM, Tom Lane wrote: >> Magnus Hagander writes: >>> On Thu, May 3, 2012 at 5:39 PM, Tom Lane wrote: I could support that with one tweak: it's only DEBUG1 if you don't have an open transaction.  Droppin

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Kevin Grittner
Tom Lane wrote: > AFAICS, all the 08 class is meant to be issued by client-side > code, not the server. I think we probably have to use nonstandard > SQLSTATEs for these messages. OK, if we're going that route, how about using "Class 2D * Invalid Transaction Termination"? I still think it m

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
Magnus Hagander writes: > Heh - we already used ERRCODE_CONNECTION_FAILURE on the errors in > copy.c. Since COPY can only happen when there is a transaction > (right?), I just changed those error messages for consistency. Agreed on changing the message texts to match, but I wonder whether we ough

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Andres Freund
On Thursday, May 03, 2012 06:12:04 PM Simon Riggs wrote: > AFAICS you'd either use transactional or session level, but to use > both seems bizarre. And if you really did need both, you can put a > wrapper around the function to check whether a session level exists > before you grant the transaction

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Tom Lane
"Kevin Grittner" writes: > I still think it might be useful to differentiate in our server log > between the case where the transaction failed and the case where the > transaction committed but we don't know that the client got the news > of that. How about something like: > 2DP01 connection_l

Re: [HACKERS] CLOG extension

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 1:27 PM, Simon Riggs wrote: > Why not switch to 1 WAL record per file, rather than 1 per page. (32 > pages, IIRC). > > We can then have the whole new file written as zeroes by a background > process, which needn't do that while holding the XidGenLock. I thought about doing

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-05-03 Thread Josh Berkus
Tom, So that I can test this properly, what is the specific use-case we'd expect to be slow with this patch? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgr

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-05-03 Thread Josh Berkus
> One thing I wanted to mention is that non-binary replication has an > added advantage over binary from a DR standpoint: if corruption occurs > on a master it is more likely to make it into your replicas thanks to > full page writes. You might want to consider that depending on how > sensitive yo

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Josh Berkus
> (on standby) > INSERT INTO s1 ; > SELECT ... FROM s1 WHERE ... > which seems to me to be actually harder than just rewriting as derived > table and isn't an option on Microstrategy etc, hence my observation > that GTTs don't help HS much. What I would like to see, one day, is > for temp tables t

Re: [HACKERS] CLOG extension

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 2:50 PM, Robert Haas wrote: > Doing it a background process, though, may make sense.  What I'm a > little worried about is that - on a busy system - we've only got about > 2 seconds to complete each CLOG extension, and we must do an fsync in > order to get there. Scratch th

Re: [HACKERS] CLOG extension

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 7:50 PM, Robert Haas wrote: > On Thu, May 3, 2012 at 1:27 PM, Simon Riggs wrote: >> Why not switch to 1 WAL record per file, rather than 1 per page. (32 >> pages, IIRC). >> >> We can then have the whole new file written as zeroes by a background >> process, which needn't do

Re: [HACKERS] "unexpected EOF" messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 7:48 PM, Tom Lane wrote: > Magnus Hagander writes: >> Heh - we already used ERRCODE_CONNECTION_FAILURE on the errors in >> copy.c. Since COPY can only happen when there is a transaction >> (right?), I just changed those error messages for consistency. > > Agreed on changing

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-05-03 Thread Kevin Grittner
Josh Berkus wrote: >> One thing I wanted to mention is that non-binary replication has >> an added advantage over binary from a DR standpoint: if >> corruption occurs on a master it is more likely to make it into >> your replicas thanks to full page writes. You might want to >> consider that dep

Re: [HACKERS] Uppercase tab completion keywords in psql?

2012-05-03 Thread Bruce Momjian
Peter, where are we on this? --- On Fri, Mar 30, 2012 at 08:16:59PM +0300, Peter Eisentraut wrote: > On fre, 2012-03-23 at 07:52 -0700, David Fetter wrote: > > On Thu, Mar 22, 2012 at 06:05:30PM -0400, Andrew Dunstan wrote:

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
I haven't tried quex, but I have tried lemon (which can be broken out of SQLite) and re2c and ragel. I like ragel and lemon, but the combination supports a push-parser style from memory, and many tools are inconvenient unless you are prepared to suck in a whole message before parsing, or let t

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
I wrote: > Robert Haas writes: >> In fact I'm a bit confused by the original complaint for the same >> reason - if LockRelationOid and LockRelationIdForSession can coexist, >> why doesn't the same thing work for advisory locks? > The problem (or problems) is bad implementation, not the specificat

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
Doesn't that imply that a plan cache might be worthwhile? But no matter: didn't the OP really have issue with packaging and Windows support - and there are a lot of Windows users, and in general there are many Windows devs: making it easier for them to contribute has to be good doesn't it? -

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-05-03 Thread Josh Berkus
> That's an interesting point. Out of curiosity, how did the > corruption originate? We're still not sure. It appears to be in the system catalogs, though. Note that the original master developed memory issues. > It suggests a couple questions: > > (1) Was Slony running before the corrupti

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 12:51 PM, james wrote: > I haven't tried quex, but I have tried lemon (which can be broken out of > SQLite) and re2c and ragel. > > I like ragel and lemon, but the combination supports a push-parser style > from memory, and many tools are inconvenient unless you are prepared

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
I believe there are tools that are significantly faster than flex. I believe re2c generates code that is faster. But the key thing is to test, probably, or perhaps ask around. I'm out of touch, but from memory flex wasn't the be-all and end-all. Lemon is definitely easy to maintain/port and

Re: [HACKERS] CLOG extension

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 3:20 PM, Simon Riggs wrote: > Your two paragraphs have roughly opposite arguments... > > Doing it every 32 pages would give you 30 seconds to complete the > fsync, if you kicked it off when half way through the previous file - > at current maximum rates. So there is utility

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 3:52 PM, Tom Lane wrote: > I'm inclined to say that you can PREPARE if your session holds a given > advisory lock at either session or transaction level, but not both. > This is a bit annoying but doesn't seem likely to be a real problem in > practice, so thinking of a hack

Re: [HACKERS] CLOG extension

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 1:56 PM, Robert Haas wrote: > Possibly.  I have some fear of ending up with too many background > processes, but we may need them. I sort of care about this, but only on systems that are not very busy and could otherwise get by with fewer resources -- for example, it'd be n

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Robert Haas writes: > On Thu, May 3, 2012 at 3:52 PM, Tom Lane wrote: >> I'm inclined to say that you can PREPARE if your session holds a given >> advisory lock at either session or transaction level, but not both. >> This is a bit annoying but doesn't seem likely to be a real problem in >> pract

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
... btw, it appears to me that the "fast path" patch has broken things rather badly in LockReleaseAll. AFAICS it's not honoring either the lockmethodid restriction nor the allLocks restriction with respect to fastpath locks. Perhaps user locks and session locks are never taken fast path, but stil

  1   2   >