Re: [HACKERS] Add regression tests for COLLATE

2013-05-09 Thread Fabien COELHO
Fabien pointed out that currently does not check for non-trivial locales. Indeed, but although it was not very from my point (my wish), and as pointed out by Tom, it is not quite possible to test non trivial locales because you cannot assume that a given locale is available on any test

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Thu, May 9, 2013 at 2:26 AM, Peter Eisentraut pete...@gmx.net wrote: On Wed, 2013-05-08 at 18:24 +0100, Dave Page wrote: It's failing on Linux. Even worse, it configures fine and then builds without error. There is a message spewed out by configure, but it doesn't contain the words warning

Re: [HACKERS] Add regression tests for COLLATE

2013-05-09 Thread Fabien COELHO
Maybe some tests could be applied under some condition, say a given locale is indeed available, but ISTM that it would require to change the test infrastructure in a portable way to add such feature. I just noticed that there is a src/test/locale directory with some stuff about german,

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Thu, May 9, 2013 at 8:09 AM, Dave Page dp...@pgadmin.org wrote: On Thu, May 9, 2013 at 2:26 AM, Peter Eisentraut pete...@gmx.net wrote: On Wed, 2013-05-08 at 18:24 +0100, Dave Page wrote: It's failing on Linux. Even worse, it configures fine and then builds without error. There is a message

Re: [HACKERS] Fast promotion failure

2013-05-09 Thread Kyotaro HORIGUCHI
Hello, I think it can so happen that last checkpoint is with old timeline and there are operations with new timeline which might have caused the problem Heikki has seen. I suppose to have seen that. After adding an SQL command to modify the DB on standby-B after passive(propagated?)

Re: [HACKERS] Add regression tests for ROLE (USER)

2013-05-09 Thread Fabien COELHO
This updated version works for me and addresses previous comments. I think that such tests are definitely valuable, especially as many corner cases which must trigger errors are covered. I recommend to apply it. Please find an updated patch as per comments on Commitfest (comments

Re: [HACKERS] Fast promotion failure

2013-05-09 Thread Kyotaro HORIGUCHI
With printing some additinal logs, the situation should be more clear.. It seems that Sby-B failes to promote to TLI= 2; nevertheless the history file for TLI = 2 is somehow sent to sby-C. So sby-B remains on TLI=1 but sby-C solely switches onto TLI=2. # Come to think of this, I suspect that

Re: [HACKERS] Proposal to add --single-row to psql

2013-05-09 Thread Michael Paquier
On Thu, May 9, 2013 at 9:12 AM, David Fetter da...@fetter.org wrote: On Wed, May 08, 2013 at 06:08:28PM -0500, Jim Nasby wrote: I believe that makes it significantly harder for them to actually contribute code back that doesn't give them a business advantage, as well as making it harder to

Re: [HACKERS] Add regression tests for COLLATE

2013-05-09 Thread Michael Paquier
On Thu, May 9, 2013 at 4:19 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: However, it is not clear whether these tests run automatically or only if they are explicitely called. The README seems to suggest that it is the later. If so, maybe having them invoked automatically if possible would

Re: [HACKERS] Fast promotion failure

2013-05-09 Thread Amit Kapila
On Thursday, May 09, 2013 2:14 PM Kyotaro HORIGUCHI wrote: With printing some additinal logs, the situation should be more clear.. It seems that Sby-B failes to promote to TLI= 2; nevertheless the history file for TLI = 2 is somehow sent to sby-C. So sby-B remains on TLI=1 but sby-C solely

Re: [HACKERS] Graph datatype addition

2013-05-09 Thread Merlin Moncure
On Wed, May 8, 2013 at 2:16 PM, Atri Sharma atri.j...@gmail.com wrote: Your second drawing didn't really make any sense to me. :( I do think it would be most productive to focus on what the API for dealing with graph data would look like before trying to handle the storage aspect. The

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Peter Eisentraut
On 5/9/13 3:09 AM, Dave Page wrote: I assume you'll fix the configure script so it actually errors out if plpython cannot be built, but is explicitly requested? That is ancient behavior, which I'm not fond of, but now is not the time to change that. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Peter Eisentraut
On 5/9/13 3:25 AM, Dave Page wrote: BTW - it's always worked fine for us on 64 bit machines with the past few major releases of both PG and Python - are you saying that's pure chance? It works because ActiveState Python has PIC inside a static library. But we have no straightforward way of

Re: [HACKERS] [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
From the 9.1 cluster (port 5432): db=# SELECT relname, relfilenode, relkind from pg_class where oid = 2938685; relname| relfilenode | relkind ---+-+- substitutionlist_pkey |21446253 | i (1 row) db=# From the 9.2 cluster (port 5433):

[HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 10:20:12AM -0400, Evan D. Hoffman wrote: From the 9.1 cluster (port 5432): db=# SELECT relname, relfilenode, relkind from pg_class where oid = 2938685; relname| relfilenode | relkind ---+-+-

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut pete...@gmx.net wrote: On 5/9/13 3:09 AM, Dave Page wrote: I assume you'll fix the configure script so it actually errors out if plpython cannot be built, but is explicitly requested? That is ancient behavior, which I'm not fond of, but now is

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-05-09 Thread Robert Haas
On Sat, May 4, 2013 at 3:59 PM, Tom Lane t...@sss.pgh.pa.us wrote: Given the lack of any good alternative proposals, I think we should press ahead with this approach. We still need doc updates and fixes for the affected contrib module(s), though. Also, in view of point 2, it seems like the

[HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Pavel Stehule
Hello I am writing a article about 9.3. I found so event trigger functions is not complete. We have only pg_event_trigger_dropped_objects() function. It looks really strange and asymmetric. Can we implement similar function for CREATE as minimum to 9.3? I am expecting so this function should not

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Darren Duncan
On 2013.05.09 10:40 AM, Pavel Stehule wrote: I am writing a article about 9.3. I found so event trigger functions is not complete. We have only pg_event_trigger_dropped_objects() function. It looks really strange and asymmetric. Can we implement similar function for CREATE as minimum to 9.3? I

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Dimitri Fontaine
Darren Duncan dar...@darrenduncan.net writes: On 2013.05.09 10:40 AM, Pavel Stehule wrote: I am expecting so this function should not be too complex - and can be moved to contrib maybe (if it is too late now). My understanding is that it's too late even for contrib now. Really, the touted

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Pavel Stehule
2013/5/9 Dimitri Fontaine dimi...@2ndquadrant.fr Darren Duncan dar...@darrenduncan.net writes: On 2013.05.09 10:40 AM, Pavel Stehule wrote: I am expecting so this function should not be too complex - and can be moved to contrib maybe (if it is too late now). My understanding is that

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Alvaro Herrera
Darren Duncan wrote: On 2013.05.09 10:40 AM, Pavel Stehule wrote: I am writing a article about 9.3. I found so event trigger functions is not complete. We have only pg_event_trigger_dropped_objects() function. It looks really strange and asymmetric. Can we implement similar function for CREATE

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Misa Simic
2013/5/9 Pavel Stehule pavel.steh...@gmail.com Hello I am writing a article about 9.3. I found so event trigger functions is not complete. We have only pg_event_trigger_dropped_objects() function. It looks really strange and asymmetric. Can we implement similar function for CREATE as

Re: [HACKERS] missing event trigger support functions in 9.3

2013-05-09 Thread Darren Duncan
On 2013.05.09 11:22 AM, Alvaro Herrera wrote: Darren Duncan wrote: On 2013.05.09 10:40 AM, Pavel Stehule wrote: I am writing a article about 9.3. I found so event trigger functions is not complete. We have only pg_event_trigger_dropped_objects() function. It looks really strange and

Re: [HACKERS] [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
I just did the whole process over from the beginning. here's the full output: -bash-4.1$ date ; time /usr/pgsql-9.2/bin/pg_upgrade -b /usr/pgsql-9.1/bin/ -B /usr/pgsql-9.2/bin/ -d /var/lib/pgsql/9.1/data/ -D /var/lib/pgsql/9.2/data/ -p 50432 -P 50433 ; date Thu May 9 14:31:07 EDT 2013

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jim Nasby
On 5/8/13 7:34 PM, Jeff Davis wrote: On Wed, 2013-05-08 at 17:56 -0500, Jim Nasby wrote: Apologies if this is a stupid question, but is this mostly an issue due to torn pages? IOW, if we had a way to ensure we never see torn pages, would that mean an invalid CRC on a WAL page indicated there

[HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 03:23:20PM -0400, Evan D. Hoffman wrote: I just did the whole process over from the beginning. here's the full output: Copying user relation files /var/lib/pgsql/9.1/data/base/16406/3016054 Mismatch of relation OID in database db: old OID 2938685,

Re: [HACKERS] [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
That's correct. Here's what substitutionlist_pkey looks like in the new cluster. From this, it looks like it's actually correct (the oid for substitutionlist_pkey is correct) but pg_upgrade thinks it's wrong and dies. I'll look for the logs you requested and send them separately db=# SELECT

[HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 03:52:42PM -0400, Evan D. Hoffman wrote: That's correct. Here's what substitutionlist_pkey looks like in the new cluster. From this, it looks like it's actually correct (the oid for substitutionlist_pkey is correct) but pg_upgrade thinks it's wrong and dies. I'll

Re: [HACKERS] [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
Looks like your guess was correct: [ehoffman@dev-db2 ~]$ psql -Upostgres db -p 5433 psql (9.2.4) Type help for help. db=# SELECT oid, relname, reltoastrelid, reltoastidxid FROM pg_class db-# WHERE reltoastrelid = 299749; oid | relname | reltoastrelid | reltoastidxid

Re: [HACKERS] [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Tom Lane
Evan D. Hoffman evandhoff...@gmail.com writes: Looks like your guess was correct: Could we see the full schema (eg psql \d+) for setupinfo? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
Here it is with the interesting field names mangled for paranoia reasons: db=# \d+ bpm.setupinfo; Table bpm.setupinfo Column| Type | Modifiers | Storage | Stats target | Description

[HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 04:21:05PM -0400, Evan D. Hoffman wrote: Looks like your guess was correct: [ehoffman@dev-db2 ~]$ psql -Upostgres db -p 5433 psql (9.2.4) Type help for help. db=# SELECT oid, relname, reltoastrelid, reltoastidxid FROM pg_class db-# WHERE

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Peter Eisentraut
On 5/9/13 10:52 AM, Dave Page wrote: On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut pete...@gmx.net wrote: On 5/9/13 3:09 AM, Dave Page wrote: I assume you'll fix the configure script so it actually errors out if plpython cannot be built, but is explicitly requested? That is ancient

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: OK, that's progress. Having received the table schema privately via email, I see several 'character varying(40)' fields in the schema. So the question is how was this table able to get away without a TOAST table in 9.1, while 9.2 created one for an

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Simon Riggs
On 9 May 2013 20:28, Jim Nasby j...@nasby.net wrote: Unfortunately, it seems that doing any kind of validation to determine that we have a valid end-of-the-WAL inherently requires some kind of separate durable write somewhere. It would be a tiny amount of data (an LSN and maybe some extra

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 05:11:43PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: OK, that's progress. Having received the table schema privately via email, I see several 'character varying(40)' fields in the schema. So the question is how was this table able to get away

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Dave Page
On Thu, May 9, 2013 at 10:06 PM, Peter Eisentraut pete...@gmx.net wrote: On 5/9/13 10:52 AM, Dave Page wrote: On Thu, May 9, 2013 at 3:09 PM, Peter Eisentraut pete...@gmx.net wrote: On 5/9/13 3:09 AM, Dave Page wrote: I assume you'll fix the configure script so it actually errors out if

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: If the current WAL record is corrupt and the next WAL record is in every way valid, we can potentially continue. That seems like a seriously bad idea. regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
I believe the history of this cluster is that it started on 9.0 and was upgraded to 9.1 via pg_upgrade. The instance I'm working on was created as a streaming replica, then I broke the replication to make it a standalone master specifically for testing pg_upgrade to 9.2. On May 9, 2013, at

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Simon Riggs
On 9 May 2013 22:39, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: If the current WAL record is corrupt and the next WAL record is in every way valid, we can potentially continue. That seems like a seriously bad idea. I agree. But if you knew that were true, is

[HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Greg Stark
On Thu, May 9, 2013 at 10:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: In any case, it seems like pg_upgrade ought to have a strategy for dealing with tables acquiring toast tables like this, Acquiring toast tables seems pretty trivial to deal with. *Losing* a toast table might be a bit more

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Alvaro Herrera
Greg Stark escribió: On Thu, May 9, 2013 at 10:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: In any case, it seems like pg_upgrade ought to have a strategy for dealing with tables acquiring toast tables like this, Acquiring toast tables seems pretty trivial to deal with. *Losing* a toast

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Greg Stark
On Thu, May 9, 2013 at 10:45 PM, Simon Riggs si...@2ndquadrant.com wrote: On 9 May 2013 22:39, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: If the current WAL record is corrupt and the next WAL record is in every way valid, we can potentially continue. That

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jeff Davis
On Thu, 2013-05-09 at 14:28 -0500, Jim Nasby wrote: What about moving some critical data from the beginning of the WAL record to the end? That would make it easier to detect that we don't have a complete record. It wouldn't necessarily replace the CRC though, so maybe that's not good enough.

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 06:05:14PM -0400, Alvaro Herrera wrote: Greg Stark escribió: On Thu, May 9, 2013 at 10:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: In any case, it seems like pg_upgrade ought to have a strategy for dealing with tables acquiring toast tables like this, Acquiring

Re: [HACKERS] corrupt pages detected by enabling checksums

2013-05-09 Thread Jeff Davis
On Thu, 2013-05-09 at 23:13 +0100, Greg Stark wrote: However it is possible to reduce the window... Sounds reasonable. It's fairly limited though -- the window is already a checkpoint (typically 5-30 minutes), and we'd bring that down an order of magnitude (10s). I speculate that, if it got

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 05:41:39PM -0400, Evan D. Hoffman wrote: I believe the history of this cluster is that it started on 9.0 and was upgraded to 9.1 via pg_upgrade. The instance I'm working on was created as a streaming replica, then I broke the replication to make it a standalone master

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Evan D. Hoffman
Hmm... the database itself predates me, so I can't say for sure what encoding it was created with, but when I did a pg_dumpall -s it showed every database in the cluster uses SET client_encoding = 'UTF8'; On Thu, May 9, 2013 at 7:25 PM, Bruce Momjian br...@momjian.us wrote: On Thu, May 9, 2013

Re: [HACKERS] Add regression tests for COLLATE

2013-05-09 Thread Robins Tharakan
One workaround is to use DROP COLLATION IF EXISTS, that conveys the message without UTF8 in the message string. But three other tests use ALTER COLLATION and I see no way but to comment / disable them. Currently have them disabled (with comments as to what they do, and why they're disabled).

Re: [HACKERS] improving PL/Python builds on OS X

2013-05-09 Thread Peter Eisentraut
On Thu, 2013-05-09 at 10:11 -0400, Peter Eisentraut wrote: On 5/9/13 3:25 AM, Dave Page wrote: BTW - it's always worked fine for us on 64 bit machines with the past few major releases of both PG and Python - are you saying that's pure chance? It works because ActiveState Python has PIC

Re: [HACKERS] Re: [GENERAL] pg_upgrade fails, mismatch of relation OID - 9.1.9 to 9.2.4

2013-05-09 Thread Bruce Momjian
On Thu, May 9, 2013 at 09:22:55PM -0400, Evan D. Hoffman wrote: Hmm... the database itself predates me, so I can't say for sure what encoding it was created with, but when I did a pg_dumpall -s it showed every database in the cluster uses SET client_encoding = 'UTF8'; OK, that's good to