[HACKERS] Parse more than bind and execute when connect to database by jdbc

2013-11-18 Thread wangshuo
Hi everyone, Finally we found , the JDBC function we ever modified contributed to this phenomenon, thanks of all. Yours, Wang Shuo HighGo Software Co.,Ltd. November 18, 2013 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread David Rowley
On Wed, Oct 30, 2013 at 3:32 AM, Andres Freund and...@2ndquadrant.comwrote: On 2013-10-25 09:26:29 -0400, Robert Haas wrote: In any case, it's very far from obvious to me that CLUSTER ought to throw away information by default, which is what you're proposing. I find it odd to referring

Re: [HACKERS] writable FDWs / update targets confusion

2013-11-18 Thread Albe Laurenz
Tom Lane wrote: Tom, could you show us a rope if there is one? What is it you actually need to fetch? IIRC, the idea was that most FDWs would do the equivalent of fetching the primary-key columns to use in an update. If that's what you need, then AddForeignUpdateTargets should identify

Re: [HACKERS] freeze cannot be finished

2013-11-18 Thread Heikki Linnakangas
Committed, thanks for the report! On 16.11.2013 22:05, Миша Тюрин wrote: Hello! Could anyone review patch suggested by Jeff Janes ? Initial thread http://www.postgresql.org/message-id/flat/1384356585.995240612%40f50.i.mail.ru Thanks in advance! On Wed, Nov 13, 2013 at 3:53 PM, Sergey

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 15.11.2013 21:00, Simon Riggs wrote: On 15 November 2013 15:48, Peter Eisentraut pete...@gmx.net wrote: Also, you set this to returned with feedback in the CF app. Please verify whether that was intentional. Not sure that was me, if so, corrected. It was me, sorry. I figured this needs

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 15.11.2013 20:21, Andres Freund wrote: On 2013-11-15 20:08:30 +0200, Heikki Linnakangas wrote: It's pretty hard to review the this without seeing the other implementation you're envisioning to use this API. But I'll try: We've written a distributed sequence implementation against it, so it

Re: [HACKERS] additional json functionality

2013-11-18 Thread Hannu Krosing
On 11/18/2013 05:19 AM, Andrew Dunstan wrote: On 11/17/2013 08:49 PM, Josh Berkus wrote: Now, if it turns out that the new hstore is not dealing with json input and output, we could have json, jstore and hstore. Jstore isn't the worst name suggestion I've heard on this thread. The reason I

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-18 Thread Amit Khandekar
On 18 October 2013 17:07, Rajeev rastogi rajeev.rast...@huawei.com wrote: From the following mail, copy behaviour between stdin and normal file having some inconsistency. http://www.postgresql.org/message-id/ce85a517.4878e%tim.k...@gmail.com The issue was that if copy execute from

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Haribabu kommi
On 18 November 2013 11:19 Haribabu kommi wrote: On 17 November 2013 00:55 Fujii Masao wrote: On Sat, Nov 16, 2013 at 2:27 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: on 15 November 2013 17:26 Magnus Hagander wrote: On Fri, Nov 15, 2013 at 12:10 PM, Haribabu kommi

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Andres Freund
On 2013-11-18 10:54:42 +0200, Heikki Linnakangas wrote: On 15.11.2013 20:21, Andres Freund wrote: Well, it exposes log_sequence_tuple() - together with the added am private column of pg_squence that allows to do quite a bit of different things. I think unless we really implement pluggable

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-11-18 Thread Sameer Thakur
Hello, Please find v10 of patch attached. This patch addresses following review comments 1. Removed errcode and used elogs for error pg_stat_statements schema is not supported by its binary 2. Removed comments and other code formatting not directly relevant to patch functionality 3. changed

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 18.11.2013 11:48, Andres Freund wrote: On 2013-11-18 10:54:42 +0200, Heikki Linnakangas wrote: On 15.11.2013 20:21, Andres Freund wrote: Well, it exposes log_sequence_tuple() - together with the added am private column of pg_squence that allows to do quite a bit of different things. I think

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Haribabu kommi
On 18 October 2013 13:35 KONDO Mitsumasa wrote: Hi, I submit improvement of pg_stat_statement usage patch in CF3. In pg_stat_statement, I think buffer hit ratio is very important value. However, it is difficult to calculate it, and it need complicated SQL. This patch makes it more simple

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/18 20:16), Haribabu kommi wrote: On 18 October 2013 13:35 KONDO Mitsumasa wrote: This patch conflicts pg_stat_statement_min_max_exectime patch which I submitted, and pg_stat_statement_min_max_exectime patch also adds new columns which are min_time and max_time. So I'd like to change it

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Andres Freund
On 2013-11-18 12:50:21 +0200, Heikki Linnakangas wrote: On 18.11.2013 11:48, Andres Freund wrote: I don't think the sequence AM should be in control of 'cached'. The caching is done outside the AM. And log_cnt probably should be passed to the _alloc function directly as an argument, ie. the

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Simon Riggs
On 18 November 2013 07:50, Heikki Linnakangas hlinnakan...@vmware.com wrote: It doesn't go far enough, it's still too *low*-level. The sequence AM implementation shouldn't need to have direct access to the buffer page at all. I don't think the sequence AM should be in control of 'cached'.

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 18.11.2013 13:48, Simon Riggs wrote: On 18 November 2013 07:50, Heikki Linnakangas hlinnakan...@vmware.com wrote: It doesn't go far enough, it's still too *low*-level. The sequence AM implementation shouldn't need to have direct access to the buffer page at all. I don't think the

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2013-11-18 Thread Andres Freund
On 2013-11-18 19:52:29 +0900, Michael Paquier wrote: On Sat, Nov 16, 2013 at 5:09 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-15 11:40:17 +0900, Michael Paquier wrote: - 20131114_3_reindex_concurrently.patch, providing the core feature. Patch 3 needs to have patch 2 applied

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 14.11.2013 22:10, Simon Riggs wrote: Includes test extension which allows sequences without gaps - gapless. I realize this is just for demonstration purposes, but it's worth noting that it doesn't actually guarantee that when you use the sequence to populate a column in the table, the

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Haribabu kommi
On 17 November 2013 12:25 Amit Kapila wrote: On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 16 November 2013 09:43 Amit Kapila wrote: On Fri, Nov 15, 2013 at 10:18 PM, Peter Eisentraut pete...@gmx.net wrote: On 11/14/13, 2:50 AM, Amit Kapila wrote:

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Fujii Masao
On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 18 November 2013 11:19 Haribabu kommi wrote: On 17 November 2013 00:55 Fujii Masao wrote: On Sat, Nov 16, 2013 at 2:27 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: on 15 November 2013 17:26 Magnus

Re: [HACKERS] CREATE TABLE IF NOT EXISTS AS

2013-11-18 Thread Fabrízio de Royes Mello
On Sun, Nov 17, 2013 at 6:05 PM, David E. Wheeler da...@justatheory.comwrote: On Nov 16, 2013, at 4:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: Co-worker asked a question I could not answer: Why is IF NOT EXISTS not supported by CREATE TABLE AS? That's an even worse idea than plain CREATE

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-18 Thread Marko Kreen
On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote: On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen mark...@gmail.com wrote: I am bit suspicious of performance impact of this patch, but think that it's still worthwhile as it decreases code style where single string argument is given

Re: [HACKERS] additional json functionality

2013-11-18 Thread Peter Eisentraut
On 11/15/13, 6:15 PM, Josh Berkus wrote: Thing is, I'm not particularly concerned about *Merlin's* specific use case, which there are ways around. What I am concerned about is that we may have users who have years of data stored in JSON text fields which won't survive an upgrade to binary

[HACKERS] Force optimizer to use hash/nl/merge join?

2013-11-18 Thread Zhan Li
Hi All, Is there any way to force the optimizer to use a specific join operator? For example, in SQL Server, I can do this way select * from (A inner hash join B on A.a = B.b) inner loop join C on A.a = C.c I did some search but didn't find PostgreSQL had similar join hints except for enable_*

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Amit Kapila
On Mon, Nov 18, 2013 at 6:28 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 17 November 2013 12:25 Amit Kapila wrote: On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi Find the rebased version attached with this mail. ereport(ERROR,

Re: [HACKERS] Force optimizer to use hash/nl/merge join?

2013-11-18 Thread Amit Kapila
On Mon, Nov 18, 2013 at 7:58 PM, Zhan Li zhanl...@gmail.com wrote: Hi All, Is there any way to force the optimizer to use a specific join operator? For example, in SQL Server, I can do this way select * from (A inner hash join B on A.a = B.b) inner loop join C on A.a = C.c I did some

Re: [HACKERS] inherit support for foreign tables

2013-11-18 Thread Robert Haas
On Thu, Nov 14, 2013 at 12:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: 2) Allow foreign tables to have CHECK constraints Like NOT NULL, I think we don't need to enforce the check duroing INSERT/UPDATE against foreign table. Really? It's one thing to say that somebody who adds a CHECK

Re: [HACKERS] additional json functionality

2013-11-18 Thread Merlin Moncure
On Sun, Nov 17, 2013 at 10:19 PM, Andrew Dunstan and...@dunslane.net wrote: I don't think any name that doesn't begin with json is acceptable. I could live with jsonb. It has the merit of brevity, but maybe it's a tad too close to json to be the right answer. I think that seems right. Couple

[HACKERS] Review: pre-commit triggers

2013-11-18 Thread Ian Lawrence Barwick
Review for Andrew Dunstan's patch in CF 2013-11: https://commitfest.postgresql.org/action/patch_view?id=1312 This review is more from a usage point of view, I would like to spend more time looking at the code but only so many hours in a day, etcetera; I hope this is useful as-is. Submission

Re: [HACKERS] additional json functionality

2013-11-18 Thread Andrew Dunstan
On 11/18/2013 09:38 AM, Merlin Moncure wrote: On Sun, Nov 17, 2013 at 10:19 PM, Andrew Dunstan and...@dunslane.net wrote: I don't think any name that doesn't begin with json is acceptable. I could live with jsonb. It has the merit of brevity, but maybe it's a tad too close to json to be the

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Heikki Linnakangas
On 14.10.2013 07:12, Peter Geoghegan wrote: On Wed, Oct 9, 2013 at 1:11 PM, Peter Geoghegan p...@heroku.com wrote: Unfortunately, I have a very busy schedule in the month ahead, including travelling to Ireland and Japan, so I don't think I'm going to get the opportunity to work on this too

Re: [HACKERS] inherit support for foreign tables

2013-11-18 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Nov 14, 2013 at 12:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: 2) Allow foreign tables to have CHECK constraints Like NOT NULL, I think we don't need to enforce the check duroing INSERT/UPDATE against foreign table. Really? I think it's been

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Haribabu kommi
On 18 November 2013 18:45 Fujii Masao wrote: On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 18 November 2013 11:19 Haribabu kommi wrote: On 17 November 2013 00:55 Fujii Masao wrote: On Sat, Nov 16, 2013 at 2:27 PM, Haribabu kommi

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Haribabu kommi
On 18 November 2013 20:01 Amit Kapila wrote: On Mon, Nov 18, 2013 at 6:28 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 17 November 2013 12:25 Amit Kapila wrote: On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi Find the rebased version attached with this mail.

Re: [HACKERS] Small improvement to json out functions by using cstring_to_text_with_len instead of cstring_to_text

2013-11-18 Thread Robert Haas
On Thu, Nov 14, 2013 at 2:18 AM, David Rowley dgrowle...@gmail.com wrote: Hi, Here's a small patch which should speedup json out functions a little bit by removing a call to strlen for which could be a long string. The length of the string is already known by the StringInfoData, so there's

Re: [HACKERS] Can we add sample systemd service file to git repo?

2013-11-18 Thread Will Crawford
On 12 November 2013 20:54, Nigel Heron nhe...@querymetrics.com wrote: On Tue, Nov 12, 2013 at 11:47 AM, Devrim GÜNDÜZ dev...@gunduz.org wrote: ... http://svn.pgrpms.org/browser/rpm/redhat/9.3/postgresql/F-19/postgresql-9.3.service is an example of what we use in the RPMs. (if website fails,

Re: [HACKERS] Freezing without write I/O

2013-11-18 Thread Andres Freund
On 2013-09-25 12:31:20 +0300, Heikki Linnakangas wrote: Hmm, some of those are trivial, but others, rewrite_heap_tuple() are currently only passed the HeapTuple, with no reference to the page where the tuple came from. Instead of changing code to always pass that along with a tuple, I think we

Re: [HACKERS] nested hstore patch

2013-11-18 Thread Andres Freund
Hi, On 2013-11-12 22:35:31 +0400, Teodor Sigaev wrote: Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type. I took a quick peek at this: * You cannot simply catch and ignore errors by doing + PG_TRY(); + { +

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-18 Thread Heikki Linnakangas
On 18.11.2013 15:40, Marko Kreen wrote: On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote: On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen mark...@gmail.com wrote: I am bit suspicious of performance impact of this patch, but think that it's still worthwhile as it decreases code style

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 09:22:58PM +1300, David Rowley wrote: So now I'm wondering what the next move should be for this patch? a. Are we waiting on Robert's patch to be committed before we can apply Thomas's cluster with freeze as default? b. Are we waiting on me reviewing one or both of

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Andres Freund
On 2013-11-18 11:39:44 -0500, Bruce Momjian wrote: On Mon, Nov 18, 2013 at 09:22:58PM +1300, David Rowley wrote: So now I'm wondering what the next move should be for this patch? a. Are we waiting on Robert's patch to be committed before we can apply Thomas's cluster with freeze as

Re: [HACKERS] Review: pre-commit triggers

2013-11-18 Thread Andrew Dunstan
On 11/18/2013 09:39 AM, Ian Lawrence Barwick wrote: Review for Andrew Dunstan's patch in CF 2013-11: https://commitfest.postgresql.org/action/patch_view?id=1312 This review is more from a usage point of view, I would like to spend more time looking at the code but only so many hours in a

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-18 Thread Andres Freund
Hi, On 2013-11-15 22:38:05 -0500, Jaime Casanova wrote: sorry, i clearly misunderstood you. attached a rebased patch with those functions removed. * --write-standby-enable seems to loose quite some functionality in comparison to --write-recovery-conf since it doesn't seem to set

Re: [HACKERS] additional json functionality

2013-11-18 Thread Josh Berkus
On 11/18/2013 06:13 AM, Peter Eisentraut wrote: On 11/15/13, 6:15 PM, Josh Berkus wrote: Thing is, I'm not particularly concerned about *Merlin's* specific use case, which there are ways around. What I am concerned about is that we may have users who have years of data stored in JSON text

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Andres Freund
On 2013-11-18 15:01:42 +, Haribabu kommi wrote: /* + * Returns the malloced string of containing current working directory. + * The caller has to take care of freeing the memory. + * On failure exits with error code. + */ +static char * +get_current_working_dir() +{ + char

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Fujii Masao
On Tue, Nov 19, 2013 at 12:01 AM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 18 November 2013 18:45 Fujii Masao wrote: On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 18 November 2013 11:19 Haribabu kommi wrote: On 17 November 2013 00:55 Fujii

Re: [HACKERS] additional json functionality

2013-11-18 Thread Josh Berkus
Merlin, *) Aside from the text in and out routines, how is 'jsbonb' different from the coming 'nested hstore'? Enough to justify two code bases? In/out functions and defaults are all different. Otherwise, the two types will be accessing the same code base, so no duplication. Think of is as

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Josh Berkus
On 11/18/2013 08:39 AM, Bruce Momjian wrote: If we do add FREEZE, all we would be doing is delaying the time when all CLUSTER operations will use FREEZE, and hence debugging will be just as difficult. My point is that will full knowledge, everyone would use FREEZE unless they expect MVCC

Re: [HACKERS] Review: pre-commit triggers

2013-11-18 Thread Dimitri Fontaine
Andrew Dunstan and...@dunslane.net writes: Given that, do we want to keep the bar on these operating in single user mode? I can easily drop it and just document this way out of difficulty. Currently Event Triggers are disabled in single user mode, in parts because operating them require

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Fujii Masao
On Mon, Nov 18, 2013 at 8:36 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: (2013/11/18 20:16), Haribabu kommi wrote: On 18 October 2013 13:35 KONDO Mitsumasa wrote: This patch conflicts pg_stat_statement_min_max_exectime patch which I submitted, and

Re: [HACKERS] additional json functionality

2013-11-18 Thread Merlin Moncure
On Mon, Nov 18, 2013 at 12:10 PM, Josh Berkus j...@agliodbs.com wrote: Merlin, *) Aside from the text in and out routines, how is 'jsbonb' different from the coming 'nested hstore'? Enough to justify two code bases? In/out functions and defaults are all different. Otherwise, the two

[HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
Three times in the last two weeks, we have experience data corruption secondary servers using streaming replication on client systems. The versions involved are 9.0.14, 9.2.5, and 9.3.1. Each incident was separate; two cases they were for the same client (9.0.14 and 9.3.1), one for a

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao masao.fu...@gmail.com wrote: The same idea was proposed before but not committed because Itagaki thought that pg_stat_statements view should report only raw values. Please read the following thread. I have the same feeling with him. Anyway we

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Josh Berkus
On 11/18/2013 10:58 AM, Christophe Pettus wrote: Three times in the last two weeks, we have experience data corruption secondary servers using streaming replication on client systems. The versions involved are 9.0.14, 9.2.5, and 9.3.1. Each incident was separate; two cases they were for

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 10:58 AM, Christophe Pettus x...@thebuild.com wrote: As a note, P1 was created from another system (let's call it P0) using just WAL shipping (no streaming replication), and no data corruption was observed. As another data point, P0 was running 9.0.13, rather than 9.0.14.

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
Hi, On 2013-11-18 10:58:26 -0800, Christophe Pettus wrote: INCDIDENT #1: 9.0.14 -- A new secondary (S1) was initialized using rsync off of an existing, correct primary (P1) for the base backup, and using WAL-E for WAL segment shipping. Both the primary and secondary were running 9.0.14. S1

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:28 AM, Andres Freund and...@2ndquadrant.com wrote: Could you detail how exactly the base backup was created? Including the *exact* logic for copying? 0. Before any of this began, P1 was archiving WAL segments to AWS-S3. 1. pg_start_backup('', true) on P1. 2. Using rsync

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 11:38:43 -0800, Christophe Pettus wrote: On Nov 18, 2013, at 11:28 AM, Andres Freund and...@2ndquadrant.com wrote: Could you detail how exactly the base backup was created? Including the *exact* logic for copying? 0. Before any of this began, P1 was archiving WAL segments

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund and...@2ndquadrant.com wrote: Without deleting any data, including pg_xlog/, backup.label, anything? Correct. Did you have hot_standby enabled on all of those machines? Even on the 9.0.13 cluster? Yes. That was just recovery command and primary

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund and...@2ndquadrant.com wrote: Did you have hot_standby enabled on all of those machines? Even on the 9.0.13 cluster? Actually, it's a bit more complex than this: 1. We don't know about P0, the 9.0.13 machine. I assume it was, but it was managed

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund and...@2ndquadrant.com wrote: Without deleting any data, including pg_xlog/, backup.label, anything? One more correction: After rsync finished and the pg_base_backup() was issued, the contents of pg_xlog/ on S1 were deleted. -- -- Christophe Pettus

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 12:00 PM, Christophe Pettus x...@thebuild.com wrote: One more correction: After rsync finished and the pg_base_backup() was issued, the contents of pg_xlog/ on S1 were deleted. pg_stop_backup(), sorry. -- -- Christophe Pettus x...@thebuild.com -- Sent via

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Kevin Grittner
Josh Berkus j...@agliodbs.com wrote: On 11/18/2013 08:39 AM, Bruce Momjian wrote: If we do add FREEZE, all we would be doing is delaying the time when all CLUSTER operations will use FREEZE, and hence debugging will be just as difficult.  My point is that will full knowledge, everyone would

Re: [HACKERS] Changing pg_dump default file format

2013-11-18 Thread Bruce Momjian
On Thu, Nov 7, 2013 at 02:40:17PM -0500, Peter Eisentraut wrote: On 11/7/13, 1:00 PM, Josh Berkus wrote: If we wanted to change the defaults, I think it would be easier to create a separate bin name (e.g. pg_backup) than to change the existing parameters for pg_dump. Note the following

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Alexander Korotkov
On Fri, Nov 15, 2013 at 11:42 PM, Alexander Korotkov aekorot...@gmail.comwrote: On Fri, Nov 15, 2013 at 11:39 PM, Rod Taylor rod.tay...@gmail.com wrote: On Fri, Nov 15, 2013 at 2:26 PM, Alexander Korotkov aekorot...@gmail.com wrote: On Fri, Nov 15, 2013 at 11:18 PM, Rod Taylor

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Sat, Nov 9, 2013 at 02:15:52PM -0500, Robert Haas wrote: On Fri, Nov 8, 2013 at 5:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: [ I'm so far behind ... ] Bruce Momjian br...@momjian.us writes: Applied. Thank you for all your suggestions. I thought the suggestion had been to issue a

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 10:58:26 -0800, Christophe Pettus wrote: After promotion, it was discovered that there was significant data loss on S1. Rows that were present on P1 were missing on S1, and some rows were duplicated (including duplicates that violated primary key and other unique constraints).

Re: [HACKERS] unaccent module - two params function should be immutable

2013-11-18 Thread Bruce Momjian
On Fri, Nov 8, 2013 at 06:00:53PM -0500, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: [ mark unaccent functions immutable ] Applied. This patch is flat out wrong and needs to be reverted. The functions were correctly marked (by you!) in commit

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 12:57 PM, Andres Freund and...@2ndquadrant.com wrote: Were there any kind of patterns in the lost data? What kind of workload are they running? I have an idea what the issue might be... On the P1 S1 case, the data corrupted was data modified in the last few minutes

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
Hi, Afaics it's likely a combination/interaction of bugs and fixes between: * the initial HS code * 5a031a5556ff83b8a9646892715d7fef415b83c3 * f44eedc3f0f347a856eea8590730769125964597 But that'd mean nobody noticed it during 9.3's beta... Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
Great! If there's any further data I can supply to help, let me know. On Nov 18, 2013, at 2:15 PM, Andres Freund and...@2ndquadrant.com wrote: Hi, Afaics it's likely a combination/interaction of bugs and fixes between: * the initial HS code * 5a031a5556ff83b8a9646892715d7fef415b83c3 *

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 14:25:58 -0800, Christophe Pettus wrote: Great! If there's any further data I can supply to help, let me know. Trying to reproduce the issue with and without hot_standby=on would be very helpful, but I guess that's time consuming? Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 2:26 PM, Andres Freund and...@2ndquadrant.com wrote: Trying to reproduce the issue with and without hot_standby=on would be very helpful, but I guess that's time consuming? I've been working on it, but I haven't gotten it to fail yet. I'll keep at it. -- -- Christophe

[HACKERS] Review: HStore Gin Speedup

2013-11-18 Thread Antonin Houska
Following are my initial comments for https://commitfest.postgresql.org/action/patch_view?id=1203 1. It no longer applies, probably due to commit a18167510f4c385329697588ce5132cbf95779c3 error: contrib/hstore/hstore--1.1.sql: No such file or directory 2. Compatibility with HStore v2.0

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-11-18 Thread Jim Mlodgenski
On Mon, Nov 18, 2013 at 7:25 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: The attached patches are the revised custom-scan APIs. My initial review on this feature: - The patches apply and build, but it produces a warning: ctidscan.c: In function ‘CTidInitCustomScanPlan’: ctidscan.c:362:9:

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 6:44 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I think it's important to recap the design goals of this. Seems reasonable to list them out. * It should be usable and perform well for both large batch updates and small transactions. I think that that's a

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
On Fri, Nov 15, 2013 at 2:42 PM, Alexander Korotkov aekorot...@gmail.comwrote: On Fri, Nov 15, 2013 at 11:39 PM, Rod Taylor rod.tay...@gmail.com wrote: The patched index is 58% of the 9.4 master size. 212 MB instead of 365 MB. Good. That's meet my expectations :) You mention that both

[HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread David Johnston
Bruce Momjian wrote Considering we are doing this outside of a transaction, and WARNING or ERROR is pretty much the same, from a behavioral perspective. Should we change this and LOCK to be a warning? From the calling application's perspective an error and a warning are definitely

[HACKERS] More legacy code: pg_ctl

2013-11-18 Thread Josh Berkus
Folks, Speaking of legacy code with bad default behaviors: pg_ctl. The current utility is designed to fathfully reproduce the rather hackish shell script we originally had for postgres startup. This results in a couple of unintuitive defaults which give sysadmins and config management

Re: [HACKERS] More legacy code: pg_ctl

2013-11-18 Thread Josh Berkus
On 11/18/2013 05:09 PM, Josh Berkus wrote: Folks, Speaking of legacy code with bad default behaviors: pg_ctl. The current utility is designed to fathfully reproduce the rather hackish shell script we originally had for postgres startup. This results in a couple of unintuitive defaults

Re: [HACKERS] [PATCH] configure: allow adding a custom string to PG_VERSION

2013-11-18 Thread Peter Eisentraut
On Tue, 2013-11-05 at 18:29 +0200, Oskari Saarenmaa wrote: This can be used to tag custom built packages with an extra version string such as the git describe id or distribution package release version. I think this is a reasonable feature, and the implementation is OK, but I don't see why the

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 4:37 PM, Peter Geoghegan p...@heroku.com wrote: You're right that the value locking is scary. I think we need to very carefully consider it, once I have buy-in on the basic approach. I really do think it's the least-worst approach described to date. It isn't like we

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/19 3:56), Peter Geoghegan wrote: On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao masao.fu...@gmail.com wrote: The same idea was proposed before but not committed because Itagaki thought that pg_stat_statements view should report only raw values. Please read the following thread. I have

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 05:05:45PM -0800, David Johnston wrote: Bruce Momjian wrote Considering we are doing this outside of a transaction, and WARNING or ERROR is pretty much the same, from a behavioral perspective. Should we change this and LOCK to be a warning? From the calling

Re: [HACKERS] [PATCH] configure: allow adding a custom string to PG_VERSION

2013-11-18 Thread Michael Paquier
On Tue, Nov 19, 2013 at 10:48 AM, Peter Eisentraut pete...@gmx.net wrote: I think this is a reasonable feature, and the implementation is OK, but I don't see why the format of the extra version information needs to be exactly PG_VERSION=$PACKAGE_VERSION ($withval) I'd imagine, for

[HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread David Johnston
Bruce Momjian wrote On Mon, Nov 18, 2013 at 05:05:45PM -0800, David Johnston wrote: Bruce Momjian wrote Considering we are doing this outside of a transaction, and WARNING or ERROR is pretty much the same, from a behavioral perspective. Should we change this and LOCK to be a warning?

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/19 11:12), KONDO Mitsumasa wrote: (2013/11/19 3:56), Peter Geoghegan wrote: On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao masao.fu...@gmail.com wrote: The same idea was proposed before but not committed because Itagaki thought that pg_stat_statements view should report only raw

[HACKERS] Remove use of obsolescent Autoconf macros

2013-11-18 Thread Peter Eisentraut
According to the Autoconf documentation, the following macros are obsolescent because no reasonably current system has the defect that they are testing for. - AC_C_CONST - AC_C_STRINGIZE - AC_C_VOLATILE - AC_FUNC_MEMCMP Therefore, I propose to remove their use, with the attached patch. From

Re: [HACKERS] Transaction-lifespan memory leak with plpgsql DO blocks

2013-11-18 Thread Dilip kumar
On 13 November 2013 03:17 David Johnston wrote, Having had this same thought WRT the FOR UPDATE in LOOP bug posting the lack of a listing of outstanding bugs does leave some gaps. I would imagine people would appreciate something like: Frequency: Rare Severity: Low Fix Complexity:

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 6:12 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I confirmed that Itagaki-san and Mr Cerdic disscution. He said that raw values be just simple. However, were his changes just simple? I cannot understand his aesthetics sense and also you, too:-( It's too

[HACKERS] Standalone synchronous master

2013-11-18 Thread Rajeev rastogi
This patch implements the following TODO item: Add a new eager synchronous mode that starts out synchronous but reverts to asynchronous after a failure timeout period This would require some type of command to be executed to alert administrators of this change.

Re: [HACKERS] Add \i option to bring in the specified file as a quoted literal

2013-11-18 Thread Piotr Marcinczyk
Dnia 2013-11-13, śro o godzinie 10:26 -0500, Bruce Momjian pisze: On Wed, Nov 13, 2013 at 08:58:07AM +0530, Amit Kapila wrote: On Tue, Nov 12, 2013 at 9:37 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Oct 23, 2013 at 10:31:39AM +0530, Amit Kapila wrote: On Tue, Oct 22, 2013 at 3:04

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
I checked out master and put together a test case using a small percentage of production data for a known problem we have with Pg 9.2 and text search scans. A small percentage in this case means 10 million records randomly selected; has a few billion records. Tests ran for master successfully

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
I tried again this morning using gin-packed-postinglists-16.patch and gin-fast-scan.6.patch. No crashes. It is about a 0.1% random sample of production data (10,000,000 records) with the below structure. Pg was compiled with debug enabled in both cases. Table public.kp Column | Type |

Re: [HACKERS] pg_upgrade: delete_old_cluster.sh issues

2013-11-18 Thread Bruce Momjian
On Tue, Nov 12, 2013 at 10:35:58AM +, Marc Mamin wrote: Hello, IMHO, there is a serious issue in the script to clean the old data directory when running pg_upgrade in link mode. in short: When working with symbolic links, the first step in delete_old_cluster.sh is to delete the old

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 06:30:32PM -0800, David Johnston wrote: Personally, I am fine with changing them all to WARNING. Error makes more sense if the goal is internal consistency. That goal should be subservient to backward compatibility. Changing LOCK to warning is less problematic

[HACKERS]

2013-11-18 Thread Rajeev rastogi
On 18 November 2013, Amit Khandekar wrote: On 18 October 2013 17:07, Rajeev rastogi rajeev.rast...@huawei.commailto:rajeev.rast...@huawei.com wrote: From the following mail, copy behaviour between stdin and normal file having some inconsistency.

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-18 Thread Rajeev rastogi
On 18 November 2013, Amit Khandekar wrote: On 18 October 2013 17:07, Rajeev rastogi rajeev.rast...@huawei.commailto:rajeev.rast...@huawei.com wrote: From the following mail, copy behaviour between stdin and normal file having some inconsistency.

Re: [HACKERS] LISTEN / NOTIFY enhancement request for Postgresql

2013-11-18 Thread Sev Zaslavsky
Thank you all for considering my feature request. Dimitri's suggestion is a very good one - I feel it will accomplish the goal of allowing more granularity in the Listen. We might also want to add a flag in postgresql.conf to disable this enhancement so that we don't break existing code.

  1   2   >