Re: [HACKERS] isolation check takes a long time

2012-07-24 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On fre, 2012-07-20 at 13:15 -0400, Andrew Dunstan wrote: Meanwhile, I would like to remove the prepared_transactions test from the main isolation schedule, and add a new Make target which runs that test explicitly. Is there any objection to that?

[HACKERS] Re: Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-24 Thread Greg Stark
On Wed, Jul 18, 2012 at 1:13 AM, Craig Ringer ring...@ringerc.id.au wrote: That makes me wonder if on top of the buildfarm, extending some buildfarm machines into a crashfarm is needed: - Keep kvm instances with copy-on-write snapshot disks and the build env on them - Fire up the VM, do a

Re: [HACKERS] isolation check takes a long time

2012-07-24 Thread Alvaro Herrera
Excerpts from Noah Misch's message of dom jul 22 17:11:53 -0400 2012: I was pondering something like this: setting i-rc isolation = READ COMMITTED setting i-rr isolation = REPEATABLE READ session s1 setup{ BEGIN TRANSACTION ISOLATION LEVEL :isolation; } step

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Mon, Jul 23, 2012 at 5:07 PM, Marko Kreen mark...@gmail.com wrote: Does PQgetRowData() break the ability to call PQgetvalue() against the result as well as other functions like PQgetisnull()? If so, I object. I don't get what are you objecting to. The PQgetRowData() is called instead of

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: I'm arguing that *all* data getting must continue to do so through the result object, and bypassing the result to get at data is breaking the result abstraction in the libpq api. That's a fair point, but the single-row mode without PQgetRowData still

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 10:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: I'm arguing that *all* data getting must continue to do so through the result object, and bypassing the result to get at data is breaking the result abstraction in the libpq api.

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 6:18 PM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Jul 23, 2012 at 5:07 PM, Marko Kreen mark...@gmail.com wrote: Does PQgetRowData() break the ability to call PQgetvalue() against the result as well as other functions like PQgetisnull()? If so, I object. I

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen mark...@gmail.com wrote: I'm arguing that *all* data getting must continue to do so through the result object, and bypassing the result to get at data is breaking the result abstraction in the libpq api. I bet you can still maintain data access

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:08 PM, Marko Kreen mark...@gmail.com wrote: The PQgetRowData() is unimportant as it just exposes the rowBuf to user and thats all. So to get back to the more interesting PQgetRowData() discussion... Did you notice that it's up to app to decide whether it calls

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:25 PM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen mark...@gmail.com wrote: I'm arguing that *all* data getting must continue to do so through the result object, and bypassing the result to get at data is breaking the result

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:39 AM, Marko Kreen mark...@gmail.com wrote: On Tue, Jul 24, 2012 at 7:25 PM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 11:08 AM, Marko Kreen mark...@gmail.com wrote: I'm arguing that *all* data getting must continue to do so through the result

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 7:52 PM, Merlin Moncure mmonc...@gmail.com wrote: But, the faster rowbuf method is a generally incompatible way of dealing with data vs current libpq -- this is bad. If it's truly impossible to get those benefits without bypassing result API that then I remove my

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: I think the dummy copy of PGresult is plausible (if by that you mean optimizing PQgetResult when in single row mode). That would be even better: you'd remove the need for the rowbuf mode. I haven't spent any time looking at this, but my gut tells me

Re: [HACKERS] Checkpointer split has broken things dramatically (was Re: DELETE vs TRUNCATE explanation)

2012-07-24 Thread karavelov
- Цитат от David Fetter (da...@fetter.org), на 23.07.2012 в 15:41 - I'm not sure how you automate testing a pull-the-plug scenario. I have a dim memory of how the FreeBSD project was alleged to have done it, namely by rigging a serial port (yes, it was that long ago) to the

Re: [HACKERS] Use of rsync for data directory copying

2012-07-24 Thread Kevin Grittner
Bruce Momjian br...@momjian.us wrote: if a write happens in both the first and second half of a second, While I'm not sure whether I believe that granularity is really to the nanosecond, a stat of a table in a production database on xfs shows this: Modify: 2012-07-24 10:15:44.096415501

[HACKERS] Recent absence

2012-07-24 Thread Kevin Grittner
I have recently been laid up with a MRSA infection, which left me suddenly unable to deal with even reading my email. (I took a shot at catching up a week ago, and it was too taxing, resulting in a relapse.) I'm trying to fight though the backlog and get caught up, without pushing it too hard

Re: [HACKERS] Recent absence

2012-07-24 Thread Joshua D. Drake
On 07/24/2012 10:26 AM, Kevin Grittner wrote: I have recently been laid up with a MRSA infection, which left me suddenly unable to deal with even reading my email. (I took a shot at catching up a week ago, and it was too taxing, resulting in a relapse.) I'm trying to fight though the backlog

Re: [HACKERS] Recent absence

2012-07-24 Thread Atri Sharma
On Tue, Jul 24, 2012 at 11:05 PM, Joshua D. Drake j...@commandprompt.com wrote: On 07/24/2012 10:26 AM, Kevin Grittner wrote: I have recently been laid up with a MRSA infection, which left me suddenly unable to deal with even reading my email. (I took a shot at catching up a week ago, and

[HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
I am running into a lot of customer situations where the customer reports that canceling autovacuum task shows up in the logs, and it's unclear whether this is happening often enough to matter, and even more unclear what's causing it. Me: So, do you know what table it's getting cancelled on?

Re: [HACKERS] 9.2 release schedule

2012-07-24 Thread Fujii Masao
On Tue, Jul 24, 2012 at 4:10 AM, Noah Misch n...@leadboat.com wrote: On Mon, Jul 23, 2012 at 10:29:06AM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Seems OK, but I think we need to work a little harder on evicting some things from the list of open items. I don't think

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Andres Freund
Hi, On Tuesday, July 24, 2012 07:48:27 PM Robert Haas wrote: I am running into a lot of customer situations where the customer reports that canceling autovacuum task shows up in the logs, and it's unclear whether this is happening often enough to matter, and even more unclear what's causing

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Steve Singer
On 12-07-24 01:48 PM, Robert Haas wrote: I am running into a lot of customer situations where the customer reports that canceling autovacuum task shows up in the logs, and it's unclear whether this is happening often enough to matter, and even more unclear what's causing it. Could autovacuum

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Tue, Jul 24, 2012 at 2:11 PM, Steve Singer ssin...@ca.afilias.info wrote: On 12-07-24 01:48 PM, Robert Haas wrote: I am running into a lot of customer situations where the customer reports that canceling autovacuum task shows up in the logs, and it's unclear whether this is happening often

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Andrew Dunstan
On 07/24/2012 01:48 PM, Robert Haas wrote: I am running into a lot of customer situations where the customer reports that canceling autovacuum task shows up in the logs, and it's unclear whether this is happening often enough to matter, and even more unclear what's causing it. Me: So, do you

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 11:57 AM, Marko Kreen mark...@gmail.com wrote: On Tue, Jul 24, 2012 at 7:52 PM, Merlin Moncure mmonc...@gmail.com wrote: But, the faster rowbuf method is a generally incompatible way of dealing with data vs current libpq -- this is bad. If it's truly impossible to get

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 1:33 PM, Merlin Moncure mmonc...@gmail.com wrote: The 'source' result (or source data that would be copied into the destination result) would be stored in the PGconn, right? So, the idea is that when you set up single row mode the connection generates a template PGconn

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 1:49 PM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 1:33 PM, Merlin Moncure mmonc...@gmail.com wrote: The 'source' result (or source data that would be copied into the destination result) would be stored in the PGconn, right? So, the idea is that

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Alvaro Herrera
On Tuesday, July 24, 2012 07:48:27 PM Robert Haas wrote: I am running into a lot of customer situations where the customer reports that canceling autovacuum task shows up in the logs, and it's unclear whether this is happening often enough to matter, and even more unclear what's causing

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of mar jul 24 15:30:49 -0400 2012: On Tuesday, July 24, 2012 07:48:27 PM Robert Haas wrote: I am running into a lot of customer situations where the customer reports that canceling autovacuum task shows up in the logs, and it's unclear whether

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: Either way, it looks like there's plausible paths to optimizing repeated result fetch without having expose an alternate data-fetching API and that the proposed implementation doesn't appear to be a wall in terms of getting there. So I'm firmly in the

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Tue, Jul 24, 2012 at 3:35 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Yep, it says: ERROR: canceling autovacuum task CONTEXT: automatic vacuum of table alvherre.public.foo So at least that part seems pilot error more than anything else. Yeah, you're right. So you do get the

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar jul 24 15:52:23 -0400 2012: On Tue, Jul 24, 2012 at 3:35 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Yep, it says: ERROR: canceling autovacuum task CONTEXT: automatic vacuum of table alvherre.public.foo So at least that part

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Tue, Jul 24, 2012 at 4:03 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of mar jul 24 15:52:23 -0400 2012: On Tue, Jul 24, 2012 at 3:35 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Yep, it says: ERROR: canceling autovacuum task

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Yeah, you're right. So you do get the table name. But you don't get the cause, which is what you really need to understand why it's happening. Attached is a patch that adds some more detail. Uh, what's the added dependency on pgstat.h for? Looks

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Tue, Jul 24, 2012 at 4:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Yeah, you're right. So you do get the table name. But you don't get the cause, which is what you really need to understand why it's happening. Attached is a patch that adds some

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
Hm, I think it's possible to rig the test to do dummy copy of pgresult, thus it's possible to see what kind of speed is possible.. Will try. I added a new method (-x) to rowdump where it asks for row with PQgetRowData() and then tries to emulate super-efficient PGresult copy, then loads data

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jul 24, 2012 at 4:03 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Looks great. Are you considering backpatching this? Well, that would certainly make MY life easier. I am not sure whether it would be in line with project policy,

Re: [HACKERS] CHECK NO INHERIT syntax

2012-07-24 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of sáb jul 21 00:20:57 -0400 2012: Here's a (hopefully) complete patch. Pushed. I hope people like this one better (third time's the charm, and all that). -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc.

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 2:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: Either way, it looks like there's plausible paths to optimizing repeated result fetch without having expose an alternate data-fetching API and that the proposed implementation doesn't

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 11:34 PM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 2:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: In particular I agree that PQsetSingleRowMode is a bit ugly, so I'm wondering about your thoughts on providing PQgetSingleRowResult instead. I don't

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Tue, Jul 24, 2012 at 10:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: Either way, it looks like there's plausible paths to optimizing repeated result fetch without having expose an alternate data-fetching API and that the proposed implementation doesn't

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 3:52 PM, Marko Kreen mark...@gmail.com wrote: * Is there better API than PQsetSingleRowMode()? New PQsend* functions is my alternative. I would prefer to have PQsetSingleRowMode() over new flavor of PQsend. To consolidate my argument above: since you're throwing

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Robert Haas
On Jul 24, 2012, at 4:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jul 24, 2012 at 4:03 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Looks great. Are you considering backpatching this? Well, that would certainly make MY life easier. I

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Wed, Jul 25, 2012 at 12:35 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 3:52 PM, Marko Kreen mark...@gmail.com wrote: * Is there better API than PQsetSingleRowMode()? New PQsend* functions is my alternative. I would prefer to have PQsetSingleRowMode() over new

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen mark...@gmail.com wrote: So if we give only PQgetResult() in 9.2, I do not see that we are locked out from any interesting optimizations. Well, you are locked out of having PQgetResult reuse the conn's result since that would then introduce

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tue, Jul 24, 2012 at 5:29 PM, Merlin Moncure mmonc...@gmail.com wrote: so #2 seems like the lowest common denominator (it would permanently preclude #3 and would require #4 to introduce two new functions instead of just one). #1 of course would bolt on to #2. oops, got #1 and #2 backwards

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Marko Kreen
On Wed, Jul 25, 2012 at 1:29 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen mark...@gmail.com wrote: So if we give only PQgetResult() in 9.2, I do not see that we are locked out from any interesting optimizations. Well, you are locked out of having

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-24 Thread Merlin Moncure
On Tuesday, July 24, 2012, Marko Kreen mark...@gmail.com wrote: On Wed, Jul 25, 2012 at 1:29 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Jul 24, 2012 at 4:59 PM, Marko Kreen mark...@gmail.com wrote: So if we give only PQgetResult() in 9.2, I do not see that we are locked out from any

Re: [HACKERS] [BUGS] BUG #6748: sequence value may be conflict in some cases

2012-07-24 Thread Tom Lane
I wrote: The tuple in buffers has log_cnt = 1, is_called = false, but the initial XLOG_SEQ_LOG record shows log_cnt = 0, is_called = true. So if we crash at this point, after recovery it looks like one nextval() has already been done. However, AlterSequence generates another XLOG_SEQ_LOG

Re: [HACKERS] canceling autovacuum task woes

2012-07-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Jul 24, 2012, at 4:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... This means that with respect to (a), the connection from the process doing the kill to the AV proc may be inadequately documented by this patch, and with respect to (b), there might