Re: [HACKERS] recovery_target_time and standby_mode

2014-11-12 Thread Josh Berkus
On 11/12/2014 10:06 AM, Robert Haas wrote: >> hat *appears* to be happening is that the pause_at_recovery_target, >> > followed by the restart, on the replica causes it to advance one commit >> > on timeline 1. But *not all the time*; this doesn't happen in my >> > pgbench-based tests. >> > >> > T

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-11-12 Thread Amit Kapila
On Mon, Oct 27, 2014 at 5:26 PM, Amit Kapila wrote: > > > Going further with verification of this patch, I found below issue: > Run the testcase.sql file at below link: > http://www.postgresql.org/message-id/4205e661176a124faf891e0a6ba9135266347...@szxeml509-mbs.china.huawei.com > ./vacuumdb --ana

Re: [HACKERS] On partitioning

2014-11-12 Thread Amit Langote
> ow...@postgresql.org] On Behalf Of Amit Langote > Sent: Thursday, November 13, 2014 3:50 PM > > Greenplum uses a single table for this purpose with separate columns for range > and list cases, for example. They store allowed values per partition though. > They have 6 partitioning related catalo

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-12 Thread Jeff Davis
On Wed, 2014-11-12 at 14:16 -0500, Robert Haas wrote: > Detected deadlocks are fine. Improving the deadlock detector is the > heart of what needs to be done here. OK, great. > As you say, the lock requests > we're talking about will rarely wait, so deadlocks won't be frequent. > The issue is mak

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-12 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > src/include/rewrite/rowsecurity.h, which one would > reasonably think to be a rewriter header (nevermind its header comment > to the contrary), nonetheless includes execnodes.h (executor stuff) I'll fix the header comment. The include of execnodes.h was a

Re: [HACKERS] On partitioning

2014-11-12 Thread Amit Langote
> From: Stephen Frost [mailto:sfr...@snowman.net] > Sent: Thursday, November 13, 2014 3:40 PM > > > The point for me is just that range and list partitioning probably > > need different structure, and hash partitioning, if we want to support > > that, needs something else again. Range partitioni

Re: [HACKERS] inherit support for foreign tables

2014-11-12 Thread Ashutosh Bapat
On Thu, Nov 13, 2014 at 12:20 PM, Etsuro Fujita wrote: > Hi Ashutosh, > > Thanks for the review! > > (2014/11/13 15:23), Ashutosh Bapat wrote: > >> I tried to apply fdw-inh-3.patch on the latest head from master branch. >> It failed to apply using both patch and git apply. >> >> "patch" failed to

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-12 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Tom Lane wrote: > > I noticed that the recent custom-path commit completely ignored my > > advice about not including executor headers into planner headers or > > vice versa. On the way to fixing that, I was dismayed to discover > > that the RLS

Re: [HACKERS] inherit support for foreign tables

2014-11-12 Thread Etsuro Fujita
Hi Ashutosh, Thanks for the review! (2014/11/13 15:23), Ashutosh Bapat wrote: I tried to apply fdw-inh-3.patch on the latest head from master branch. It failed to apply using both patch and git apply. "patch" failed to apply because of rejections in contrib/file_fdw/output/file_fdw.source and

Re: [HACKERS] On partitioning

2014-11-12 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Nov 12, 2014 at 5:06 PM, Tom Lane wrote: > > Robert Haas writes: > >> Maybe as anyarray, but I think pg_node_tree > >> might even be better. That can also represent data of some arbitrary > >> type, but it doesn't enforce that everything is

Re: [HACKERS] inherit support for foreign tables

2014-11-12 Thread Ashutosh Bapat
Hi Fujita-san, I tried to apply fdw-inh-3.patch on the latest head from master branch. It failed to apply using both patch and git apply. "patch" failed to apply because of rejections in contrib/file_fdw/output/file_fdw.source and doc/src/sgml/ref/create_foreign_table.sgml On Fri, Nov 7, 2014 at

Re: [HACKERS] tracking commit timestamps

2014-11-12 Thread Michael Paquier
On Wed, Nov 12, 2014 at 10:06 PM, Petr Jelinek wrote: > Brief list of changes: > - the commit timestamp record now stores timestamp, lsn and nodeid Now that not only the commit timestamp is stored, calling that "commit timestamp", "committs" or "commit_timestamp" is strange, no? If this patch is

Re: [HACKERS] Race in "tablespace" test on Windows

2014-11-12 Thread Amit Kapila
On Thu, Nov 13, 2014 at 8:46 AM, Noah Misch wrote: > > On Tue, Nov 11, 2014 at 10:21:26AM +0530, Amit Kapila wrote: > > On Sat, Nov 8, 2014 at 10:34 AM, Noah Misch wrote: > > > Here is a briefer command sequence exhibiting the same problem: > > > > > > To make this work as well on Windows as it d

Re: [HACKERS] tracking commit timestamps

2014-11-12 Thread Michael Paquier
On Thu, Nov 13, 2014 at 7:56 AM, Jim Nasby wrote: > On 11/12/14, 7:06 AM, Petr Jelinek wrote: >> >> - if the xid passed to get interface is out of range -infinity timestamp >> is returned (I think it's bad idea to throw errors here as the valid range >> is not static and same ID can start throwi

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Michael Paquier
On Tue, Nov 11, 2014 at 3:52 PM, Peter Geoghegan wrote: > I'm pretty puzzled by this. Other than our "agree to disagree and > defer to committer" position on the question of whether or not more > than one suggestion can come from a single RTE, which you were fine > with before [1], I have only res

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-11-12 Thread Fujii Masao
Thanks for reviewing the patch! On Thu, Nov 13, 2014 at 4:05 AM, Alvaro Herrera wrote: > Fujii Masao wrote: > >> --- 127,152 >>When this option is used, pg_receivexlog will >> report >>a flush position to the server, indicating when each segment has >> been >>

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-11-12 Thread Fujii Masao
On Wed, Nov 12, 2014 at 9:30 PM, Fujii Masao wrote: > On Wed, Nov 12, 2014 at 12:40 AM, Tom Lane wrote: >> Fujii Masao writes: >>> On Tue, Nov 11, 2014 at 10:14 PM, Robert Haas wrote: Not to kibitz too much after-the-fact, but wouldn't it be better to give this a name that has "GIN" i

Re: [HACKERS] Race in "tablespace" test on Windows

2014-11-12 Thread Noah Misch
On Tue, Nov 11, 2014 at 10:21:26AM +0530, Amit Kapila wrote: > On Sat, Nov 8, 2014 at 10:34 AM, Noah Misch wrote: > > Here is a briefer command sequence exhibiting the same problem: > > > > CREATE TABLESPACE testspace LOCATION '...somewhere...'; > > CREATE TABLE atable (c int) tablespace testspace

Re: [HACKERS] pg_multixact not getting truncated

2014-11-12 Thread Alvaro Herrera
Jim Nasby wrote: > On 11/10/14, 12:16 AM, Josh Berkus wrote: > >On 11/09/2014 08:00 PM, Josh Berkus wrote: > >On 11/08/2014 01:46 PM, Andres Freund wrote: > >>>I'm these days suggesting that people should add manual vacuuming for > "older" relations during off peak hours on busy databases. Ther

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-12 Thread Alvaro Herrera
Andreas Karlsson wrote: > On 11/13/2014 02:03 AM, Andreas Karlsson wrote: > >Here is version 2 of the patch which detects the presence of gcc/clang > >style 128-bit integers and has been cleaned up to a reviewable state. I > >have not added support for any other compilers since I found no > >docume

Re: [HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-12 Thread Alvaro Herrera
Tom Lane wrote: > I noticed that the recent custom-path commit completely ignored my > advice about not including executor headers into planner headers or > vice versa. On the way to fixing that, I was dismayed to discover > that the RLS patch has utterly bollixed all semblance of modularization >

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-12 Thread Andreas Karlsson
On 11/13/2014 02:03 AM, Andreas Karlsson wrote: Here is version 2 of the patch which detects the presence of gcc/clang style 128-bit integers and has been cleaned up to a reviewable state. I have not added support for any other compilers since I found no documentation 128-bit support with icc or

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Michael Paquier
On Thu, Nov 13, 2014 at 10:26 AM, Michael Paquier wrote: > On Thu, Nov 13, 2014 at 9:31 AM, Andres Freund wrote: >> I don't recall what the problem with just swapping the names was - but >> I'm pretty sure there was one... Hm. The index relation oids are >> referred to by constraints and dependen

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Michael Paquier
On Thu, Nov 13, 2014 at 9:31 AM, Andres Freund wrote: > I don't recall what the problem with just swapping the names was - but > I'm pretty sure there was one... Hm. The index relation oids are > referred to by constraints and dependencies. That's somewhat > solvable. But I think there was somethi

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-11-12 18:23:38 -0500, Robert Haas wrote: > > > The problem is that it's very hard to avoid the wrong index's > > > relfilenode being used when swapping the relfilenodes between two > > > indexes. > > > > How about storing both the old and new relfilenodes in the sam

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-11-12 Thread Andreas Karlsson
Hi, Here is version 2 of the patch which detects the presence of gcc/clang style 128-bit integers and has been cleaned up to a reviewable state. I have not added support for any other compilers since I found no documentation 128-bit support with icc or MSVC. I do not have access to any Window

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Peter Geoghegan
On Wed, Nov 12, 2014 at 4:54 PM, Peter Geoghegan wrote: > Attached patch moves the Levenshtein distance implementation into core. Oops. Somehow managed to send a *.patch.swp file. :-) Here is the actual patch. -- Peter Geoghegan From b7df918f1a52107637600f3b22d1cff18bd07ae1 Mon Sep 17 00:00:0

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Peter Geoghegan
On Wed, Nov 12, 2014 at 1:13 PM, Peter Geoghegan wrote: > On Wed, Nov 12, 2014 at 12:59 PM, Robert Haas wrote: >> I agree with your proposed approach to moving Levenshtein into core. >> However, I think this should be separated into two patches, one of >> them moving the Levenshtein functionality

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Andres Freund
On 2014-11-12 18:23:38 -0500, Robert Haas wrote: > On Wed, Nov 12, 2014 at 4:39 PM, Andres Freund wrote: > > On 2014-11-12 16:11:58 -0500, Robert Haas wrote: > >> On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas wrote: > >> > On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut wrote: > >> >> If REINDEX

[HACKERS] On the warpath again about ill-considered inclusion nests

2014-11-12 Thread Tom Lane
I noticed that the recent custom-path commit completely ignored my advice about not including executor headers into planner headers or vice versa. On the way to fixing that, I was dismayed to discover that the RLS patch has utterly bollixed all semblance of modularization of the headers. src/incl

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-12 Thread Michael Paquier
On Wed, Nov 12, 2014 at 9:49 PM, Robert Haas wrote: > On Tue, Nov 11, 2014 at 7:48 PM, Kouhei Kaigai wrote: >> Isn't provolatile = PROVOLATILE_IMMUTABLE sufficient? > > There are certainly things that are parallel-safe that are not > immutable. It might be the case that everything immutable is >

Re: [HACKERS] Teaching pg_dump to use NOT VALID constraints

2014-11-12 Thread Jim Nasby
On 11/10/14, 12:00 PM, Simon Riggs wrote: On 10 November 2014 17:33, Alvaro Herrera wrote: pg_dump --no-revalidaton will add "NOT VALID" onto the recreation SQL for any FKs, but only for ones that were already known to be valid. Well. Constraints that haven't been validated already have

Re: [HACKERS] pg_multixact not getting truncated

2014-11-12 Thread Jim Nasby
On 11/10/14, 12:16 AM, Josh Berkus wrote: On 11/09/2014 08:00 PM, Josh Berkus wrote: On 11/08/2014 01:46 PM, Andres Freund wrote: I'm these days suggesting that people should add manual vacuuming for "older" relations during off peak hours on busy databases. There's too many sites which service

Re: [HACKERS] On partitioning

2014-11-12 Thread Jim Nasby
On 11/12/14, 5:27 PM, Robert Haas wrote: Maybe as anyarray, but I think pg_node_tree >>might even be better. That can also represent data of some arbitrary >>type, but it doesn't enforce that everything is uniform. > >Of course, the more general you make it, the more likely that it'll be >impos

Re: [HACKERS] Unintended restart after recovery error

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 4:52 PM, Antonin Houska wrote: > Fujii Masao wrote: > >> On Wed, Nov 12, 2014 at 6:52 PM, Antonin Houska wrote: >> > While looking at postmaster.c:reaper(), one problematic case occurred to >> > me. >> > >> > >> > 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. >>

Re: [HACKERS] On partitioning

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 5:06 PM, Tom Lane wrote: > Robert Haas writes: >> I thought putting the partition boundaries into pg_inherits was a >> strange choice. I'd put it in pg_class, or in pg_partition if we >> decide to create that. > > Yeah. I rather doubt that we want this mechanism to be ve

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 4:39 PM, Andres Freund wrote: > On 2014-11-12 16:11:58 -0500, Robert Haas wrote: >> On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas wrote: >> > On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut wrote: >> >> If REINDEX cannot work without an exclusive lock, we should invent so

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Jim Nasby
On 11/12/14, 9:47 AM, Tom Lane wrote: Heikki Linnakangas writes: On 11/12/2014 05:20 PM, Tom Lane wrote: On reconsideration I think the "RBM_ZERO returns page already locked" alternative may be the less ugly. That has the advantage that any code that doesn't get updated will fail clearly and

Re: [HACKERS] tracking commit timestamps

2014-11-12 Thread Jim Nasby
On 11/12/14, 7:06 AM, Petr Jelinek wrote: - if the xid passed to get interface is out of range -infinity timestamp is returned (I think it's bad idea to throw errors here as the valid range is not static and same ID can start throwing errors between calls theoretically) Wouldn't NULL be mor

Re: [HACKERS] On partitioning

2014-11-12 Thread Tom Lane
Robert Haas writes: > I thought putting the partition boundaries into pg_inherits was a > strange choice. I'd put it in pg_class, or in pg_partition if we > decide to create that. Yeah. I rather doubt that we want this mechanism to be very closely tied to the existing inheritance features. If

Re: [HACKERS] Add CREATE support to event triggers

2014-11-12 Thread Andres Freund
On 2014-11-12 16:36:30 -0500, Robert Haas wrote: > On Mon, Nov 10, 2014 at 9:02 PM, Jim Nasby wrote: > > +1. Adding columns is a PITA, you have to manually ensure you do it on all > > slaves first. > > > > Drop is somewhat worse, because you have to do it on the master first, > > opposite of the (

Re: [HACKERS] Unintended restart after recovery error

2014-11-12 Thread Antonin Houska
Fujii Masao wrote: > On Wed, Nov 12, 2014 at 6:52 PM, Antonin Houska wrote: > > While looking at postmaster.c:reaper(), one problematic case occurred to me. > > > > > > 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. > > > > 2. Checkpointer process is forked and immediately dies. > > > > 3

Re: [HACKERS] On partitioning

2014-11-12 Thread Robert Haas
On Mon, Nov 10, 2014 at 8:53 PM, Amit Langote wrote: > Above commands are merely transformed into ALTER TABLE subcommands that > arrange > partitioned table and partitions into inheritance hierarchy, but with extra > information, that is, allowed values for the partition in a new anyarray > colu

Re: [HACKERS] Reverse Engineering - search constraints are not explicitly stated in the tables from the VIEW

2014-11-12 Thread Tom Lane
nill writes: > I am analyzing query plans generated by the view in the database PostgreSQL > 8.3, looking for missing information "constraints not explicitly registrants > in the tables." You realize of course that 8.3 is nearly 7 years old and has been out of support for awhile. > In the analys

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Andres Freund
On 2014-11-12 16:11:58 -0500, Robert Haas wrote: > On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas wrote: > > On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut wrote: > >> If REINDEX cannot work without an exclusive lock, we should invent some > >> other qualifier, like WITH FEWER LOCKS. > > > > What

Re: [HACKERS] Add CREATE support to event triggers

2014-11-12 Thread Robert Haas
On Mon, Nov 10, 2014 at 9:02 PM, Jim Nasby wrote: > +1. Adding columns is a PITA, you have to manually ensure you do it on all > slaves first. > > Drop is somewhat worse, because you have to do it on the master first, > opposite of the (more usual) case of adding a column. > > RENAME is a complete

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Peter Geoghegan
On Wed, Nov 12, 2014 at 12:59 PM, Robert Haas wrote: > I agree with your proposed approach to moving Levenshtein into core. > However, I think this should be separated into two patches, one of > them moving the Levenshtein functionality into core, and the other > adding the new treatment for missi

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas wrote: > On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut wrote: >> If REINDEX cannot work without an exclusive lock, we should invent some >> other qualifier, like WITH FEWER LOCKS. > > What he said. But more to the point why, precisely, can't t

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-12 Thread Robert Haas
On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut wrote: > If REINDEX cannot work without an exclusive lock, we should invent some > other qualifier, like WITH FEWER LOCKS. What he said. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgs

Re: [HACKERS] Error building the EnterpriseDB mysql_fdw on OSX

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 3:48 PM, Kirk Roybal wrote: > I'm running 10.9.5 of OSX. > > I got the MySQL and PostgreSQL dependencies installed (I think). > > Checked out the git repo for mysql_fdw from > git://github.com/EnterpriseDB/mysql_fdw > > USE_PGXS=1 make > > and got the error: > > mysql_fdw.

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-12 Thread Robert Haas
On Sun, Nov 9, 2014 at 11:48 PM, Peter Geoghegan wrote: > On Fri, Nov 7, 2014 at 12:57 PM, Robert Haas wrote: >> Based on this review from a month ago, I'm going to mark this Waiting >> on Author. If nobody updates the patch in a few days, I'll mark it >> Returned with Feedback. Thanks. > > Att

[HACKERS] Error building the EnterpriseDB mysql_fdw on OSX

2014-11-12 Thread Kirk Roybal
I'm running 10.9.5 of OSX. I got the MySQL and PostgreSQL dependencies installed (I think). Checked out the git repo for mysql_fdw from git://github.com/EnterpriseDB/mysql_fdw USE_PGXS=1 make and got the error: mysql_fdw.c mysql_fdw.c:153:56: error: use of undeclared identifier 'RTLD_

Re: [HACKERS] pg_prewarm really needs some CHECK_FOR_INTERRUPTS

2014-11-12 Thread Andres Freund
On 2014-11-11 12:17:11 +0100, Andres Freund wrote: > pg_prewarm() currently can't be cannot be interrupted - which seems odd > given that it's intended to read large amounts of data from disk. A > rather slow process. > > Unless somebody protests I'm going to add a check to the top of each of > th

[HACKERS] Reverse Engineering - search constraints are not explicitly stated in the tables from the VIEW

2014-11-12 Thread nill
I am analyzing query plans generated by the view in the database PostgreSQL 8.3, looking for missing information "constraints not explicitly registrants in the tables." In nested queries, (ex. IN clause, ...), the query plan consist in the evaluation of the subplane derived from clause (SELECT * ..

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 2:57 AM, Jeff Davis wrote: > Trying to catch up on this thread, please excuse me if these questions > are already covered. Welcome to the party. The more, the merrier. :-) > You mention the possibility of undetected deadlocks, which is surely > unacceptable. But why not

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-11-12 Thread Alvaro Herrera
Fujii Masao wrote: > --- 127,152 >When this option is used, pg_receivexlog will > report >a flush position to the server, indicating when each segment has > been >synchronized to disk so that the server can remove that segment if > it > ! is not

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 12:12 PM, Josh Berkus wrote: > On 11/07/2014 02:03 PM, Josh Berkus wrote: >> But, like I said, there's a serviceable workaround. > > Some update on this. We've seen a problem in production with this setup > which I can't reproduce as a test case, but which may jog Heikki's

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 12:47 PM, Andres Freund wrote: >> >> > So maybe 'Encountered xl_running_xacts record with xcnt = 0.'? >> >> >> >> That's not very user-facing, is it -- I mean, why bother the user with >> >> the names of structs and members thereof? It seems better to describe >> >> what t

Re: [HACKERS] alter user set local_preload_libraries.

2014-11-12 Thread Fujii Masao
On Wed, Nov 5, 2014 at 1:22 AM, Peter Eisentraut wrote: > On 10/9/14 1:58 PM, Fujii Masao wrote: >> Also I think that it's useful to allow ALTER ROLE/DATABASE SET to >> set PGC_BACKEND and PGC_SU_BACKEND parameters. So, what >> about applying the attached patch? This patch allows that and >> chang

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 12:37 PM, Andres Freund wrote: > Stop overdesigning this. > > Add it to the existing mesage and let us be done with this. This thread > has already wasted far too much time. That's a little harsh, but I agree. Producing a warning here is just going to be log-spam. We've

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-12 Thread Andres Freund
On 2014-11-12 12:40:41 -0500, Robert Haas wrote: > On Wed, Nov 12, 2014 at 10:59 AM, Andres Freund > wrote: > > On 2014-11-12 11:56:01 -0300, Alvaro Herrera wrote: > >> Andres Freund wrote: > >> > Hi, > >> > > >> > On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: > >> > > Could someone trans

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 10:59 AM, Andres Freund wrote: > On 2014-11-12 11:56:01 -0300, Alvaro Herrera wrote: >> Andres Freund wrote: >> > Hi, >> > >> > On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: >> > > Could someone translate this detail message to English: >> > > >> > > ereport

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 11:36 AM, Robert Haas wrote: > On Wed, Nov 12, 2014 at 11:19 AM, Andres Freund > wrote: >> The question is whether the library is actually loaded in that case? >> Because that normally only happens early during startup - which is why >> it's a PGC_BACKEND guc. > > It look

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-12 Thread Andres Freund
On 2014-11-12 11:34:04 -0600, Jim Nasby wrote: > On 11/11/14, 2:01 AM, Andres Freund wrote: > >On 2014-11-10 19:36:18 -0600, Jim Nasby wrote: > >>Towards that simple end, I'm a bit torn. My preference would be to > >>simply log, and throw a warning if it's over some threshold. I believe > >>that wo

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-12 Thread Jim Nasby
On 11/11/14, 2:01 AM, Andres Freund wrote: On 2014-11-10 19:36:18 -0600, Jim Nasby wrote: On 11/10/14, 12:56 PM, Andres Freund wrote: On 2014-11-10 12:37:29 -0600, Jim Nasby wrote: On 11/10/14, 12:15 PM, Andres Freund wrote: If what we want is to quantify the extent of the issue, would it be

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Andres Freund
On 2014-11-12 11:36:14 -0500, Robert Haas wrote: > On Wed, Nov 12, 2014 at 11:19 AM, Andres Freund > wrote: > > The question is whether the library is actually loaded in that case? > > Because that normally only happens early during startup - which is why > > it's a PGC_BACKEND guc. > > It looks

Re: [HACKERS] recovery_target_time and standby_mode

2014-11-12 Thread Josh Berkus
On 11/07/2014 02:03 PM, Josh Berkus wrote: > But, like I said, there's a serviceable workaround. Some update on this. We've seen a problem in production with this setup which I can't reproduce as a test case, but which may jog Heikki's memory for something to fix. 1. Recover master to 2014-11-10

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-11-12 Thread Fujii Masao
On Mon, Nov 10, 2014 at 7:19 PM, wrote: >> On Fri, Oct 31, 2014 at 5:46 PM, wrote: >> >> > We seem to be going in circles. You suggested having two options, >> >> > --feedback, and --fsync, which is almost exactly what Furuya posted >> >> > originally. I objected to that, because I think that u

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 11:19 AM, Andres Freund wrote: > The question is whether the library is actually loaded in that case? > Because that normally only happens early during startup - which is why > it's a PGC_BACKEND guc. It looks like that does not work. [rhaas pgsql]$ PGOPTIONS='-c local_pr

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Andres Freund
On 2014-11-12 11:15:26 -0500, Robert Haas wrote: > On Mon, Nov 10, 2014 at 4:58 PM, Andres Freund wrote: > > No, that's not what I was thinking of. Consider: > > > > export pg_libdir=$(pg_config --libdir) > > mkdir $pg_libdir/plugins > > ln -s $pg_libdir/auto_explain.so $pg_libdir/plugins/ > > PG_

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-12 Thread Robert Haas
On Mon, Nov 10, 2014 at 4:58 PM, Andres Freund wrote: > No, that's not what I was thinking of. Consider: > > export pg_libdir=$(pg_config --libdir) > mkdir $pg_libdir/plugins > ln -s $pg_libdir/auto_explain.so $pg_libdir/plugins/ > PG_OPTIONS='-c local_preload_libraries=auto_explain' psql > > and

Re: [HACKERS] Unintended restart after recovery error

2014-11-12 Thread Fujii Masao
On Wed, Nov 12, 2014 at 6:52 PM, Antonin Houska wrote: > While looking at postmaster.c:reaper(), one problematic case occurred to me. > > > 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. > > 2. Checkpointer process is forked and immediately dies. > > 3. reaper() catches this failure, calls

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-12 Thread Andres Freund
On 2014-11-12 11:56:01 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi, > > > > On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: > > > Could someone translate this detail message to English: > > > > > > ereport(LOG, > > > (errmsg("logical decoding found con

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Tom Lane
Heikki Linnakangas writes: > On 11/12/2014 05:20 PM, Tom Lane wrote: >> On reconsideration I think the "RBM_ZERO returns page already locked" >> alternative may be the less ugly. That has the advantage that any code >> that doesn't get updated will fail clearly and reliably. > Yeah, I'm leaning

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Heikki Linnakangas
On 11/12/2014 05:20 PM, Tom Lane wrote: Heikki Linnakangas writes: On 11/12/2014 04:56 PM, Tom Lane wrote: Not great either. What about an RBM_NOERROR mode that is like RBM_ZERO in terms of handling error conditions, but does not forcibly zero the page if it's already valid? Anyway, you do

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Heikki Linnakangas
On 11/12/2014 05:08 PM, Robert Haas wrote: On Wed, Nov 12, 2014 at 7:39 AM, Heikki Linnakangas wrote: 2. When ReadBufferExtended doesn't find the page in cache, it returns the buffer in !BM_VALID state (i.e. still in I/O in-progress state). Require the caller to call a second function, after lo

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Tom Lane
Heikki Linnakangas writes: > On 11/12/2014 04:56 PM, Tom Lane wrote: >> Not great either. What about an RBM_NOERROR mode that is like RBM_ZERO >> in terms of handling error conditions, but does not forcibly zero the page >> if it's already valid? > Anyway, you don't want to read the page from di

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-11-12 Thread Andres Freund
On 2014-11-12 10:13:18 -0500, Robert Haas wrote: > On Tue, Nov 11, 2014 at 4:27 AM, Andres Freund wrote: > > The more important thing here is that I see little chance of this > > getting in before Heikki's larger rework of the wal format gets > > in. Since that'll change everything around anyay I'

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Heikki Linnakangas
On 11/12/2014 04:56 PM, Tom Lane wrote: Heikki Linnakangas writes: There's a race condition between a backend running queries in hot standby mode, and restoring a full-page image from a WAL record. It's present in all supported versions. I can think of two ways to fix this: 1. Have ReadBu

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-11-12 Thread Robert Haas
On Tue, Nov 11, 2014 at 4:27 AM, Andres Freund wrote: > The more important thing here is that I see little chance of this > getting in before Heikki's larger rework of the wal format gets > in. Since that'll change everything around anyay I'm unsure how much > point there is to iterate till that's

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Robert Haas
On Wed, Nov 12, 2014 at 7:39 AM, Heikki Linnakangas wrote: > 2. When ReadBufferExtended doesn't find the page in cache, it returns the > buffer in !BM_VALID state (i.e. still in I/O in-progress state). Require the > caller to call a second function, after locking the page, to finish the I/O. This

Re: [HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Tom Lane
Heikki Linnakangas writes: > There's a race condition between a backend running queries in hot > standby mode, and restoring a full-page image from a WAL record. It's > present in all supported versions. > I can think of two ways to fix this: > 1. Have ReadBufferExtended lock the page in RBM_Z

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-12 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: > > Could someone translate this detail message to English: > > > > ereport(LOG, > > (errmsg("logical decoding found consistent point at %X/%X", > > (uint32) (lsn >

Re: [HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-12 Thread Andres Freund
Hi, On 2014-11-12 09:03:40 -0500, Peter Eisentraut wrote: > Could someone translate this detail message to English: > > ereport(LOG, > (errmsg("logical decoding found consistent point at %X/%X", > (uint32) (lsn >> 32), (uint32) lsn), >

[HACKERS] what does this mean: "running xacts with xcnt == 0"

2014-11-12 Thread Peter Eisentraut
Could someone translate this detail message to English: ereport(LOG, (errmsg("logical decoding found consistent point at %X/%X", (uint32) (lsn >> 32), (uint32) lsn), errdetail("running xacts with xcnt == 0"))); (or downgrade to debu

Re: [HACKERS] Failback to old master

2014-11-12 Thread Ants Aasma
On Tue, Nov 11, 2014 at 11:52 PM, Maeldron T. wrote: > As far as I remember (I can’t test it right now but I am 99% sure) promoting > the slave makes it impossible to connect the old master to the new one > without making a base_backup. The reason is the timeline change. It complains. A safely

Re: [HACKERS] tracking commit timestamps

2014-11-12 Thread Petr Jelinek
On 10/11/14 14:53, Robert Haas wrote: On Mon, Nov 10, 2014 at 8:39 AM, Petr Jelinek wrote: I did the calculation above wrong btw, it's actually 20 bytes not 24 bytes per record, I am inclined to just say we can live with that. If you do it as 20 bytes, you'll have to do some work to squeeze o

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-12 Thread Robert Haas
On Tue, Nov 11, 2014 at 7:48 PM, Kouhei Kaigai wrote: > Isn't provolatile = PROVOLATILE_IMMUTABLE sufficient? There are certainly things that are parallel-safe that are not immutable. It might be the case that everything immutable is parallel-safe. -- Robert Haas EnterpriseDB: http://www.enter

[HACKERS] Race condition between hot standby and restoring a FPW

2014-11-12 Thread Heikki Linnakangas
There's a race condition between a backend running queries in hot standby mode, and restoring a full-page image from a WAL record. It's present in all supported versions. RestoreBackupBlockContents does this: buffer = XLogReadBufferExtended(bkpb.node, bkpb.fork, bkpb.block,

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-11-12 Thread Fujii Masao
On Wed, Nov 12, 2014 at 12:40 AM, Tom Lane wrote: > Fujii Masao writes: >> On Tue, Nov 11, 2014 at 10:14 PM, Robert Haas wrote: >>> Not to kibitz too much after-the-fact, but wouldn't it be better to >>> give this a name that has "GIN" in it somewhere? > >> Maybe. gin_pending_list_cleanup_size?

[HACKERS] how to determine clause for new keyword, how to add new clause in gram.y

2014-11-12 Thread Pankaj Bagul
Please tell me, if want to add 'selectivity' keyword, which clause will it categorize ? Can you please tell me in detail, how should i modify gram.y, as I am new to parser writing ? and use it thereafter. Thank you. -- Pankaj A. Bagul

Re: [HACKERS] inherit support for foreign tables

2014-11-12 Thread Ashutosh Bapat
Hi Fujita-san, I reviewed fdw-chk-3 patch. Here are my comments Sanity 1. The patch applies on the latest master using "patch but not by git apply 2. it compiles clean 3. Regression run is clean, including the contrib module regressions Tests --- 1. The tests added in file_fdw module

Re: [HACKERS] Unintended restart after recovery error

2014-11-12 Thread Antonin Houska
Antonin Houska wrote: > While looking at postmaster.c:reaper(), one problematic case occurred to me. > > > 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. > > 2. Checkpointer process is forked and immediately dies. > > 3. reaper() catches this failure, calls HandleChildCrash() and thus

[HACKERS] Unintended restart after recovery error

2014-11-12 Thread Antonin Houska
While looking at postmaster.c:reaper(), one problematic case occurred to me. 1. Startup process signals PMSIGNAL_RECOVERY_STARTED. 2. Checkpointer process is forked and immediately dies. 3. reaper() catches this failure, calls HandleChildCrash() and thus sets FatalError to true. 4. Startup pro

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-12 Thread Atri Sharma
On Wed, Nov 12, 2014 at 1:24 PM, David Rowley wrote: > > On Tue, Nov 11, 2014 at 9:29 PM, Simon Riggs > wrote: > >> >> This plan type is widely used in reporting queries, so will hit the >> mainline of BI applications and many Mat View creations. >> This will allow SELECT count(*) FROM foo to go