Re: [HACKERS] [PoC] Asynchronous execution again (which is not parallel)

2015-12-11 Thread Amit Kapila
On Tue, Dec 8, 2015 at 9:10 PM, Robert Haas wrote: > > On Mon, Nov 30, 2015 at 7:47 AM, Kyotaro HORIGUCHI > wrote: > > But is it important enough to be worthwhile? Maybe, maybe not. I > think we should be working toward a world where the Gather is at the > top of the plan tree as often as possi

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Mark Dilger
> On Dec 11, 2015, at 2:54 PM, Caleb Welton wrote: > > The current semantic level is pretty low level, somewhat cumbersome, and > requires filling in values that most of the time the system has a pretty good > idea how to fill in default values. > > Compare: > CREATE FUNCTION lo_export(oid, t

Re: [HACKERS] Disabling an index temporarily

2015-12-11 Thread Tatsuo Ishii
> On 12/11/2015 06:25 PM, Tatsuo Ishii wrote: > >> What about inventing a new SET command something like: >> >> SET disabled_index to >> >> This adds to "disabled index list". The disabled index >> list let the planner to disregard the indexes in the list. >> >> SET enabled_index to >> >> This

Re: [HACKERS] Disabling an index temporarily

2015-12-11 Thread Joshua D. Drake
On 12/11/2015 06:25 PM, Tatsuo Ishii wrote: What about inventing a new SET command something like: SET disabled_index to This adds to "disabled index list". The disabled index list let the planner to disregard the indexes in the list. SET enabled_index to This removes from the disabled i

Re: [HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.

2015-12-11 Thread Noah Misch
On Fri, Dec 11, 2015 at 09:34:34PM +0900, Michael Paquier wrote: > On Fri, Dec 11, 2015 at 8:48 PM, Alvaro Herrera > wrote: > > Michael Paquier wrote: > >> On Fri, Dec 11, 2015 at 5:35 AM, Alvaro Herrera > >> wrote: > >> I guess that to complete your idea we could allow PostgresNode to get > >> a

Re: [HACKERS] More on the libxml2 update situation

2015-12-11 Thread Tom Lane
Noah Misch writes: > On Fri, Dec 11, 2015 at 10:55:40AM -0500, Tom Lane wrote: >> I'm starting to think that maybe we'd better knuckle under and provide >> a variant expected file that matches this behavior. We're likely to be >> seeing it in the wild for some time to come. > I would look at han

[HACKERS] Disabling an index temporarily

2015-12-11 Thread Tatsuo Ishii
Sometimes I need to repeat creating and dropping indexes while doing an SQL tuning work. As you might know, creating a large index takes long time. So dropping the index and re-creating it is pain and counter productive. What about inventing a new SET command something like: SET disabled_index to

Re: [HACKERS] More on the libxml2 update situation

2015-12-11 Thread Noah Misch
On Fri, Dec 11, 2015 at 10:55:40AM -0500, Tom Lane wrote: > So I did a routine software update on my RHEL6 workstation, and noticed > a security update for libxml2 go by. And guess what: now an XML-enabled > build of Postgres fails regression tests for me, just as previously > discussed in > http:

[HACKERS] strange CREATE INDEX tab completion cases

2015-12-11 Thread Peter Eisentraut
These two tab completion pieces look strange to me: /* If we have CREATE|UNIQUE INDEX CONCURRENTLY, then add "ON" */ else if ((pg_strcasecmp(prev3_wd, "INDEX") == 0 || pg_strcasecmp(prev2_wd, "INDEX") == 0) && pg_strcasecmp(prev_wd, "CONCURRENTLY") == 0)

Re: [HACKERS] Rework the way multixact truncations work

2015-12-11 Thread Noah Misch
On Thu, Dec 10, 2015 at 08:55:54AM -0500, Robert Haas wrote: > I don't know have anything to add to what others have said in response > to this point, except this: the whole point of using a source code > management system is to tell you what changed and when. What you are > proposing to do makes

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Tom Lane
Andres Freund writes: > On 2015-12-11 19:26:38 -0500, Tom Lane wrote: >> I believe it's soluble, but it's going to take something more like >> loading up all the data at once and then doing lookups as we write >> out the .bki entries for each catalog. Fortunately, the volume of >> bootstrap data

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Andres Freund
On 2015-12-11 19:26:38 -0500, Tom Lane wrote: > I believe it's soluble, but it's going to take something more like > loading up all the data at once and then doing lookups as we write > out the .bki entries for each catalog. Fortunately, the volume of > bootstrap data is small enough that that won

Re: [HACKERS] Fwd: [GENERAL] pgxs/config/missing is... missing

2015-12-11 Thread Tom Lane
Jim Nasby writes: > Grr, right after sending this I found the thread you were talking about. > I'm not really sure our missing is better than just letting the error > bubble up. If folks think that's better then lets just rip missing out > entirely. Well, that's what I was suggesting in the ot

Re: [HACKERS] Fwd: [GENERAL] pgxs/config/missing is... missing

2015-12-11 Thread Jim Nasby
On 12/11/15 6:25 PM, Jim Nasby wrote: On 12/10/15 7:09 PM, Tom Lane wrote: Jim Nasby writes: AFAICT the problem is that missing wasn't included in install or uninstall in config/Makefile. Attached patch fixes that, and results in missing being properly installed in lib/pgxs/config. I thought

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Tom Lane
Mark Dilger writes: > Now, if we know that pg_type.dat will be processed before pg_proc.dat, > we can replace all the Oids representing datatypes in pg_proc.dat with the > names for those types, given that we already have a name <=> oid > mapping for types. I don't think this is quite as simple a

Re: [HACKERS] Fwd: [GENERAL] pgxs/config/missing is... missing

2015-12-11 Thread Jim Nasby
On 12/10/15 7:09 PM, Tom Lane wrote: Jim Nasby writes: AFAICT the problem is that missing wasn't included in install or uninstall in config/Makefile. Attached patch fixes that, and results in missing being properly installed in lib/pgxs/config. I thought we'd more or less rejected that approa

Re: [HACKERS] Add IS (NOT) DISTINCT to subquery_Op

2015-12-11 Thread Jim Nasby
On 12/10/15 7:03 PM, Tom Lane wrote: Jim Nasby writes: Is there any reason we couldn't/shouldn't support IS DISTINCT in subquery_Op? (Or really, just add support to ANY()/ALL()/(SELECT ...)?) It's not an operator (in the sense of something with a pg_operator OID), which means this would be qu

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Andres Freund
On 2015-12-11 18:12:16 -0500, Tom Lane wrote: > I think what Mark is proposing is to do the lookups while preparing the > .bki file, which would eliminate the circularity ... at the cost of having > to, essentially, reimplement regprocedure_in and friends in Perl. FWIW, I did that, when this came

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Mark Dilger
> On Dec 11, 2015, at 3:02 PM, Tom Lane wrote: > > Mark Dilger writes: >>> On Dec 11, 2015, at 2:40 PM, Tom Lane wrote: >>> Huh? Those files are the definition of that mapping, no? Isn't what >>> you're proposing circular? > >> No, there are far more references to Oids than there are defini

Re: [HACKERS] Using quicksort for every external sort run

2015-12-11 Thread Peter Geoghegan
On Fri, Dec 11, 2015 at 2:41 PM, Greg Stark wrote: > However the number of comparisons is significantly higher. And in the > non-"abbreviated keys" case where the compare is going to be a > function pointer call the number of comparisons is probably more > important than the actual time spent when

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Tom Lane
Caleb Welton writes: > ... but there is some circularity especially with respect to type > definitions and the functions that define those types. If you changed the > definition of prorettype into a regtype then bootstrap would try to lookup > the type before the pg_type entry exists and throw a

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Tom Lane
Mark Dilger writes: >> On Dec 11, 2015, at 2:40 PM, Tom Lane wrote: >> Huh? Those files are the definition of that mapping, no? Isn't what >> you're proposing circular? > No, there are far more references to Oids than there are definitions of them. Well, you're still not being very clear, but

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Caleb Welton
Yes, that alone without any other changes would be a marked improvement and could be implemented in many places, pg_operator is a good example. ... but there is some circularity especially with respect to type definitions and the functions that define those types. If you changed the definition of

Re: [HACKERS] Using quicksort for every external sort run

2015-12-11 Thread Peter Geoghegan
On Fri, Dec 11, 2015 at 2:52 PM, Greg Stark wrote: > Heh. And if I comment out the presorted check the breakeven point is > *exactly* where the threshold is today at 7 elements -- presumably > because Hoare chose it on purpose. I think it was Sedgewick, but yes. I'd be very hesitant to mess with

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Caleb Welton
The current semantic level is pretty low level, somewhat cumbersome, and requires filling in values that most of the time the system has a pretty good idea how to fill in default values. Compare: CREATE FUNCTION lo_export(oid, text) RETURNS integer LANGUAGE internal STRICT AS 'lo_export' WITH (OI

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Mark Dilger
> On Dec 11, 2015, at 2:40 PM, Tom Lane wrote: > > Mark Dilger writes: >>> On Dec 11, 2015, at 1:46 PM, Tom Lane wrote: >>> That's an interesting proposal. It would mean that the catalog files >>> stay at more or less their current semantic level (direct representations >>> of bootstrap catal

Re: [HACKERS] Using quicksort for every external sort run

2015-12-11 Thread Greg Stark
On Fri, Dec 11, 2015 at 10:41 PM, Greg Stark wrote: > > Interestingly it looks like we could raise the threshold to switching > to insertion sort. At least on my machine the insertion sort is faster > in real time as well as fewer comparisons up to 9 elements. It's > actually faster up to 16 eleme

Re: [HACKERS] Using quicksort for every external sort run

2015-12-11 Thread Greg Stark
On Wed, Dec 9, 2015 at 2:44 AM, Peter Geoghegan wrote: > On Tue, Dec 8, 2015 at 6:39 PM, Greg Stark wrote: > > I guess you mean insertion sort. What's the theoretical justification > for the change? Well my thinking was that hard coding a series of comparisons would be faster than a loop doing a

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Caleb Welton
I took a look at a few of the most recent bulk edit cases for pg_proc.h: There were two this year: * The addition of proparallel [1] * The addition of protransform [2] And prior to that the most recent seems to be from 2012: * The addition of proleakproof [3] Quick TLDR - the changes needed to

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Tom Lane
Mark Dilger writes: >> On Dec 11, 2015, at 1:46 PM, Tom Lane wrote: >> That's an interesting proposal. It would mean that the catalog files >> stay at more or less their current semantic level (direct representations >> of bootstrap catalog contents), but it does sound like a more attractive >>

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-11 Thread Peter Geoghegan
On Fri, Dec 11, 2015 at 2:26 PM, Corey Huinker wrote: > Sure, the machine we called "ninefivealpha", which incidentally, failed to > find a single bug in alpha2 thru beta2, is currently idle, and concurrent > index creation times are a bugbear around these parts. Can somebody, either > in this thr

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-11 Thread Corey Huinker
On Fri, Dec 11, 2015 at 12:13 PM, Robert Haas wrote: > On Wed, Dec 9, 2015 at 8:16 PM, Peter Geoghegan wrote: > > On Tue, Nov 17, 2015 at 7:33 PM, Corey Huinker > wrote: > >> I'm willing, but I'm too new to the codebase to be an effective reviewer > >> (without guidance). The one thing I can of

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Mark Dilger
> On Dec 11, 2015, at 1:46 PM, Tom Lane wrote: > > Alvaro Herrera writes: >> Crazy idea: we could just have a CSV file which can be loaded into a >> table for mass changes using regular DDL commands, then dumped back from >> there into the file. We already know how to do these things, using >>

Re: [HACKERS] REASSIGN OWNED doesn't know how to deal with USER MAPPINGs

2015-12-11 Thread Alvaro Herrera
Jaime Casanova wrote: > On 10 December 2015 at 13:04, Jaime Casanova > wrote: > > Hi, > > > > We just notice $SUBJECT. Attached patch fixes it by ignoring USER > > MAPPINGs in shdepReassignOwned() just like it happens with default > > ACLs. Yep, I had already posted this patch elsewhere, and I ju

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Tom Lane
Alvaro Herrera writes: > Crazy idea: we could just have a CSV file which can be loaded into a > table for mass changes using regular DDL commands, then dumped back from > there into the file. We already know how to do these things, using > \copy etc. Since CSV uses one line per entry, there woul

Re: [HACKERS] Parallel Aggregate

2015-12-11 Thread David Rowley
On 12 December 2015 at 04:00, Robert Haas wrote: > > I'd like to commit David Rowley's patch from the other thread first, > and then deal with this one afterwards. The only thing I feel > strongly needs to be changed in that patch is CFUNC -> COMBINEFUNC, > for clarity. I have addressed that in

Re: [HACKERS] Remaining 9.5 open items

2015-12-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Dec 4, 2015 at 3:22 PM, Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > >> Stephen Frost wrote: > >> > Still, I'll get a patch worked up for it and then we can discuss the > >> > merits of that patch going in to 9

Re: [HACKERS] Remove array_nulls?

2015-12-11 Thread Tom Lane
Jim Nasby writes: > A quick doc search indicates this config was created in 9.0, though the > docs state it's for a change that happened in 8.2[1]. Don't know what you're looking at, but the GUC is definitely there (and documented) in 8.2. > Is it time to remove this GUC? Perhaps, but I'd like

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Caleb Welton
Makes sense. During my own prototyping what I did was generate the sql statements via sql querying the existing catalog. Way easier than hand writing 1000+ function definitions and not difficult to modify for future changes. As affirmed that it was very easy to adapt my existing sql to acco

Re: [HACKERS] [sqlsmith] Failed to generate plan on lateral subqueries

2015-12-11 Thread Greg Stark
There may be other errors that would be surprising for Tom or Robert. What I did with the string argument fuzzer was printed counts for each sqlstate for the errors and watched for errors that only occurred occasionally or didn't make sense to me. Also, do you have any timeouts? Do you have any s

Re: [HACKERS] Logical replication and multimaster

2015-12-11 Thread Petr Jelinek
On 2015-12-11 19:24, Robert Haas wrote: On Fri, Dec 11, 2015 at 5:16 AM, Andres Freund wrote: On 2015-12-11 18:12:55 +0800, Craig Ringer wrote: On 10 December 2015 at 03:19, Robert Haas wrote: On Sun, Dec 6, 2015 at 10:24 PM, Craig Ringer wrote: * A way to securely make a libpq connection

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Alvaro Herrera
Caleb Welton wrote: > I'm happy working these ideas forward if there is interest. > > Basic design proposal is: > - keep a minimal amount of bootstrap to avoid intrusive changes to core > components > - Add capabilities of creating objects with specific OIDs via DDL during > initdb > - Updat

Re: [HACKERS] [sqlsmith] Failed to generate plan on lateral subqueries

2015-12-11 Thread Peter Geoghegan
On Fri, Dec 11, 2015 at 7:58 AM, Tom Lane wrote: >> I guess that means I have to go back to extending the grammar again :-). > > I await the results with interest. Did you note the suggestion about > trying to stress the ON CONFLICT code with this? You'd need it to > issue non-SELECT queries, wh

Re: [HACKERS] Bootstrap DATA is a pita

2015-12-11 Thread Caleb Welton
I'm happy working these ideas forward if there is interest. Basic design proposal is: - keep a minimal amount of bootstrap to avoid intrusive changes to core components - Add capabilities of creating objects with specific OIDs via DDL during initdb - Update the caching/resolution mechanism f

Re: [HACKERS] Logical replication and multimaster

2015-12-11 Thread Robert Haas
On Fri, Dec 11, 2015 at 5:16 AM, Andres Freund wrote: > On 2015-12-11 18:12:55 +0800, Craig Ringer wrote: >> On 10 December 2015 at 03:19, Robert Haas wrote: >> > On Sun, Dec 6, 2015 at 10:24 PM, Craig Ringer >> > wrote: >> > > * A way to securely make a libpq connection from a bgworker without

Re: [HACKERS] Uninterruptible slow geo_ops.c

2015-12-11 Thread Marco Nenciarini
On 11/12/15 18:48, Alvaro Herrera wrote: > Hi, > > A customer of ours hit some very slow code while running the > @>(polygon, polygon) operator with some big polygons. I'm not familiar > with this stuff but I think the problem is that the algorithm converges > too slowly to a solution and also ha

Re: [HACKERS] Cluster "stuck" in "not accepting commands to avoid wraparound data loss"

2015-12-11 Thread Jeff Janes
On Thu, Dec 10, 2015 at 1:55 PM, Andres Freund wrote: > Hi, > > I recently started a pgbench benchmark (to evaluate a piece of hardware, > not postgres) with master. Unfortunately, by accident, I started > postgres in a shell, not screen like pgbench. > > Just logged back in and saw: > client 71 a

[HACKERS] Remove array_nulls?

2015-12-11 Thread Jim Nasby
A quick doc search indicates this config was created in 9.0, though the docs state it's for a change that happened in 8.2[1]. Both versions are now supported, and 8.2 is obviously ancient. Is it time to remove this GUC? [1] http://www.postgresql.org/docs/9.0/static/runtime-config-compatible.ht

[HACKERS] Uninterruptible slow geo_ops.c

2015-12-11 Thread Alvaro Herrera
Hi, A customer of ours hit some very slow code while running the @>(polygon, polygon) operator with some big polygons. I'm not familiar with this stuff but I think the problem is that the algorithm converges too slowly to a solution and also has some pretty expensive calls somewhere. (Perhaps th

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-11 Thread Aleksander Alekseev
Oops. s/approve or disapprove/confirm or deny/ On Fri, 11 Dec 2015 19:14:41 +0300 Aleksander Alekseev wrote: > Hello, Tom > > I see your point, but I would like to clarify a few things. > > 1. Do we consider described measurement method good enough to conclude > that sometimes PostgreSQL reall

Re: [HACKERS] [DOCS] max_worker_processes on the standby

2015-12-11 Thread Alvaro Herrera
Alvaro Herrera wrote: > Not sure what is going on; my reading of the code certainly says that > the data should be there. I'm looking into it. > > I also noticed that I didn't actually push the whole of the patch > yesterday -- I neglected to "git add" the latest changes, the ones that > fix the

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-11 Thread Robert Haas
On Wed, Dec 9, 2015 at 8:16 PM, Peter Geoghegan wrote: > On Tue, Nov 17, 2015 at 7:33 PM, Corey Huinker > wrote: >> I'm willing, but I'm too new to the codebase to be an effective reviewer >> (without guidance). The one thing I can offer in the mean time is this: my >> company/client nearly alwa

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-12-11 Thread Robert Haas
On Fri, Dec 11, 2015 at 1:25 AM, Michael Paquier wrote: >> For another thing, there are definitely going to be >> some people that want the detailed information - and I can practically >> guarantee that if we don't make it available, at least one person will >> write a tool that tries to reverse-e

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-12-11 Thread Masahiko Sawada
On Wed, Dec 9, 2015 at 8:59 PM, Masahiko Sawada wrote: > On Wed, Nov 18, 2015 at 2:06 PM, Masahiko Sawada > wrote: >> On Tue, Nov 17, 2015 at 7:52 PM, Kyotaro HORIGUCHI >> wrote: >>> Oops. >>> >>> At Tue, 17 Nov 2015 19:40:10 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI >>> wrote in >>> <2

Re: [HACKERS] Tab-comletion for RLS

2015-12-11 Thread Masahiko Sawada
On Thu, Dec 10, 2015 at 11:07 PM, Robert Haas wrote: > On Tue, Dec 8, 2015 at 8:32 AM, Masahiko Sawada wrote: >> I found some lacks of tab-completion for RLS in 9.5. >> >> * ALTER POLICY [TAB] >> I expected to appear the list of policy name, but nothing is appeared. >> >> * ALTER POLICY hoge_poli

Re: [HACKERS] [sqlsmith] Failed to generate plan on lateral subqueries

2015-12-11 Thread Andreas Seltenreich
Peter Geoghegan writes: > On Sun, Dec 6, 2015 at 9:52 AM, Andreas Seltenreich > wrote: >> I've added new grammar rules to sqlsmith and improved some older ones. > > Could you possibly teach sqlsmith about INSERT ... ON CONFLICT DO > UPDATE/IGNORE? I think that that could be very helpful, especia

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-11 Thread Aleksander Alekseev
Hello, Tom I see your point, but I would like to clarify a few things. 1. Do we consider described measurement method good enough to conclude that sometimes PostgreSQL really spends 3 ms in a spinlock (like a RTT between two Internet hosts in the same city)? If not, what method should be used to

Re: [HACKERS] Rework the way multixact truncations work

2015-12-11 Thread Robert Haas
On Thu, Dec 10, 2015 at 9:32 AM, Robert Haas wrote: > On Thu, Dec 10, 2015 at 9:04 AM, Andres Freund wrote: >> On 2015-12-10 08:55:54 -0500, Robert Haas wrote: >>> Maybe. But I think we could use a little more vigorous discussion of >>> that issue, since Andres doesn't seem to be very convinced

Re: [HACKERS] [sqlsmith] Failed to generate plan on lateral subqueries

2015-12-11 Thread Tom Lane
Andreas Seltenreich writes: > Tom Lane writes: >> [2. transitive-lateral-fixes-2.patch ] >> [2. remove-lateraljoininfo-2.patch ] > They seem to have fixed the issue for good now. No errors have been > logged for 2e8 queries since applying the first patch. (The second one > was applied later and

[HACKERS] More on the libxml2 update situation

2015-12-11 Thread Tom Lane
So I did a routine software update on my RHEL6 workstation, and noticed a security update for libxml2 go by. And guess what: now an XML-enabled build of Postgres fails regression tests for me, just as previously discussed in http://www.postgresql.org/message-id/flat/cafj8pra4xjqfgnqcqmcygx-umgmr3s

Re: [HACKERS] [sqlsmith] Failed to generate plan on lateral subqueries

2015-12-11 Thread Andreas Seltenreich
Tom Lane writes: > [2. transitive-lateral-fixes-2.patch ] > [2. remove-lateraljoininfo-2.patch ] They seem to have fixed the issue for good now. No errors have been logged for 2e8 queries since applying the first patch. (The second one was applied later and didn't get as much exposure.) I guess

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-11 Thread Tom Lane
Aleksander Alekseev writes: > Turns out PostgreSQL can spend a lot of time waiting for a lock in this > particular place, especially if you are running PostgreSQL on 60-core > server. Which obviously is a pretty bad sign. > ... > I managed to fix this behaviour by modifying choose_nelem_alloc > pr

Re: [HACKERS] Remaining 9.5 open items

2015-12-11 Thread Robert Haas
On Fri, Dec 4, 2015 at 3:22 PM, Stephen Frost wrote: > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: >> Stephen Frost wrote: >> > Still, I'll get a patch worked up for it and then we can discuss the >> > merits of that patch going in to 9.5 now versus just into HEAD. >> >> Cool. > > While wor

Re: [HACKERS] [DOCS] max_worker_processes on the standby

2015-12-11 Thread Robert Haas
On Wed, Dec 9, 2015 at 4:21 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Mon, Dec 7, 2015 at 8:33 AM, Fujii Masao wrote: > >> > So firstly you will push those "latest" changes soon? >> >> It seems like these changes haven't been pushed yet, and unfortunately >> that's probably a beta bloc

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-12-11 Thread Tom Lane
Greg Stark writes: > Fwiw I poked at the bison output to see if it would be possible to do. > I think it's theoretically possible but a huge project and would > create dependencies on bison internals that we would be unlikelly to > accept. That's the impression I got when I looked at it briefly,

Re: [HACKERS] Parallel Aggregate

2015-12-11 Thread Robert Haas
On Fri, Dec 11, 2015 at 1:42 AM, Haribabu Kommi wrote: > Here I attached a POC patch of parallel aggregate based on combine > aggregate patch. This patch contains the combine aggregate changes > also. This patch generates and executes the parallel aggregate plan > as discussed in earlier threads.

[HACKERS] Patch: fix lock contention for HASHHDR.mutex

2015-12-11 Thread Aleksander Alekseev
Hello all, Consider following stacktrace: (gdb) bt #0 0x7f77c81fae87 in semop () syscall-template.S:81 #1 0x0063b721 in PGSemaphoreLock pg_sema.c:387 #2 0x00692e1b in LWLockAcquire lwlock.c:1026 #3 0x0068d4c5 in LockAcquireExtended lock.c:881 #4 0x0068dcc1

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-12-11 Thread Robert Haas
On Fri, Dec 11, 2015 at 12:16 AM, Ashutosh Bapat wrote: >> +1. >> > I think there is still a lot functionality that is offered without > EvalPlanQual fix. Sure. But I think that the EvalPlanQual-related fixes might have some impact on the overall design, and I don't want to commit this with one

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-12-11 Thread Greg Stark
On Fri, Dec 11, 2015 at 8:12 AM, Michael Paquier wrote: > Also, if > we prioritize a dynamically generated tab completion using gram.y, so > be it and let's reject both patches then... Fwiw I poked at the bison output to see if it would be possible to do. I think it's theoretically possible but

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2015-12-11 Thread Andres Freund
On 2015-12-11 15:56:46 +0300, Alexander Korotkov wrote: > On Thu, Dec 10, 2015 at 9:26 AM, Amit Kapila > wrote: > We did see this on big Intel machine in practice. pgbench -S gets > shared ProcArrayLock very frequently. Since some number of connections is > achieved, new connections

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2015-12-11 Thread Alexander Korotkov
On Thu, Dec 10, 2015 at 9:26 AM, Amit Kapila wrote: > On Wed, Dec 9, 2015 at 2:17 PM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> On Tue, Dec 8, 2015 at 6:00 PM, Amit Kapila >> wrote: >> >>> On Tue, Dec 8, 2015 at 3:56 PM, Alexander Korotkov < >>> a.korot...@postgrespro.ru> wro

Re: [HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.

2015-12-11 Thread Michael Paquier
On Fri, Dec 11, 2015 at 8:48 PM, Alvaro Herrera wrote: > Michael Paquier wrote: >> On Fri, Dec 11, 2015 at 5:35 AM, Alvaro Herrera >> wrote: >> I guess that to complete your idea we could allow PostgresNode to get >> a custom name for its log file through an optional parameter like >> logfile =>

Re: [HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.

2015-12-11 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Dec 11, 2015 at 5:35 AM, Alvaro Herrera > wrote: > > Since we now have the node name in the log file name, perhaps we no > > longer need the port number in there > > There is no node name in the log file name as of now, they are built > using the port number, and

Re: [HACKERS] Logical replication and multimaster

2015-12-11 Thread Andres Freund
On 2015-12-11 18:12:55 +0800, Craig Ringer wrote: > On 10 December 2015 at 03:19, Robert Haas wrote: > > On Sun, Dec 6, 2015 at 10:24 PM, Craig Ringer > > wrote: > > > * A way to securely make a libpq connection from a bgworker without > > messing > > > with passwords etc. Generate one-time cooki

Re: [HACKERS] Logical replication and multimaster

2015-12-11 Thread Craig Ringer
On 10 December 2015 at 03:19, Robert Haas wrote: > On Sun, Dec 6, 2015 at 10:24 PM, Craig Ringer > wrote: > > > * A way to securely make a libpq connection from a bgworker without > messing > > with passwords etc. Generate one-time cookies, sometihng like that. > > Why would you have the bgwor

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-11 Thread Aleksander Alekseev
> It would be InvalidBuffer for buffers, -1 for files and NULL for all > void*-types. Does such solution sounds OK? On second thought I believe there is no reason for storing anything for void*-like types. I could just hardcode NULL in PointerResourceArray. Best regards, Aleksander -- Sent v

Re: [HACKERS] Patch: ResourceOwner optimization for tables with many partitions

2015-12-11 Thread Aleksander Alekseev
>> To be honest, I think this patch is really ugly. [...] I'm not sure >> exactly what to do about that, but it seems like a problem. I have an idea. There are actually two types of resources - int-like (buffers, files) and void*-like (RelationRef, TupleDesc, ...). What if I split ResourceArray in

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2015-12-11 Thread Etsuro Fujita
On 2015/12/11 14:16, Ashutosh Bapat wrote: On Thu, Dec 10, 2015 at 11:20 PM, Robert Haas mailto:robertmh...@gmail.com>> wrote: On Tue, Dec 8, 2015 at 6:40 AM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: > IMO I want to see the EvalPlanQual fix in the first version

Re: [HACKERS] Remaining 9.5 open items

2015-12-11 Thread Etsuro Fujita
On 2015/12/11 1:18, Robert Haas wrote: On Wed, Dec 9, 2015 at 2:52 AM, Etsuro Fujita wrote: Thank you for committing the patch! Sorry, I overlooked a typo in docs: s/more that one/more than one/ Please find attached a patch. Committed, thanks. Thanks! Best regards, Etsuro Fujita -- S

[HACKERS] use_remote_estimate usage for join pushdown in postgres_fdw

2015-12-11 Thread Ashutosh Bapat
Hi All, postgres_fdw documentation says following about use_remote_estimate ( http://www.postgresql.org/docs/devel/static/postgres-fdw.html) -- use_remote_estimate This option, which can be specified for a foreign table or a foreign server, controls whether postgres_fdw issues remote EXPLAIN comman

Re: [HACKERS] Error with index on unlogged table

2015-12-11 Thread Andres Freund
On 2015-12-11 15:43:24 +0900, Kyotaro HORIGUCHI wrote: > What it is doing seems to me reasonable but copying_initfork > doesn't seems to be necessary. Kicking both of log_newpage() and > smgrimmedsync() by use_wal, which has the value considering > INIT_FORKNUM would be more descriptive. (more read

Re: [HACKERS] Isolation of table creation

2015-12-11 Thread Andres Freund
On 2015-12-11 11:46:11 +0300, Alexander Korotkov wrote: > I discovered interesting issue with PostgreSQL transaction isolation. > When transaction is in repeatable read isolation level, I can't see table > which was created after transaction obtained snapshot. But I can run DML > statements with th

[HACKERS] Isolation of table creation

2015-12-11 Thread Alexander Korotkov
Hackers, I discovered interesting issue with PostgreSQL transaction isolation. When transaction is in repeatable read isolation level, I can't see table which was created after transaction obtained snapshot. But I can run DML statements with this table. See example below. Session 1 # begin transa

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-12-11 Thread Michael Paquier
On Thu, Dec 10, 2015 at 5:38 PM, Kyotaro HORIGUCHI wrote: > I'm unhappy with context matching using previous_words in two > points. Current code needs human-readable comments describing > almost all matchings. It is hard to maintain and some of them > actually are wrong. The hardness is largely al