Re: [HACKERS] Choosing parallel_degree

2016-04-11 Thread Amit Kapila
On Tue, Apr 12, 2016 at 3:15 AM, Simon Riggs wrote: > On 8 April 2016 at 17:49, Robert Haas wrote: > > >> With the patch, you can - if you wish - substitute >> some other number for the one the planner comes up with. > > > I saw you're using AccessExclusiveLock, the reason being it affects > SEL

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-11 Thread Noah Misch
On Thu, Apr 07, 2016 at 05:53:54PM -0700, Jeff Janes wrote: > On Thu, Apr 7, 2016 at 4:33 PM, Tom Lane wrote: > > Jeff Janes writes: > >> To summarize the behavior change: > > > >> In the released code, an inserting backend that violates the pending > >> list limit will try to clean the list, eve

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-04-11 Thread Noah Misch
On Mon, Apr 11, 2016 at 11:22:27AM +0200, Magnus Hagander wrote: > On Thu, Apr 7, 2016 at 5:15 AM, Noah Misch wrote: > > Unless you especially want to self-impose the same tight resolution > > schedule > > that 9.6 regressions experience, let's move this to the "Non-bugs" section. > > Which do you

Re: [HACKERS] Preprocessor condition fix

2016-04-11 Thread Christian Ullrich
* Tom Lane wrote: Christian Ullrich writes: According to git grep, this is the only place where WIN64 is used without the leading underscore. Hm, my grep found another one ... Oh, sorry. I saw that one, but thought it was intentional because _WIN64 is defined automatically anyway. --

[HACKERS] Re: Parallel Aggregate costs don't consider combine/serial/deserial funcs

2016-04-11 Thread Noah Misch
On Mon, Apr 11, 2016 at 12:47:29AM +1200, David Rowley wrote: > Hi, > > I realised a few days ago that the parallel aggregate code does not > cost for the combine, serialisation and deserialisation functions at > all. > > I've attached a patch which fixes this. > > One small point which I was a

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-11 Thread Amit Kapila
On Mon, Apr 11, 2016 at 7:33 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Sun, Apr 10, 2016 at 2:24 PM, Amit Kapila > wrote: >> >> I also tried to run perf top during pgbench and get some interesting >>> results. >>> >>> Without 5364b357: >>>5,69% postgres

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-11 Thread Tom Lane
Craig Ringer writes: > The other area where there's room for extension without throwing out the > whole thing and rebuilding is handling of new top-level statements. We can > probably dispatch the statement text to a sub-parser provided by an > extension that registers interest in that statement n

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-11 Thread Craig Ringer
On 12 April 2016 at 13:28, David G. Johnston wrote: > ​As recently discovered there is more than one reason why an intelligent > driver, like the JDBC standard at least requires in a few instances, > requires knowledge of at least some basic structure​ > > ​of the statements it sees before sendi

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-11 Thread David G. Johnston
On Mon, Apr 11, 2016 at 9:58 PM, Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > *From:* pgsql-hackers-ow...@postgresql.org [mailto: > pgsql-hackers-ow...@postgresql.org] * On Behalf Of *Arcadiy Ivanov > > Currently the parser and lexer are fully fixed at compile-time and not > amen

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-11 Thread Craig Ringer
On 12 April 2016 at 13:10, Tom Lane wrote: > > I'm interested in possible solutions to this problem, but it's far > harder than it looks. > > Exactly. Limited extension points where we accept runtime errors and confine the extension points can be OK; e.g. SOME STATEMENT ... WITH (THINGY, OTHER_T

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-11 Thread Tom Lane
Arcadiy Ivanov writes: > Is there any interest and/or tips to allow a pluggable parser or at > least allow some syntactical pluggability by extensions? There is a fair amount of discussion of this in the archives. The short answer is that bison can't do it, and "let's replace bison" is a propos

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-11 Thread Andres Freund
On 2016-04-12 00:32:13 -0400, Tom Lane wrote: > I wrote: > > This commit has broken buildfarm member gaur, and no doubt pademelon > > will be equally unhappy once it catches up to HEAD. > > Spoke too soon, actually: pademelon did not get as far as initdb. > > cc: "bufmgr.c", line 4032: error 1521

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-11 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Arcadiy Ivanov Currently the parser and lexer are fully fixed at compile-time and not amenable to the extensions - extensions are only capable of introducing functions etc. There is, however, an ad

Re: [HACKERS] Parser extensions (maybe for 10?)

2016-04-11 Thread Craig Ringer
On 12 April 2016 at 12:36, Arcadiy Ivanov wrote: > > Is there any interest and/or tips to allow a pluggable parser or at least > allow some syntactical pluggability by extensions? > I think this may allow some projects to move towards becoming an extension > as opposed to forking the project ent

[HACKERS] Parser extensions (maybe for 10?)

2016-04-11 Thread Arcadiy Ivanov
I thought I'd float this idea and see if this gets any traction. Please forgive my ignorance if this has been discussed already. Currently the parser and lexer are fully fixed at compile-time and not amenable to the extensions - extensions are only capable of introducing functions etc. There

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-11 Thread Tom Lane
I wrote: > This commit has broken buildfarm member gaur, and no doubt pademelon > will be equally unhappy once it catches up to HEAD. Spoke too soon, actually: pademelon did not get as far as initdb. cc: "bufmgr.c", line 4032: error 1521: Incorrect initialization. cc: "bufmgr.c", line 4058: error

Re: [HACKERS] Re: [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-11 Thread Andres Freund
On 2016-04-11 23:59:21 -0400, Tom Lane wrote: > Andres Freund writes: > > Will fix (both initialization and use of pg_atomic_fetch_or_u32), and > > expand the documentation on why only atomic read/write are supposed to > > be used. > > FWIW, I'd vote against adding a SpinLockInit there. Well, it

Re: [HACKERS] Re: [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-11 Thread Tom Lane
Andres Freund writes: >>> The issue is likely that either Alexander or I somehow made >>> MarkLocalBufferDirty() use pg_atomic_fetch_or_u32(), instead of the >>> proper pg_atomic_read_u32()/pg_atomic_write_u32(). > Ok, so the theory above fits. Yah, especially in view of localbuf.c:297 ;-) > Wi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-11 Thread Andres Freund
On 2016-04-11 23:41:10 -0400, Tom Lane wrote: > Andres Freund writes: > > The issue is likely that either Alexander or I somehow made > > MarkLocalBufferDirty() use pg_atomic_fetch_or_u32(), instead of the > > proper pg_atomic_read_u32()/pg_atomic_write_u32(). > > The stack trace I'm seeing is > >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-11 Thread Tom Lane
Andres Freund writes: > The issue is likely that either Alexander or I somehow made > MarkLocalBufferDirty() use pg_atomic_fetch_or_u32(), instead of the > proper pg_atomic_read_u32()/pg_atomic_write_u32(). The stack trace I'm seeing is #5 0x7279cc in elog_finish (elevel=6, fmt=0x40057cf8

[HACKERS] Re: [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-11 Thread Andres Freund
On 2016-04-11 23:24:36 -0400, Tom Lane wrote: > Andres Freund writes: > > Allow Pin/UnpinBuffer to operate in a lockfree manner. > > This commit has broken buildfarm member gaur, and no doubt pademelon > will be equally unhappy once it catches up to HEAD. The reason is that > you've caused local

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Craig Ringer
On 12 April 2016 at 10:09, Michael Paquier wrote: > On Tue, Apr 12, 2016 at 10:54 AM, Craig Ringer > wrote: > > > If you rely on shared_preload_libraries, then "oops, I forgot to put > > myextension in shared_preload_libraries on the downstream" becomes a Bad > > Thing. There's no way to go bac

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-11 Thread Tom Lane
Andres Freund writes: > Allow Pin/UnpinBuffer to operate in a lockfree manner. This commit has broken buildfarm member gaur, and no doubt pademelon will be equally unhappy once it catches up to HEAD. The reason is that you've caused localbuf.c to perform a whole bunch of atomic operations on its

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-11 Thread Andres Freund
On 2016-04-11 14:40:29 -0700, Andres Freund wrote: > On 2016-04-11 12:17:20 -0700, Andres Freund wrote: > I did get access to the machine (thanks!). My testing shows that > performance is sensitive to various parameters influencing memory > allocation. E.g. twiddling with max_connections changes >

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Michael Paquier
On Tue, Apr 12, 2016 at 10:54 AM, Craig Ringer wrote: > On 12 April 2016 at 08:36, Michael Paquier wrote: > The only way we really have at the moment is shared_preload_libraries. That's exactly my point. > If you rely on shared_preload_libraries, then "oops, I forgot to put > myextension in shar

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Craig Ringer
On 12 April 2016 at 08:36, Michael Paquier wrote: > > > Also, the entire point here is that extensions DON'T > > get to have custom apply routines, because we have no way for > > replay to know about such apply routines. (It surely cannot look > > in the catalogs for them, but there's no other

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-11 Thread Michael Paquier
On Mon, Apr 11, 2016 at 5:16 PM, Etsuro Fujita wrote: > On 2016/04/11 12:30, Michael Paquier wrote: >> >> + if ((cancel = PQgetCancel(entry->conn))) >> + { >> + PQcancel(cancel, errbuf, sizeof(errbuf)); >> +

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-04-11 Thread Craig Ringer
On 12 April 2016 at 00:39, Justin Clift wrote: > Moving over a conversation from the pgsql-advocacy mailing list. In it > Simon (CC'd) raised the issue of potentially creating a > backwards-compatibility > breaking release at some point in the future, to deal with things that > might have no oth

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Michael Paquier
On Tue, Apr 12, 2016 at 9:33 AM, Tom Lane wrote: > ... BTW, with respect to the documentation angle, it seems to me > that it'd be better if GenericXLogRegister were renamed to > GenericXLogRegisterBuffer, or perhaps GenericXLogRegisterPage. > I think this would make the documentation clearer, and

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Michael Paquier
On Tue, Apr 12, 2016 at 9:21 AM, Tom Lane wrote: > Michael Paquier writes: >> What I thought we should be able to do with that should not be only >> limited to indexes, aka to: >> - Be able to register and log full pages >> - Be able to log data associated to a block >> - Be able to have record d

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Tom Lane
... BTW, with respect to the documentation angle, it seems to me that it'd be better if GenericXLogRegister were renamed to GenericXLogRegisterBuffer, or perhaps GenericXLogRegisterPage. I think this would make the documentation clearer, and it would also make it easier to add other sorts of Regist

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Tom Lane
Michael Paquier writes: > As given out now, we are able to do the following: > - Log a full page > - Log a delta of a full page, which is actually data associated to a page. > - At recovery, replay those full pages with a delta. Right. > What I thought we should be able to do with that should no

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Michael Paquier
On Mon, Apr 11, 2016 at 11:29 PM, Tom Lane wrote: > Michael Paquier writes: >> Actually, as I look at this code for the first time, I find that >> GenericXLogFinish() is a very confusing interface. It makes no >> distinction between a page and the meta data associated to this page, >> this is con

Re: [HACKERS] 'Create table if not exists as' breaks SPI_execute

2016-04-11 Thread Tom Lane
Stas Kelvich writes: > SPI_execute assumes that CreateTableAsStmt always have completionTag == > “completionTag”. > But it isn’t true in case of ‘IF NOT EXISTS’ present. Pushed, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-11 Thread Masahiko Sawada
On Mon, Apr 11, 2016 at 8:47 PM, Fujii Masao wrote: > On Mon, Apr 11, 2016 at 5:52 PM, Masahiko Sawada > wrote: >> On Mon, Apr 11, 2016 at 1:31 PM, Fujii Masao wrote: >>> On Mon, Apr 11, 2016 at 10:58 AM, Masahiko Sawada >>> wrote: On Sat, Apr 9, 2016 at 12:32 PM, Tom Lane wrote: >

Re: [HACKERS] Preprocessor condition fix

2016-04-11 Thread Tom Lane
Christian Ullrich writes: > Here is a one-line patch to fix a wrong preprocessor condition in > pg_regress, found because the VS 2015 compiler warns on the cast in the > 32-bit branch where apparently earlier versions did not. Pushed, thanks. > According to git grep, this is the only place whe

Re: [HACKERS] Removing the TRACE_SORT macro

2016-04-11 Thread Peter Geoghegan
On Mon, Apr 11, 2016 at 2:54 PM, Robert Haas wrote: > I'm not excited about this change. It doesn't really buy us anything > that I can see. For one thing, I think we've arguably got too much > trace_sort output from sorts now and should look to reduce that > instead of further normalizing it.

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-11 Thread Andres Freund
On 2016-04-11 14:40:29 -0700, Andres Freund wrote: > On 2016-04-11 12:17:20 -0700, Andres Freund wrote: > > On 2016-04-11 22:08:15 +0300, Alexander Korotkov wrote: > > > On Mon, Apr 11, 2016 at 5:04 PM, Alexander Korotkov < > > > a.korot...@postgrespro.ru> wrote: > > > > > > > On Mon, Apr 11, 2016

Re: [HACKERS] Removing the TRACE_SORT macro

2016-04-11 Thread Robert Haas
On Mon, Apr 11, 2016 at 4:43 PM, Peter Geoghegan wrote: > Currently, debug instrumentation for sorting is only available if the > TRACE_SORT macro was defined when PostgreSQL was compiled. It is > defined by default, and so in practice it's always available; there is > really no upside to disablin

Re: [HACKERS] Removing the TRACE_SORT macro

2016-04-11 Thread Peter Geoghegan
On Mon, Apr 11, 2016 at 2:35 PM, Simon Riggs wrote: > Yeh, sort has changed enough now that fixes weren't going to backpatch > cleanly, so its a good time to do cleanup. I wonder if the category of "Developer Options" is appropriate for trace_sort. trace_sort is closer to log_executor_stats, whic

Re: [HACKERS] Choosing parallel_degree

2016-04-11 Thread Simon Riggs
On 8 April 2016 at 17:49, Robert Haas wrote: > With the patch, you can - if you wish - substitute > some other number for the one the planner comes up with. I saw you're using AccessExclusiveLock, the reason being it affects SELECTs. That is supposed to apply when things might change the answ

Re: [HACKERS] Choosing parallel_degree

2016-04-11 Thread Simon Riggs
On 8 April 2016 at 17:27, Paul Ramsey wrote: > PostGIS is just one voice... > We're listening. > >> Functions have very unequal CPU costs, and we're talking here about > >> using CPUs more effectively, why are costs being given the see-no-evil > >> treatment? This is as true in core as it is

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-11 Thread Andres Freund
On 2016-04-11 12:17:20 -0700, Andres Freund wrote: > On 2016-04-11 22:08:15 +0300, Alexander Korotkov wrote: > > On Mon, Apr 11, 2016 at 5:04 PM, Alexander Korotkov < > > a.korot...@postgrespro.ru> wrote: > > > > > On Mon, Apr 11, 2016 at 8:10 AM, Andres Freund wrote: > > > > > >> Could you retry

Re: [HACKERS] Remove unused function from walsender.c

2016-04-11 Thread Simon Riggs
On 11 April 2016 at 08:05, Fujii Masao wrote: > There is an unused function GetOldestWALSendPointer() in walsender.c. > Per comment, it was introduced because we may need it in the future for > synchronous replication. > > Now we have very similar function SyncRepGetOldestSyncRecPtr() in > syncr

Re: [HACKERS] Removing the TRACE_SORT macro

2016-04-11 Thread Simon Riggs
On 11 April 2016 at 21:43, Peter Geoghegan wrote: > Currently, debug instrumentation for sorting is only available if the > TRACE_SORT macro was defined when PostgreSQL was compiled. It is > defined by default, and so in practice it's always available; there is > really no upside to disabling it.

Re: [HACKERS] Choosing parallel_degree

2016-04-11 Thread Julien Rouhaud
On 11/04/2016 17:44, Robert Haas wrote: > On Mon, Apr 11, 2016 at 11:27 AM, Julien Rouhaud > wrote: >> On 11/04/2016 15:56, tushar wrote: >>> >>> I am assuming parallel_degree=0 is as same as not using it , i.e >>> create table fok2(n int) with (parallel_degree=0); = create table >>> fok2(n int)

[HACKERS] Removing the TRACE_SORT macro

2016-04-11 Thread Peter Geoghegan
Currently, debug instrumentation for sorting is only available if the TRACE_SORT macro was defined when PostgreSQL was compiled. It is defined by default, and so in practice it's always available; there is really no upside to disabling it. "18.17. Developer Options" notes this restriction for trace

[HACKERS] Remove unused function from walsender.c

2016-04-11 Thread Fujii Masao
Hi, There is an unused function GetOldestWALSendPointer() in walsender.c. Per comment, it was introduced because we may need it in the future for synchronous replication. Now we have very similar function SyncRepGetOldestSyncRecPtr() in syncrep.c. Which makes me think that GetOldestWALSendPointer

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-04-11 Thread Robert Haas
On Mon, Apr 11, 2016 at 3:23 PM, Robbie Harwood wrote: > I'm sure this won't be a popular suggestion, but in the interest of > advocating for more cryptography: if we land GSSAPI auth+encryption, I'd > like the auth-only codepath to go away. I can't think of a reason that would be a good idea. O

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-04-11 Thread Robbie Harwood
Justin Clift writes: > Moving over a conversation from the pgsql-advocacy mailing list. In it > Simon (CC'd) raised the issue of potentially creating a > backwards-compatibility > breaking release at some point in the future, to deal with things that > might have no other solution (my wording).

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-11 Thread Andres Freund
On 2016-04-11 22:08:15 +0300, Alexander Korotkov wrote: > On Mon, Apr 11, 2016 at 5:04 PM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > > > On Mon, Apr 11, 2016 at 8:10 AM, Andres Freund wrote: > > > >> Could you retry after applying the attached series of patches? > >> > > > > Yes,

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-11 Thread Alexander Korotkov
On Mon, Apr 11, 2016 at 5:04 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Mon, Apr 11, 2016 at 8:10 AM, Andres Freund wrote: > >> Could you retry after applying the attached series of patches? >> > > Yes, I will try with these patches and snapshot too old reverted. > I've run

Re: [HACKERS] PQsendQuery+PQgetResult+PQsetSingleRowMode limitations and support

2016-04-11 Thread Karl O. Pinc
On Mon, 11 Apr 2016 19:25:20 +0200 "Shulgin, Oleksandr" wrote: > On Mon, Apr 11, 2016 at 7:15 PM, Karl O. Pinc wrote: > > > Not sure about the part > > > where you call PQsetSingleRowMode() again after seeing > > > PGRES_TUPLES_OK: doesn't look to me like you need or want to do > > > that. You

Re: [HACKERS] pgsql: Add the "snapshot too old" feature

2016-04-11 Thread Kevin Grittner
On Mon, Apr 11, 2016 at 8:20 AM, Alexander Korotkov wrote: > 1) We uglify buffer manager interface a lot. This patch adds 3 more > arguments to BufferGetPage(). It looks weird. Should we try to find less > invasive way for doing this? As already pointed out, I originally touched about 450 few

Re: [HACKERS] PQsendQuery+PQgetResult+PQsetSingleRowMode limitations and support

2016-04-11 Thread Shulgin, Oleksandr
On Mon, Apr 11, 2016 at 7:15 PM, Karl O. Pinc wrote: > > Should I submit a regression test or something to ensure > that this usage is officially supported? (A grep for > PQsetSingleRowMode in src/test/ finds no hits.) > Can I assume because it's documented it'll continue to work? Pretty much.

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-11 Thread Andres Freund
On 2016-04-11 13:04:48 -0400, Robert Haas wrote: > You're right, but I think that's more because I didn't say it > correctly than because you haven't done something novel. Could be. > DROP and > relation truncation know about shared buffers, and they go clear > blocks that that might be affected

Re: [HACKERS] PQsendQuery+PQgetResult+PQsetSingleRowMode limitations and support

2016-04-11 Thread Karl O. Pinc
On Mon, 11 Apr 2016 15:55:53 +0200 "Shulgin, Oleksandr" wrote: > On Fri, Apr 1, 2016 at 7:53 PM, Karl O. Pinc wrote: > > > > On Fri, 1 Apr 2016 05:57:33 +0200 > > "Shulgin, Oleksandr" wrote: > > > > > On Apr 1, 2016 02:57, "Karl O. Pinc" wrote: > > > > > > > > I assume there are no questio

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-11 Thread Robert Haas
On Mon, Apr 11, 2016 at 12:50 PM, Andres Freund wrote: > On 2016-04-04 12:45:25 -0400, Robert Haas wrote: >> Well, I agree that it's pretty strange that _mdfd_getseg() makes no >> such check, but I still don't think I understand what's going on here. >> Backends shouldn't be requesting nonexistent

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-11 Thread Andres Freund
On 2016-04-04 12:45:25 -0400, Robert Haas wrote: > Well, I agree that it's pretty strange that _mdfd_getseg() makes no > such check, but I still don't think I understand what's going on here. > Backends shouldn't be requesting nonexistent blocks from a relation - > higher-level safeguards, like hol

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-04-11 Thread Robert Haas
On Mon, Apr 11, 2016 at 5:22 AM, Magnus Hagander wrote: > Well, if we *don't* do the rewrite before we release it, then we have to > instead put information about the new version of the functions into the old > structure I think. I think that you should have done that in the patch as committed. M

[HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-04-11 Thread Justin Clift
Moving over a conversation from the pgsql-advocacy mailing list. In it Simon (CC'd) raised the issue of potentially creating a backwards-compatibility breaking release at some point in the future, to deal with things that might have no other solution (my wording). Relevant part of that thread the

[HACKERS] plan for beta1 & open issues

2016-04-11 Thread Robert Haas
The PostgreSQL 9.6 release management team has determined that PostgreSQL 9.6beta1 should wrap on May 9, 2016 for release on May 12,2016, subject to the availability of the packaging team to perform a release at that time. The release management team believes that there are no currently-known prob

Re: [HACKERS] 'Create table if not exists as' breaks SPI_execute

2016-04-11 Thread Stas Kelvich
> On 11 Apr 2016, at 18:41, Stas Kelvich wrote: > > Hi. > > SPI_execute assumes that CreateTableAsStmt always have completionTag == > “completionTag”. > But it isn’t true in case of ‘IF NOT EXISTS’ present. > > > Sorry, I meant completionTag == “SELECT”. -- Stas Kelvich Postgres Professio

Re: [HACKERS] 2016-03 Commitfest

2016-04-11 Thread Robert Haas
On Mon, Apr 11, 2016 at 8:37 AM, Andreas Karlsson wrote: > On 04/11/2016 01:35 PM, David Steele wrote: >> I've marked this committed so the 2016-03 CF is now complete! > > Thanks to you and everyone else involved in running this CF. You did an > excellent job. Yeah, David deserves mad props. --

Re: [HACKERS] Choosing parallel_degree

2016-04-11 Thread Robert Haas
On Mon, Apr 11, 2016 at 11:27 AM, Julien Rouhaud wrote: > On 11/04/2016 15:56, tushar wrote: >> On 04/08/2016 08:53 PM, Robert Haas wrote: >>> On Fri, Apr 8, 2016 at 1:22 AM, Amit Kapila wrote: Other than that, patch looks good and I have marked it as Ready For Committer. Hope, we get

[HACKERS] 'Create table if not exists as' breaks SPI_execute

2016-04-11 Thread Stas Kelvich
Hi. SPI_execute assumes that CreateTableAsStmt always have completionTag == “completionTag”. But it isn’t true in case of ‘IF NOT EXISTS’ present. spi-cta.patch Description: Binary data -- Stas Kelvich Postgres Professional: http://www.postgrespro.com Russian Postgres Company -- Sent vi

Re: [HACKERS] Choosing parallel_degree

2016-04-11 Thread Julien Rouhaud
On 11/04/2016 15:56, tushar wrote: > On 04/08/2016 08:53 PM, Robert Haas wrote: >> On Fri, Apr 8, 2016 at 1:22 AM, Amit Kapila wrote: >>> Other than that, patch looks good and I have marked it as Ready For >>> Committer. Hope, we get this for 9.6. >> Committed. I think this is likely to make par

Re: [HACKERS] [COMMITTERS] pgsql: Add the "snapshot too old" feature

2016-04-11 Thread Alexander Korotkov
On Mon, Apr 11, 2016 at 4:27 PM, Alvaro Herrera wrote: > Alexander Korotkov wrote: > > Kevin, > > > > This commit makes me very uneasy. I didn't much care about this patch > > mainly because I didn't imagine its consequences. Now, I see following: > > > > 1) We uglify buffer manager interface a

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Tom Lane
Teodor Sigaev writes: >> 2. Unless I'm missing something, contrib/bloom is making no effort >> to ensure that bloom index pages can be distinguished from other pages >> by pg_filedump. That's fine if your expectation is that bloom will always >> be a toy with no use in production; but otherwise,

Re: [HACKERS] Execute ignoring cursor?

2016-04-11 Thread Pavel Stehule
2016-04-11 16:31 GMT+02:00 nummervet nummervet : > Oh. That doesn't work for me as i generate the query dynamically and don't > know their structure... > Maybe there is an easy way to get the cursor structure (column - value, > column - value)? > Or should i give up on cursors and try somethin

[HACKERS] Re[2]: [HACKERS] Execute ignoring cursor?

2016-04-11 Thread nummervet nummervet
Oh. That doesn't work for me as i generate the query dynamically and don't know their structure... Maybe there is an easy way to get the cursor structure (column - value, column - value)? Or should i give up on cursors and try something else? Some Google search hint that hstore could be my

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Tom Lane
Michael Paquier writes: > Actually, as I look at this code for the first time, I find that > GenericXLogFinish() is a very confusing interface. It makes no > distinction between a page and the meta data associated to this page, > this is controlled by a flag isNew and buffer data is saved as some

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Alexander Korotkov
Hi, Tom! On Sun, Apr 10, 2016 at 7:49 AM, Tom Lane wrote: > 1. It doesn't seem like generic_xlog.c has thought very carefully about > the semantics of the "hole" between pd_lower and pd_upper. The mainline > XLOG code goes to some lengths to ensure that the hole stays all-zeroes; > for example

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Teodor Sigaev
mainline XLOG replay. Shouldn't generic_xlog.c take the same trouble? Yes, it should. Alexander now works on it. 2. Unless I'm missing something, contrib/bloom is making no effort to ensure that bloom index pages can be distinguished from other pages by pg_filedump. That's fine if your expec

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-11 Thread Alexander Korotkov
On Mon, Apr 11, 2016 at 8:10 AM, Andres Freund wrote: > On 2016-04-10 09:03:37 +0300, Alexander Korotkov wrote: > > On Sun, Apr 10, 2016 at 8:36 AM, Alexander Korotkov < > > a.korot...@postgrespro.ru> wrote: > > > > > On Sat, Apr 9, 2016 at 10:49 PM, Andres Freund > wrote: > > > > > >> > > >> >

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-11 Thread Alexander Korotkov
On Sun, Apr 10, 2016 at 2:24 PM, Amit Kapila wrote: > On Sun, Apr 10, 2016 at 11:33 AM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> On Sun, Apr 10, 2016 at 8:36 AM, Alexander Korotkov < >> a.korot...@postgrespro.ru> wrote: >> >>> On Sat, Apr 9, 2016 at 10:49 PM, Andres Freund >

Re: [HACKERS] PQsendQuery+PQgetResult+PQsetSingleRowMode limitations and support

2016-04-11 Thread Shulgin, Oleksandr
On Fri, Apr 1, 2016 at 7:53 PM, Karl O. Pinc wrote: > > On Fri, 1 Apr 2016 05:57:33 +0200 > "Shulgin, Oleksandr" wrote: > > > On Apr 1, 2016 02:57, "Karl O. Pinc" wrote: > > > > > > I assume there are no questions about supporting a > > > similar functionality only without PQsetSingleRowMode, >

Re: [HACKERS] Choosing parallel_degree

2016-04-11 Thread tushar
On 04/08/2016 08:53 PM, Robert Haas wrote: On Fri, Apr 8, 2016 at 1:22 AM, Amit Kapila wrote: Other than that, patch looks good and I have marked it as Ready For Committer. Hope, we get this for 9.6. Committed. I think this is likely to make parallel query significantly more usable in 9.6.

[HACKERS] Preprocessor condition fix

2016-04-11 Thread Christian Ullrich
Here is a one-line patch to fix a wrong preprocessor condition in pg_regress, found because the VS 2015 compiler warns on the cast in the 32-bit branch where apparently earlier versions did not. According to git grep, this is the only place where WIN64 is used without the leading underscore.

Re: [HACKERS] Relax requirement for INTO with SELECT in pl/pgsql

2016-04-11 Thread Merlin Moncure
On Sun, Apr 10, 2016 at 3:13 AM, Pavel Stehule wrote: > > Hi > > 2016-03-21 22:13 GMT+01:00 Pavel Stehule : >> >> Hi >> >> 2016-03-21 21:24 GMT+01:00 Merlin Moncure : >>> >>> Patch is trivial (see below), discussion is not :-). >>> >>> I see no useful reason to require INTO when returning data wit

Re: [HACKERS] [COMMITTERS] pgsql: Add the "snapshot too old" feature

2016-04-11 Thread Alvaro Herrera
Alexander Korotkov wrote: > Kevin, > > This commit makes me very uneasy. I didn't much care about this patch > mainly because I didn't imagine its consequences. Now, I see following: > > 1) We uglify buffer manager interface a lot. This patch adds 3 more > arguments to BufferGetPage(). It look

Re: [HACKERS] [COMMITTERS] pgsql: Add the "snapshot too old" feature

2016-04-11 Thread Alexander Korotkov
Kevin, This commit makes me very uneasy. I didn't much care about this patch mainly because I didn't imagine its consequences. Now, I see following: 1) We uglify buffer manager interface a lot. This patch adds 3 more arguments to BufferGetPage(). It looks weird. Should we try to find less inv

Re: [HACKERS] Execute ignoring cursor?

2016-04-11 Thread Pavel Stehule
2016-04-11 13:11 GMT+02:00 nummervet nummervet : > Ok, now i am getting this: > ERROR: could not identify column "151" in record data type > > Raise notice show that the column exists. > Any other way around it? > > hmm - it doesn't work for generic record - it should be typed row value. postgre

Re: [HACKERS] 2016-03 Commitfest

2016-04-11 Thread Andreas Karlsson
On 04/11/2016 01:35 PM, David Steele wrote: I've marked this committed so the 2016-03 CF is now complete! Thanks to you and everyone else involved in running this CF. You did an excellent job. Andreas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] 2016-03 Commitfest

2016-04-11 Thread Magnus Hagander
On Mon, Apr 11, 2016 at 1:35 PM, David Steele wrote: > On 4/10/16 11:14 PM, Andres Freund wrote: > > > On 2016-04-08 11:52:45 -0400, Robert Haas wrote: > > > >> In view of these circumstances, the RMT has voted to extend the > >> deadline for this particular patch by 2.5 days; that is, this patch

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-11 Thread Fujii Masao
On Mon, Apr 11, 2016 at 5:52 PM, Masahiko Sawada wrote: > On Mon, Apr 11, 2016 at 1:31 PM, Fujii Masao wrote: >> On Mon, Apr 11, 2016 at 10:58 AM, Masahiko Sawada >> wrote: >>> On Sat, Apr 9, 2016 at 12:32 PM, Tom Lane wrote: Jeff Janes writes: > When I compile now without cassert, I

Re: [HACKERS] 2016-03 Commitfest

2016-04-11 Thread David Steele
On 4/10/16 11:14 PM, Andres Freund wrote: > On 2016-04-08 11:52:45 -0400, Robert Haas wrote: > >> In view of these circumstances, the RMT has voted to extend the >> deadline for this particular patch by 2.5 days; that is, this patch >> may be committed with RMT approval no later than 2016-04-11 12

Re: [HACKERS] [COMMITTERS] pgsql: Move each SLRU's lwlocks to a separate tranche.

2016-04-11 Thread Andres Freund
Hi, On 2016-04-11 07:09:18 -0400, Robert Haas wrote: > Attached patch does that. Thanks. > > Additionally, doesn't this obsolete > > > > /* > > * Preferred alignment for disk I/O buffers. On some CPUs, copies between > > * user space and kernel space are significantly faster if the user buff

[HACKERS] Re[4]: [HACKERS] Execute ignoring cursor?

2016-04-11 Thread nummervet nummervet
Ok, now i am getting this: ERROR:  could not identify column "151" in record data type Raise notice show that the column exists. Any other way around it? >Пятница, 8 апреля 2016, 18:24 +03:00 от Pavel Stehule >: > > > >2016-04-08 16:46 GMT+02:00 nummervet nummervet < nummer...@mail.ru > : >>

Re: [HACKERS] [COMMITTERS] pgsql: Move each SLRU's lwlocks to a separate tranche.

2016-04-11 Thread Robert Haas
On Mon, Apr 11, 2016 at 12:01 AM, Andres Freund wrote: >> InitBufferPool() manually fixes up alignment; that should probably be >> removed now. Attached patch does that. >> I also wonder if it doesn't make sense to fix PG_CACHE_LINE_SIZE to >> 64byte on x86. I personally think a manual ifdef in

Re: [HACKERS] Speedup twophase transactions

2016-04-11 Thread Stas Kelvich
> On 08 Apr 2016, at 16:09, Stas Kelvich wrote: > > Tried on linux and os x 10.11 and 10.4. > > Still can’t reproduce, but have played around with your backtrace. > > I see in xlodump on slave following sequence of records: > > rmgr: Storage len (rec/tot): 16/42, tx: 0, ls

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-04-11 Thread Magnus Hagander
On Thu, Apr 7, 2016 at 5:15 AM, Noah Misch wrote: > On Wed, Apr 06, 2016 at 09:17:22AM +0200, Magnus Hagander wrote: > > On Wed, Apr 6, 2016 at 6:42 AM, Noah Misch wrote: > > > On Tue, Apr 05, 2016 at 08:15:16PM +0200, Magnus Hagander wrote: > > > > I've pushed this version, and also added the i

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-11 Thread Masahiko Sawada
On Mon, Apr 11, 2016 at 1:31 PM, Fujii Masao wrote: > On Mon, Apr 11, 2016 at 10:58 AM, Masahiko Sawada > wrote: >> On Sat, Apr 9, 2016 at 12:32 PM, Tom Lane wrote: >>> Jeff Janes writes: When I compile now without cassert, I get the compiler warning: >>> syncrep.c: In function 'Sync

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-11 Thread Etsuro Fujita
On 2016/04/11 12:30, Michael Paquier wrote: + if ((cancel = PQgetCancel(entry->conn))) + { + PQcancel(cancel, errbuf, sizeof(errbuf)); + PQfreeCancel(cancel); + } Wouldn't it be b

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-11 Thread Michael Paquier
On Mon, Apr 11, 2016 at 3:25 PM, Michael Paquier wrote: > Now, I have produced two patches: > - 0001-Support-for-VS-2015-locale-hack.patch, which makes use of > __crt_locale_data_public in ucrt/corecrt.h. This is definitely an ugly > hack, though I am coming to think that this may be the approach

Re: [HACKERS] Some other things about contrib/bloom and generic_xlog.c

2016-04-11 Thread Michael Paquier
On Sun, Apr 10, 2016 at 1:49 PM, Tom Lane wrote: > 1. It doesn't seem like generic_xlog.c has thought very carefully about > the semantics of the "hole" between pd_lower and pd_upper. The mainline > XLOG code goes to some lengths to ensure that the hole stays all-zeroes; > for example RestoreBloc