Re: Online enabling of checksums

2018-03-26 Thread Magnus Hagander
On Mon, Mar 26, 2018 at 10:09 PM, Tomas Vondra wrote: > Hi, > > I see enable_data_checksums() does this: > > if (cost_limit <= 0) > ereport(ERROR, > (errmsg("cost limit must be a positive value"))); > > Is there a reason not to allow -1 (no limit), just like for vacuum

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-26 Thread Michael Paquier
On Tue, Mar 27, 2018 at 04:47:41PM +1100, Haribabu Kommi wrote: > updated patch attached with additional doc updates as per the suggestion > from the upthreads. Thanks Hari for the quick update. It looks to me that this is shaped as suggested. Any input from other folks? I don't have more to sa

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-26 Thread Jeevan Chalke
On Tue, Mar 27, 2018 at 3:33 AM, Andres Freund wrote: > Hi, > > On 2018-03-23 17:01:54 +0530, Jeevan Chalke wrote: > > Attached patch which fixes that. > > Thanks, will push. For the future, I'd be more likely to notice if you > CC me ;) > Sure. Thanks. > > > > However, I am not sure whether i

Re: A design for amcheck heapam verification

2018-03-26 Thread Andrey Borodin
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed Hi! This patch adds handy code data structure (bloom filter) and

Re: GSOC 2018 Proposal review

2018-03-26 Thread Garima Natani
Hi Mark, Thanks a lot for your feedback towards improving my proposal. I have updated timeline as per changes suggested by you. Can you please review it again? Below is the link to shared google docs. *https://docs.google.com/document/d/1W2N8zRXvylj5cxW8aKHvPDKFgj0Qk59UIJr9SPwFsa8/edit#

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-26 Thread Haribabu Kommi
On Tue, Mar 27, 2018 at 3:03 PM, David G. Johnston < david.g.johns...@gmail.com> wrote: > On Mon, Mar 26, 2018 at 8:24 PM, Michael Paquier > wrote: > >> On Tue, Mar 27, 2018 at 11:43:27AM +1100, Haribabu Kommi wrote: >> > Patch attached with the above behavior along with other comments from >> >

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-26 Thread Amit Kapila
On Tue, Mar 27, 2018 at 3:08 AM, Robert Haas wrote: > On Sat, Mar 24, 2018 at 9:40 AM, Amit Kapila wrote: >> For me, it is equivalent to the master. The average of ten runs on >> the master is 20664.3683 and with all the patches applied it is >> 20590.4734. I think there is some run-to-run vari

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-26 Thread Michael Paquier
On Mon, Mar 26, 2018 at 09:03:17PM -0700, David G. Johnston wrote: > And yes, while I'm not sure right now what the multi-value condition logic > results in it should be mentioned - at least if the goal of the docs is to > be a sufficient resource for using these functions. In particular what > ha

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-03-26 Thread Amit Langote
On 2018/03/26 23:20, Alvaro Herrera wrote: > Pushed now. Thank you! > Amit Langote wrote: >> On 2018/03/24 9:23, Alvaro Herrera wrote: > >>> To fix this, I had to completely rework the "get partition parent root" >>> stuff into "get list of ancestors of this partition". >> >> I wondered if a is_

Re: idea - custom menu

2018-03-26 Thread Pavel Stehule
2018-03-27 2:44 GMT+02:00 Nikolay Samokhvalov : > On Sat, Mar 3, 2018 at 1:08 PM, Pavel Stehule > wrote: > >> Hi >> > > Hi Pavel, I don't know how I missed this email, but finally I found it :-) > > >> I am looking on project https://github.com/NikolayS/postgres_dba >> >> Nice idea, and perfect p

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-26 Thread David G. Johnston
On Mon, Mar 26, 2018 at 8:24 PM, Michael Paquier wrote: > On Tue, Mar 27, 2018 at 11:43:27AM +1100, Haribabu Kommi wrote: > > Patch attached with the above behavior along with other comments from > > upthread. > > Thanks for the updated version. > > The function changes look logically good to me.

Re: [HACKERS] pg_serial early wraparound

2018-03-26 Thread Thomas Munro
On Tue, Mar 27, 2018 at 5:50 AM, Tom Lane wrote: > Thomas Munro writes: >> Rebased again, now with a commit message. That assertion has since >> been removed (commit ec99dd5a) so the attached test script can once >> again be used to see the contents of pg_serial as the xid goes all the >> way ar

Re: PostgreSQL crashes with SIGSEGV

2018-03-26 Thread Peter Geoghegan
On Mon, Mar 26, 2018 at 5:14 AM, Kyotaro HORIGUCHI wrote: >> Attached patches do it that way. I'm happy with what I came up with, >> which is a lot simpler than my first approach. The extra copying seems >> likely to be well worth it, since it is fairly isolated in practice, >> especially on 9.6.

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-26 Thread Michael Paquier
On Tue, Mar 27, 2018 at 11:43:27AM +1100, Haribabu Kommi wrote: > Patch attached with the above behavior along with other comments from > upthread. Thanks for the updated version. The function changes look logically good to me. + + The PQhost function returns NULL when the + in

Re: [HACKERS] A design for amcheck heapam verification

2018-03-26 Thread Peter Geoghegan
On Fri, Mar 23, 2018 at 7:13 AM, Andrey Borodin wrote: > I've just flipped patch to WoA. But if above issues will be fixed I think > that patch is ready for committer. Attached is v7, which has the small tweaks that you suggested. Thank you for the review. I hope that this can be committed shor

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-26 Thread Chapman Flack
On 03/26/18 12:34, Tom Lane wrote: > If that's the argument, why is the WALInsertLockUpdateInsertingAt(CurrPos) > call still there? GetXLogBuffer() would do that too. "Because I hadn't noticed that," he said, sheepishly. > In any case, the new comment ... fails to > explain what's going on, and

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread David Rowley
On 27 March 2018 at 13:45, David Rowley wrote: > On 27 March 2018 at 12:49, Tom Lane wrote: >> Oh, I thought of another thing that would need to get done, if we decide >> to commit this. array_agg_serialize/deserialize only work if the array >> element type has send/receive functions. The plann

Re: [PATCH] Add missing type conversion functions for PL/Python

2018-03-26 Thread Haozhou Wang
Thanks Nikita! On Tue, Mar 27, 2018 at 12:07 AM, Nikita Glukhov wrote: > On 26.03.2018 17:19, David Steele wrote: > > On 2/20/18 10:14 AM, Haozhou Wang wrote: >> >>> Thank you very much for your review! >>> >>> I attached a new patch with typo fixed. >>> >> I think it's a bit premature to mark t

Fix for pg_stat_activity putting client hostaddr into appname field

2018-03-26 Thread Edmund Horner
I noticed when querying pg_stat_activity (in 10.1): $ SELECT pid, application_name, client_hostname, backend_type FROM pg_stat_activity; pid | application_name |client_hostname| backend_type ---+--+---+

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-26 Thread Thomas Munro
On Tue, Mar 27, 2018 at 3:30 PM, Thomas Munro wrote: > I hacked something up in Python # otool -tvV | \ In case anyone is interested in trying that, it should be "otool -tvV [path to postgres executable compiled with -O0]" (meaning disassemble it). I was removing my home directory from the pa

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-26 Thread Thomas Munro
On Tue, Mar 27, 2018 at 12:23 PM, Tom Lane wrote: > Querying for other functions marked 'r' leaves me with some other related > doubts: > > 1. Why are the various flavors of pg_get_viewdef() marked 'r'? Surely > reading the catalogs is a thing parallel children are allowed to do. > If there is a

Re: ppc64le support in 9.3 branch?

2018-03-26 Thread Mark Wong
On Tue, Mar 27, 2018 at 11:09:37AM +1030, Andrew Dunstan wrote: > On Tue, Mar 27, 2018 at 10:57 AM, Alvaro Herrera > wrote: > > Andrew Dunstan wrote: > > > >> I am in discussions with Mark, he's going to disable the animals from > >> building 9.3. (by setting branches_to_build to 'HEAD_PLUS_LATEST

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-26 Thread Michael Paquier
On Sat, Mar 24, 2018 at 09:12:09PM -0400, Stephen Frost wrote: > Yeah, neither 2 or 3 really appeals to me. Option 1 does touch a number > of places but in a pretty straight-forward way- and if there's a typo > there, the compiler is likely to complain, so it seems like the risk is > relatively lo

Re: GSOC 2018 Proposal review

2018-03-26 Thread Mark Wong
Hi Garima, On Tue, Mar 20, 2018 at 10:36:05PM +0530, Garima Natani wrote: > Hi All, > > I am looking forward to working with PostgreSQL organization In GSoC 2018. Glad to see you're interested in this project! > I have created Proposal for "Develop Performance Farm Database and Website" > Proje

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-26 Thread Michael Paquier
On Tue, Mar 27, 2018 at 01:32:41AM +0900, Fujii Masao wrote: > +1. It's better for us to focus on the code change of the fillter on pg_rewind > rather than such "refactoring". (filter takes one 'l', not two) Okay. I had my mind mostly focused on how to shape the exclusion list and get it shared

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread David Rowley
On 27 March 2018 at 12:49, Tom Lane wrote: > I wrote: >> The main thing that remains undone is to get some test coverage --- >> AFAICS, none of these new functions get exercised in the standard >> regression tests. > > Oh, I thought of another thing that would need to get done, if we decide > to c

Re: idea - custom menu

2018-03-26 Thread Nikolay Samokhvalov
On Sat, Mar 3, 2018 at 1:08 PM, Pavel Stehule wrote: > Hi > Hi Pavel, I don't know how I missed this email, but finally I found it :-) > I am looking on project https://github.com/NikolayS/postgres_dba > > Nice idea, and perfect publicity of typical PostgreSQL maintenance task. > Thank you! I

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-26 Thread Haribabu Kommi
On Tue, Mar 27, 2018 at 12:23 AM, Michael Paquier wrote: > On Mon, Mar 26, 2018 at 11:39:51PM +1100, Haribabu Kommi wrote: > > And if we are not going to change the above, then PQhost() function > > returns 3 values, > > - NULL when the conn==NULL > > - Actual host or hostaddr of the active conne

Re: ppc64le support in 9.3 branch?

2018-03-26 Thread Andrew Dunstan
On Tue, Mar 27, 2018 at 10:57 AM, Alvaro Herrera wrote: > Andrew Dunstan wrote: > >> I am in discussions with Mark, he's going to disable the animals from >> building 9.3. (by setting branches_to_build to 'HEAD_PLUS_LATEST4' >> instead of 'ALL'). > > So once we desupport 9.3, they'll stop building

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread David Rowley
On 27 March 2018 at 13:26, Alvaro Herrera wrote: > David Rowley wrote: > >> Anyway, the options are not zero for anyone who is strongly affected >> with no other workaround. They just need to disable parallel query, >> which to me seems fairly similar to the 8.4 release note's "the >> previous beh

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Tomas Vondra
Hi Dean, Here is an updated patch (hopefully) fixing the bugs you've reported so far. In particular, it fixes this: 1) mostly harmless memset bug in UpdateStatisticsForTypeChange 2) passing the right list (stat_clauses) to mcv_clauselist_selectivity 3) corrections to a couple of outdated commen

Re: ppc64le support in 9.3 branch?

2018-03-26 Thread Alvaro Herrera
Andrew Dunstan wrote: > I am in discussions with Mark, he's going to disable the animals from > building 9.3. (by setting branches_to_build to 'HEAD_PLUS_LATEST4' > instead of 'ALL'). So once we desupport 9.3, they'll stop building 9.4? :-) -- Álvaro Herrerahttps://www.2ndQuadra

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Alvaro Herrera
David Rowley wrote: > Anyway, the options are not zero for anyone who is strongly affected > with no other workaround. They just need to disable parallel query, > which to me seems fairly similar to the 8.4 release note's "the > previous behavior can be restored by disabling enable_hashagg" synch

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Andres Freund
Hi, On 2018-03-27 13:14:15 +1300, David Rowley wrote: > I have to say, it really would be a shame to have this concern block > us from future optimisations in aggregation. Yea, I think that's an important point. By the dint of Tom's argument we're never going to be able to provide parallelism for

Re: ppc64le support in 9.3 branch?

2018-03-26 Thread Andrew Dunstan
On Mon, Mar 26, 2018 at 4:50 PM, Michael Paquier wrote: > On Fri, Mar 23, 2018 at 08:01:53PM +0100, Magnus Hagander wrote: >> On Fri, Mar 23, 2018 at 7:58 PM, Andres Freund wrote: >>> I'm inclined to just ask them to stop running the animals on that >>> branch. There are no pre-existing users on

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread David Rowley
On 27 March 2018 at 11:28, Tom Lane wrote: > David Rowley writes: >> This very much reminds me of something that exists in the 8.4 release notes: >>> SELECT DISTINCT and UNION/INTERSECT/EXCEPT no longer always produce sorted >>> output (Tom) > > That's a completely false analogy: we got a signif

Re: CALL optional in PL/pgSQL

2018-03-26 Thread Andrew Dunstan
On Fri, Mar 2, 2018 at 2:01 AM, Tom Lane wrote: > Pavel Stehule writes: >> 2018-03-01 5:51 GMT+01:00 Peter Eisentraut >> This seems to be a popular issue when porting from PL/SQL, so I'll throw >>> it out here for discussion. Apparently, in PL/SQL you can call another >>> procedure without the C

Re: Proposal: http2 wire format

2018-03-26 Thread Craig Ringer
On 26 March 2018 at 22:56, Tom Lane wrote: > Damir Simunic writes: > >> On 26 Mar 2018, at 11:06, Vladimir Sitnikov < > sitnikov.vladi...@gmail.com> wrote: > >>> If anyone finds the idea of Postgres speaking http2 appealing > > TBH, this sounds like a proposal to expend a whole lot of work (much

Re: Proposal: http2 wire format

2018-03-26 Thread Stephen Frost
Greetings, * Craig Ringer (cr...@2ndquadrant.com) wrote: > On 26 March 2018 at 21:05, Damir Simunic > wrote: > > The same goes for the ‘authorization’ header. Postgres does not support > > Bearer token authorization today. But maybe you’ll be able to define a > > function that knows how to deal w

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Tom Lane
I wrote: > The main thing that remains undone is to get some test coverage --- > AFAICS, none of these new functions get exercised in the standard > regression tests. Oh, I thought of another thing that would need to get done, if we decide to commit this. array_agg_serialize/deserialize only work

Re: Proposal: http2 wire format

2018-03-26 Thread Craig Ringer
On 26 March 2018 at 21:05, Damir Simunic wrote: > > On 26 Mar 2018, at 11:06, Vladimir Sitnikov > wrote: > > > > Hi, > > > > >If anyone finds the idea of Postgres speaking http2 appealing > > > > HTTP/2 sounds interesting. > > What do you think of https://grpc.io/ ? > > > > Have you evaluated it

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-26 Thread Tom Lane
Thomas Munro writes: > On Tue, Mar 27, 2018 at 11:58 AM, Tom Lane wrote: >> I wonder whether we shouldn't mark *all* of these parallel-unsafe. >> I'm not exactly convinced that 'restricted' is sufficient for the >> others, and even if it is, there's certainly little if any upside >> for letting t

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-26 Thread Thomas Munro
On Tue, Mar 27, 2018 at 11:58 AM, Tom Lane wrote: > # select proname, proparallel from pg_proc where proname like 'binary_upg%'; > proname | proparallel > +- > binary_upgrade_create_empty_extension |

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-26 Thread Tom Lane
... BTW: # select proname, proparallel from pg_proc where proname like 'binary_upg%'; proname | proparallel +- binary_upgrade_create_empty_extension | r binary_upgrade_set_next_array_pg_type_oid |

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-26 Thread Tom Lane
Thomas Munro writes: > Here's a single character patch to mark > that function PARALLEL UNSAFE. Ugh. Clearly a bug. > Obviously that'll affect only newly > initdb'd clusters after this patch, but that's what people have in a > pg_upgrade scenario. We're a bit fortunate on that. I wonder if th

Parallel safety of binary_upgrade_create_empty_extension

2018-03-26 Thread Thomas Munro
Hi, My colleague Richard Yen came across this situation: pg_restore: [archiver (db)] could not execute query: ERROR: cannot assign XIDs during a parallel operation Command was: -- For binary upgrade, create an empty extension and insert objects into it DROP EXTENSION IF EXISTS "btree_gin"; S

Re: Proposal: http2 wire format

2018-03-26 Thread David G. Johnston
On Mon, Mar 26, 2018 at 1:05 PM, Damir Simunic wrote: > Would it be the only protocol supported? What if I wanted JSON or CSV > returned, or just plain old Postgres v3 binary format, since I already have > the parser written for it? Wouldn’t you need to first solve the problem of > content negoti

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Tom Lane
David Rowley writes: > On 27 March 2018 at 09:27, Tom Lane wrote: >> I do not think it is accidental that these aggregates are exactly the ones >> that do not have parallelism support today. Rather, that's because you >> just about always have an interest in the order in which the inputs get >>

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Tomas Vondra
On 03/26/2018 11:19 PM, Tom Lane wrote: > Tomas Vondra writes: >> On 03/26/2018 10:27 PM, Tom Lane wrote: >>> I fear that what will happen, if we commit this, is that something like >>> 0.01% of the users of array_agg and string_agg will be pleased, another >>> maybe 20% will be unaffected because

Re: Backend memory dump analysis

2018-03-26 Thread Tom Lane
Vladimir Sitnikov writes: > It takes just a single pass to compute "total" (and it takes no memory), so > it would be much better if "TopMemoryContext: ..." was replaced with > "Total memory used by all contexts is XXX bytes" > Current TopMemoryContext row is extremely misleading. This may or may

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-26 Thread Peter Geoghegan
On Mon, Mar 26, 2018 at 12:17 PM, Simon Riggs wrote: >> As far as I >> know, the proposed MERGE patch has that issue an existing DML commands >> don't; but someone else may have better information. > > I will look deeper and report back. It's quite clear that the problem exists with the MERGE pat

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread David Rowley
On 27 March 2018 at 09:27, Tom Lane wrote: > I spent a fair amount of time hacking on this with intent to commit, > but just as I was getting to code that I liked, I started to have second > thoughts about whether this is a good idea at all. I quote from the fine > manual: > > The aggregate f

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-26 Thread Andres Freund
Hi, On 2018-03-23 17:01:54 +0530, Jeevan Chalke wrote: > Attached patch which fixes that. Thanks, will push. For the future, I'd be more likely to notice if you CC me ;) > However, I am not sure whether it is expected to have stable regression run > with installcheck having local settings. > Fo

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread David Rowley
On 27 March 2018 at 10:20, Stephen Frost wrote: > In the end, I do tend to agree that we probably should add parallel > support to these aggregates, but it'd also be nice to hear from those > who had worked to add parallelism to the various aggregates as to if > there was some reason these were sk

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-26 Thread Robert Haas
On Sat, Mar 24, 2018 at 9:40 AM, Amit Kapila wrote: > For me, it is equivalent to the master. The average of ten runs on > the master is 20664.3683 and with all the patches applied it is > 20590.4734. I think there is some run-to-run variation, but more or > less there is no visible degradation.

Re: Proposal: http2 wire format

2018-03-26 Thread Damir Simunic
> Currently it is implemented via different v3 messages (parse, bind, execute, > row description, data row, etc etc). > > The claim is *any* implementation "on top of HTTP/2" would basically require > to implement those "parse, bind, execute, row data, etc" *messages*. Why? Wouldn’t you be abl

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Tom Lane
Stephen Frost writes: > Yeah, there certainly seems like a lot of opportunity for the ordering > to end up being volatile already and queries depending on it not > changing really shouldn't be making that assumption. I do think it was > probably a bad move on our part to say that ordering a subqu

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Tom Lane
Tomas Vondra writes: > On 03/26/2018 10:27 PM, Tom Lane wrote: >> I fear that what will happen, if we commit this, is that something like >> 0.01% of the users of array_agg and string_agg will be pleased, another >> maybe 20% will be unaffected because they wrote ORDER BY which prevents >> paralle

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On 03/26/2018 10:27 PM, Tom Lane wrote: > > I fear that what will happen, if we commit this, is that something like > > 0.01% of the users of array_agg and string_agg will be pleased, another > > maybe 20% will be unaffected because

Re: Proposal: http2 wire format

2018-03-26 Thread Alvaro Hernandez
On 26/03/18 21:57, Damir Simunic wrote: On 26 Mar 2018, at 15:42, Alvaro Hernandez > wrote: On 26/03/18 13:11, Damir Simunic wrote: On 26 Mar 2018, at 11:13, Vladimir Sitnikov mailto:sitnikov.vladi...@gmail.com>> wrote: Damir> * What are the criteria for getting

Re: Index scan prefetch?

2018-03-26 Thread Claudio Freire
On Mon, Mar 26, 2018 at 4:59 PM, Justin Pryzby wrote: >> But now effective_io_concurrency parameter is applicable only for bitmap > ... >> Will it be useful to support it also for index scan? >> Or there are some other ways to address this problem? > > Does your case perform well with bitmap heap

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Dean Rasheed
On 26 March 2018 at 20:17, Tomas Vondra wrote: > On 03/26/2018 09:01 PM, Dean Rasheed wrote: >> Also, just above that, in statext_clauselist_selectivity(), it >> computes the list stat_clauses, then doesn't appear to use it >> anywhere. I think that would have been the appropriate thing to pass >>

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Tomas Vondra
On 03/26/2018 10:27 PM, Tom Lane wrote: > David Rowley writes: >> [ combinefn_for_string_and_array_aggs_v7.patch ] > > I spent a fair amount of time hacking on this with intent to commit, > but just as I was getting to code that I liked, I started to have second > thoughts about whether this is a

Re: Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2018-03-26 Thread Mark Rofail
On 3/7/18 5:43 AM, Alvaro Herrera wrote: I searched for the new GIN operator so that I > could brush it up for commit ahead of the rest -- only to find out that > it was eviscerated from the patch between v5 and v5.1. > > The latest version of the patch which contained the new GIN operator is vers

Re: Proposal: http2 wire format

2018-03-26 Thread Damir Simunic
Hi Andres, > > At least I do *NOT* want many protocols in core. We've a hard enough > time to keep up with integrating patches and maintenance to not just > willy nilly integrate multiple new features with unclear lifetimes. Admire your effort in applying all these patches—this commitfest thing

Re: Proposal: http2 wire format

2018-03-26 Thread Vladimir Sitnikov
Damir>Wouldn’t that be protocol semantics? Framing is already taken care of by the wire protocol. Apparently I'm using the wrong word. I do mean protocol semantics. Damir>But can you pull off grpc.. Would it be the only protocol supported? Of course there will be lots of "older clients" For inst

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Tom Lane
David Rowley writes: > [ combinefn_for_string_and_array_aggs_v7.patch ] I spent a fair amount of time hacking on this with intent to commit, but just as I was getting to code that I liked, I started to have second thoughts about whether this is a good idea at all. I quote from the fine manual:

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Andres Freund
On 2018-03-26 11:31:36 -0700, Andres Freund wrote: > On 2018-03-26 11:14:03 -0400, Robert Haas wrote: > > On Sat, Mar 24, 2018 at 4:33 PM, Andres Freund wrote: > > > I don't think it really changes the need, but it's worthwhile to note > > > that gcc-8 warns about this now: > > > /home/andres/src/

Re: Proposal: http2 wire format

2018-03-26 Thread Vladimir Sitnikov
It could make sense to arrange a Google Hangouts conversation (or alike). Hangouts allows to record sessions with up to 10 speakers and unlimited listeners. The recording can be shared via YouTube. Damir>Funny you agree with that—for someone having the experience of writing a driver and having a l

Re: Online enabling of checksums

2018-03-26 Thread Tomas Vondra
Hi, I see enable_data_checksums() does this: if (cost_limit <= 0) ereport(ERROR, (errmsg("cost limit must be a positive value"))); Is there a reason not to allow -1 (no limit), just like for vacuum_cost? regards -- Tomas Vondra http://www.2ndQuadr

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Andres Freund
On 2018-03-26 15:26:00 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-02-06 15:43:29 -0500, Tom Lane wrote: > >> void* isn't necessarily compatible with function pointers --- there are > >> platforms where they're physically different widths, though possibly > >> you'd never get PG to

Re: Proposal: http2 wire format

2018-03-26 Thread Damir Simunic
> On 26 Mar 2018, at 18:09, Vladimir Sitnikov > wrote: > > Damir>Postgres doesn’t know about grpc, s > > I'm afraid you are missing the point. > I would say PostgreSQL doesn't know about HTTP/2. > It is the same as "PostgreSQL doesn't know about grpc". > > Here's a quote from your pg_h2 repo:

Re: JIT compiling with LLVM v12.2

2018-03-26 Thread Andres Freund
Hi, On 2018-03-25 00:07:11 +1300, Thomas Munro wrote: > I spotted a couple of typos and some very minor coding details -- see > please see attached. Thanks, applying 0001 in a bit. > From 648e303072c77e781eca2bb06f488f6be9ccac84 Mon Sep 17 00:00:00 2001 > From: Thomas Munro > Date: Sat, 24 Mar

Re: Index scan prefetch?

2018-03-26 Thread Justin Pryzby
On Mon, Mar 26, 2018 at 12:43:02PM +0300, Konstantin Knizhnik wrote: > Hi, hackers. > > I was faced with the following bad performance use case with Postgres: there > is a huge append-only table with serial key (ID) > which is permanently appended using multithreaded pgloader. I think this could

Re: WIP: Covering + unique indexes.

2018-03-26 Thread Peter Geoghegan
On Mon, Mar 26, 2018 at 3:10 AM, Alexander Korotkov wrote: > So, as I get you're proposing to introduce INDEX_ALT_TID_MASK flag > which would indicate that we're storing something special in the t_tid > offset. And that should help us not only for covering indexes, but also for > further btree en

Re: Re: csv format for psql

2018-03-26 Thread Pavel Stehule
2018-03-26 20:26 GMT+02:00 Daniel Verite : > Pavel Stehule wrote: > > > b) the list of pset options is bloating - every possible new format can > > introduce fieldsep_X option > > What new format? > The usefulness of fieldsep does not extend outside of xSV, and it's > no suprise that there

Re: committing inside cursor loop

2018-03-26 Thread Peter Eisentraut
On 3/19/18 20:40, Peter Eisentraut wrote: > On 3/14/18 08:05, Ildus Kurbangaliev wrote: >>> The ROLLBACK call in the first loop iteration undoes the UPDATE >>> command that drives the loop. Is it then sensible to continue the >>> loop? >>> >> I think that in the first place ROLLBACK was prohibited

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Tom Lane
Andres Freund writes: > On 2018-02-06 15:43:29 -0500, Tom Lane wrote: >> void* isn't necessarily compatible with function pointers --- there are >> platforms where they're physically different widths, though possibly >> you'd never get PG to run on such hardware anyway. > Btw, given that we store

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Tomas Vondra
On 03/26/2018 09:01 PM, Dean Rasheed wrote: > On 18 March 2018 at 23:57, Tomas Vondra wrote: >> Attached is an updated version of the patch series, addressing issues >> pointed out by Alvaro. > > I've just been reading the new code in > statext_clauselist_selectivity() and mcv_clauselist_select

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-26 Thread Simon Riggs
On 26 March 2018 at 17:52, Robert Haas wrote: > On Mon, Mar 26, 2018 at 12:16 PM, Simon Riggs wrote: >> On 26 March 2018 at 16:09, Robert Haas wrote: >>> On Mon, Mar 26, 2018 at 5:53 AM, Simon Riggs wrote: Since we now have MVCC catalog scans, all the name lookups are performed using

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Tomas Vondra
On 03/26/2018 06:21 PM, Dean Rasheed wrote: > On 26 March 2018 at 14:08, Tomas Vondra wrote: >> On 03/26/2018 12:31 PM, Dean Rasheed wrote: >>> A wider concern I have is that I think this function is trying to be >>> too clever by only resetting selected stats. IMO it should just reset >>> all st

Re: Proposal: http2 wire format

2018-03-26 Thread Andres Freund
Hi, On 2018-03-26 20:36:09 +0200, Damir Simunic wrote: > If so, I’m not suggesting we get rid of FEBE, but leave it as is and > complement it with a widely understood and supported protocol, that in fact > takes compatibility way more seriously than FEBE. Just leave v3 frozen. Seems > like ulti

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Dean Rasheed
On 18 March 2018 at 23:57, Tomas Vondra wrote: > Attached is an updated version of the patch series, addressing issues > pointed out by Alvaro. I've just been reading the new code in statext_clauselist_selectivity() and mcv_clauselist_selectivity(), and I'm having a hard time convincing myself th

Re: Proposal: http2 wire format

2018-03-26 Thread Damir Simunic
> On 26 Mar 2018, at 15:42, Alvaro Hernandez wrote: > > > > On 26/03/18 13:11, Damir Simunic wrote: >>> On 26 Mar 2018, at 11:13, Vladimir Sitnikov >> > wrote: >>> >>> Damir> * What are the criteria for getting this into the core? >>> Craig>Mine would be:

Re: Proposal: http2 wire format

2018-03-26 Thread Damir Simunic
> On 26 Mar 2018, at 18:19, Vladimir Sitnikov > wrote: > > Tom>But starting from the assumption that HTTP2 solves our problems seems to > me to be "Here's a hammer. > > Agree. Funny you agree with that—for someone having the experience of writing a driver and having a long list of things th

Re: Proposal: http2 wire format

2018-03-26 Thread Damir Simunic
> On 26 Mar 2018, at 16:56, Tom Lane wrote: > > Damir Simunic writes: >>> On 26 Mar 2018, at 11:06, Vladimir Sitnikov >>> wrote: If anyone finds the idea of Postgres speaking http2 appealing > > TBH, this sounds like a proposal to expend a whole lot of work (much of it > outside the cor

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Andres Freund
Hi, On 2018-02-06 15:43:29 -0500, Tom Lane wrote: > void* isn't necessarily compatible with function pointers --- there are > platforms where they're physically different widths, though possibly > you'd never get PG to run on such hardware anyway. Btw, given that we store function pointers in dat

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Andres Freund
On 2018-03-26 11:14:03 -0400, Robert Haas wrote: > On Sat, Mar 24, 2018 at 4:33 PM, Andres Freund wrote: > > I don't think it really changes the need, but it's worthwhile to note > > that gcc-8 warns about this now: > > /home/andres/src/postgresql/src/backend/postmaster/bgworker.c: In function >

Re: Re: csv format for psql

2018-03-26 Thread Daniel Verite
Pavel Stehule wrote: > b) the list of pset options is bloating - every possible new format can > introduce fieldsep_X option What new format? The usefulness of fieldsep does not extend outside of xSV, and it's no suprise that there have been no other use for a fieldsep-like variable until

Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently

2018-03-26 Thread Claudio Freire
On Mon, Mar 26, 2018 at 2:46 PM, Claudio Freire wrote: > On Mon, Mar 26, 2018 at 11:26 AM, Claudio Freire > wrote: >> On Mon, Mar 26, 2018 at 11:19 AM, Tom Lane wrote: >>> Claudio Freire writes: On Sat, Mar 24, 2018 at 4:17 PM, Tom Lane wrote: > I hadn't paid any attention to this pa

Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently

2018-03-26 Thread Claudio Freire
On Mon, Mar 26, 2018 at 11:26 AM, Claudio Freire wrote: > On Mon, Mar 26, 2018 at 11:19 AM, Tom Lane wrote: >> Claudio Freire writes: >>> On Sat, Mar 24, 2018 at 4:17 PM, Tom Lane wrote: I hadn't paid any attention to this patch previously, so maybe I'm missing something ... but this

Re: Proposal: http2 wire format

2018-03-26 Thread Alvaro Hernandez
On 26/03/18 13:11, Damir Simunic wrote: On 26 Mar 2018, at 11:13, Vladimir Sitnikov mailto:sitnikov.vladi...@gmail.com>> wrote: Damir> * What are the criteria for getting this into the core? Craig>Mine would be: +1 There's a relevant list as well: https://github.com/pgjdbc/pgjdbc/blob/mast

Re: WIP: a way forward on bootstrap data

2018-03-26 Thread John Naylor
> On Mar 26, 2018, at 10:44 PM, Tom Lane wrote > Layout of .dat files seems generally reasonable, but I don't understand > the proposed make rule: > > +reformat-dat-files: > +$(PERL) -I $(catalogdir) $< catalog/rewrite_dat.pl -o catalog > catalog/pg_*.dat > > This rule has no prerequisit

Re: PATCH: Exclude temp relations from base backup

2018-03-26 Thread David Steele
On 3/26/18 1:06 PM, Stephen Frost wrote: > > * Teodor Sigaev (teo...@sigaev.ru) wrote: >> Will autovacuum (or something else) complain about absense of relfile during >> orphan table deleting? I mean, you get a base backup without temp tables, >> then you try to run postgres on it and will it comp

Re: PATCH: Exclude temp relations from base backup

2018-03-26 Thread Stephen Frost
Greetings, * Teodor Sigaev (teo...@sigaev.ru) wrote: > Will autovacuum (or something else) complain about absense of relfile during > orphan table deleting? I mean, you get a base backup without temp tables, > then you try to run postgres on it and will it complain about existing > record in pg_cl

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-26 Thread Robert Haas
On Mon, Mar 26, 2018 at 12:16 PM, Simon Riggs wrote: > On 26 March 2018 at 16:09, Robert Haas wrote: >> On Mon, Mar 26, 2018 at 5:53 AM, Simon Riggs wrote: >>> Since we now have MVCC catalog scans, all the name lookups are >>> performed using the same snapshot so in the above scenario the newly

Re: [HACKERS] pg_serial early wraparound

2018-03-26 Thread Tom Lane
Thomas Munro writes: > Rebased again, now with a commit message. That assertion has since > been removed (commit ec99dd5a) so the attached test script can once > again be used to see the contents of pg_serial as the xid goes all the > way around, if you build with TEST_OLDSERXID defined so that >

Re: new function for tsquery creartion

2018-03-26 Thread Dmitry Ivanov
Patch 03 (the documentation) needed some proof-reading. I've attached a new version of that patch with some small suggested improvements. Thanks, I'm definitely going to use this. Is there anything to_tsquery() can do that websearch_to_tsquery() can't? Currently, no. Would it be OK to use

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-26 Thread Tom Lane
Chapman Flack writes: > On 03/25/18 23:27, Stephen Frost wrote: >> AdvanceXLInsertBuffer() does quite a bit, so I'm a bit surprised to see >> this simply removing that call, you're confident there's nothing done >> which still needs doing..? > My belief from looking at the code was that AdvanceXL

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-26 Thread Fujii Masao
On Sun, Mar 25, 2018 at 5:06 PM, Michael Paquier wrote: > On Sat, Mar 24, 2018 at 11:14:34PM -0400, Tom Lane wrote: >> Stephen Frost writes: >>> I don't completely buy off on the argument that having these #define's >>> would make it easier for forks (we've had quite a few folks fork PG, but >>>

  1   2   >