Re: Make pg_checksums complain if compiled BLCKSZ and data folder's block size differ

2019-03-16 Thread Fabien COELHO
Bonjour Michaël, If the block size the tool is compiled with does not match the data folder block size, then users would get incorrect checksums failures, Or worse, incorrect checksump writing under "enabling"? Initial proposal: "%s: data directory block size %d is different to compiled-in

Re: libpq environment variables in the server

2019-03-16 Thread Peter Eisentraut
On 2019-03-15 16:01, Noah Misch wrote: { local %ENV; delete $ENV{PGAPPNAME}; ... } >>> >>> That doesn't work because the first line clears the entire environment. >> >> The solution to that is to do 'local %ENV = %ENV;', to assign a copy of >> the original

RE: Timeout parameters

2019-03-16 Thread Fabien COELHO
Hello, Fabien, I was wondering whether you can apply TCP_USER_TIMEOUT patch and continue discussion about 'socket_timeout'? I can apply nothing, I'm just a small-time reviewer. Committers on the thread are Michaël-san and Robert, however I'm not sure that they are very sensitive to "please

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Heikki Linnakangas
On 16/03/2019 06:16, Peter Geoghegan wrote: On Thu, Mar 14, 2019 at 2:21 AM Heikki Linnakangas wrote: It doesn't matter how often it happens, the code still needs to deal with it. So let's try to make it as readable as possible. Well, IMHO holding the buffer and the bounds in the new struct

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Peter Geoghegan
On Sat, Mar 16, 2019 at 1:44 AM Heikki Linnakangas wrote: > > It would be nice if you could take a look at the amcheck "relocate" > > patch > When I started looking at this, I thought that "relocate" means "move". > So I thought that the new mode would actually move tuples, i.e. that it > would mo

Re: jsonpath

2019-03-16 Thread Alexander Korotkov
On Thu, Mar 14, 2019 at 12:07 PM Alexander Korotkov wrote: > On Sun, Mar 10, 2019 at 1:51 PM Alexander Korotkov > wrote: > > On Wed, Mar 6, 2019 at 12:40 AM Nikita Glukhov > > wrote: > > > Attached 36th version of the patches. > > > > Thank yo for the revision! > > > > In the attached revision

Re: seems like a bug in pgbench -R

2019-03-16 Thread Fabien COELHO
Hello Tomas, So I guess this is a bug in 12788ae49e1933f463bc59a6efe46c4a01701b76, or one of the other commits touching this part of the code. I could not reproduce this issue on head, but I confirm on 11.2. AFAICS on head it's fixed by 3bac77c48f166b9024a5ead984df73347466ae12 Thanks for

Re: Make pg_checksums complain if compiled BLCKSZ and data folder's block size differ

2019-03-16 Thread Magnus Hagander
On Sat, Mar 16, 2019 at 2:22 AM Michael Paquier wrote: > Hi all, > (related folks in CC) > > Sergei Kornilov has reported here an issue with pg_checksums: > > https://www.postgresql.org/message-id/5217311552474...@myt2-66bcb87429e6.qloud-c.yandex.net > > If the block size the tool is compiled wit

Re: [PATCH v20] GSSAPI encryption support

2019-03-16 Thread Stephen Frost
Greetings! * Robbie Harwood (rharw...@redhat.com) wrote: > Stephen Frost writes: > > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > >> Or maybe we avoid that, and you rename be-secure-gssapi.c to just > >> be-gssapi.c and also combine that with the contents of > >> be-gssapi-commo

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-16 Thread Dean Rasheed
On Fri, 15 Mar 2019 at 00:06, Tomas Vondra wrote: > I've noticed an annoying thing when modifying type of column not > included in a statistics... > > That is, we don't remove the statistics, but the estimate still changes. > But that's because the ALTER TABLE also resets reltuples/relpages: > > T

Re: Change ereport level for QueuePartitionConstraintValidation

2019-03-16 Thread Justin Pryzby
On Fri, Mar 15, 2019 at 12:55:36PM +0300, Sergei Kornilov wrote: > We have INFO ereport messages in alter table attach partition like this: > > partition constraint for table \"%s\" is implied by existing constraints > > So now I am +1 to idea of change error level for this messages. I attach > p

Re: Rare SSL failures on eelpout

2019-03-16 Thread Thomas Munro
On Wed, Mar 6, 2019 at 9:21 AM Tom Lane wrote: > Thomas Munro writes: > > Bleugh. I think this OpenSSL package might just be buggy on ARM. On > > x86, apparently the same version of OpenSSL and all other details of > > the test the same, I can see that SSL_connect() returns <= 0 > > (failure),

Re: What is a savepointLevel ?

2019-03-16 Thread Edmund Horner
On Fri, 15 Mar 2019 at 18:18, Chapman Flack wrote: > What exactly is a savepointLevel? > > They seem to have been there for 15 years[1], diligently copied from > parent transactions to children, fastidiously checked to avoid crossing > a level on rollback or release, but does anything ever change

Re: pg_basebackup ignores the existing data directory permissions

2019-03-16 Thread Robert Haas
On Thu, Mar 14, 2019 at 7:34 PM Peter Eisentraut wrote: > I think the potential problems of getting this wrong are bigger than the > issue we are trying to fix. I think the question is: how do we know what the user intended? If the user wants the directory to be accessible only to the owner, the

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-03-16 Thread Robert Haas
On Fri, Mar 15, 2019 at 9:50 PM Tom Lane wrote: > Yun Li writes: > > Do you think if we can add queryId into the pg_stat_get_activity function > > and ultimatly expose it in the view? It would be easier to track "similar" > > query's performance over time easier. > > No, we're not likely to do th

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-16 Thread Tomas Vondra
On 3/16/19 11:55 AM, Dean Rasheed wrote: > On Fri, 15 Mar 2019 at 00:06, Tomas Vondra > wrote: >> I've noticed an annoying thing when modifying type of column not >> included in a statistics... >> >> That is, we don't remove the statistics, but the estimate still changes. >> But that's because

Unduly short fuse in RequestCheckpoint

2019-03-16 Thread Tom Lane
I noticed an odd buildfarm failure today: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sungazer&dt=2019-03-16%2012%3A12%3A20 of which the key bit seems to be 2019-03-16 15:20:43.835 UTC [10879304] 003_promote.pl LOG: received replication command: BASE_BACKUP LABEL 'pg_basebackup bas

Re: Index Skip Scan

2019-03-16 Thread Dmitry Dolgov
> On Fri, Mar 15, 2019 at 4:55 AM Kyotaro HORIGUCHI > wrote: > I have some comments on the latest v11 patch. Thank you! > L619: > > +indexstate->ioss_NumDistinctKeys = node->distinctPrefix; > > The number of distinct prefix keys has various names in this > patch. They should be unified as

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Heikki Linnakangas
On 16/03/2019 10:51, Peter Geoghegan wrote: On Sat, Mar 16, 2019 at 1:44 AM Heikki Linnakangas wrote: It would be nice if you could take a look at the amcheck "relocate" patch When I started looking at this, I thought that "relocate" means "move". So I thought that the new mode would actually

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-03-16 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 15, 2019 at 9:50 PM Tom Lane wrote: >> pg_stat_statements has a notion of query ID, but that notion might be >> quite inappropriate for other usages, which is why it's an extension >> and not core. > Having written an extension that also wanted a query ID, I dis

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Peter Geoghegan
On Sat, Mar 16, 2019 at 9:17 AM Heikki Linnakangas wrote: > Hmm. "re-find", maybe? We use that term in a few error messages already, > to mean something similar. WFM. > At first, I thought this would be horrendously expensive, but thinking > about it a bit more, nearby tuples will always follow

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Peter Geoghegan
On Sat, Mar 16, 2019 at 9:55 AM Peter Geoghegan wrote: > On Sat, Mar 16, 2019 at 9:17 AM Heikki Linnakangas wrote: > > Hmm. "re-find", maybe? We use that term in a few error messages already, > > to mean something similar. > > WFM. Actually, how about "rootsearch", or "rootdescend"? You're suppo

Re: What is a savepointLevel ?

2019-03-16 Thread Tom Lane
Edmund Horner writes: > On Fri, 15 Mar 2019 at 18:18, Chapman Flack wrote: >> What exactly is a savepointLevel? >> >> They seem to have been there for 15 years[1], diligently copied from >> parent transactions to children, fastidiously checked to avoid crossing >> a level on rollback or release,

Re: Willing to fix a TODO case in libpq module

2019-03-16 Thread Tom Lane
"Wu, Fei" writes: > On website: https://wiki.postgresql.org/wiki/Todo#libpq > I found that in libpq module,there is a todo case: > --- > Prevent PQfnumber() from lowercasing unquoted column names > PQfnumber() should never

Re: jsonpath

2019-03-16 Thread Jeff Janes
On Sat, Mar 16, 2019 at 5:36 AM Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > > So, pushed. Many thanks to reviewers and authors! > I think these files have to be cleaned up by "make maintainer-clean" ./src/backend/utils/adt/jsonpath_gram.c ./src/backend/utils/adt/jsonpath_scan.c Ch

Re: jsonpath

2019-03-16 Thread Alexander Korotkov
сб, 16 мар. 2019 г., 20:52 Jeff Janes : > On Sat, Mar 16, 2019 at 5:36 AM Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> >> So, pushed. Many thanks to reviewers and authors! >> > > I think these files have to be cleaned up by "make maintainer-clean" > > ./src/backend/utils/adt/json

Re: jsonpath

2019-03-16 Thread Tom Lane
Jeff Janes writes: > I think these files have to be cleaned up by "make maintainer-clean" > ./src/backend/utils/adt/jsonpath_gram.c > ./src/backend/utils/adt/jsonpath_scan.c Good point. I also see jsonpath_gram.h left behind after maintainer-clean: $ git status --ignored On branch master Your

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-03-16 Thread Julien Rouhaud
On Sat, Mar 16, 2019 at 5:20 PM Tom Lane wrote: > > Robert Haas writes: > > On Fri, Mar 15, 2019 at 9:50 PM Tom Lane wrote: > >> pg_stat_statements has a notion of query ID, but that notion might be > >> quite inappropriate for other usages, which is why it's an extension > >> and not core. > >

Re: jsonpath

2019-03-16 Thread Tom Lane
I wrote: > Good point. I also see jsonpath_gram.h left behind after maintainer-clean: Oh, and of potentially more significance: after maintainer-clean and re-configure, make fails with In file included from jsonpath_gram.y:24: ../../../../src/include/utils/jsonpath_scanner.h:25:33: error: utils

Re: jsonpath

2019-03-16 Thread Alexander Korotkov
сб, 16 мар. 2019 г., 21:12 Tom Lane : > I wrote: > > Good point. I also see jsonpath_gram.h left behind after > maintainer-clean: > > Oh, and of potentially more significance: after maintainer-clean and > re-configure, make fails with > > In file included from jsonpath_gram.y:24: > ../../../../sr

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-03-16 Thread legrand legrand
Hello, This is available in https://github.com/legrandlegrand/pg_stat_sql_plans extension with a specific function pgssp_backend_queryid(pid) that permits to join pg_stat_activity with pg_stat_sql_plans (that is similar to pg_stat_statements) and also permits to collect samples of wait events per

Re: jsonpath

2019-03-16 Thread Pavel Stehule
so 16. 3. 2019 v 10:36 odesílatel Alexander Korotkov < a.korot...@postgrespro.ru> napsal: > On Thu, Mar 14, 2019 at 12:07 PM Alexander Korotkov > wrote: > > On Sun, Mar 10, 2019 at 1:51 PM Alexander Korotkov > > wrote: > > > On Wed, Mar 6, 2019 at 12:40 AM Nikita Glukhov < > n.glu...@postgrespro

Keyword table constness in jsonpath scanner.

2019-03-16 Thread Mark G
Hello all, While looking over the new jsonpath stuff I noticed the keyword table wasn't declared const. Shouldn't the table and the actual keyword strings both be declared const? Perhaps something like the attached (untested) patch. -Mark diff --git a/src/backend/utils/adt/jsonpath_scan.l b/src/b

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-16 Thread Tom Lane
Andrew Dunstan writes: > On 3/14/19 3:08 PM, Tom Lane wrote: >> I feel therefore that what we should do (barring new evidence) is either >> 1. Remove all the inf/nan test cases for the hyoerbolic functions ... >> 2. Just comment out the one failing test, with a note about why. > 2. would help us

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Heikki Linnakangas
On 16/03/2019 19:32, Peter Geoghegan wrote: On Sat, Mar 16, 2019 at 9:55 AM Peter Geoghegan wrote: On Sat, Mar 16, 2019 at 9:17 AM Heikki Linnakangas wrote: Hmm. "re-find", maybe? We use that term in a few error messages already, to mean something similar. WFM. Actually, how about "rootse

Fix XML handling with DOCTYPE

2019-03-16 Thread Ryan Lambert
Hi all, I'm investigating the issue I reported here: https://www.postgresql.org/message-id/flat/153478795159.1302.9617586466368699403%40wrigleys.postgresql.org As Tom Lane mentioned there, the docs (8.13) indicate xmloption = CONTENT should accept all valid XML. At this time, XML with a DOCTYPE

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Chapman Flack
On 03/16/19 16:10, Ryan Lambert wrote: > As Tom Lane mentioned there, the docs (8.13) indicate xmloption = CONTENT > should accept all valid XML. At this time, XML with a DOCTYPE declaration > is not accepted with this setting even though it is considered valid XML. Hello Ryan, A patch for your

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Heikki Linnakangas
On 16/03/2019 18:55, Peter Geoghegan wrote: On Sat, Mar 16, 2019 at 9:17 AM Heikki Linnakangas wrote: Then, a cosmic ray changes the 20 on the root page to 18. That causes the the leaf tuple 19 to become non-re-findable; if you descend the for 19, you'll incorrectly land on page 6. But it also

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Tom Lane
Ryan Lambert writes: > I'm investigating the issue I reported here: > https://www.postgresql.org/message-id/flat/153478795159.1302.9617586466368699403%40wrigleys.postgresql.org > I'd like to work on a patch to address this issue and make it work as > advertised. Good idea, because it doesn't seem

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Peter Geoghegan
On Sat, Mar 16, 2019 at 1:33 PM Heikki Linnakangas wrote: > AFAICS, there is a copy of every page's high key in its immediate > parent. Either in the downlink of the right sibling, or as the high key > of the parent page itself. Cross-checking those would catch any > corruption in high keys. I ag

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Chapman Flack
On 03/16/19 16:42, Tom Lane wrote: > Ryan Lambert writes: >> I'm investigating the issue I reported here: >> https://www.postgresql.org/message-id/flat/153478795159.1302.9617586466368699403%40wrigleys.postgresql.org >> I'd like to work on a patch to address this issue and make it work as >> advert

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Tom Lane
Chapman Flack writes: > A patch for your issue is currently registered in the 2019-03 commitfest[1]. Oh! I apologize for saying nobody was working on this issue. Taking a very quick look at your patch, though, I dunno --- it seems like it adds a boatload of new assumptions about libxml's data s

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Peter Geoghegan
On Sat, Mar 16, 2019 at 1:47 PM Peter Geoghegan wrote: > I agree that it's always true that the high key is also in the parent, > and we could cross-check that within the child. Actually, I should > probably figure out a way of arranging for the Bloom filter used > within bt_relocate_from_root() (

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-16 Thread Peter Geoghegan
On Sat, Mar 16, 2019 at 2:01 PM Peter Geoghegan wrote: > > On Sat, Mar 16, 2019 at 1:47 PM Peter Geoghegan wrote: > > I agree that it's always true that the high key is also in the parent, > > and we could cross-check that within the child. Actually, I should > > probably figure out a way of arra

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Chapman Flack
On 03/16/19 16:55, Tom Lane wrote: > What do you think of the idea I just posted about parsing off the DOCTYPE > thing for ourselves, and not letting libxml see it? The principled way of doing that would be to pre-parse to find a DOCTYPE, and if there is one, leave it there and parse the input as

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Tom Lane
Chapman Flack writes: > On 03/16/19 16:55, Tom Lane wrote: >> What do you think of the idea I just posted about parsing off the DOCTYPE >> thing for ourselves, and not letting libxml see it? > The principled way of doing that would be to pre-parse to find a DOCTYPE, > and if there is one, leave i

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-16 Thread Dean Rasheed
On Fri, 15 Mar 2019 at 00:06, Tomas Vondra wrote: > ... attached patch ... Some more review comments, carrying on from where I left off: 16). This regression test fails for me: @@ -654,11 +654,11 @@ -- check change of unrelated column type does not reset the MCV statistics ALTER TABLE mcv_lis

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Chapman Flack
On 03/16/19 17:21, Tom Lane wrote: > Chapman Flack writes: >> On 03/16/19 16:55, Tom Lane wrote: >>> What do you think of the idea I just posted about parsing off the DOCTYPE >>> thing for ourselves, and not letting libxml see it? > >> The principled way of doing that would be to pre-parse to fin

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Ryan Lambert
Thank you both! I had glanced at that item in the commitfest but didn't notice it would fix this issue. I'll try to test/review this before the end of the month, much better than starting from scratch myself. A quick glance at the patch looks logical and looks like it should work for my use case

Re: proposal: pg_restore --convert-to-text

2019-03-16 Thread Euler Taveira
Em qua, 27 de fev de 2019 às 23:48, Imai, Yoshikazu escreveu: > > Is there no need to rewrite the Description in the Doc to state we should > specify either -d or -f option? > (and also it might be better to write if -l option is given, neither -d nor > -f option isn't necessarily needed.) > I d

Possible to modify query language in an extension?

2019-03-16 Thread Chris Cleveland
Noob here. I'm getting started on building a Postgres extension. I'd like to add some keywords/clauses to the SELECT statement. For my particular application, the syntax with new keywords would be way better than trying to do it through functions alone. I would add some new keywords followed by ex

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-16 Thread Tomas Vondra
On 3/16/19 10:26 PM, Dean Rasheed wrote: > On Fri, 15 Mar 2019 at 00:06, Tomas Vondra > wrote: >> ... attached patch ... > > Some more review comments, carrying on from where I left off: > > 16). This regression test fails for me: > > @@ -654,11 +654,11 @@ > -- check change of unrelated col

Re: Unaccent extension python script Issue in Windows

2019-03-16 Thread Ramanarayana
Hi Hugh, I have abstracted out the windows compatibility changes from your patch to a new patch and tested it. Added the patch to https://commitfest.postgresql.org/23/ Please feel free to change it if it requires any changes. Cheers Ram 4.0 v1_unaccent_windows_compatibility.patch Description:

Re: Fix XML handling with DOCTYPE

2019-03-16 Thread Chapman Flack
On 03/16/19 18:33, Chapman Flack wrote: > The pre-scan is a simple linear search and will ordinarily say yes or no > within a couple dozen characters--you could *have* an input with 20k of > leading whitespace and comments, but it's hardly the norm. Just trying to If the available regexp functions

Re: Possible to modify query language in an extension?

2019-03-16 Thread Tom Lane
Chris Cleveland writes: > I'd like to add some keywords/clauses to the SELECT statement. Yeah, you'll have to modify gram.y (and a pile of other places) if you want to do that. That's certainly something we do all the time, but bison doesn't provide any way to add grammar productions on-the-fly,

Re: Parallel query vs smart shutdown and Postmaster death

2019-03-16 Thread Arseny Sher
Hi, Thomas Munro writes: > 1. Why does pmdie()'s SIGTERM case terminate parallel workers > immediately? That breaks aborts running parallel queries, so they > don't get to end their work normally. > 2. Why are new parallel workers not allowed to be started while in > this state? That hangs f

Re: Make pg_checksums complain if compiled BLCKSZ and data folder's block size differ

2019-03-16 Thread Michael Paquier
On Sat, Mar 16, 2019 at 09:18:34AM +0100, Fabien COELHO wrote: >> If the block size the tool is compiled with does not match the data >> folder block size, then users would get incorrect checksums failures, > > Or worse, incorrect checksump writing under "enabling"? Let's hope that we make that p