Re: Remove useless associativity/precedence from parsers

2019-05-22 Thread Akim Demaille
Hi Tom, > Le 21 mai 2019 à 21:06, Tom Lane a écrit : > > Akim Demaille writes: >>> Le 20 mai 2019 à 15:54, Tom Lane a écrit : >>> 2013? Certainly not. We have a lot of buildfarm critters running >>> older platforms than that. > >> This I fully understand. However, Bison is a source

Excessive memory usage in multi-statement queries w/ partitioning

2019-05-22 Thread Andreas Seltenreich
Hi, a customer reported excessive memory usage and out-of-memory ERRORs after introducing native partitioning in one of their databases. We could narrow it down to the overhead introduced by the partitioning when issuing multiple statements in a single query. I could reduce the problem to the

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-22 Thread David Rowley
On Thu, 23 May 2019 at 15:31, Jonathan S. Katz wrote: > Attached is > v3 of the patch, along with a diff. Minor details, but this query is not valid: > WITH c AS MATERIALIZED ( > SELECT * FROM a WHERE a.x % 4 > ) > SELECT * FROM c JOIN d ON d.y = a.x; a.x % 4 is not a boolean clause, and "a"

Re: nitpick about poor style in MergeAttributes

2019-05-22 Thread Michael Paquier
On Wed, May 22, 2019 at 06:20:01PM -0700, Mark Dilger wrote: > What to do about this is harder to say. In the following > patch, I'm just doing what I think is standard for callers > of list_delete_cell, and assigning the return value back > to the list (similar to how a call to repalloc should

Re: Patch to fix write after end of array in hashed agg initialization

2019-05-22 Thread Andrew Gierth
> "Tom" == Tom Lane writes: > Andrew Gierth writes: >> My inclination is to fix this in the planner rather than the >> executor; there seems no good reason to actually hash a duplicate >> column more than once. Tom> Sounds reasonable --- but would it make sense to introduce some Tom>

Re: Patch to fix write after end of array in hashed agg initialization

2019-05-22 Thread Tom Lane
Andrew Gierth writes: > My inclination is to fix this in the planner rather than the executor; > there seems no good reason to actually hash a duplicate column more than > once. Sounds reasonable --- but would it make sense to introduce some assertions, or other cheap tests, into the executor to

Re: Remove useless associativity/precedence from parsers

2019-05-22 Thread Tom Lane
Robert Haas writes: > Another thing is that it would be nice to have a better way of > resolving conflicts than attaching precedence declarations. Some > problems can't be solved that way at all, and others can only be > solved that way at the risk of unforeseen side effects. Yeah, we've

nitpick about useless floating point division in gimme_edge_table

2019-05-22 Thread Mark Dilger
Hackers, The return value of gimme_edge_table is not used anywhere in the core code, so far as I can see. But the value is computed as /* return average number of edges per index */ return ((float) (edge_total * 2) / (float) num_gene); which involves some floating point math. I'm not sure

Re: Patch to fix write after end of array in hashed agg initialization

2019-05-22 Thread Andrew Gierth
> "Andrew" == Andrew Gierth writes: >>> Attached is a patch for a write after allocated memory which we >>> found in testing. Its an obscure case but can happen if the same >>> column is used in different grouping keys, as in the example below, >>> which uses tables from the regress test

Re: Remove useless associativity/precedence from parsers

2019-05-22 Thread Robert Haas
On Tue, May 21, 2019 at 3:07 PM Tom Lane wrote: > Other PG hackers might have a different laundry list, but that's mine. Good list. Another thing is that it would be nice to have a better way of resolving conflicts than attaching precedence declarations. Some problems can't be solved that way

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-22 Thread Jonathan S. Katz
Hi Ilmari, On 5/22/19 6:52 PM, Dagfinn Ilmari Mannsåker wrote: > Hi Jonathan, > > "Jonathan S. Katz" writes: > >> If you have additional feedback please provide it before 7am EDT >> tomorrow. > > Thanks for writing this up. Below are some things I noticed when > reading through (disclaimer:

Re: docs about FKs referencing partitioned tables

2019-05-22 Thread Michael Paquier
On Mon, May 20, 2019 at 10:35:31PM -0700, Paul A Jungwirth wrote: > I agree that sounds better. To avoid repeating it I changed the second > instance to just "inherited tables". New patch attached. Looking closer, you are adding that: + + + While primary keys are supported on

Re: Minor typos and copyright year slippage

2019-05-22 Thread Michael Paquier
On Thu, May 23, 2019 at 01:28:45PM +1200, Thomas Munro wrote: > Here are some tiny things I noticed in passing. Good catches. And you have spotted all the blank spots for the copyright notices. -- Michael signature.asc Description: PGP signature

Re: ACL dump ordering broken as well for tablespaces

2019-05-22 Thread Michael Paquier
On Wed, May 22, 2019 at 06:35:31PM +, Bossart, Nathan wrote: > The patch looks good to me. Thanks for double-checking. I have applied and back-patched. The good thing here is that there were zero conflicts. > A bit of digging led me to the commit that removed databases and > tablespaces

Re: MSVC Build support with visual studio 2019

2019-05-22 Thread Haribabu Kommi
On Wed, May 22, 2019 at 7:36 AM Juanjo Santamaria Flecha < juanjo.santama...@gmail.com> wrote: > I have gone through path > '0001-Support-building-with-visual-studio-2019.patch' only, but I am sure > some comments will also apply to back branches. > Thanks for the review. > 1. The

Minor typos and copyright year slippage

2019-05-22 Thread Thomas Munro
Hi, Here are some tiny things I noticed in passing. -- Thomas Munro https://enterprisedb.com 0001-Fix-typos.patch Description: Binary data 0002-Update-copyright-year.patch Description: Binary data

nitpick about poor style in MergeAttributes

2019-05-22 Thread Mark Dilger
Hackers, I have been auditing the v12 source code for places which inappropriately ignore the return value of a function and have found another example which seems to me a fertile source of future bugs. In src/backend/nodes/list.c, list_delete_cell frees the list and returns NIL when you delete

Re: Typo: llvm*.cpp files identified as llvm*.c

2019-05-22 Thread Thomas Munro
On Wed, Jan 23, 2019 at 7:24 PM Andrew Gierth wrote: > > "Andres" == Andres Freund writes: > > [IDENTIFICATION] > > Andres> I think we should just rip them out. It's useless noise. > > +1 Here's a patch like that. 948 files changed, 3215 deletions(-) -- Thomas Munro

Re: with oids option not removed in pg_dumpall

2019-05-22 Thread Michael Paquier
On Tue, May 21, 2019 at 05:24:57PM +0900, Michael Paquier wrote: > Good catch. Your cleanup looks correct to me. Andres, perhaps you > would prefer doing the cleanup yourself? As I am cleaning up the area for another issue, applied. -- Michael signature.asc Description: PGP signature

Re: Patch to fix write after end of array in hashed agg initialization

2019-05-22 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> Attached is a patch for a write after allocated memory which we >> found in testing. Its an obscure case but can happen if the same >> column is used in different grouping keys, as in the example below, >> which uses tables from the regress test suite

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Michael Paquier
On Wed, May 22, 2019 at 02:39:54PM -0400, Tom Lane wrote: > Andres Freund writes: >> Well, if we explicitly have to check for -1, it's not really an error of >> omission for everything. Yes, we could forget returning the amname in a >> newer version of the query, but given that we usually just

Suppressing noise in successful check-world runs

2019-05-22 Thread Tom Lane
[ redirected from a thread in pgsql-committers[1] ] As of commit eb9812f27 you can run a manual check-world with stdout dumped to /dev/null, and get fairly clean results: $ time make check-world -j10 >/dev/null NOTICE: database "regression" does not exist, skipping real1m43.875s user

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-22 Thread Dagfinn Ilmari Mannsåker
Hi Jonathan, "Jonathan S. Katz" writes: > If you have additional feedback please provide it before 7am EDT > tomorrow. Thanks for writing this up. Below are some things I noticed when reading through (disclaimer: I'm not a native speaker). > PostgreSQL 12 extends the abilities of several of

Re: Remove useless associativity/precedence from parsers

2019-05-22 Thread Tom Lane
Andrew Dunstan writes: > On 5/21/19 11:49 AM, Akim Demaille wrote: >> Usually users of Bison build tarballs with the generated parsers >> in them, and ship/test from that. > The buildfarm client does not build from tarballs, it builds from git, > meaning it has to run bison. Thus Tom's objection

Re: Remove useless associativity/precedence from parsers

2019-05-22 Thread Andrew Dunstan
On 5/21/19 11:49 AM, Akim Demaille wrote: > Hi Tom! > >> Le 20 mai 2019 à 15:54, Tom Lane a écrit : >> >> Akim Demaille writes: >>> It is for the same reasons that I would recommend not using associativity >>> directives (%left, %right, %nonassoc) where associativity plays no role: >>>

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-22 Thread Jonathan S. Katz
On 5/21/19 11:39 PM, Jonathan S. Katz wrote: > Hi, > > Attached is a draft of the PG12 Beta 1 press release that is going out > this Thursday. The primary goals of this release announcement are to > introduce new features, enhancements, and changes that are available in > PG12, as well as

Re: Remove useless associativity/precedence from parsers

2019-05-22 Thread Daniel Gustafsson
> On 22 May 2019, at 23:25, Tom Lane wrote: > > Akim Demaille writes: >> Honestly, I seriously doubt that you have contributors that don't >> have MacPorts or Brew installed, and both are pretty up to date on >> Bison. > > Hm, well, I'm a counterexample ;-) And one more. While I do have brew

Re: Remove useless associativity/precedence from parsers

2019-05-22 Thread Tom Lane
Akim Demaille writes: > Honestly, I seriously doubt that you have contributors that don't > have MacPorts or Brew installed, and both are pretty up to date on > Bison. Hm, well, I'm a counterexample ;-). Right now you can develop PG on a Mac just fine without any additional stuff, excepting

Re: pgindent run next week?

2019-05-22 Thread Euler Taveira
Em qua, 22 de mai de 2019 às 14:08, Tom Lane escreveu: > > I wrote: > > Hearing no objections, I'll plan on running pgindent tomorrow sometime. > > And done. > > > The new underlying pg_bsd_indent (2.1) is available now from > > https://git.postgresql.org/git/pg_bsd_indent.git > > Please update

Re: Is it safe to ignore the return value of SPI_finish and SPI_execute?

2019-05-22 Thread Mark Dilger
On Wed, May 22, 2019 at 1:52 PM Tom Lane wrote: > > Mark Dilger writes: > > On Fri, May 17, 2019 at 6:12 PM Tom Lane wrote: > >> One reasonable solution would be to change the callers that got this > >> wrong. Another one would be to reconsider whether the error-return-code > >> convention

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-22 Thread Finnerty, Jim
Fwiw, I had an intern do some testing on the JOB last year, and he reported that geqo sometimes produced plans of lower cost than the standard planner (we were on PG10 at the time). I filed it under "unexplained things that we need to investigate when we have time", but alas... In any case,

Re: Is it safe to ignore the return value of SPI_finish and SPI_execute?

2019-05-22 Thread Tom Lane
Mark Dilger writes: > On Fri, May 17, 2019 at 6:12 PM Tom Lane wrote: >> One reasonable solution would be to change the callers that got this >> wrong. Another one would be to reconsider whether the error-return-code >> convention makes any sense at all here. If we changed the above-quoted >>

Re: FullTransactionId changes are causing portability issues

2019-05-22 Thread Tom Lane
Andres Freund writes: > On 2019-05-22 15:55:50 -0400, Tom Lane wrote: >>> So the disturbing thing here is that we no longer have any active >>> buildfarm members that can build HEAD but have the won't-elide- >>> unused-static-functions problem. Clearly we'd better close that >>> gap somehow ...

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Andres Freund
On 2019-05-22 16:04:34 -0400, Andrew Dunstan wrote: > > On 5/22/19 2:42 PM, Andres Freund wrote: > > Hi, > > > > On 2019-05-22 14:27:51 -0400, Tom Lane wrote: > >> Andres Freund writes: > >>> On 2019-05-22 14:06:47 -0400, Tom Lane wrote: > Not sure about that last bit. pg_upgrade has the

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Andrew Dunstan
On 5/22/19 2:42 PM, Andres Freund wrote: > Hi, > > On 2019-05-22 14:27:51 -0400, Tom Lane wrote: >> Andres Freund writes: >>> On 2019-05-22 14:06:47 -0400, Tom Lane wrote: Not sure about that last bit. pg_upgrade has the issue of possibly wanting to deal with 2 installations, unlike

Re: FullTransactionId changes are causing portability issues

2019-05-22 Thread Andres Freund
Hi, On 2019-05-22 15:55:50 -0400, Tom Lane wrote: > I wrote: > >> Our Solaris packager reports that 12beta1 is failing to build for him > >> on some Solaris variants: > >>> The link failure is: > >>> Undefined first referenced > >>> symbolin file > >>>

Re: FullTransactionId changes are causing portability issues

2019-05-22 Thread Tom Lane
I wrote: >> Our Solaris packager reports that 12beta1 is failing to build for him >> on some Solaris variants: >>> The link failure is: >>> Undefined first referenced >>> symbol in file >>> ReadNextFullTransactionId pg_checksums.o > On looking

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-22 Thread Donald Dong
On May 22, 2019, at 11:42 AM, Tom Lane wrote: > > Donald Dong writes: >> I find the cost from cheapest_total_path->total_cost is different >> from the cost from queryDesc->planstate->total_cost. What I saw was >> that GEQO tends to form paths with lower >> cheapest_total_path->total_cost (aka

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-22 Thread Andres Freund
Hi, > ### Indexing Performance, Functionality, and Management > > PostgreSQL 12 improves the overall performance of the standard B-tree indexes > with improvements to the overall space management of these indexes as well. > These improvements also provide an overall reduction of bloating when

Re: pgindent run next week?

2019-05-22 Thread Tom Lane
Peter Eisentraut writes: > In my experience, changes to function declarations in header files > happen a lot in forks. So applying the pgindent change to backbranches > would cause some trouble. > On the other hand, it seems to me that patches that we backpatch between > PostgreSQL branches

Re: "long" type is not appropriate for counting tuples

2019-05-22 Thread Tom Lane
Peter Eisentraut writes: > Could we define int64 to be long long int on all platforms and just > always use %lld? Hmmm ... maybe. Once upon a time we had to cope with compilers that didn't have "long long", but perhaps that time is past. Another conceivable hazard is somebody deciding that the

Re: pgindent run next week?

2019-05-22 Thread Peter Eisentraut
On 2019-05-21 23:46, Tom Lane wrote: >> Would we want to also apply this to the back branches to avoid spurious >> conflicts? > I think we should hold off on any talk of that until we get some results > from Mark Dilger (or anyone else) on how much pain it would cause for > people carrying private

Re: Is it safe to ignore the return value of SPI_finish and SPI_execute?

2019-05-22 Thread Mark Dilger
On Fri, May 17, 2019 at 6:12 PM Tom Lane wrote: > > Mark Dilger writes: > > most places that use SPI_connect ... SPI_finish check the > > return value of SPI_finish and elog if it failed. There > > are a few places that do not, and it is unclear to me > > why this is safe. SPI_finish appears

Re: "long" type is not appropriate for counting tuples

2019-05-22 Thread Peter Eisentraut
On 2019-05-22 17:52, Tom Lane wrote: > I don't really see how controlling snprintf is enough to get somewhere > on this. Sure we could invent some new always-64-bit length modifier > and teach snprintf.c about it, but none of the other tools we use > would know about it. I don't want to give up

Re: "long" type is not appropriate for counting tuples

2019-05-22 Thread Peter Eisentraut
On 2019-05-22 18:59, Andres Freund wrote: > On May 22, 2019 7:40:28 AM PDT, Peter Eisentraut > wrote: >> On 2019-04-29 19:52, Andres Freund wrote: >>> Hm. It appears that gettext supports expanding PRId64 PRIu64 etc in >>> translated strings. >> >> That won't work in non-GNU gettext. > > Which

Re: pgindent run next week?

2019-05-22 Thread Mark Dilger
On Wed, May 22, 2019 at 10:07 AM Tom Lane wrote: > > I wrote: > > Hearing no objections, I'll plan on running pgindent tomorrow sometime. > > And done. > > > The new underlying pg_bsd_indent (2.1) is available now from > > https://git.postgresql.org/git/pg_bsd_indent.git > > Please update your

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Andrew Dunstan
On 5/22/19 9:46 AM, Tom Lane wrote: > Michael Paquier writes: >> Trying to do pg_dump[all] on a 9.5 or older server results in spurious >> failures: >> pg_dump: column number -1 is out of range 0..36 >> After looking around, the problem comes from >> check_tuple_field_number(), more

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Andres Freund
Hi, On 2019-05-22 14:27:51 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-05-22 14:06:47 -0400, Tom Lane wrote: > >> Not sure about that last bit. pg_upgrade has the issue of possibly > >> wanting to deal with 2 installations, unlike the rest of the tree, > >> so I'm not sure that

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-22 Thread Tom Lane
Donald Dong writes: > I find the cost from cheapest_total_path->total_cost is different > from the cost from queryDesc->planstate->total_cost. What I saw was > that GEQO tends to form paths with lower > cheapest_total_path->total_cost (aka the fitness of the children). > However,

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Tom Lane
Andres Freund writes: > On 2019-05-22 14:17:41 -0400, Tom Lane wrote: >> FWIW, I think that's a pretty awful idea, and the fact that some >> people have had it before doesn't make it less awful. It's giving >> up the ability to detect errors-of-omission, which might easily >> be harmful rather

Re: ACL dump ordering broken as well for tablespaces

2019-05-22 Thread Bossart, Nathan
On 5/22/19, 12:16 AM, "Michael Paquier" wrote: > Attached is a patch to fix that, so as pg_dumpall does not complain > when piling up GRANT commands using WITH GRANT OPTION. Are there any > complains to apply that down to 9.6? The patch looks good to me. > As the problem is kind of different

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-22 Thread Donald Dong
Hi, Thank you very much for the explanation! I think the join order benchmark I used [1] is somewhat representative, however, I probably didn't use the most accurate cost estimation. I find the cost from cheapest_total_path->total_cost is different from the cost from

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Tom Lane
Andres Freund writes: > On 2019-05-22 14:06:47 -0400, Tom Lane wrote: >> Not sure about that last bit. pg_upgrade has the issue of possibly >> wanting to deal with 2 installations, unlike the rest of the tree, >> so I'm not sure that fixing its problem means there's something we >> need to

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Andres Freund
Hi, On 2019-05-22 14:17:41 -0400, Tom Lane wrote: > Andres Freund writes: > > Wouldn't the better fix be to change > > if (PQgetisnull(res, i, i_amname)) > > tblinfo[i].amname = NULL; > > into > > if (i_amname == -1 || PQgetisnull(res, i, i_amname)) >

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Andres Freund
Hi, On 2019-05-22 14:06:47 -0400, Tom Lane wrote: > Andres Freund writes: > > Seems what we need to fix the immediate issue is to ressurect: > > > # We need to make it use psql from our temporary installation, > > # because otherwise the installcheck run below would try to > > # use psql from

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Tom Lane
Andres Freund writes: > Wouldn't the better fix be to change > if (PQgetisnull(res, i, i_amname)) > tblinfo[i].amname = NULL; > into > if (i_amname == -1 || PQgetisnull(res, i, i_amname)) > tblinfo[i].amname = NULL; > it's

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Tom Lane
Andres Freund writes: > Seems what we need to fix the immediate issue is to ressurect: > # We need to make it use psql from our temporary installation, > # because otherwise the installcheck run below would try to > # use psql from the proper installation directory, which might > # be outdated

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Andres Freund
Hi, On 2019-05-22 09:46:19 -0400, Tom Lane wrote: > Michael Paquier writes: > > Trying to do pg_dump[all] on a 9.5 or older server results in spurious > > failures: > > pg_dump: column number -1 is out of range 0..36 > > > After looking around, the problem comes from > >

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Andres Freund
Hi, On 2019-05-22 10:58:54 -0400, Tom Lane wrote: > Thomas Munro writes: > > After these commits (and Tom's commit "Un-break pg_upgrade regression > > test."), cfbot broke: I should just have finished working two hours earlier yesterday :(. > > sh: 1: /usr/local/pgsql/bin/psql: not found > >

Re: pgindent run next week?

2019-05-22 Thread Tom Lane
I wrote: > Hearing no objections, I'll plan on running pgindent tomorrow sometime. And done. > The new underlying pg_bsd_indent (2.1) is available now from > https://git.postgresql.org/git/pg_bsd_indent.git Please update your local copy if you have one. regards, tom

Re: "long" type is not appropriate for counting tuples

2019-05-22 Thread Andres Freund
Hi, On May 22, 2019 7:40:28 AM PDT, Peter Eisentraut wrote: >On 2019-04-29 19:52, Andres Freund wrote: >> Hm. It appears that gettext supports expanding PRId64 PRIu64 etc in >> translated strings. > >That won't work in non-GNU gettext. Which of those do currently work with postgres? Andres --

Re: Why does ExecComputeStoredGenerated() form a heap tuple

2019-05-22 Thread Peter Eisentraut
On 2019-05-20 04:23, David Rowley wrote: > On Thu, 16 May 2019 at 05:44, Peter Eisentraut > wrote: >> Updated patch attached. > > This patch looks okay to me. committed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: VACUUM fails to parse 0 and 1 as boolean value

2019-05-22 Thread Fujii Masao
On Wed, May 22, 2019 at 4:47 PM Michael Paquier wrote: > > On Wed, May 22, 2019 at 04:32:38AM +0900, Fujii Masao wrote: > > I found that tab-completion also needs to be updated for ANALYZE > > boolean options. I added that change for tab-completion into > > the patch and am thinking to apply the

Re: "long" type is not appropriate for counting tuples

2019-05-22 Thread Tom Lane
Andres Freund writes: > On May 22, 2019 7:39:41 AM PDT, Peter Eisentraut > wrote: >> On 2019-04-29 19:32, Tom Lane wrote: >>> Another problem is that while "%lu" format specifiers are portable, >>> INT64_FORMAT is a *big* pain, not least because you can't put it into >>> translatable strings

Re: Read-only access to temp tables for 2PC transactions

2019-05-22 Thread Stas Kelvich
> On 14 May 2019, at 12:53, Stas Kelvich wrote: > > Hi, > > That is an attempt number N+1 to relax checks for a temporary table access > in a transaction that is going to be prepared. > Konstantin Knizhnik made off-list review of this patch and spotted few problems. * Incorrect reasoning

upgrades in row-level locks can deadlock

2019-05-22 Thread Oleksii Kliukin
Hello, I have recently observed a deadlock on one of our production servers related to locking only a single row in a job table. There were two functions involved in the deadlock, the first one acquires a “for key share” lock on the row that represents the job it works on and subsequently

Re: "long" type is not appropriate for counting tuples

2019-05-22 Thread Andres Freund
Hi, On May 22, 2019 7:39:41 AM PDT, Peter Eisentraut wrote: >On 2019-04-29 19:32, Tom Lane wrote: >> Another problem is that while "%lu" format specifiers are portable, >> INT64_FORMAT is a *big* pain, not least because you can't put it into >> translatable strings without causing problems. To

Re: PG 12 draft release notes

2019-05-22 Thread Bruce Momjian
On Tue, May 21, 2019 at 05:00:31PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Mon, May 20, 2019 at 08:48:15PM -0400, Tom Lane wrote: > >> Yes, this should be in "source code". I think it should be merged > >> with a391ff3c and 74dfe58a into something like > >> > >> Allow extensions

Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3

2019-05-22 Thread Robert Haas
On Wed, Apr 24, 2019 at 10:48 AM Antonin Houska wrote: > Attached is the next version. It tries to address various problems pointed out > upthread, including documentation. It's not immediately evident to me, perhaps because I haven't looked at this stuff in a while, what the purpose of 0001 and

Re: Patch to fix write after end of array in hashed agg initialization

2019-05-22 Thread Tom Lane
Colm McHugh writes: > Attached is a patch for a write after allocated memory which we found in > testing. Its an obscure case but can happen if the same column is used in > different grouping keys, as in the example below, which uses tables from > the regress test suite (build with

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Tom Lane
Thomas Munro writes: > After these commits (and Tom's commit "Un-break pg_upgrade regression > test."), cfbot broke: > sh: 1: /usr/local/pgsql/bin/psql: not found I can confirm that here: check-world passes as long as I've done "make install" beforehand ... but of course that should not be

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-22 Thread Robbie Harwood
"Jonathan S. Katz" writes: > Attached is a draft of the PG12 Beta 1 press release that is going out > this Thursday. The primary goals of this release announcement are to > introduce new features, enhancements, and changes that are available in > PG12, as well as encourage our users to test and

Re: FullTransactionId changes are causing portability issues

2019-05-22 Thread Tom Lane
I wrote: > Our Solaris packager reports that 12beta1 is failing to build for him > on some Solaris variants: >> The link failure is: >> --- >> Undefinedfirst referenced >> symbol in file >> ReadNextFullTransactionId pg_checksums.o >> ld:

Re: "long" type is not appropriate for counting tuples

2019-05-22 Thread Peter Eisentraut
On 2019-04-29 19:52, Andres Freund wrote: > Hm. It appears that gettext supports expanding PRId64 PRIu64 etc in > translated strings. That won't work in non-GNU gettext. > Perhaps we should implement them in our printf, and > then replace all use of INT64_FORMAT with that? But this might. --

Re: "long" type is not appropriate for counting tuples

2019-05-22 Thread Peter Eisentraut
On 2019-04-29 19:32, Tom Lane wrote: > Another problem is that while "%lu" format specifiers are portable, > INT64_FORMAT is a *big* pain, not least because you can't put it into > translatable strings without causing problems. To the extent that > we could go over to "%zu" instead, maybe this

FullTransactionId changes are causing portability issues

2019-05-22 Thread Tom Lane
Our Solaris packager reports that 12beta1 is failing to build for him on some Solaris variants: > The link failure is: > --- > Undefined first referenced > symbol in file > ReadNextFullTransactionId pg_checksums.o > ld: fatal: symbol

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Tom Lane
Michael Paquier writes: > Trying to do pg_dump[all] on a 9.5 or older server results in spurious > failures: > pg_dump: column number -1 is out of range 0..36 > After looking around, the problem comes from > check_tuple_field_number(), more specifically from getTables() where > someone has

Re: SQL statement PREPARE does not work in ECPG

2019-05-22 Thread Tom Lane
Michael Paquier writes: > This patch seems to have little incidence on the stability, but FWIW I > am not cool with the concept of asking for objections and commit a > patch only 4 hours after-the-fact, particularly after feature freeze. FWIW, I figured it was okay since ECPG has essentially no

Re: POC: Cleaning up orphaned files using undo logs

2019-05-22 Thread Amit Kapila
On Wed, May 22, 2019 at 5:47 PM Robert Haas wrote: > > On Wed, May 22, 2019 at 7:17 AM Amit Kapila wrote: > > > > +/* Extract xid from a value comprised of epoch and xid */ > > > > +#define GetXidFromEpochXid(epochxid) \ > > > > + ((uint32) (epochxid) & 0X) > > > > +

Re: POC: Cleaning up orphaned files using undo logs

2019-05-22 Thread Robert Haas
On Wed, May 22, 2019 at 7:17 AM Amit Kapila wrote: > > > +/* Extract xid from a value comprised of epoch and xid */ > > > +#define GetXidFromEpochXid(epochxid) \ > > > + ((uint32) (epochxid) & 0X) > > > + > > > +/* Extract epoch from a value comprised of epoch and xid

Re: Remove page-read callback from XLogReaderState.

2019-05-22 Thread Antonin Houska
Kyotaro HORIGUCHI wrote: > Hello. Thank you for looking this. > ... > Yeah, I'll register this, maybe the week after next week. I've checked the new version. One more thing I noticed now is that XLR_STATE.j is initialized to zero, either by XLogReaderAllocate() which zeroes the whole reader

Re: accounting for memory used for BufFile during hash joins

2019-05-22 Thread Tomas Vondra
On Tue, May 21, 2019 at 05:38:50PM -0700, Melanie Plageman wrote: On Wed, May 8, 2019 at 8:08 AM Tomas Vondra wrote: On Tue, May 07, 2019 at 05:30:27PM -0700, Melanie Plageman wrote: > One thing I was a little confused by was the nbatch_inmemory member > of the hashtable. The comment in

Re: POC: Cleaning up orphaned files using undo logs

2019-05-22 Thread Amit Kapila
On Tue, May 21, 2019 at 10:47 PM Andres Freund wrote: > > Hi, > > On 2019-05-05 10:28:21 +0530, Amit Kapila wrote: > > From 5d9e179bd481b5ed574b6e7117bf3eb62b5dc003 Mon Sep 17 00:00:00 2001 > > From: Amit Kapila > > Date: Sat, 4 May 2019 16:52:01 +0530 > > Subject: [PATCH] Allow undo actions to

RE: Global shared meta cache

2019-05-22 Thread Ideriha, Takeshi
>From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >[TL; DR] >The basic idea is following 4 points: >A. User can choose which database to put a cache (relation and catalog) on >shared >memory and how much memory is used >B. Caches of committed data are on the >shared memory. Caches

Re: SQL statement PREPARE does not work in ECPG

2019-05-22 Thread Michael Meskes
> This patch seems to have little incidence on the stability, but FWIW > I > am not cool with the concept of asking for objections and commit a > patch only 4 hours after-the-fact, particularly after feature freeze. This was only done to beat the pg_indent run as Tom pointed out. I figured worse

Re: Minimal logical decoding on standbys

2019-05-22 Thread Amit Khandekar
On Tue, 9 Apr 2019 at 22:23, Amit Khandekar wrote: > > On Sat, 6 Apr 2019 at 04:45, Andres Freund wrote: > > > diff --git a/src/backend/replication/slot.c > > > b/src/backend/replication/slot.c > > > index 006446b..5785d2f 100644 > > > --- a/src/backend/replication/slot.c > > > +++

Re: Minimal logical decoding on standbys

2019-05-22 Thread Amit Khandekar
Hi, I am going through you comments. Meanwhile, attached is a rebased version of the v4 patch. On Tue, 21 May 2019 at 21:49, Andres Freund wrote: > > Hi, > > Sorry for the late response. > > On 2019-04-16 12:27:46 +0530, Amit Khandekar wrote: > > On Sat, 13 Apr 2019 at 00:57, Andres Freund

Re: Teach pg_upgrade test to honor NO_TEMP_INSTALL

2019-05-22 Thread Thomas Munro
On Wed, May 22, 2019 at 7:41 AM Andres Freund wrote:> > On 2019-05-21 12:19:18 -0700, Andres Freund wrote: > > Roughly like in the attached? > > > -check: test.sh all > > - MAKE=$(MAKE) bindir="$(tbindir)" libdir="$(tlibdir)" > > EXTRA_REGRESS_OPTS="$(EXTRA_REGRESS_OPTS)" $(SHELL) $<

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Dmitry Dolgov
> On Wed, May 22, 2019 at 10:34 AM Michael Paquier wrote: > > Trying to do pg_dump[all] on a 9.5 or older server results in spurious > failures: > pg_dump: column number -1 is out of range 0..36 > > After looking around, the problem comes from > check_tuple_field_number(), more specifically from

pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-22 Thread Michael Paquier
Hi all, Trying to do pg_dump[all] on a 9.5 or older server results in spurious failures: pg_dump: column number -1 is out of range 0..36 After looking around, the problem comes from check_tuple_field_number(), more specifically from getTables() where someone has forgotten to add NULL values for

Re: Ought to use heap_multi_insert() for pg_attribute/depend insertions?

2019-05-22 Thread Daniel Gustafsson
> On 13 Feb 2019, at 19:27, Andres Freund wrote: > > Hi, > > Turns out in portions of the regression tests a good chunk of the > runtime is inside AddNewAttributeTuples() and > recordMultipleDependencies()'s heap insertions. Looking at a few > profiles I had lying around I found that in some

Re: PG 12 draft release notes

2019-05-22 Thread Ian Barwick
On 5/22/19 4:26 PM, Michael Paquier wrote: On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: the last two items are performance improvements not related to authentication; presumably the VACUUM item would be better off in the "Utility Commands" section and the TRUNCATE item in

Re: VACUUM fails to parse 0 and 1 as boolean value

2019-05-22 Thread Michael Paquier
On Wed, May 22, 2019 at 04:32:38AM +0900, Fujii Masao wrote: > I found that tab-completion also needs to be updated for ANALYZE > boolean options. I added that change for tab-completion into > the patch and am thinking to apply the attached patch. Looks fine to me at quick glance. -- Michael

Re: Should MSVC 2019 support be an open item for v12?

2019-05-22 Thread Michael Paquier
On Tue, May 21, 2019 at 11:53:51PM +0200, Juan José Santamaría Flecha wrote: > El mar., 21 may. 2019 23:06, Tom Lane escribió: >> Given the number of different people that have sent in patches >> for building with VS2019, it doesn't seem to me that we ought >> to let that wait for v13. > > I am

Re: PG 12 draft release notes

2019-05-22 Thread Michael Paquier
On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: > the last two items are performance improvements not related to authentication; > presumably the VACUUM item would be better off in the "Utility Commands" > section and the TRUNCATE item in "General Performance"? I agree with removing

ACL dump ordering broken as well for tablespaces

2019-05-22 Thread Michael Paquier
Hi all, As some may have noticed, I have been looking at the ACL dump ordering for databases, and I have noticed the same issue with tablespaces: https://www.postgresql.org/message-id/20190522062626.gc1...@paquier.xyz For the sake of avoiding looking at the other email, here is how to reproduce

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-22 Thread Amit Langote
On 2019/05/22 15:47, David Rowley wrote: > On Wed, 22 May 2019 at 15:39, Jonathan S. Katz wrote: >> PostgreSQL 12 also provides improvements to the performance of both >> using `COPY` with a partitioned table as well as the `ATTACH PARTITION` >> operation. Additionally, the ability to use foreign

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-22 Thread David Rowley
On Wed, 22 May 2019 at 15:39, Jonathan S. Katz wrote: > Speaking of feedback, please provide me with your feedback on the > technical correctness of this announcement so I can incorporate changes > prior to the release. Seems like a pretty good summary. Thanks for writing that up. Couple notes

Re: SQL statement PREPARE does not work in ECPG

2019-05-22 Thread Michael Paquier
On Wed, May 22, 2019 at 05:10:14AM +0200, Michael Meskes wrote: > Thanks for the heads-up Tom, pushed. > > And thanks to Matsumura-san for the patch. This patch seems to have little incidence on the stability, but FWIW I am not cool with the concept of asking for objections and commit a patch