pgsql: doc: Clarify use of ACCESS EXCLUSIVE lock in various sections

2021-03-31 Thread Michael Paquier
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections Some sections of the documentation used "exclusive lock" to describe that an ACCESS EXCLUSIVE lock is taken during a given operation. This can be confusing to the reader as ACCESS SHARE is allowed with an EXCLUSIVE lock is used, but th

pgsql: doc: Clarify use of ACCESS EXCLUSIVE lock in various sections

2021-03-31 Thread Michael Paquier
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections Some sections of the documentation used "exclusive lock" to describe that an ACCESS EXCLUSIVE lock is taken during a given operation. This can be confusing to the reader as ACCESS SHARE is allowed with an EXCLUSIVE lock is used, but th

pgsql: doc: Clarify use of ACCESS EXCLUSIVE lock in various sections

2021-03-31 Thread Michael Paquier
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections Some sections of the documentation used "exclusive lock" to describe that an ACCESS EXCLUSIVE lock is taken during a given operation. This can be confusing to the reader as ACCESS SHARE is allowed with an EXCLUSIVE lock is used, but th

pgsql: doc: Clarify use of ACCESS EXCLUSIVE lock in various sections

2021-03-31 Thread Michael Paquier
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections Some sections of the documentation used "exclusive lock" to describe that an ACCESS EXCLUSIVE lock is taken during a given operation. This can be confusing to the reader as ACCESS SHARE is allowed with an EXCLUSIVE lock is used, but th

pgsql: doc: Clarify use of ACCESS EXCLUSIVE lock in various sections

2021-03-31 Thread Michael Paquier
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections Some sections of the documentation used "exclusive lock" to describe that an ACCESS EXCLUSIVE lock is taken during a given operation. This can be confusing to the reader as ACCESS SHARE is allowed with an EXCLUSIVE lock is used, but th

pgsql: doc: Clarify use of ACCESS EXCLUSIVE lock in various sections

2021-03-31 Thread Michael Paquier
doc: Clarify use of ACCESS EXCLUSIVE lock in various sections Some sections of the documentation used "exclusive lock" to describe that an ACCESS EXCLUSIVE lock is taken during a given operation. This can be confusing to the reader as ACCESS SHARE is allowed with an EXCLUSIVE lock is used, but th

pgsql: Ensure to send a prepare after we detect concurrent abort during

2021-03-31 Thread Amit Kapila
Ensure to send a prepare after we detect concurrent abort during decoding. It is possible that while decoding a prepared transaction, it gets aborted concurrently via a ROLLBACK PREPARED command. In that case, we were skipping all the changes and directly sending Rollback Prepared when we find the

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread David Rowley
On Thu, 1 Apr 2021 at 14:49, Tomas Vondra wrote: > I think that's probably sufficient for now. More testing may be useful > once I refactor the code a bit to make it more readable. Just to let you know, I left it running a bit longer. About 70 mins now. No crashes yet. David

Re: pgsql: Add Result Cache executor node

2021-03-31 Thread David Rowley
On Thu, 1 Apr 2021 at 14:36, Tom Lane wrote: > > Tomas Vondra writes: > > On 4/1/21 2:52 AM, Tom Lane wrote: > >> Anyway, it looks like I can probably reproduce it on florican's > >> host, if you still need help understanding it tomorrow. But > >> I remain really dubious that this can work at al

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Tomas Vondra
On 4/1/21 3:43 AM, David Rowley wrote: > On Thu, 1 Apr 2021 at 11:07, Tomas Vondra > wrote: >> The attached patch fixes this for me. David, can you check if this >> resolves the issue for you? > > Thanks for taking a look at this. > > I applied your patch and kicked off sqlancer with 64 threads

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread David Rowley
On Thu, 1 Apr 2021 at 11:07, Tomas Vondra wrote: > The attached patch fixes this for me. David, can you check if this > resolves the issue for you? Thanks for taking a look at this. I applied your patch and kicked off sqlancer with 64 threads. It's been going about 30 mins without any crashes so

Re: pgsql: Add Result Cache executor node

2021-03-31 Thread Tom Lane
Tomas Vondra writes: > On 4/1/21 2:52 AM, Tom Lane wrote: >> Anyway, it looks like I can probably reproduce it on florican's >> host, if you still need help understanding it tomorrow. But >> I remain really dubious that this can work at all. Question: >> have you tried that test under CLOBBER_CA

Re: pgsql: Add Result Cache executor node

2021-03-31 Thread Tomas Vondra
On 4/1/21 2:52 AM, Tom Lane wrote: > David Rowley writes: >> On Thu, 1 Apr 2021 at 12:32, David Rowley wrote: >>> Add Result Cache executor node > >> I'm not really sure why yet why many buildfarm members don't like this. > > Something that struck me is that the animals that stayed green wer

pgsql: Move some client-specific routines from SSLServer to PostgresNod

2021-03-31 Thread Michael Paquier
Move some client-specific routines from SSLServer to PostgresNode test_connect_ok() and test_connect_fails() have always been part of the SSL tests, and check if a connection to the backend should work or not, and there are sanity checks done on specific error patterns dropped by libpq if the conn

Re: pgsql: Add Result Cache executor node

2021-03-31 Thread Tom Lane
David Rowley writes: > On Thu, 1 Apr 2021 at 12:32, David Rowley wrote: >> Add Result Cache executor node > I'm not really sure why yet why many buildfarm members don't like this. Something that struck me is that the animals that stayed green were almost exclusively Debian and macOS, while the

pgsql: Revert b6002a796

2021-03-31 Thread David Rowley
Revert b6002a796 This removes "Add Result Cache executor node". It seems that something weird is going on with the tracking of cache hits and misses as highlighted by many buildfarm animals. It's not yet clear what the problem is as other parts of the plan indicate that the cache did work correc

Re: pgsql: Add Result Cache executor node

2021-03-31 Thread David Rowley
On Thu, 1 Apr 2021 at 12:32, David Rowley wrote: > Add Result Cache executor node I'm not really sure why yet why many buildfarm members don't like this. You can see below that the Index scan was executed 20 times (loops=20) -> Result Cache (actual rows=1 loops=1000)

Re: pgsql: Add Result Cache executor node

2021-03-31 Thread David Rowley
On Thu, 1 Apr 2021 at 12:47, Tom Lane wrote: > > David Rowley writes: > > Add Result Cache executor node > > You didn't seriously believe that EXPLAIN ANALYZE numbers > would be stable in the buildfarm, did you? If you look a bit harder you can probably see which ones I thought would be stable a

Re: pgsql: Add Result Cache executor node

2021-03-31 Thread Tom Lane
David Rowley writes: > Add Result Cache executor node You didn't seriously believe that EXPLAIN ANALYZE numbers would be stable in the buildfarm, did you? regards, tom lane

pgsql: Add Result Cache executor node

2021-03-31 Thread David Rowley
Add Result Cache executor node Here we add a new executor node type named "Result Cache". The planner can include this node type in the plan to have the executor cache the results from the inner side of parameterized nested loop joins. This allows caching of tuples for sets of parameters so that

pgsql: Remove setvbuf() call from PQtrace()

2021-03-31 Thread Alvaro Herrera
Remove setvbuf() call from PQtrace() It's misplaced there -- it's not libpq's output stream to tweak in that way. In particular, POSIX says that it has to be called before any other operation on the file, so if a stream previously used by the calling application, bad things may happen. Put setvb

pgsql: Initialize conn->Pfdebug to NULL when creating a connection

2021-03-31 Thread Alvaro Herrera
Initialize conn->Pfdebug to NULL when creating a connection Failing to do this can cause a crash, and I suspect is what has happened with a buildfarm member reporting mysterious failures. This is an ancient bug, but I'm not backpatching since evidently nobody cares about PQtrace in older releases

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Tomas Vondra
On 3/31/21 7:54 PM, Tomas Vondra wrote: > On 3/31/21 7:08 PM, Alvaro Herrera wrote: >> On 2021-Mar-31, Tomas Vondra wrote: >> >>> Thanks for the report, I'll take a look. You're right this seems like an >>> out-of-bounds access, but mcv_match_expression is only expected to be >>> run on expressio

pgsql: Fix has_column_privilege function corner case

2021-03-31 Thread Joe Conway
Fix has_column_privilege function corner case According to the comments, when an invalid or dropped column oid is passed to has_column_privilege(), the intention has always been to return NULL. However, when the caller had table level privilege the invalid/missing column was never discovered, beca

pgsql: Disable force_parallel_mode in libpq_pipeline

2021-03-31 Thread Alvaro Herrera
Disable force_parallel_mode in libpq_pipeline Some buildfarm animals with force_parallel_mode=regress were failing this test because the error is reported in a parallel worker quicker than the rows that succeed. Take the opportunity to move the SET of lc_messages out of the traced section, becaus

pgsql: Fix unportable use of isprint().

2021-03-31 Thread Tom Lane
Fix unportable use of isprint(). We must cast the arguments of functions to unsigned char to avoid problems where char is signed. Speaking of which, considering that this *is* a function, it's rather remarkable that we aren't seeing more complaints about not having included that header. Per bu

pgsql: Silence compiler warning in non-assert builds.

2021-03-31 Thread Tom Lane
Silence compiler warning in non-assert builds. Per buildfarm. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/8998e3cafa23632790787b8cc726998e84067259 Modified Files -- contrib/postgres_fdw/postgres_fdw.c | 2 +- 1 file changed, 1 insertion(+), 1 deletio

pgsql: Fix portability and safety issues in pqTraceFormatTimestamp.

2021-03-31 Thread Tom Lane
Fix portability and safety issues in pqTraceFormatTimestamp. Remove confusion between time_t and pg_time_t; neither gettimeofday() nor localtime() deal in the latter. libpq indeed has no business using at all. Use snprintf not sprintf, to ensure we can't overrun the supplied buffer. (Unlikely,

pgsql: Don't prematurely cram a value into a short int.

2021-03-31 Thread Tom Lane
Don't prematurely cram a value into a short int. Since a4d75c86b, some buildfarm members have been warning that Assert(attnum <= MaxAttrNumber); is useless if attnum is an AttrNumber. I'm not certain how plausible it is that the value coming out of the bitmap could actually exceed

pgsql: Add a docs section for obsoleted and renamed functions and setti

2021-03-31 Thread Stephen Frost
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced fo

pgsql: Add a docs section for obsoleted and renamed functions and setti

2021-03-31 Thread Stephen Frost
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced fo

pgsql: Add a docs section for obsoleted and renamed functions and setti

2021-03-31 Thread Stephen Frost
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced fo

pgsql: Add a docs section for obsoleted and renamed functions and setti

2021-03-31 Thread Stephen Frost
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced fo

pgsql: Add a docs section for obsoleted and renamed functions and setti

2021-03-31 Thread Stephen Frost
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced fo

pgsql: Suppress compiler warning in libpq_pipeline.c.

2021-03-31 Thread Tom Lane
Suppress compiler warning in libpq_pipeline.c. Some compilers seem to be concerned about the possibility that recv_step is not any of the defined enum values. Silence warnings about uninitialized cmdtag in a different way than I did in 9fb9691a8. Branch -- master Details --- https://git

pgsql: Improve style of some replication-related error messages.

2021-03-31 Thread Tom Lane
Improve style of some replication-related error messages. Put the remote end's error message into the primary error string, instead of relegating it to errdetail(). Although this could end up being awkward if the remote sends us a really long error message, it seems more in keeping with our messa

pgsql: Fix some libpq_pipeline test problems

2021-03-31 Thread Alvaro Herrera
Fix some libpq_pipeline test problems Test pipeline_abort was not checking that it got the rows it expected in one mode; make it do so. This doesn't fix the actual problem (no idea what that is, yet) but at least it should make it more obvious rather than being visible only as a difference in the

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Tomas Vondra
On 3/31/21 7:08 PM, Alvaro Herrera wrote: > On 2021-Mar-31, Tomas Vondra wrote: > >> Thanks for the report, I'll take a look. You're right this seems like an >> out-of-bounds access, but mcv_match_expression is only expected to be >> run on expressions we know are in the statistics (because we pic

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Alvaro Herrera
On 2021-Mar-31, Tomas Vondra wrote: > Thanks for the report, I'll take a look. You're right this seems like an > out-of-bounds access, but mcv_match_expression is only expected to be > run on expressions we know are in the statistics (because we pick the > statistics like that). Clearly, that does

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread Tomas Vondra
On 3/31/21 10:05 AM, David Rowley wrote: > Hi Tomas, > > I'm debugging a crash after running sqllancer on current master. The > first bad commit seems to be this one. > > The crash stack trace is: > > Program received signal SIGSEGV, Segmentation fault. > pg_detoast_datum_packed (datum=0x5) a

pgsql: Rework planning and execution of UPDATE and DELETE.

2021-03-31 Thread Tom Lane
Rework planning and execution of UPDATE and DELETE. This patch makes two closely related sets of changes: 1. For UPDATE, the subplan of the ModifyTable node now only delivers the new values of the changed columns (i.e., the expressions computed in the query's SET clause) plus row identity informa

pgsql: Allow an alias to be attached to a JOIN ... USING

2021-03-31 Thread Peter Eisentraut
Allow an alias to be attached to a JOIN ... USING This allows something like SELECT ... FROM t1 JOIN t2 USING (a, b, c) AS x where x has the columns a, b, c and unlike a regular alias it does not hide the range variables of the tables being joined t1 and t2. Per SQL:2016 feature F404 "Range

pgsql: Add support for asynchronous execution.

2021-03-31 Thread Etsuro Fujita
Add support for asynchronous execution. This implements asynchronous execution, which runs multiple parts of a non-parallel-aware Append concurrently rather than serially to improve performance when possible. Currently, the only node type that can be run concurrently is a ForeignScan that is an i

pgsql: Add p_names field to ParseNamespaceItem

2021-03-31 Thread Peter Eisentraut
Add p_names field to ParseNamespaceItem ParseNamespaceItem had a wired-in assumption that p_rte->eref describes the table and column aliases exposed by the nsitem. This relaxes this by creating a separate p_names field in an nsitem. This is mainly preparation for a patch for JOIN USING aliases,

Re: pgsql: Extended statistics on expressions

2021-03-31 Thread David Rowley
Hi Tomas, I'm debugging a crash after running sqllancer on current master. The first bad commit seems to be this one. The crash stack trace is: Program received signal SIGSEGV, Segmentation fault. pg_detoast_datum_packed (datum=0x5) at fmgr.c:1759 1759 if (VARATT_IS_COMPRESSED(datum) || VARATT_I

pgsql: Add errhint_plural() function and make use of it

2021-03-31 Thread Peter Eisentraut
Add errhint_plural() function and make use of it Similar to existing errmsg_plural() and errdetail_plural(). Some errhint() calls hadn't received the proper plural treatment yet. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/91c5a8caaa61055959aa5fb68a00e5f690e39a