Re: SEARCH and CYCLE clauses

2020-10-09 Thread Pavel Stehule
Hi pá 9. 10. 2020 v 12:17 odesílatel Pavel Stehule napsal: > > > pá 9. 10. 2020 v 11:40 odesílatel Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> napsal: > >> On 2020-09-22 20:29, Pavel Stehule wrote: >> > The result is correct. When I tried to use UNION instead UNION ALL, the >> > pg cr

Re: [PATCH] Add `truncate` option to subscription commands

2020-10-09 Thread Amit Kapila
On Sat, Oct 10, 2020 at 12:24 AM David Christensen wrote: > > -hackers, > > Enclosed find a patch to add a “truncate” option to subscription commands. > > When adding new tables to a subscription (either via `CREATE SUBSCRIPTION` or > `REFRESH PUBLICATION`), tables on the target which are being n

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 7:32 PM Greg Nancarrow wrote: > > On Fri, Oct 9, 2020 at 11:57 PM Amit Kapila wrote: > > > > > > > > Sure but in that case 'top_plan->parallel_safe' should be false and it > > > should stick Gather node atop Insert node. > > > > > > > /should/should not. > > > > OK, for the

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread David G. Johnston
On Fri, Oct 9, 2020 at 6:41 PM Ranier Vilela wrote: > The problem is not only in nodeIncrementalSort.c, but in several others > too, where people are using TupIsNull with ExecCopySlot. > I would call this a design flaw. > If (TupIsNull) > ExecCopySlot > > The callers, think they are using Tu

Use list_delete_xxxcell O(1) instead of list_delete_ptr O(N) in some places

2020-10-09 Thread Hou, Zhijie
Hi I found some code places call list_delete_ptr can be replaced by list_delete_xxxcell which can be faster. diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c index db54a6b..61ef7c8 100644 --- a/src/backend/optimizer/path/joinpath.c +++ b/src/backend/opt

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Ranier Vilela
Em sex., 9 de out. de 2020 às 19:45, Tom Lane escreveu: > Tomas Vondra writes: > > My (admittedly very subjective) opinion is that it looks much worse. The > > TupIsNull is pretty self-descriptive, unlike this proposed code. > > +1 > > > That could be fixed by defining a new macro, perhaps somet

Re: pg_upgrade: fail early if a tablespace dir already exists for new cluster version

2020-10-09 Thread Bruce Momjian
On Fri, Oct 9, 2020 at 02:23:10PM -0500, Justin Pryzby wrote: > In my local branch, I had revised this comment to say: > > + * Note, v8.4 has no tablespace_suffix, which is fine so long as the version > we > + * being upgraded *to* has a suffix, since it's not allowed to pg_upgrade > from > + *

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Tom Lane
Tomas Vondra writes: > My (admittedly very subjective) opinion is that it looks much worse. The > TupIsNull is pretty self-descriptive, unlike this proposed code. +1 > That could be fixed by defining a new macro, perhaps something like > SlotIsEmpty() or so. But as was already explained, Increme

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Tomas Vondra
On Fri, Oct 09, 2020 at 05:50:09PM -0300, Ranier Vilela wrote: Em sex., 9 de out. de 2020 às 17:47, Tom Lane escreveu: Ranier Vilela writes: > The trap is not on the second part of expression. Is in the first. > If the pointer is NULL, ExecCopySlot will be called. Your initial comment indica

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Tomas Vondra
On Fri, Oct 09, 2020 at 05:25:02PM -0300, Ranier Vilela wrote: Em sex., 9 de out. de 2020 às 14:05, Tomas Vondra < tomas.von...@2ndquadrant.com> escreveu: On Fri, Oct 09, 2020 at 12:24:16PM -0300, Ranier Vilela wrote: >I think that TupIsNull macro is no longer appropriate, to protect >ExecCopyS

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Peter Geoghegan
On Fri, Oct 9, 2020 at 2:04 PM Ranier Vilela wrote: >> > The author of deduplication claimed that he thinks the problem may be in >> > IncrementalSort, >> > he did not specify which part. >> >> No I didn't. > > https://www.postgresql.org/message-id/CAH2-Wz=ae84z0pxtbc+ssgi9ec8ngkn9d16op-dgh47jcrv

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Stephen Frost
Greetings, * Ranier Vilela (ranier...@gmail.com) wrote: > Em sex., 9 de out. de 2020 às 18:02, Stephen Frost > escreveu: > > As a side-note, this kind of further analysis and looking for other, > > similar, cases that might be problematic is really helpful and important > > to do whenever you com

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Ranier Vilela
Em sex., 9 de out. de 2020 às 18:02, Stephen Frost escreveu: > Greetings, > > * Ranier Vilela (ranier...@gmail.com) wrote: > > Em sex., 9 de out. de 2020 às 14:05, Tomas Vondra < > > tomas.von...@2ndquadrant.com> escreveu: > > > > > On Fri, Oct 09, 2020 at 12:24:16PM -0300, Ranier Vilela wrote: >

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Tom Lane
Ranier Vilela writes: > So I said that TupIsNull was not the most appropriate. [ shrug... ] You're entitled to your opinion, but I see essentially no value in running around and trying to figure out which TupIsNull calls actually can see a null pointer and which never will. It'd likely introduc

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Ranier Vilela
Em sex., 9 de out. de 2020 às 17:58, Peter Geoghegan escreveu: > On Fri, Oct 9, 2020 at 1:28 PM Ranier Vilela wrote: > > Sorry, can't find the thread. > > The author of deduplication claimed that he thinks the problem may be in > IncrementalSort, > > he did not specify which part. > > No I didn'

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Stephen Frost
Greetings, * Ranier Vilela (ranier...@gmail.com) wrote: > Em sex., 9 de out. de 2020 às 14:05, Tomas Vondra < > tomas.von...@2ndquadrant.com> escreveu: > > > On Fri, Oct 09, 2020 at 12:24:16PM -0300, Ranier Vilela wrote: > > >I think that TupIsNull macro is no longer appropriate, to protect > > >

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Peter Geoghegan
On Fri, Oct 9, 2020 at 1:28 PM Ranier Vilela wrote: > Sorry, can't find the thread. > The author of deduplication claimed that he thinks the problem may be in > IncrementalSort, > he did not specify which part. No I didn't. -- Peter Geoghegan

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Ranier Vilela
Em sex., 9 de out. de 2020 às 17:47, Tom Lane escreveu: > Ranier Vilela writes: > > The trap is not on the second part of expression. Is in the first. > > If the pointer is NULL, ExecCopySlot will be called. > > Your initial comment indicated that you were worried about > IncrementalSortState's

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Tom Lane
Ranier Vilela writes: > The trap is not on the second part of expression. Is in the first. > If the pointer is NULL, ExecCopySlot will be called. Your initial comment indicated that you were worried about IncrementalSortState's group_pivot slot, but that is never going to be null in any execution

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Thomas Munro
On Fri, Oct 9, 2020 at 11:58 PM Greg Nancarrow wrote: > On Fri, Oct 9, 2020 at 8:41 PM Amit Kapila wrote: > > That will be true for the number of rows/pages we need to scan not for > > the number of tuples we need to return as a result. The formula here > > considers the number of rows the parall

Re: Improving connection scalability: GetSnapshotData()

2020-10-09 Thread Andrew Dunstan
On 10/5/20 10:33 PM, Andres Freund wrote: > Hi, > > On 2020-10-01 19:21:14 -0400, Andrew Dunstan wrote: >> On 10/1/20 4:22 PM, Andres Freund wrote: >>> # Note: on Windows, IPC::Run seems to convert \r\n to \n in program >>> output >>> # if we're using native Perl, but not if we're using

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Ranier Vilela
Em sex., 9 de out. de 2020 às 14:05, Tomas Vondra < tomas.von...@2ndquadrant.com> escreveu: > On Fri, Oct 09, 2020 at 12:24:16PM -0300, Ranier Vilela wrote: > >I think that TupIsNull macro is no longer appropriate, to protect > >ExecCopySlot. > > > >See at tuptable.h: > >#define TupIsNull(slot) \

Re: BUG #15858: could not stat file - over 4GB

2020-10-09 Thread Tom Lane
Emil Iggland writes: > I tested the patch at hand, and it performs as expected. Files larger than > 4GB can be imported. Thanks for testing! I'd been expecting one of our Windows-savvy committers to pick this up, but since nothing has been happening, I took it on myself to push it. I'll probabl

Re: [PATCH] ecpg: fix progname memory leak

2020-10-09 Thread John Naylor
On Fri, Oct 9, 2020 at 12:50 PM Tom Lane wrote: > Consulting the wiki page's edit history shows that Bruce has been > doing some maintenance work, but almost nobody else does. Evidently > that's not enough. > > I'd be +1 for the "aggressive culling" suggested upthread, but > I'm not particularly

Re: pg_upgrade: fail early if a tablespace dir already exists for new cluster version

2020-10-09 Thread Justin Pryzby
On Fri, Oct 09, 2020 at 02:53:25PM -0400, Bruce Momjian wrote: > On Thu, Sep 24, 2020 at 07:55:31PM -0500, Justin Pryzby wrote: > > This should be caught during --check, or earlier in the upgrade, rather than > > only after dumping the schema. > > > > Typically, the tablespace dir would be left be

Re: dynamic result sets support in extended query protocol

2020-10-09 Thread Dave Cramer
On Fri, 9 Oct 2020 at 14:59, Andres Freund wrote: > Hi, > > On 2020-10-09 14:49:11 -0400, Dave Cramer wrote: > > On Fri, 9 Oct 2020 at 14:46, Andres Freund wrote: > > > > (I suspect what would be more useful in practice is to designate > > > > output formats per data type.) > > > > > > Yea, that

Re: dynamic result sets support in extended query protocol

2020-10-09 Thread Andres Freund
Hi, On 2020-10-09 14:49:11 -0400, Dave Cramer wrote: > On Fri, 9 Oct 2020 at 14:46, Andres Freund wrote: > > > (I suspect what would be more useful in practice is to designate > > > output formats per data type.) > > > > Yea, that'd be *really* useful. It sucks that we basically require > > multi

Re: [PATCH] ecpg: fix progname memory leak

2020-10-09 Thread Bruce Momjian
On Fri, Oct 9, 2020 at 12:50:36PM -0400, Tom Lane wrote: > Magnus Hagander writes: > > That said, having this in a different format would in no way fix the fact > > that the information is mislabeled and obsolete. It would likely be > > equally mislabeled and obsolete in an issue tracker. Just l

[PATCH] Add `truncate` option to subscription commands

2020-10-09 Thread David Christensen
-hackers, Enclosed find a patch to add a “truncate” option to subscription commands. When adding new tables to a subscription (either via `CREATE SUBSCRIPTION` or `REFRESH PUBLICATION`), tables on the target which are being newly subscribed will be truncated before the data copy step. This sav

Re: pg_upgrade: fail early if a tablespace dir already exists for new cluster version

2020-10-09 Thread Bruce Momjian
On Thu, Sep 24, 2020 at 07:55:31PM -0500, Justin Pryzby wrote: > This should be caught during --check, or earlier in the upgrade, rather than > only after dumping the schema. > > Typically, the tablespace dir would be left behind by a previous failed > upgrade > attempt, causing a cascade of fail

Re: dynamic result sets support in extended query protocol

2020-10-09 Thread Dave Cramer
Hi, On Fri, 9 Oct 2020 at 14:46, Andres Freund wrote: > Hi, > > On 2020-10-08 09:46:38 +0200, Peter Eisentraut wrote: > > New would be that the server would now also respond with a new message, > say, > > > > S: DynamicResultInfo > > > Now, if the client has seen DynamicResultInfo earlier, it s

Re: dynamic result sets support in extended query protocol

2020-10-09 Thread Andres Freund
Hi, On 2020-10-08 09:46:38 +0200, Peter Eisentraut wrote: > New would be that the server would now also respond with a new message, say, > > S: DynamicResultInfo > Now, if the client has seen DynamicResultInfo earlier, it should now go into > a new subsequence to get the remaining result sets, l

Re: dynamic result sets support in extended query protocol

2020-10-09 Thread Dave Cramer
On Fri, 9 Oct 2020 at 13:33, Andrew Dunstan wrote: > > On 10/8/20 3:46 AM, Peter Eisentraut wrote: > > I want to progress work on stored procedures returning multiple result > > sets. Examples of how this could work on the SQL side have previously > > been shown [0]. We also have ongoing work t

Re: Batching page logging during B-tree build

2020-10-09 Thread Andrey Borodin
> 23 сент. 2020 г., в 23:19, Peter Geoghegan написал(а): > > On Fri, Sep 18, 2020 at 8:39 AM Andrey M. Borodin > wrote: >> Here is PoC with porting that same routine to B-tree. It allows to build >> B-trees ~10% faster on my machine. > > It doesn't seem to make any difference on my machine

Re: dynamic result sets support in extended query protocol

2020-10-09 Thread Andrew Dunstan
On 10/8/20 3:46 AM, Peter Eisentraut wrote: > I want to progress work on stored procedures returning multiple result > sets.  Examples of how this could work on the SQL side have previously > been shown [0].  We also have ongoing work to make psql show multiple > result sets [1].  This appears to

Re: Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Tomas Vondra
On Fri, Oct 09, 2020 at 12:24:16PM -0300, Ranier Vilela wrote: I think that TupIsNull macro is no longer appropriate, to protect ExecCopySlot. See at tuptable.h: #define TupIsNull(slot) \ ((slot) == NULL || TTS_EMPTY(slot)) If var node->group_pivot is NULL, ExecCopySlot will dereference a null

Re: [PATCH] ecpg: fix progname memory leak

2020-10-09 Thread Tom Lane
Magnus Hagander writes: > That said, having this in a different format would in no way fix the fact > that the information is mislabeled and obsolete. It would likely be > equally mislabeled and obsolete in an issue tracker. Just look at almost > any of the other projects out there that *do* use

Re: Expansion of our checks for connection-loss errors

2020-10-09 Thread Tom Lane
I wrote: > Hmm, excellent point. While our code response to all these errors > should be the same, you are right that that doesn't extend to emitting > identical error texts. For EHOSTUNREACH/ENETDOWN/ENETUNREACH, we > should say something like "connection to server lost", without claiming > that

Re: [PATCH] ecpg: fix progname memory leak

2020-10-09 Thread Magnus Hagander
On Fri, Oct 9, 2020 at 1:08 PM John Naylor wrote: > > On Fri, Oct 9, 2020 at 4:47 AM Daniel Gustafsson wrote: > >> Now, I don't actually suggest we *remove* it, as there is valuable curated >> content, but that we rename it to something which won't encourage >> newcomers to >> pick something fro

Possible NULL dereferencing null pointer (src/backend/executor/nodeIncrementalSort.c)

2020-10-09 Thread Ranier Vilela
I think that TupIsNull macro is no longer appropriate, to protect ExecCopySlot. See at tuptable.h: #define TupIsNull(slot) \ ((slot) == NULL || TTS_EMPTY(slot)) If var node->group_pivot is NULL, ExecCopySlot will dereference a null pointer (first arg). Maybe, this can be related to a bug reporte

Re: Wrong example in the bloom documentation

2020-10-09 Thread Bruce Momjian
On Fri, Oct 9, 2020 at 05:44:57AM +, Daniel Westermann (DWE) wrote: > Hi Bruce, Tom, > > On Thu, Oct  8, 2020 at 03:43:32PM -0400, Tom Lane wrote: > >> "Daniel Westermann (DWE)" writes: > >> >> I was hoping someone more experienced with this would comment, but > >> >> seeing none, I will app

Re: libpq debug log

2020-10-09 Thread Alvaro Herrera
On 2020-Oct-09, Kyotaro Horiguchi wrote: > I saw that version and have some comments. > > +pqGetProtocolMsgType(unsigned char c, PGCommSource commsource) > +{ > + const char *message_type; > > Compiler complains that this is unused. > > +static const char * > +pqGetProtocolMsgType(unsigned

Re: Uninitialized var utilized (src/backend/tsearch/spell.c)

2020-10-09 Thread Ranier Vilela
Em sex., 9 de out. de 2020 às 11:37, Tom Lane escreveu: > Ranier Vilela writes: > > Em sex., 9 de out. de 2020 às 11:08, Daniel Gustafsson > > escreveu: > >> To help reviewers, your report should contain an explanation of when > that > >> can happen. > > > When option "flag" is not handled. > >

Re: Uninitialized var utilized (src/backend/tsearch/spell.c)

2020-10-09 Thread Tom Lane
Ranier Vilela writes: > Em sex., 9 de out. de 2020 às 11:08, Daniel Gustafsson > escreveu: >> To help reviewers, your report should contain an explanation of when that >> can happen. > When option "flag" is not handled. > if (STRNCMP(pstr, "flag") == 0) I think what he means is that if the file

Re: Expansion of our checks for connection-loss errors

2020-10-09 Thread Tom Lane
Fujii Masao writes: > On 2020/10/09 4:15, Tom Lane wrote: >> -#ifdef ECONNRESET >> -case ECONNRESET: >> +case ALL_CONNECTION_LOSS_ERRNOS: >> printfPQExpBuffer(&conn->errorMessage, >>

Re: Uninitialized var utilized (src/backend/tsearch/spell.c)

2020-10-09 Thread Ranier Vilela
Em sex., 9 de out. de 2020 às 11:08, Daniel Gustafsson escreveu: > > On 9 Oct 2020, at 14:36, Ranier Vilela wrote: > > > At function NIImportAffixes (src/backend/tsearch/spell.c). > > > > If option "flag" is not handled, variable char flag[BUFSIZE] will remain > uninitialized. > > To help review

Re: Uninitialized var utilized (src/backend/tsearch/spell.c)

2020-10-09 Thread Daniel Gustafsson
> On 9 Oct 2020, at 14:36, Ranier Vilela wrote: > At function NIImportAffixes (src/backend/tsearch/spell.c). > > If option "flag" is not handled, variable char flag[BUFSIZE] will remain > uninitialized. To help reviewers, your report should contain an explanation of when that can happen. chee

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Greg Nancarrow
On Fri, Oct 9, 2020 at 11:57 PM Amit Kapila wrote: > > > > > Sure but in that case 'top_plan->parallel_safe' should be false and it > > should stick Gather node atop Insert node. > > > > /should/should not. > OK, for the minimal patch, just allowing INSERT with parallel SELECT, you're right, neit

Re: Two fsync related performance issues?

2020-10-09 Thread Michael Banck
Hi, Am Mittwoch, den 07.10.2020, 18:17 +1300 schrieb Thomas Munro: > On Mon, Oct 5, 2020 at 2:38 PM Thomas Munro wrote: > > On Wed, Sep 9, 2020 at 3:49 PM Thomas Munro wrote: > > > For the record, Andres Freund mentioned a few problems with this > > > off-list and suggested we consider calling L

Re: Expansion of our checks for connection-loss errors

2020-10-09 Thread Fujii Masao
On 2020/10/09 4:15, Tom Lane wrote: Over in the thread at [1], we've tentatively determined that the reason buildfarm member lorikeet is currently failing is that its network stack returns ECONNABORTED for (some?) connection failures, whereas our code is only expecting ECONNRESET. Fujii Masao

Re: Assertion failure with LEFT JOINs among >500 relations

2020-10-09 Thread Tom Lane
David Rowley writes: > On Fri, 9 Oct 2020 at 15:06, Tom Lane wrote: >> I notice there are some other ad-hoc isnan() checks scattered >> about costsize.c, too. Maybe we should indeed consider fixing >> clamp_row_estimate to get rid of inf (and nan too, I suppose) >> so that we'd not need those.

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 6:26 PM Amit Kapila wrote: > > On Fri, Oct 9, 2020 at 3:51 PM Greg Nancarrow wrote: > > > > On Fri, Oct 9, 2020 at 8:09 PM Amit Kapila wrote: > > > > OK, I will update the comments for this. > > Basically, up to now, the "force_parallel_mode" has only ever operated > > on

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 3:51 PM Greg Nancarrow wrote: > > On Fri, Oct 9, 2020 at 8:09 PM Amit Kapila wrote: > > > > 0001-InsertParallelSelect > > 1. > > ParallelContext *pcxt; > > > > + /* > > + * We need to avoid an attempt on INSERT to assign a > > + * FullTransactionId whilst in parallel mode (

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 5:54 PM Greg Nancarrow wrote: > > On Fri, Oct 9, 2020 at 8:09 PM Amit Kapila wrote: > > > > + /* > > + * We need to avoid an attempt on INSERT to assign a > > + * FullTransactionId whilst in parallel mode (which is in > > + * effect due to the underlying parallel query) - s

Re: Transactions involving multiple postgres foreign servers, take 2

2020-10-09 Thread Masahiko Sawada
On Fri, 9 Oct 2020 at 14:55, Kyotaro Horiguchi wrote: > > At Fri, 9 Oct 2020 02:33:37 +, "tsunakawa.ta...@fujitsu.com" > wrote in > > From: Masahiko Sawada > > > What about temporary network failures? I think there are users who > > > don't want to give up resolving foreign transactions fai

Uninitialized var utilized (src/backend/tsearch/spell.c)

2020-10-09 Thread Ranier Vilela
At function NIImportAffixes (src/backend/tsearch/spell.c). If option "flag" is not handled, variable char flag[BUFSIZE] will remain uninitialized. regards, Ranier Vilela fix_uninitialized_var_flag_spell.patch Description: Binary data

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Greg Nancarrow
On Fri, Oct 9, 2020 at 8:09 PM Amit Kapila wrote: > > + /* > + * We need to avoid an attempt on INSERT to assign a > + * FullTransactionId whilst in parallel mode (which is in > + * effect due to the underlying parallel query) - so the > + * FullTransactionId is assigned here. Parallel mode must >

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 4:28 PM Greg Nancarrow wrote: > > On Fri, Oct 9, 2020 at 8:41 PM Amit Kapila wrote: > > > > On Fri, Oct 9, 2020 at 2:37 PM Greg Nancarrow wrote: > > > > > > Speaking of costing, I'm not sure I really agree with the current > > > costing of a Gather node. Just considering a

Re: [PATCH] ecpg: fix progname memory leak

2020-10-09 Thread John Naylor
On Fri, Oct 9, 2020 at 4:47 AM Daniel Gustafsson wrote: > Now, I don't actually suggest we *remove* it, as there is valuable curated > content, but that we rename it to something which won't encourage > newcomers to > pick something from the list simply because it exists. The name "TODO" > impli

Re: Transactions involving multiple postgres foreign servers, take 2

2020-10-09 Thread Masahiko Sawada
On Fri, 9 Oct 2020 at 11:33, tsunakawa.ta...@fujitsu.com wrote: > > From: Masahiko Sawada > > What about temporary network failures? I think there are users who > > don't want to give up resolving foreign transactions failed due to a > > temporary network failure. Or even they might want to wait

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Greg Nancarrow
On Fri, Oct 9, 2020 at 8:41 PM Amit Kapila wrote: > > On Fri, Oct 9, 2020 at 2:37 PM Greg Nancarrow wrote: > > > > Speaking of costing, I'm not sure I really agree with the current > > costing of a Gather node. Just considering a simple Parallel SeqScan > > case, the "run_cost += parallel_tuple_c

Re: Parallel copy

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 3:50 PM Bharath Rupireddy wrote: > > On Fri, Oct 9, 2020 at 3:26 PM Amit Kapila wrote: > > > > On Fri, Oct 9, 2020 at 2:52 PM Bharath Rupireddy > > wrote: > > > > > > On Tue, Sep 29, 2020 at 6:30 PM Amit Kapila > > > wrote: > > > > > > > > From the testing perspective, >

Re: Parallel copy

2020-10-09 Thread Bharath Rupireddy
On Fri, Oct 9, 2020 at 3:26 PM Amit Kapila wrote: > > On Fri, Oct 9, 2020 at 2:52 PM Bharath Rupireddy > wrote: > > > > On Tue, Sep 29, 2020 at 6:30 PM Amit Kapila wrote: > > > > > > From the testing perspective, > > > 1. Test by having something force_parallel_mode = regress which means > > > t

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Greg Nancarrow
On Fri, Oct 9, 2020 at 8:09 PM Amit Kapila wrote: > > 0001-InsertParallelSelect > 1. > ParallelContext *pcxt; > > + /* > + * We need to avoid an attempt on INSERT to assign a > + * FullTransactionId whilst in parallel mode (which is in > + * effect due to the underlying parallel query) - so the >

Re: SEARCH and CYCLE clauses

2020-10-09 Thread Pavel Stehule
pá 9. 10. 2020 v 11:40 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2020-09-22 20:29, Pavel Stehule wrote: > > The result is correct. When I tried to use UNION instead UNION ALL, the > > pg crash > > I fixed the crash, but UNION [DISTINCT] won't actually work here b

Re: powerpc pg_atomic_compare_exchange_u32_impl: error: comparison of integer expressions of different signedness (Re: pgsql: For all ppc compilers, implement compare_exchange and) fetch_add

2020-10-09 Thread Noah Misch
On Fri, Oct 09, 2020 at 11:28:25AM +0200, Christoph Berg wrote: > pg-cron on powerpc/ppc64/ppc64el is raising this warning inside the > ppc atomics: > > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement > -Werror=vla -Wendif-labels -Wmissing-format-attribute > -Wimplic

Re: Parallel copy

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 2:52 PM Bharath Rupireddy wrote: > > On Tue, Sep 29, 2020 at 6:30 PM Amit Kapila wrote: > > > > From the testing perspective, > > 1. Test by having something force_parallel_mode = regress which means > > that all existing Copy tests in the regression will be executed via >

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 2:37 PM Greg Nancarrow wrote: > > Speaking of costing, I'm not sure I really agree with the current > costing of a Gather node. Just considering a simple Parallel SeqScan > case, the "run_cost += parallel_tuple_cost * path->path.rows;" part of > Gather cost always completely

Re: SEARCH and CYCLE clauses

2020-10-09 Thread Peter Eisentraut
On 2020-09-22 20:29, Pavel Stehule wrote: The result is correct. When I tried to use UNION instead UNION ALL, the pg crash I fixed the crash, but UNION [DISTINCT] won't actually work here because row/record types are not hashable. I'm leaving the partial support in, but I'm documenting it as

Re: [HACKERS] Custom compression methods

2020-10-09 Thread Dilip Kumar
On Fri, Oct 9, 2020 at 3:24 AM Tomas Vondra wrote: > > On Thu, Oct 08, 2020 at 02:38:27PM +0530, Dilip Kumar wrote: > >On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar wrote: > >> > >> On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar wrote: > >> > > >> > On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra > >> >

powerpc pg_atomic_compare_exchange_u32_impl: error: comparison of integer expressions of different signedness (Re: pgsql: For all ppc compilers, implement compare_exchange and) fetch_add

2020-10-09 Thread Christoph Berg
Re: Noah Misch > For all ppc compilers, implement compare_exchange and fetch_add with asm. > > This is more like how we handle s_lock.h and arch-x86.h. > > Reviewed by Tom Lane. > > Discussion: https://postgr.es/m/20191005173400.ga3979...@rfd.leadboat.com Hi, pg-cron on powerpc/ppc64/ppc64el i

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Amit Kapila
On Tue, Oct 6, 2020 at 3:08 PM Greg Nancarrow wrote: > > On Mon, Oct 5, 2020 at 10:36 PM Dilip Kumar wrote: > > Also, I have attached a separate patch (requested by Andres Freund) > that just allows the underlying SELECT part of "INSERT INTO ... SELECT > ..." to be parallel. > It might be a good

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Greg Nancarrow
On Fri, Oct 9, 2020 at 6:31 PM Thomas Munro wrote: > > A couple more observations: > > + pathnode->path.parallel_aware = parallel_workers > 0 ? true : false; > > Hmm, I think this may be bogus window dressing only affecting EXPLAIN. > If you change it to assign false always, it works just th

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-09 Thread Amit Kapila
On Fri, Oct 9, 2020 at 2:19 PM Simon Riggs wrote: > > On Fri, 9 Oct 2020 at 04:10, Amit Kapila wrote: > > > > On Thu, Oct 8, 2020 at 2:34 PM Simon Riggs wrote: > > > > > > On Thu, 8 Oct 2020 at 09:47, Dilip Kumar wrote: > > > > > > > > This script will wait 10 seconds after INSERT exits > > > >

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-09 Thread Simon Riggs
On Fri, 9 Oct 2020 at 04:10, Amit Kapila wrote: > > On Thu, Oct 8, 2020 at 2:34 PM Simon Riggs wrote: > > > > On Thu, 8 Oct 2020 at 09:47, Dilip Kumar wrote: > > > > > > This script will wait 10 seconds after INSERT exits > > > > before executing TRUNCATE, please wait for it to run. > > > > Has

Re: [PATCH] ecpg: fix progname memory leak

2020-10-09 Thread Daniel Gustafsson
> On 8 Oct 2020, at 19:08, Bruce Momjian wrote: > OK, TODO removed. Potentially controversial proposal: Should we perhaps remove (for some value of) the TODO list altogether? The value of the list is questionable as it's not actually a TODO list for established developers, and for aspiring new

Re: Wrong statistics for size of XLOG_SWITCH during pg_waldump.

2020-10-09 Thread Kyotaro Horiguchi
At Fri, 9 Oct 2020 13:41:25 +0800, "movead...@highgo.ca" wrote in > Hello hackers, > > We know that pg_waldump can statistics size for every kind of records. When I > use > the feature I find it misses some size for XLOG_SWITCH records. When a user > does > a pg_wal_switch(), then postgres wi

Add a description to the documentation that toast_tuple_target affects "Main"

2020-10-09 Thread Shinya Okano
Hi, Regarding the toast_tuple_target parameter of CREATE TABLE, the documentation says that it only affects External or Extended, but it actually affects the compression of Main as well. The attached patch modifies the document to match the actual behavior. Regards, -- Shinya Okano diff --g

Re: libpq debug log

2020-10-09 Thread Kyotaro Horiguchi
At Wed, 16 Sep 2020 17:41:55 -0300, Alvaro Herrera wrote in > Hello Aya Iwata, > > I like this patch, and I think we should have it. I have updated it, as > it had conflicts. > > In 0002, I remove use of ftime(), because that function is obsolete. > However, with that change we lose printing

Re: Parallel copy

2020-10-09 Thread Greg Nancarrow
On Fri, Oct 9, 2020 at 5:40 PM Amit Kapila wrote: > > > Looking a bit deeper into this, I'm wondering if in fact your > > EstimateStringSize() and EstimateNodeSize() functions should be using > > BUFFERALIGN() for EACH stored string/node (rather than just calling > > shm_toc_estimate_chunk() once

Wrong statistics for size of XLOG_SWITCH during pg_waldump.

2020-10-09 Thread movead...@highgo.ca
Hello hackers, We know that pg_waldump can statistics size for every kind of records. When I use the feature I find it misses some size for XLOG_SWITCH records. When a user does a pg_wal_switch(), then postgres will discard the remaining size in the current wal segment, and the pg_waldump tool m

Re: dynamic result sets support in extended query protocol

2020-10-09 Thread Peter Eisentraut
On 2020-10-08 10:23, Tatsuo Ishii wrote: Are you proposing to bump up the protocol version (either major or minor)? I am asking because it seems you are going to introduce some new message types. It wouldn't be a new major version. It could either be a new minor version, or it would be guard

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-09 Thread Thomas Munro
On Fri, Oct 9, 2020 at 3:48 PM Greg Nancarrow wrote: > It does give me the incentive to look beyond that issue and see > whether parallel Update and parallel Delete are indeed possible. I'll > be sure to give it a go! Cool! A couple more observations: + pathnode->path.parallel_aware = par

abstract Unix-domain sockets

2020-10-09 Thread Peter Eisentraut
During the discussion on Unix-domain sockets on Windows, someone pointed out[0] abstract Unix-domain sockets. This is a variant of the normal Unix-domain sockets that don't use the file system but a separate "abstract" namespace. At the user interface, such sockets are represented by names st

RE: Transactions involving multiple postgres foreign servers, take 2

2020-10-09 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > I don't understand why we hate ERRORs from fdw-2pc-commit routine so > much. I think remote-commits should be performed before local commit > passes the point-of-no-return and the v26-0002 actually places > AtEOXact_FdwXact() before the critical section. I don't hate ERR