Re: Identify missing publications from publisher while create/alter subscription.

2021-05-06 Thread Dilip Kumar
report(ERROR, + (errmsg("could not connect to the publisher: %s", err))); Instead of using global wrconn, I think you should use a local variable? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Race condition in recovery?

2021-05-07 Thread Dilip Kumar
On Fri, May 7, 2021 at 2:33 PM Kyotaro Horiguchi wrote: > > Thanks. > > > At Fri, 7 May 2021 11:04:53 +0530, Dilip Kumar wrote > in > > On Fri, May 7, 2021 at 8:23 AM Kyotaro Horiguchi > > wrote: > > > > > > At Tue, 4 May 2021 17:41:06 +05

Re: Identify missing publications from publisher while create/alter subscription.

2021-05-07 Thread Dilip Kumar
On Fri, May 7, 2021 at 5:38 PM Bharath Rupireddy wrote: > > On Fri, May 7, 2021 at 11:50 AM Dilip Kumar wrote: > > > > On Thu, May 6, 2021 at 7:22 PM vignesh C wrote: > > > > > > > Some comments: > > 1. > > I don't see any change in p

Re: Logical Replication - behavior of TRUNCATE ... CASCADE

2021-05-07 Thread Dilip Kumar
to set that. I think the documentation should clearly say the impact of setting this to true. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-07 Thread Dilip Kumar
this thread again. Thanks! -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Inaccurate error message when set fdw batch_size to 0

2021-05-07 Thread Dilip Kumar
eger value” ? > > I also found fetch_size has the similar issue, attaching a patch to fix this. Yes, it should be a positive integer, so your change makes sense. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-08 Thread Dilip Kumar
hanges. I will send the other patches soon, after removing the doc part which you have already included here. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-08 Thread Dilip Kumar
On Sat, May 8, 2021 at 2:08 PM Michael Paquier wrote: > > On Thu, May 06, 2021 at 09:33:53PM +0530, Dilip Kumar wrote: > > On Thu, May 6, 2021 at 5:42 PM Michael Paquier wrote: > > > > > > On Thu, May 06, 2021 at 10:45:53AM +0530, Dilip Kumar wrote: > > >

Re: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-09 Thread Dilip Kumar
teps. > > Now, the \help command is for commands, which is a different thing as the > command in that case is DELETE not DELETE FROM, even if you will have to > follow > your DELETE with a FROM. I agree with Julien. But, I also agree with the consistency point from Tang. So maybe we can fix the insert and add INSERT INTO in the tab completion? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel INSERT SELECT take 2

2021-05-09 Thread Dilip Kumar
tputs objects that are not parallel safe. > > > > So, users need to check count(*) for this to determine > parallel-safety? How about if we provide a wrapper function on top of > this function or a separate function that returns char to indicate > whether it is safe, unsafe, or restricted to perform a DML operation > on the table? Such wrapper function make sense. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-09 Thread Dilip Kumar
On Mon, May 10, 2021 at 11:27 AM Michael Paquier wrote: > > On Sat, May 08, 2021 at 08:19:03PM +0530, Dilip Kumar wrote: > > I have fixed some of them, I could not write the test cases where we > > have to ensure that 'ALTER TABLE COMPRESSION' is not appearing in the &

Re: Race condition in recovery?

2021-05-10 Thread Dilip Kumar
WAL, this patch is required for that to work. = > > I believe the 004_timeline_switch.pl detects your issue. And the > attached change fixes it. I think this fix looks better to me, but I will think more about it and give my feedback. Thanks for quickly coming up with the repro

Re: GetSubscriptionRelations declares too many scan keys

2021-05-10 Thread Dilip Kumar
pattern as every other ScanData skey[n] code. > > > > Please search PG source code for "ScanData skey[1];" - there are > > dozens of precedents where other people felt the same as me for > > declaring single keys. > > AFAICT there are 73 occurences vs 62 of the "Scandata skey;". I don't think > there's a huge consensus for one over the other. I think both Scandata skey[1]; and Scandata skey; are used. But IMHO using Scandata skey; looks better. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Proposal] Global temporary tables

2021-05-10 Thread Dilip Kumar
true for the vacuum no? I will try to do a more detailed review. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-10 Thread Dilip Kumar
Doc said. > CREATE POLICY > CREATE [ OR REPLACE ] RULE > CREATE [ OR REPLACE ] TRIGGER > ALTER DEFAULT PRIVILEGES > > After applying the patch, the tap-tests for psql is passed. > Please be free to tell me anything insufficient you found in my fix. Thanks. LGTM. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-10 Thread Dilip Kumar
t least this will give people an option to test LZ4 on > Windows. Perhaps this will require some adjustments, but let's see if > that's necessary when that comes up. Thanks, make sense. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-10 Thread Dilip Kumar
ritten the WAL in XLogAcceptWrites but later if we could not set the state to WALPROHIBIT_STATE_READ_WRITE? Then maybe we can inform all the backend first but before setting the state to WALPROHIBIT_STATE_READ_WRITE, we can call XLogAcceptWrites? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Race condition in recovery?

2021-05-11 Thread Dilip Kumar
On Tue, May 11, 2021 at 1:42 PM Kyotaro Horiguchi wrote: > > At Mon, 10 May 2021 14:27:21 +0530, Dilip Kumar wrote > in > > On Mon, May 10, 2021 at 2:05 PM Kyotaro Horiguchi > > wrote: > > > > > I thought that the reason using receiveTLI instead of > >

Re: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-11 Thread Dilip Kumar
... USING" > is not as good as it is now.(I guess that's why the comment was added). And > for now, IMHO, we can remove the comment directly. But your patch is doing nothing to add the implementation for DELETE.. USING. Basically, the tab completion support for DELETEUSING is still pending right? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-11 Thread Dilip Kumar
d memory state is changed to WALPROHIBIT_STATE_READ_WRITE? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-11 Thread Dilip Kumar
On Tue, May 11, 2021 at 3:03 PM tanghy.f...@fujitsu.com wrote: > > On Tuesday, May 11, 2021 5:44 PM, Dilip Kumar wrote: > >But your patch is doing nothing to add the implementation for DELETE.. > >USING. Basically, the tab completion support for DELETEUSING is > >st

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-11 Thread Dilip Kumar
On Tue, May 11, 2021 at 3:38 PM Amul Sul wrote: > > On Tue, May 11, 2021 at 2:26 PM Dilip Kumar wrote: > > > > On Tue, May 11, 2021 at 2:16 PM Amul Sul wrote: > > > > > I get why you think that, I wasn't very precise in briefing the problem. > > >

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-11 Thread Dilip Kumar
On Tue, May 11, 2021 at 4:50 PM Amul Sul wrote: > > On Tue, May 11, 2021 at 4:13 PM Dilip Kumar wrote: > > I might be missing something, but assume the behavior should be like this > > > > 1. If the state is getting changed from WALPROHIBIT_STATE_READ_WRITE > >

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-11 Thread Dilip Kumar
On Tue, May 11, 2021 at 6:56 PM Amul Sul wrote: > > On Tue, May 11, 2021 at 6:48 PM Dilip Kumar wrote: > > > > On Tue, May 11, 2021 at 4:50 PM Amul Sul wrote: > > > > > > On Tue, May 11, 2021 at 4:13 PM Dilip Kumar wrote: > > > > I might be miss

Re: parallel vacuum - few questions on docs, comments and code

2021-05-11 Thread Dilip Kumar
rform vacuum in parallel */ > if (parallel_workers <= 0) Yes it should if (parallel_workers == 0) > 8) Is it still true that if parallel workers are specified as 0 the > parallelism will not be picked up? > From the docs: This feature is known as parallel vacuum. To disable

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-11 Thread Dilip Kumar
on't need separate "if checks" whether the XLogAcceptWrites() is called or not, instead we can just rely on the state, if it is #4 then we have to call XLogAcceptWrites(). If so then I think it's okay to have an additional state, just wanted to know what idea you had in mind? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: OOM in spgist insert

2021-05-12 Thread Dilip Kumar
On Wed, 12 May 2021 at 1:43 PM, Pavel Borisov wrote: > ср, 12 мая 2021 г. в 11:09, Dilip Kumar : > >> While testing something on spgist I found that at certain point while >> inserting in spgist it is going for doPickSplit, but even after split >> is is not able to find a

Re: OOM in spgist insert

2021-05-12 Thread Dilip Kumar
On Wed, May 12, 2021 at 2:21 PM Pavel Borisov wrote: >> >> PFA v1 patch. Does this help? > > I've made a mistake in attributes count in v1. PFA v2 V2 works. Thanks for fixing this quickly, I think you can add a comment for the new error condition you added. -

Re: OOM in spgist insert

2021-05-12 Thread Dilip Kumar
ow whether we can reduce + * index tuple size by suffixing its key part or we will go into the + * endless loop on pick-split (in case included columns data is big enough INCLUDEd -> why you have used a mixed case here? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: OOM in spgist insert

2021-05-12 Thread Dilip Kumar
v4 Okay, that makes sense. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: alter subscription drop publication fixes

2021-05-12 Thread Dilip Kumar
if (!oldpublist) > > ereport(ERROR, > > (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), > > errmsg("subscription must contain at least one > > publication"), > > errdetail("Dropping all the publications from a > > subscription is not supported"))); > > Or how about just errmsg("cannot drop all the publications of the > subscriber \"%s\"", subname) without any error detail? IMHO, this message without errdetail looks much better. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-12 Thread Dilip Kumar
On Thu, May 13, 2021 at 2:26 AM Robert Haas wrote: > > On Wed, May 12, 2021 at 1:39 AM Dilip Kumar wrote: > > Your idea makes sense, but IMHO, if we are first writing > > XLogAcceptWrites() and then pushing out the barrier, then I don't > > understand the meanin

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-13 Thread Dilip Kumar
now IMHO, we don't need to keep the system in recovery until pg_prohibit_wal(false) is called, right? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
hose pid you have passed, the connected client to that backend might not have superuser privileges and if you use elevel LOG then that message will be sent to that connected client as well and I don't think that is secure. So can we use LOG_SERVER_ONLY so that we can prevent it from sending to the client. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 2:54 PM Amul Sul wrote: > > On Thu, May 13, 2021 at 12:36 PM Dilip Kumar wrote: > > > > On Wed, May 12, 2021 at 5:55 PM Amul Sul wrote: > > > > > > > Thanks for the updated patch, while going through I noticed this comment. > >

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 3:06 PM Bharath Rupireddy wrote: > > On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy > wrote: > > On Thu, May 13, 2021 at 2:44 PM Dilip Kumar wrote: > > > +1 for the idea. I did not read the complete patch but while reading > > > thro

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
> Yes, that was my exact point, that in this particular code log with LOG_SERVER_ONLY. Like this. /* Log immediately if dictated by log_statement */ if (check_log_statement(parsetree_list)) { ereport(LOG_SERVER_ONLY, . -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy wrote: > > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar wrote: > > > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy > > wrote: > > > > > > I'm saying that - currently, queries are logged wit

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 5:18 PM Dilip Kumar wrote: > > On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy > wrote: > > > > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar wrote: > > > > > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy > > > wro

Re: Race condition in recovery?

2021-05-13 Thread Dilip Kumar
ory(recoveryTargetTLI); - - /* - * If the location of the checkpoint record is not on the expected - * timeline in the history of the requested timeline, we cannot proceed: - * the backup is not part of the history of the requested timeline. - */ - if (tliOfPointInHistory(ControlFile->checkPoint, expectedTLEs) != - ControlFile->checkPointCopy.ThisTimeLineID) - { = -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: parallel vacuum - few questions on docs, comments and code

2021-05-13 Thread Dilip Kumar
thing there yet. - pg_log_error("parallel vacuum degree must be a non-negative integer"); + pg_log_error("parallel workers for vacuum must be greater than or equal to zero"); exit(1); [1] https://www.postgresql.org/message-id/os0pr01mb5716415335a06b489f1b3a8194...@os0pr0

Re: OOM in spgist insert

2021-05-13 Thread Dilip Kumar
ng back-branches > > unfixed is not great. > > Hm. Index bloat is not something to totally ignore, though, so I'm > not sure what the best cutoff is. > > Anyway, here is a patch set teased apart into committable bites, > and with your other points addr

Re: Support for VACUUMing Foreign Tables

2021-05-13 Thread Dilip Kumar
sm they are using) on the remote server and what is the vacuuming need for that database. So maybe garbage collection should be controlled locally by the DBA on that server. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Race condition in recovery?

2021-05-14 Thread Dilip Kumar
l if we directly fix this one line to initialize "expectedTLEs" from "recoveryTargetTLI" then it will expose to the same problem this commit tried to fix. [2] if (tliOfPointInHistory(ControlFile->checkPoint, expectedTLEs) != ControlFile->checkPointCopy.ThisTimeLineID) { error() } -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-15 Thread Dilip Kumar
On Thu, May 13, 2021 at 2:56 PM Dilip Kumar wrote: > > Great thanks. I will review the remaining patch soon. I have reviewed v28-0003, and I have some comments on this. === @@ -126,9 +127,14 @@ XLogBeginInsert(void) Assert(mainrdata_last == (XLogRecData *) &mainrdata_head);

Re: Race condition in recovery?

2021-05-16 Thread Dilip Kumar
f history was missing. > The fix caused the issue for the case (1). Basically, before this commit expectedTLEs and recoveryTargetTLI were in always in sync which this patch broke. > The proposed fix fixes the case (1), caused by the commit. Right, I agree with the fix. So fix should be j

Re: Race condition in recovery?

2021-05-16 Thread Dilip Kumar
On Mon, May 17, 2021 at 10:09 AM Dilip Kumar wrote: > > On Mon, May 17, 2021 at 8:50 AM Kyotaro Horiguchi > wrote: > > > > Before the commit expectedTLEs is always initialized with just one > > entry for the TLI of the last checkpoint record. > > Right > >

Re: pg_dumpall misses --no-toast-compression

2021-05-16 Thread Dilip Kumar
simple but that's perhaps too > late for 14, so I am adding it to the next CF. +1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-17 Thread Dilip Kumar
On Mon, May 17, 2021 at 11:48 AM Amul Sul wrote: > > On Sat, May 15, 2021 at 3:12 PM Dilip Kumar wrote: > > > > On Thu, May 13, 2021 at 2:56 PM Dilip Kumar wrote: > > > > > > Great thanks. I will review the remaining patch soon. > > > > I hav

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Dilip Kumar
s scenario. I am not sure this test case is exactly targeting the problematic behavior because that will depends upon the order of execution of the apply workers right? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Dilip Kumar
On Mon, May 17, 2021 at 3:43 PM Amit Kapila wrote: > > On Mon, May 17, 2021 at 3:05 PM Dilip Kumar wrote: > > > > On Mon, May 17, 2021 at 12:30 PM Peter Smith wrote: > > > > > PSA a patch adding a test for this scenario. > > > > I am not sure this t

Re: Race condition in recovery?

2021-05-17 Thread Dilip Kumar
with the recovery target timeline is the right fix. - We still have some differences of opinion about what was the original problem in the base code which was fixed by the commit (ee994272ca50f70b53074f0febaec97e28f83c4e). -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-17 Thread Dilip Kumar
then someone can pause again and we will be in loop so better to exit as soon as promotion is requested, see attached patch. Should be applied along with your patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com diff --git a/src/backend/access/transam/xlog.c b/src/backend/access

Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-18 Thread Dilip Kumar
tion is requested in that case, like when > pg_wal_replay_resume() is executed in that case. Thought? Yeah, you are right, I missed that. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-18 Thread Dilip Kumar
ly because > pg_get_wal_replay_pause_state() needs to be executed > before the promotion finishes. Even for #2, we can not ensure that whether it will be 'paused' or 'pause requested'. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-19 Thread Dilip Kumar
On Wed, May 19, 2021 at 11:55 AM Kyotaro Horiguchi wrote: > > At Wed, 19 May 2021 11:19:13 +0530, Dilip Kumar wrote > in > > On Wed, May 19, 2021 at 10:16 AM Fujii Masao > > wrote: > > > > > > On 2021/05/18 15:46, Michael Paquier wrote: > > > &

Re: Race condition in recovery?

2021-05-19 Thread Dilip Kumar
TimeLineHistory(recoveryTargetTLI); [1] if (tliOfPointInHistory(ControlFile->checkPoint, expectedTLEs) != ControlFile->checkPointCopy.ThisTimeLineID) { report(FATAL.. } -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-20 Thread Dilip Kumar
t; version bump between betas. > > This is still an open item. FWIW, I can get behind the reordering > proposed by Tom for the consistency gained with pg_type, leading to > the attached to reduce the size of FormData_pg_attribute from 116b to > 112b. This makes sense, thanks fo

Re: Race condition in recovery?

2021-05-21 Thread Dilip Kumar
On Thu, May 20, 2021 at 11:19 PM Robert Haas wrote: > > On Tue, May 18, 2021 at 1:33 AM Dilip Kumar wrote: > > Yeah, it will be a fake 1-element list. But just to be clear that > > 1-element can only be "ControlFile->checkPointCopy.ThisTimeLineID" and > &g

Re: Race condition in recovery?

2021-05-21 Thread Dilip Kumar
ll just initialize expectedTLE based on the online entry (ControlFile->checkPointCopy.ThisTimeLineID) and later we will fail to find the checkpoint record on this timeline because the checkpoint LSN is smaller than the start LSN of this timeline. Right? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Race condition in recovery?

2021-05-21 Thread Dilip Kumar
tream with old TL. Now walrecievr is under impression that it has read from the old TL. And, we know the rest of the story that we will set the expectedTLEs based on the old history file and never be able to go to the new TL. Anyways now we understand the issue and there are many ways we can r

Re: Logical Replication - behavior of TRUNCATE ... CASCADE

2021-05-21 Thread Dilip Kumar
On Thu, May 20, 2021 at 5:03 PM Amit Kapila wrote: > > On Fri, May 7, 2021 at 6:06 PM Dilip Kumar wrote: > > > > On Mon, May 3, 2021 at 6:08 PM Bharath Rupireddy > > wrote: > > > > > > Having said that, isn't it good if we can provide a sub

Re: Race condition in recovery?

2021-05-21 Thread Dilip Kumar
On Sat, May 22, 2021 at 10:15 AM Dilip Kumar wrote: > > On Sat, May 22, 2021 at 1:14 AM Robert Haas wrote: > > > > The attached test script, test.sh seems to reliably reproduce this. > > Put that file and the recalcitrant_cp script, also attached, into an > > I have

Re: Race condition in recovery?

2021-05-23 Thread Dilip Kumar
ting fixed after putting the fix we discussed[1] [1] - expectedTLEs = readTimeLineHistory(receiveTLI); + expectedTLEs = readTimeLineHistory(recoveryTargetTLI); -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com test2.sh Description: application/shellscript

Re: Race condition in recovery?

2021-05-23 Thread Dilip Kumar
On Sun, May 23, 2021 at 2:19 PM Dilip Kumar wrote: > > On Sat, May 22, 2021 at 8:33 PM Robert Haas wrote: I have created a tap test based on Robert's test.sh script. It reproduces the issue. I am new with perl so this still needs some cleanup/improvement, but at least it sho

Re: Race condition in recovery?

2021-05-23 Thread Dilip Kumar
On Mon, May 24, 2021 at 10:17 AM Kyotaro Horiguchi wrote: > > At Sun, 23 May 2021 21:37:58 +0530, Dilip Kumar wrote > in > > On Sun, May 23, 2021 at 2:19 PM Dilip Kumar wrote: > > > > > > On Sat, May 22, 2021 at 8:33 PM Robert Haas wrote: > > > >

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-23 Thread Dilip Kumar
y in "ATExecAddColumn()" right? But we can not ALTER TABLE ADD COLUMN to matviews right? I agree that even if we don't skip matview it will not create any issue as matview will not reach here. Am I missing something? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-23 Thread Dilip Kumar
t need to initialize tup_values[i] with the values[i];, other than that looks fine to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-24 Thread Dilip Kumar
On Mon, May 24, 2021 at 2:23 PM Michael Paquier wrote: > > On Mon, May 24, 2021 at 11:32:22AM +0530, Dilip Kumar wrote: > > I think you don't need to initialize tup_values[i] with the > > values[i];, other than that looks fine to me. > > You mean because heap_deform

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-24 Thread Dilip Kumar
On Tue, 25 May 2021 at 11:16 AM, Michael Paquier wrote: > On Mon, May 24, 2021 at 02:46:11PM +0530, Dilip Kumar wrote: > > Like this. > > if (TupleDescAttr(newTupDesc, i)->attisdropped) > > isnull[i] = true; > > else > > tup_values[i] = v

Re: Different compression methods for FPI

2021-05-24 Thread Dilip Kumar
e same as LZ4_compress_fast with acceleration=1. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Assertion failure while streaming toasted data

2021-05-24 Thread Dilip Kumar
t; transaction, but fails to adjust the buffer size for toast chunks. Maybe we > are missing a call to `ReorderBufferToastReset()` somewhere? > > From what I see, the assertion only triggers when data is inserted via COPY > (multi-insert). > > Let me know if anything else is needed to reproduce this. Thanks, I will look into this and let you know if need some help. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
On Tue, May 25, 2021 at 12:12 PM Dilip Kumar wrote: > > On Tue, May 25, 2021 at 12:06 PM Pavan Deolasee > wrote: > > > > Hi, > > > > While working on an output plugin that uses streaming protocol, I hit an > > assertion failure. Further investi

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
he > test_decoding plugin, but if you have some other mechanism to test that > change with an actual downstream > node receiving and applying changes, it will be useful to test with that. Okay, I will test that. Thanks. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
On Tue, May 25, 2021 at 1:59 PM Pavan Deolasee wrote: > > On Tue, May 25, 2021 at 1:49 PM Dilip Kumar wrote: >> >> On Tue, May 25, 2021 at 1:45 PM Pavan Deolasee >> wrote: >> > >> > I am not entirely sure if it works correctly. I'd tried somethi

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
On Tue, May 25, 2021 at 2:34 PM Dilip Kumar wrote: > > > When the transaction is streamed, I see: > > ``` > > + opening a streamed block for transaction > > + table public.toasted: INSERT: id[integer]:9001 other[text]:'bbb' > > data[text]:'ccc

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
eaming code kicks in very often and even > for small transactions. Thanks for confirming, I will come up with the test and add that to the next version of the patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
am unless we get the last tuple of multi insert WAL. How about > changing the code such that when we are clearing the toast flag, we > additionally check 'clear_toast_afterwards' flag? Yes, that can be done, I will fix this in the next version of the patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
On Tue, May 25, 2021 at 5:46 PM Dilip Kumar wrote: > > On Tue, May 25, 2021 at 4:50 PM Amit Kapila wrote: > > > > Your patch will fix the reported scenario but I don't like the way > > multi_insert flag is used to detect incomplete tuple. One problem > > c

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
size 6000 bytes so that is big enough to cross 64kB. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
On Wed, May 26, 2021 at 11:19 AM Amit Kapila wrote: > > On Tue, May 25, 2021 at 6:43 PM Dilip Kumar wrote: > > > > On Tue, May 25, 2021 at 5:46 PM Dilip Kumar wrote: > > > > > > On Tue, May 25, 2021 at 4:50 PM Amit Kapila > > > wrote: > > &g

Re: Assertion failure while streaming toasted data

2021-05-25 Thread Dilip Kumar
> acceptable or not. In the test decoding config, it is already set to a minimum value which is 64k. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Race condition in recovery?

2021-05-25 Thread Dilip Kumar
check whether the cascade standby is able to get that or not, that will guarantee that it is actually following the promoted standby, I have added the test for that so that it matches the comments. > Let's not call the command skip_cp. It's not very descriptive. If you > don't

Re: Assertion failure while streaming toasted data

2021-05-26 Thread Dilip Kumar
On Wed, May 26, 2021 at 11:53 AM Dilip Kumar wrote: > > On Wed, May 26, 2021 at 11:19 AM Amit Kapila wrote: > > > > > I searched and didn't find any similar existing tests. Can we think of > > any other way to test this code path? We already have one copy

Re: Assertion failure while streaming toasted data

2021-05-26 Thread Dilip Kumar
On Wed, May 26, 2021 at 5:28 PM Amit Kapila wrote: > > On Wed, May 26, 2021 at 1:47 PM Dilip Kumar wrote: > > > > On Wed, May 26, 2021 at 11:53 AM Dilip Kumar wrote: > > > > > > On Wed, May 26, 2021 at 11:19 AM Amit Kapila > > > wrote: > > &

Re: Race condition in recovery?

2021-05-26 Thread Dilip Kumar
On Wed, May 26, 2021 at 9:40 PM Robert Haas wrote: > > On Wed, May 26, 2021 at 2:44 AM Dilip Kumar wrote: > > I think we need to create some content on promoted standby and check > > whether the cascade standby is able to get that or not, that will > > guarantee that it i

Re: Race condition in recovery?

2021-05-26 Thread Dilip Kumar
On Wed, 26 May 2021 at 10:06 PM, Robert Haas wrote: > On Wed, May 26, 2021 at 12:26 PM Dilip Kumar > wrote: > > I will check if there is any timing dependency in the test case. > > There is. I explained it in the second part of my email, which you may > have failed to no

Re: Decoding speculative insert with toast leaks memory

2021-05-26 Thread Dilip Kumar
in > ReorderBufferCleanupTXN/ReorderBufferTruncateTXN? It will delay the > clean up of memory till the end of stream or txn but there won't be > any memory leak. Currently, we are ignoring XLH_DELETE_IS_SUPER, so maybe we can do something based on this flag? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Decoding speculative insert with toast leaks memory

2021-05-26 Thread Dilip Kumar
maining, just clean in up, it > * can't have been successful, otherwise we'd gotten a confirmation > * record. > */ > if (specinsert) > { > ReorderBufferReturnChange(rb, specinsert, true); > specinsert = NULL; > } > > But I guess we might miss cleaning it up in case of an error. A > similar problem could be there in the idea where we will try to tie > the clean up with the next change. In error case also we can handle it in the CATCH block no? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Decoding speculative insert with toast leaks memory

2021-05-26 Thread Dilip Kumar
On Thu, May 27, 2021 at 9:47 AM Amit Kapila wrote: > > On Thu, May 27, 2021 at 9:40 AM Dilip Kumar wrote: > > True, but if you do this clean-up in ReorderBufferCleanupTXN then you > don't need to take care at separate places. Also, toast_hash is stored > in txn so it appe

Re: Parallel Inserts in CREATE TABLE AS

2021-05-26 Thread Dilip Kumar
lex condition and also it select fewer record say 50%, 40%...10% and see what are the numbers. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Race condition in recovery?

2021-05-26 Thread Dilip Kumar
On Thu, May 27, 2021 at 6:19 AM Kyotaro Horiguchi wrote: > > At Wed, 26 May 2021 22:08:32 +0530, Dilip Kumar wrote > in > > On Wed, 26 May 2021 at 10:06 PM, Robert Haas wrote: > > > > > On Wed, May 26, 2021 at 12:26 PM Dilip Kumar > > > wrote: >

Re: Race condition in recovery?

2021-05-26 Thread Dilip Kumar
t/recovery/t/025_stuck_on_old_timeline.pl > I suggest we rename the test to something a bit more descriptive. Like > instead of 025_timeline_issue.pl, perhaps > 025_stuck_on_old_timeline.pl? Or I'm open to other suggestions, but > "timeline issue" is a bit too vague for my taste. Changed

Re: Parallel Inserts in CREATE TABLE AS

2021-05-26 Thread Dilip Kumar
On Thu, 27 May 2021 at 11:32 AM, tsunakawa.ta...@fujitsu.com < tsunakawa.ta...@fujitsu.com> wrote: > From: Dilip Kumar > > I think some other cause of contention on relation extension locks are > > 1. CTAS is using a buffer strategy and due to that, it might need to >

Re: Race condition in recovery?

2021-05-27 Thread Dilip Kumar
On Thu, May 27, 2021 at 12:09 PM Kyotaro Horiguchi wrote: > > At Thu, 27 May 2021 11:44:47 +0530, Dilip Kumar wrote > in > > Maybe we can somehow achieve that without a broken archive command, > > but I am not sure how it is enough to just delete WAL from pg_wal? I >

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Dilip Kumar
he user wants to get rid of the dependency on the external library then IMHO, there should be some way to do it by recompressing all the data. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [BUG]Update Toast data failure in logical replication

2021-05-28 Thread Dilip Kumar
or ALTER TABLE toasted_key REPLICA IDENTITY FULL. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Decoding speculative insert with toast leaks memory

2021-05-28 Thread Dilip Kumar
On Fri, May 28, 2021 at 5:16 PM Tomas Vondra wrote: > On 5/27/21 6:36 AM, Dilip Kumar wrote: > > On Thu, May 27, 2021 at 9:47 AM Amit Kapila wrote: > >> > >> On Thu, May 27, 2021 at 9:40 AM Dilip Kumar wrote: > >> > >> True, but if you do this

Re: Decoding speculative insert with toast leaks memory

2021-05-30 Thread Dilip Kumar
mitigation, so that we don't leak the memory > > forever. > > > > Right. > > > So let's get this committed, perhaps with a comment explaining that it > > might be possible to reset earlier if needed. > > > > Okay, I think it would be better if we can test this once for the > streaming case as well. Dilip, would you like to do that and send the > updated patch as per one of the comments by Tomas? I will do that in sometime. > -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [BUG]Update Toast data failure in logical replication

2021-05-30 Thread Dilip Kumar
On Mon, May 31, 2021 at 8:04 AM tanghy.f...@fujitsu.com wrote: > > On Friday, May 28, 2021 6:51 PM, Dilip Kumar wrote: > > Seems you did not set the replica identity for updating the tuple. > > Try this before updating, and it should work. > > Thanks for your reply. I tri

<    2   3   4   5   6   7   8   9   10   11   >