Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-07 Thread Amit Kapila
On Wed, Dec 7, 2022 at 9:35 AM shiy.f...@fujitsu.com wrote: > > On Wed, Dec 7, 2022 12:01 PM Dilip Kumar wrote: > > > > > > > > Thanks. I think it is better to go ahead with this patch and once we > > > decide what is the right thing to do in terms of GUC then we can try > > > to add additional

RE: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-06 Thread shiy.f...@fujitsu.com
On Wed, Dec 7, 2022 12:01 PM Dilip Kumar wrote: > > On Wed, Dec 7, 2022 at 9:28 AM Amit Kapila > wrote: > > > > On Tue, Dec 6, 2022 at 11:55 AM shiy.f...@fujitsu.com > > wrote: > > > > > > On Mon, Dec 5, 2022 6:57 PM Amit Kapila > wrote: > > > > > > > > On Mon, Dec 5, 2022 at 3:41 PM Dilip

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-06 Thread Dilip Kumar
On Wed, Dec 7, 2022 at 9:28 AM Amit Kapila wrote: > > On Tue, Dec 6, 2022 at 11:55 AM shiy.f...@fujitsu.com > wrote: > > > > On Mon, Dec 5, 2022 6:57 PM Amit Kapila wrote: > > > > > > On Mon, Dec 5, 2022 at 3:41 PM Dilip Kumar wrote: > > > > > > > > I think we need something like this[1] so

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-06 Thread Amit Kapila
On Tue, Dec 6, 2022 at 11:55 AM shiy.f...@fujitsu.com wrote: > > On Mon, Dec 5, 2022 6:57 PM Amit Kapila wrote: > > > > On Mon, Dec 5, 2022 at 3:41 PM Dilip Kumar wrote: > > > > > > I think we need something like this[1] so that we can better control > > > the streaming. > > > > > > > +1. The

RE: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-05 Thread shiy.f...@fujitsu.com
On Mon, Dec 5, 2022 6:57 PM Amit Kapila wrote: > > On Mon, Dec 5, 2022 at 3:41 PM Dilip Kumar wrote: > > > > I think we need something like this[1] so that we can better control > > the streaming. > > > > +1. The additional advantage would be that we can generate parallel > apply and new

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-05 Thread Amit Kapila
On Mon, Dec 5, 2022 at 3:41 PM Dilip Kumar wrote: > > On Mon, Dec 5, 2022 at 9:21 AM Dilip Kumar wrote: > > > > On Mon, Dec 5, 2022 at 8:59 AM Amit Kapila wrote: > > > > > > On Sun, Dec 4, 2022 at 5:14 PM houzj.f...@fujitsu.com > > > wrote: > > > > > > > > On Saturday, December 3, 2022 7:37 PM

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-05 Thread Dilip Kumar
On Mon, Dec 5, 2022 at 9:21 AM Dilip Kumar wrote: > > On Mon, Dec 5, 2022 at 8:59 AM Amit Kapila wrote: > > > > On Sun, Dec 4, 2022 at 5:14 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Saturday, December 3, 2022 7:37 PM Amit Kapila > > > wrote: > > > > Apart from the above, I have

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-04 Thread Dilip Kumar
On Mon, Dec 5, 2022 at 8:59 AM Amit Kapila wrote: > > On Sun, Dec 4, 2022 at 5:14 PM houzj.f...@fujitsu.com > wrote: > > > > On Saturday, December 3, 2022 7:37 PM Amit Kapila > > wrote: > > > Apart from the above, I have slightly adjusted the comments in the > > > attached. Do > > > let me

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-04 Thread Amit Kapila
On Sun, Dec 4, 2022 at 5:14 PM houzj.f...@fujitsu.com wrote: > > On Saturday, December 3, 2022 7:37 PM Amit Kapila > wrote: > > Apart from the above, I have slightly adjusted the comments in the > > attached. Do > > let me know what you think of the attached. > > Thanks for updating the patch.

RE: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-04 Thread houzj.f...@fujitsu.com
On Saturday, December 3, 2022 7:37 PM Amit Kapila wrote: > > On Tue, Nov 29, 2022 at 12:23 PM houzj.f...@fujitsu.com > wrote: > > > > On Tuesday, November 29, 2022 12:08 PM Dilip Kumar > wrote: > > > > I have few comments about the patch. > > > > 1. > > > > 1.1. > > - /* For streamed

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-03 Thread Amit Kapila
On Tue, Nov 29, 2022 at 12:23 PM houzj.f...@fujitsu.com wrote: > > On Tuesday, November 29, 2022 12:08 PM Dilip Kumar > wrote: > > I have few comments about the patch. > > 1. > > 1.1. > - /* For streamed transactions notify the remote node about the abort. > */ > - if

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-02 Thread Amit Kapila
On Fri, Dec 2, 2022 at 4:58 PM Ashutosh Bapat wrote: > > Hi Dilip, > > On Tue, Nov 29, 2022 at 9:38 AM Dilip Kumar wrote: > > > > > > > > You are right we need this in ReorderBufferProcessPartialChange() as > > > well. I will fix this in the next version. > > > > Fixed this in the attached

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-12-02 Thread Ashutosh Bapat
Hi Dilip, On Tue, Nov 29, 2022 at 9:38 AM Dilip Kumar wrote: > > > > > You are right we need this in ReorderBufferProcessPartialChange() as > > well. I will fix this in the next version. > > Fixed this in the attached patch. > I focused my attention on SnapBuildXactNeedsSkip() usages and I see

RE: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-28 Thread houzj.f...@fujitsu.com
On Tuesday, November 29, 2022 12:08 PM Dilip Kumar wrote: Hi, > > On Mon, Nov 28, 2022 at 3:19 PM Dilip Kumar wrote: > > > > On Mon, Nov 28, 2022 at 1:46 PM shiy.f...@fujitsu.com > > wrote: > > > > > > Thanks for your patch. > > > > > > I saw that the patch added a check when selecting

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-28 Thread Dilip Kumar
On Mon, Nov 28, 2022 at 3:19 PM Dilip Kumar wrote: > > On Mon, Nov 28, 2022 at 1:46 PM shiy.f...@fujitsu.com > wrote: > > > > Thanks for your patch. > > > > I saw that the patch added a check when selecting largest transaction, but > > in > > addition to ReorderBufferCheckMemoryLimit(), the

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-28 Thread Dilip Kumar
On Mon, Nov 28, 2022 at 1:46 PM shiy.f...@fujitsu.com wrote: > > Thanks for your patch. > > I saw that the patch added a check when selecting largest transaction, but in > addition to ReorderBufferCheckMemoryLimit(), the transaction can also be > streamed in ReorderBufferProcessPartialChange().

RE: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-28 Thread shiy.f...@fujitsu.com
On Sun, Nov 27, 2022 1:33 PM Dilip Kumar wrote: > > On Sat, Nov 26, 2022 at 12:15 PM Amit Kapila > wrote: > > > > On Fri, Nov 25, 2022 at 5:38 PM Amit Kapila > wrote: > > > > > > On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar > wrote: > > > > > > > > During DecodeCommit() for skipping a

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-26 Thread Dilip Kumar
On Sat, Nov 26, 2022 at 12:15 PM Amit Kapila wrote: > > On Fri, Nov 25, 2022 at 5:38 PM Amit Kapila wrote: > > > > On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar wrote: > > > > > > During DecodeCommit() for skipping a transaction we use ReadRecPtr to > > > check whether to skip this transaction or

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-26 Thread Amit Kapila
On Sat, Nov 26, 2022 at 10:59 AM Dilip Kumar wrote: > > On Fri, Nov 25, 2022 at 4:04 PM Ashutosh Bapat > wrote: > > > > Excellent catch. We were looking at this code last week and wondered > > the purpose of this abort. Probably we should have some macro or > > function to decided whether to

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Amit Kapila
On Fri, Nov 25, 2022 at 5:38 PM Amit Kapila wrote: > > On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar wrote: > > > > During DecodeCommit() for skipping a transaction we use ReadRecPtr to > > check whether to skip this transaction or not. Whereas in > > ReorderBufferCanStartStreaming() we use

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Dilip Kumar
On Fri, Nov 25, 2022 at 4:04 PM Ashutosh Bapat wrote: > > Excellent catch. We were looking at this code last week and wondered > the purpose of this abort. Probably we should have some macro or > function to decided whether to skip a transaction based on log record. > That will avoid using

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Amit Kapila
On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar wrote: > > During DecodeCommit() for skipping a transaction we use ReadRecPtr to > check whether to skip this transaction or not. Whereas in > ReorderBufferCanStartStreaming() we use EndRecPtr to check whether to > stream or not. Generally it will not

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Ashutosh Bapat
Excellent catch. We were looking at this code last week and wondered the purpose of this abort. Probably we should have some macro or function to decided whether to skip a transaction based on log record. That will avoid using different values in different places. On Fri, Nov 25, 2022 at 1:35 PM

Avoid streaming the transaction which are skipped (in corner cases)

2022-11-25 Thread Dilip Kumar
During DecodeCommit() for skipping a transaction we use ReadRecPtr to check whether to skip this transaction or not. Whereas in ReorderBufferCanStartStreaming() we use EndRecPtr to check whether to stream or not. Generally it will not create a problem but if the commit record itself is adding