Re: Skipping logical replication transactions on subscriber side

2022-06-23 Thread Robert Haas
On Wed, Jun 22, 2022 at 10:48 PM Noah Misch wrote: > Here's a more-verbose description of (2), with additions about what it does > and doesn't achieve: > > 2. On systems where double alignment differs from int64 alignment, require >NAMEDATALEN%8==0. Modify the test from commits 79b716c and

Re: Skipping logical replication transactions on subscriber side

2022-06-22 Thread Noah Misch
On Wed, Jun 22, 2022 at 09:50:02AM -0400, Robert Haas wrote: > On Wed, Jun 22, 2022 at 12:28 AM Noah Misch wrote: > > Here's how I rank the options, from most-preferred to least-preferred: > > > > 1. Put new eight-byte fields at the front of each catalog, when in doubt. > > 2. On systems where

Re: Skipping logical replication transactions on subscriber side

2022-06-22 Thread Robert Haas
On Wed, Jun 22, 2022 at 11:01 AM Tom Lane wrote: > Given that we haven't run into this before, it seems like a reasonable > bet that the problem will seldom arise. So as long as we have a > cross-check I'm all right with calling it good and moving on. Expending > a whole lot of work to improve

Re: Skipping logical replication transactions on subscriber side

2022-06-22 Thread Tom Lane
Robert Haas writes: > On Wed, Jun 22, 2022 at 10:39 AM Tom Lane wrote: >> I don't especially want to >> invent an additional typalign code that we could only test on legacy >> platforms. > I agree with that, but I don't think that having the developers > enforce alignment rules by reordering

Re: Skipping logical replication transactions on subscriber side

2022-06-22 Thread Robert Haas
On Wed, Jun 22, 2022 at 10:39 AM Tom Lane wrote: > That's a fundamental misreading of the situation. typalign is essential > on alignment-picky architectures, else you will get a SIGBUS fault > when trying to fetch a multibyte value (whether it's just going to get > stored into a Datum array is

Re: Skipping logical replication transactions on subscriber side

2022-06-22 Thread Tom Lane
[ sorry for not having tracked this thread more closely ... ] Robert Haas writes: > Regarding (1), it is my opinion that the only real value of typalign > is for system catalogs, and specifically that it lets you put the > fields in an order that is aesthetically pleasing rather than worrying >

Re: Skipping logical replication transactions on subscriber side

2022-06-22 Thread Robert Haas
On Wed, Jun 22, 2022 at 12:28 AM Noah Misch wrote: > "Everything" isn't limited to PostgreSQL. The Perl ABI exposes large structs > to plperl; a field of type double could require the AIX user to rebuild Perl > with the same compiler option. Oh, that isn't so great, then. > Here's how I rank

Re: Skipping logical replication transactions on subscriber side

2022-06-21 Thread Noah Misch
On Mon, Jun 13, 2022 at 10:25:24AM -0400, Robert Haas wrote: > On Sun, Apr 17, 2022 at 11:22 PM Noah Misch wrote: > > > Yes, but it could be false positives in some cases. For instance, the > > > column {oid, bool, XLogRecPtr} should be okay on ALIGNOF_DOUBLE == 4 > > > and 8 platforms but the

Re: Skipping logical replication transactions on subscriber side

2022-06-20 Thread Robert Haas
On Mon, Jun 20, 2022 at 9:52 AM Peter Eisentraut wrote: > That means changing the system's ABI, so in the extreme case you then > need to compile everything else to match as well. I think we wouldn't want to do that in a minor release, but doing it in a new major release seems fine -- especially

Re: Skipping logical replication transactions on subscriber side

2022-06-20 Thread Peter Eisentraut
On 16.06.22 18:35, Robert Haas wrote: On Thu, Jun 16, 2022 at 3:26 AM Masahiko Sawada wrote: FWIW, looking at the manual, there might have been a solution for AIX to specify -qalign=natural compiler option in order to enforce the alignment of double to 8. Well if that can work it sure seems

Re: Skipping logical replication transactions on subscriber side

2022-06-16 Thread Robert Haas
On Thu, Jun 16, 2022 at 3:26 AM Masahiko Sawada wrote: > FWIW, looking at the manual, there might have > been a solution for AIX to specify -qalign=natural compiler option in > order to enforce the alignment of double to 8. Well if that can work it sure seems better. -- Robert Haas EDB:

Re: Skipping logical replication transactions on subscriber side

2022-06-16 Thread Masahiko Sawada
On Thu, Jun 16, 2022 at 2:27 AM Robert Haas wrote: > > On Tue, Jun 14, 2022 at 3:54 AM Masahiko Sawada wrote: > > > AFAICS, we could do that by: > > > > > > 1. De-supporting platforms that have this problem, or > > > 2. Introducing new typalign values, as Noah proposed back on April 2, or > > >

Re: Skipping logical replication transactions on subscriber side

2022-06-15 Thread Robert Haas
On Tue, Jun 14, 2022 at 3:54 AM Masahiko Sawada wrote: > > AFAICS, we could do that by: > > > > 1. De-supporting platforms that have this problem, or > > 2. Introducing new typalign values, as Noah proposed back on April 2, or > > 3. Somehow forcing values that are sometimes 4-byte aligned and >

Re: Skipping logical replication transactions on subscriber side

2022-06-14 Thread Masahiko Sawada
On Mon, Jun 13, 2022 at 11:25 PM Robert Haas wrote: > > On Sun, Apr 17, 2022 at 11:22 PM Noah Misch wrote: > > > Yes, but it could be false positives in some cases. For instance, the > > > column {oid, bool, XLogRecPtr} should be okay on ALIGNOF_DOUBLE == 4 > > > and 8 platforms but the new test

Re: Skipping logical replication transactions on subscriber side

2022-06-13 Thread Robert Haas
On Sun, Apr 17, 2022 at 11:22 PM Noah Misch wrote: > > Yes, but it could be false positives in some cases. For instance, the > > column {oid, bool, XLogRecPtr} should be okay on ALIGNOF_DOUBLE == 4 > > and 8 platforms but the new test fails. > > I'm happy with that, because the affected author

Re: Skipping logical replication transactions on subscriber side

2022-04-17 Thread Masahiko Sawada
On Mon, Apr 18, 2022 at 12:22 PM Noah Misch wrote: > > On Mon, Apr 18, 2022 at 10:45:50AM +0900, Masahiko Sawada wrote: > > On Fri, Apr 15, 2022 at 4:26 PM Noah Misch wrote: > > > On Thu, Apr 07, 2022 at 08:39:58PM +0900, Masahiko Sawada wrote: > > > > On Thu, Apr 7, 2022 at 7:28 PM Amit Kapila

Re: Skipping logical replication transactions on subscriber side

2022-04-17 Thread Noah Misch
On Mon, Apr 18, 2022 at 10:45:50AM +0900, Masahiko Sawada wrote: > On Fri, Apr 15, 2022 at 4:26 PM Noah Misch wrote: > > On Thu, Apr 07, 2022 at 08:39:58PM +0900, Masahiko Sawada wrote: > > > On Thu, Apr 7, 2022 at 7:28 PM Amit Kapila > > > wrote: > > > > On Thu, Apr 7, 2022 at 8:25 AM Amit

Re: Skipping logical replication transactions on subscriber side

2022-04-17 Thread Masahiko Sawada
On Fri, Apr 15, 2022 at 4:26 PM Noah Misch wrote: > > On Thu, Apr 07, 2022 at 08:39:58PM +0900, Masahiko Sawada wrote: > > On Thu, Apr 7, 2022 at 7:28 PM Amit Kapila wrote: > > > On Thu, Apr 7, 2022 at 8:25 AM Amit Kapila > > > wrote: > > > > I'll take care of this today. I think we can mark

Re: Skipping logical replication transactions on subscriber side

2022-04-15 Thread Noah Misch
On Thu, Apr 07, 2022 at 08:39:58PM +0900, Masahiko Sawada wrote: > On Thu, Apr 7, 2022 at 7:28 PM Amit Kapila wrote: > > On Thu, Apr 7, 2022 at 8:25 AM Amit Kapila wrote: > > > I'll take care of this today. I think we can mark the new function > > > get_column_offset() being introduced by this

Re: Skipping logical replication transactions on subscriber side

2022-04-07 Thread Masahiko Sawada
On Thu, Apr 7, 2022 at 7:28 PM Amit Kapila wrote: > > On Thu, Apr 7, 2022 at 8:25 AM Amit Kapila wrote: > > > > I'll take care of this today. I think we can mark the new function > > get_column_offset() being introduced by this patch as parallel safe. > > > > Pushed. Thanks! Regards, --

Re: Skipping logical replication transactions on subscriber side

2022-04-07 Thread Amit Kapila
On Thu, Apr 7, 2022 at 8:25 AM Amit Kapila wrote: > > I'll take care of this today. I think we can mark the new function > get_column_offset() being introduced by this patch as parallel safe. > Pushed. -- With Regards, Amit Kapila.

Re: Skipping logical replication transactions on subscriber side

2022-04-06 Thread Amit Kapila
On Wed, Apr 6, 2022 at 10:01 AM Amit Kapila wrote: > > On Wed, Apr 6, 2022 at 9:25 AM Masahiko Sawada wrote: > > > > On Wed, Apr 6, 2022 at 12:21 PM Noah Misch wrote: > > > > Right. I've attached an updated patch. > > > > Thanks, this looks good to me as well. Noah, would you like to commit it?

Re: Skipping logical replication transactions on subscriber side

2022-04-06 Thread Peter Eisentraut
On 02.04.22 10:13, Noah Misch wrote: uint64 and pg_lsn use TYPALIGN_DOUBLE. For AIX, they really need a typalign corresponding to ALIGNOF_LONG. Hence, the C struct layout doesn't match the tuple layout. Columns potentially affected: [local] test=*# select attrelid::regclass, attname from

Re: Skipping logical replication transactions on subscriber side

2022-04-05 Thread Amit Kapila
On Wed, Apr 6, 2022 at 9:25 AM Masahiko Sawada wrote: > > On Wed, Apr 6, 2022 at 12:21 PM Noah Misch wrote: > > Right. I've attached an updated patch. > Thanks, this looks good to me as well. Noah, would you like to commit it? -- With Regards, Amit Kapila.

Re: Skipping logical replication transactions on subscriber side

2022-04-05 Thread Masahiko Sawada
On Wed, Apr 6, 2022 at 12:21 PM Noah Misch wrote: > > On Tue, Apr 05, 2022 at 04:41:28PM +0900, Masahiko Sawada wrote: > > On Tue, Apr 5, 2022 at 4:08 PM Noah Misch wrote: > > > On Tue, Apr 05, 2022 at 03:05:10PM +0900, Masahiko Sawada wrote: > > > > I've attached an updated patch. The patch

Re: Skipping logical replication transactions on subscriber side

2022-04-05 Thread Noah Misch
On Tue, Apr 05, 2022 at 04:41:28PM +0900, Masahiko Sawada wrote: > On Tue, Apr 5, 2022 at 4:08 PM Noah Misch wrote: > > On Tue, Apr 05, 2022 at 03:05:10PM +0900, Masahiko Sawada wrote: > > > I've attached an updated patch. The patch includes a regression test > > > to detect the new violation as

Re: Skipping logical replication transactions on subscriber side

2022-04-05 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 4:08 PM Noah Misch wrote: > > On Tue, Apr 05, 2022 at 03:05:10PM +0900, Masahiko Sawada wrote: > > I've attached an updated patch. The patch includes a regression test > > to detect the new violation as we discussed. I've confirmed that > > Cirrus CI tests pass. Please

Re: Skipping logical replication transactions on subscriber side

2022-04-05 Thread Noah Misch
On Tue, Apr 05, 2022 at 03:05:10PM +0900, Masahiko Sawada wrote: > I've attached an updated patch. The patch includes a regression test > to detect the new violation as we discussed. I've confirmed that > Cirrus CI tests pass. Please confirm on AIX and review the patch. When the context of a "git

Re: Skipping logical replication transactions on subscriber side

2022-04-05 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 12:38 PM Masahiko Sawada wrote: > > On Tue, Apr 5, 2022 at 10:46 AM Noah Misch wrote: > > > > On Tue, Apr 05, 2022 at 10:13:06AM +0900, Masahiko Sawada wrote: > > > On Tue, Apr 5, 2022 at 9:21 AM Noah Misch wrote: > > > > On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 10:46 AM Noah Misch wrote: > > On Tue, Apr 05, 2022 at 10:13:06AM +0900, Masahiko Sawada wrote: > > On Tue, Apr 5, 2022 at 9:21 AM Noah Misch wrote: > > > On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > > > > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Noah Misch
On Tue, Apr 05, 2022 at 10:13:06AM +0900, Masahiko Sawada wrote: > On Tue, Apr 5, 2022 at 9:21 AM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > > > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 9:21 AM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > > > > How about a comment like: "It has to be kept at

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Noah Misch
On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > > > How about a comment like: "It has to be kept at 8-byte alignment > > > boundary so as to be accessed directly

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Masahiko Sawada
On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > > On Mon, Apr 4, 2022 at 8:01 AM Noah Misch wrote: > > > On Mon, Apr 04, 2022 at 10:28:30AM +0900, Masahiko Sawada wrote: > > > > On Sun, Apr 3, 2022 at 9:45 AM Noah Misch wrote:

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Noah Misch
On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > On Mon, Apr 4, 2022 at 8:01 AM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 10:28:30AM +0900, Masahiko Sawada wrote: > > > On Sun, Apr 3, 2022 at 9:45 AM Noah Misch wrote: > > > > On Sat, Apr 02, 2022 at 08:44:45PM +0900, Masahiko

Re: Skipping logical replication transactions on subscriber side

2022-04-03 Thread Amit Kapila
On Mon, Apr 4, 2022 at 8:41 AM Masahiko Sawada wrote: > > On Mon, Apr 4, 2022 at 11:50 AM Amit Kapila wrote: > > > > Another minor point is that I think it is better to use DatumGetLSN to > > read this in GetSubscription as we use LSNGetDatum while storing it. I > > am not sure if there is any

Re: Skipping logical replication transactions on subscriber side

2022-04-03 Thread Masahiko Sawada
On Mon, Apr 4, 2022 at 11:50 AM Amit Kapila wrote: > > On Mon, Apr 4, 2022 at 8:01 AM Noah Misch wrote: > > > > On Mon, Apr 04, 2022 at 10:28:30AM +0900, Masahiko Sawada wrote: > > > On Sun, Apr 3, 2022 at 9:45 AM Noah Misch wrote: > > > > On Sat, Apr 02, 2022 at 08:44:45PM +0900, Masahiko

Re: Skipping logical replication transactions on subscriber side

2022-04-03 Thread Amit Kapila
On Mon, Apr 4, 2022 at 8:01 AM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 10:28:30AM +0900, Masahiko Sawada wrote: > > On Sun, Apr 3, 2022 at 9:45 AM Noah Misch wrote: > > > On Sat, Apr 02, 2022 at 08:44:45PM +0900, Masahiko Sawada wrote: > > > > On Sat, Apr 2, 2022 at 7:04 PM Amit Kapila >

Re: Skipping logical replication transactions on subscriber side

2022-04-03 Thread Noah Misch
On Mon, Apr 04, 2022 at 10:28:30AM +0900, Masahiko Sawada wrote: > On Sun, Apr 3, 2022 at 9:45 AM Noah Misch wrote: > > On Sat, Apr 02, 2022 at 08:44:45PM +0900, Masahiko Sawada wrote: > > > On Sat, Apr 2, 2022 at 7:04 PM Amit Kapila > > > wrote: > > > > On Sat, Apr 2, 2022 at 1:43 PM Noah

Re: Skipping logical replication transactions on subscriber side

2022-04-03 Thread Masahiko Sawada
On Sun, Apr 3, 2022 at 9:45 AM Noah Misch wrote: > > On Sat, Apr 02, 2022 at 08:44:45PM +0900, Masahiko Sawada wrote: > > On Sat, Apr 2, 2022 at 7:04 PM Amit Kapila wrote: > > > On Sat, Apr 2, 2022 at 1:43 PM Noah Misch wrote: > > > > Some options: > > > > - Move subskiplsn after subdbid, so

Re: Skipping logical replication transactions on subscriber side

2022-04-02 Thread Noah Misch
On Sat, Apr 02, 2022 at 08:44:45PM +0900, Masahiko Sawada wrote: > On Sat, Apr 2, 2022 at 7:04 PM Amit Kapila wrote: > > On Sat, Apr 2, 2022 at 1:43 PM Noah Misch wrote: > > > Some options: > > > - Move subskiplsn after subdbid, so it's always aligned anyway. I've > > > confirmed that this

Re: Skipping logical replication transactions on subscriber side

2022-04-02 Thread Masahiko Sawada
On Sat, Apr 2, 2022 at 7:04 PM Amit Kapila wrote: > > On Sat, Apr 2, 2022 at 1:43 PM Noah Misch wrote: > > > > On Sat, Apr 02, 2022 at 04:33:44PM +0900, Masahiko Sawada wrote: > > > It seems that 0/B0706F72 is not a random value. Two subscriber logs > > > show the same value. Since 0x70 = 'p',

Re: Skipping logical replication transactions on subscriber side

2022-04-02 Thread Amit Kapila
On Sat, Apr 2, 2022 at 1:43 PM Noah Misch wrote: > > On Sat, Apr 02, 2022 at 04:33:44PM +0900, Masahiko Sawada wrote: > > It seems that 0/B0706F72 is not a random value. Two subscriber logs > > show the same value. Since 0x70 = 'p', 0x6F = 'o', and 0x72 = 'r', it > > might show the next field in

Re: Skipping logical replication transactions on subscriber side

2022-04-02 Thread Noah Misch
On Sat, Apr 02, 2022 at 04:33:44PM +0900, Masahiko Sawada wrote: > It seems that 0/B0706F72 is not a random value. Two subscriber logs > show the same value. Since 0x70 = 'p', 0x6F = 'o', and 0x72 = 'r', it > might show the next field in the pg_subscription catalog, i.e., > subconninfo. The

Re: Skipping logical replication transactions on subscriber side

2022-04-02 Thread Masahiko Sawada
On Sat, Apr 2, 2022 at 1:08 PM Amit Kapila wrote: > > On Sat, Apr 2, 2022 at 7:29 AM Noah Misch wrote: > > > > On Sat, Apr 02, 2022 at 06:49:20AM +0530, Amit Kapila wrote: > > > > After applying datum_to_lsn_skiplsn_1.patch, I get another failure. Logs > > attached. > > > > The failure is for

Re: Skipping logical replication transactions on subscriber side

2022-04-01 Thread Amit Kapila
On Sat, Apr 2, 2022 at 7:29 AM Noah Misch wrote: > > On Sat, Apr 02, 2022 at 06:49:20AM +0530, Amit Kapila wrote: > > After applying datum_to_lsn_skiplsn_1.patch, I get another failure. Logs > attached. > The failure is for the same reason. I noticed that even when skip lsn value should be 0/0,

Re: Skipping logical replication transactions on subscriber side

2022-04-01 Thread Noah Misch
On Sat, Apr 02, 2022 at 06:49:20AM +0530, Amit Kapila wrote: > On Sat, Apr 2, 2022 at 5:41 AM Noah Misch wrote: > > > > On Fri, Apr 01, 2022 at 09:25:52PM +0900, Masahiko Sawada wrote: > > > > On Fri, Apr 1, 2022 at 4:44 PM Noah Misch wrote: > > > > > src/test/subscription/t/029_on_error.pl has

Re: Skipping logical replication transactions on subscriber side

2022-04-01 Thread Amit Kapila
On Sat, Apr 2, 2022 at 5:41 AM Noah Misch wrote: > > On Fri, Apr 01, 2022 at 09:25:52PM +0900, Masahiko Sawada wrote: > > > On Fri, Apr 1, 2022 at 4:44 PM Noah Misch wrote: > > > > src/test/subscription/t/029_on_error.pl has been failing reliably on > > > > the five > > > > AIX buildfarm

Re: Skipping logical replication transactions on subscriber side

2022-04-01 Thread Masahiko Sawada
On Fri, Apr 1, 2022 at 5:10 PM Masahiko Sawada wrote: > > On Fri, Apr 1, 2022 at 4:44 PM Noah Misch wrote: > > > > On Tue, Mar 29, 2022 at 10:43:00AM +0530, Amit Kapila wrote: > > > On Mon, Mar 21, 2022 at 5:51 PM Euler Taveira wrote: > > > > On Mon, Mar 21, 2022, at 12:25 AM, Amit Kapila

Re: Skipping logical replication transactions on subscriber side

2022-04-01 Thread Masahiko Sawada
On Fri, Apr 1, 2022 at 4:44 PM Noah Misch wrote: > > On Tue, Mar 29, 2022 at 10:43:00AM +0530, Amit Kapila wrote: > > On Mon, Mar 21, 2022 at 5:51 PM Euler Taveira wrote: > > > On Mon, Mar 21, 2022, at 12:25 AM, Amit Kapila wrote: > > > I have fixed all the above comments as per your suggestion

Re: Skipping logical replication transactions on subscriber side

2022-04-01 Thread Noah Misch
On Tue, Mar 29, 2022 at 10:43:00AM +0530, Amit Kapila wrote: > On Mon, Mar 21, 2022 at 5:51 PM Euler Taveira wrote: > > On Mon, Mar 21, 2022, at 12:25 AM, Amit Kapila wrote: > > I have fixed all the above comments as per your suggestion in the > > attached. Do let me know if something is missed?

Re: Skipping logical replication transactions on subscriber side

2022-03-28 Thread Amit Kapila
On Mon, Mar 21, 2022 at 5:51 PM Euler Taveira wrote: > > On Mon, Mar 21, 2022, at 12:25 AM, Amit Kapila wrote: > > I have fixed all the above comments as per your suggestion in the > attached. Do let me know if something is missed? > > Looks good to me. > This patch is committed

Re: Skipping logical replication transactions on subscriber side

2022-03-21 Thread Euler Taveira
On Mon, Mar 21, 2022, at 12:25 AM, Amit Kapila wrote: > I have fixed all the above comments as per your suggestion in the > attached. Do let me know if something is missed? Looks good to me. > > src/test/subscription/t/029_disable_on_error.pl | 94 -- > >

Re: Skipping logical replication transactions on subscriber side

2022-03-20 Thread Amit Kapila
On Mon, Mar 21, 2022 at 7:09 AM Euler Taveira wrote: > > On Thu, Mar 17, 2022, at 3:03 AM, Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. > > > > The patch LGTM. I have made minor changes in comments and docs in the >

Re: Skipping logical replication transactions on subscriber side

2022-03-20 Thread Amit Kapila
On Mon, Mar 21, 2022 at 7:09 AM Euler Taveira wrote: > > src/test/subscription/t/029_disable_on_error.pl | 94 -- > src/test/subscription/t/029_on_error.pl | 183 +++ > > It seems you are removing a test for 705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33. > We have

Re: Skipping logical replication transactions on subscriber side

2022-03-20 Thread Euler Taveira
On Thu, Mar 17, 2022, at 3:03 AM, Amit Kapila wrote: > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. > > > > The patch LGTM. I have made minor changes in comments and docs in the > attached patch. Kindly let me know what you think of the

Re: Skipping logical replication transactions on subscriber side

2022-03-17 Thread Masahiko Sawada
On Thu, Mar 17, 2022 at 3:03 PM Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. > > > > The patch LGTM. I have made minor changes in comments and docs in the > attached patch. Kindly let me know what you think of the

RE: Skipping logical replication transactions on subscriber side

2022-03-17 Thread osumi.takami...@fujitsu.com
On Thursday, March 17, 2022 7:56 PM Masahiko Sawada wrote: On Thu, Mar 17, 2022 at 5:52 PM Amit Kapila > wrote: > > > > On Thu, Mar 17, 2022 at 12:39 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > On Thursday, March 17, 2022 3:04 PM Amit Kapila > wrote: > > > > On Wed, Mar 16, 2022 at

Re: Skipping logical replication transactions on subscriber side

2022-03-17 Thread Masahiko Sawada
On Thu, Mar 17, 2022 at 5:52 PM Amit Kapila wrote: > > On Thu, Mar 17, 2022 at 12:39 PM osumi.takami...@fujitsu.com > wrote: > > > > On Thursday, March 17, 2022 3:04 PM Amit Kapila > > wrote: > > > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada > > > wrote: > > > > > > > > I've attached an

Re: Skipping logical replication transactions on subscriber side

2022-03-17 Thread Amit Kapila
On Thu, Mar 17, 2022 at 12:39 PM osumi.takami...@fujitsu.com wrote: > > On Thursday, March 17, 2022 3:04 PM Amit Kapila > wrote: > > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada > > wrote: > > > > > > I've attached an updated version patch. > > > > > > > The patch LGTM. I have made minor

RE: Skipping logical replication transactions on subscriber side

2022-03-17 Thread osumi.takami...@fujitsu.com
On Thursday, March 17, 2022 3:04 PM Amit Kapila wrote: > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada > wrote: > > > > I've attached an updated version patch. > > > > The patch LGTM. I have made minor changes in comments and docs in the > attached patch. Kindly let me know what you think of

Re: Skipping logical replication transactions on subscriber side

2022-03-17 Thread Amit Kapila
On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada wrote: > > I've attached an updated version patch. > The patch LGTM. I have made minor changes in comments and docs in the attached patch. Kindly let me know what you think of the attached? I am planning to commit this early next week (on Monday)

Re: Skipping logical replication transactions on subscriber side

2022-03-16 Thread Amit Kapila
On Thu, Mar 17, 2022 at 8:13 AM shiy.f...@fujitsu.com wrote: > > On Wed, Mar 16, 2022 4:23 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. > > > > Thanks for updating the patch. Here are some comments for the v15 patch. > > 1. src/backend/replication/logical/worker.c >

RE: Skipping logical replication transactions on subscriber side

2022-03-16 Thread shiy.f...@fujitsu.com
On Wed, Mar 16, 2022 4:23 PM Masahiko Sawada wrote: > > I've attached an updated version patch. > Thanks for updating the patch. Here are some comments for the v15 patch. 1. src/backend/replication/logical/worker.c + * to skip applying the changes when starting to apply changes. The

Re: Skipping logical replication transactions on subscriber side

2022-03-16 Thread Masahiko Sawada
On Wed, Mar 16, 2022 at 1:20 PM Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 7:58 AM Amit Kapila wrote: > > > > On Wed, Mar 16, 2022 at 6:03 AM Masahiko Sawada > > wrote: > > > > > > On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Mar 15, 2022 at 11:43 AM

Re: Skipping logical replication transactions on subscriber side

2022-03-16 Thread Masahiko Sawada
On Wed, Mar 16, 2022 at 11:28 AM Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 6:03 AM Masahiko Sawada wrote: > > > > On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila wrote: > > > > > > On Tue, Mar 15, 2022 at 11:43 AM Masahiko Sawada > > > wrote: > > > > > > > > > > 6. > > > @@ -1583,7 +1649,8

RE: Skipping logical replication transactions on subscriber side

2022-03-16 Thread osumi.takami...@fujitsu.com
On Wednesday, March 16, 2022 3:37 PM I wrote: > On Wednesday, March 16, 2022 11:33 AM Amit Kapila > wrote: > > On Tue, Mar 15, 2022 at 7:30 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > On Tuesday, March 15, 2022 3:13 PM Masahiko Sawada > > wrote: > > > > I've attached an updated

RE: Skipping logical replication transactions on subscriber side

2022-03-16 Thread osumi.takami...@fujitsu.com
On Wednesday, March 16, 2022 11:33 AM Amit Kapila wrote: > On Tue, Mar 15, 2022 at 7:30 PM osumi.takami...@fujitsu.com > wrote: > > > > On Tuesday, March 15, 2022 3:13 PM Masahiko Sawada > wrote: > > > I've attached an updated version patch. > > > > A couple of minor comments on v14. > > > >

Re: Skipping logical replication transactions on subscriber side

2022-03-16 Thread Masahiko Sawada
On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila wrote: > > On Tue, Mar 15, 2022 at 11:43 AM Masahiko Sawada > wrote: > > > > I've attached an updated version patch. > > > > Review: > === Thank you for the comments. > 1. > +++ b/doc/src/sgml/logical-replication.sgml > @@ -366,15 +366,19 @@

Re: Skipping logical replication transactions on subscriber side

2022-03-15 Thread Amit Kapila
On Wed, Mar 16, 2022 at 7:58 AM Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 6:03 AM Masahiko Sawada wrote: > > > > On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila wrote: > > > > > > On Tue, Mar 15, 2022 at 11:43 AM Masahiko Sawada > > > wrote: > > > > > > > > > > 6. > > > @@ -1583,7 +1649,8 @@

Re: Skipping logical replication transactions on subscriber side

2022-03-15 Thread Amit Kapila
On Wed, Mar 16, 2022 at 7:58 AM Amit Kapila wrote: > > On Wed, Mar 16, 2022 at 6:03 AM Masahiko Sawada wrote: > > > > On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila wrote: > > > > > > On Tue, Mar 15, 2022 at 11:43 AM Masahiko Sawada > > > wrote: > > > > > > > > > > 6. > > > @@ -1583,7 +1649,8 @@

Re: Skipping logical replication transactions on subscriber side

2022-03-15 Thread Amit Kapila
On Tue, Mar 15, 2022 at 7:30 PM osumi.takami...@fujitsu.com wrote: > > On Tuesday, March 15, 2022 3:13 PM Masahiko Sawada > wrote: > > I've attached an updated version patch. > > A couple of minor comments on v14. > > (1) apply_handle_commit_internal > > > + if (is_skipping_changes()) > +

Re: Skipping logical replication transactions on subscriber side

2022-03-15 Thread Amit Kapila
On Wed, Mar 16, 2022 at 6:03 AM Masahiko Sawada wrote: > > On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila wrote: > > > > On Tue, Mar 15, 2022 at 11:43 AM Masahiko Sawada > > wrote: > > > > > > > 6. > > @@ -1583,7 +1649,8 @@ apply_handle_insert(StringInfo s) > > TupleTableSlot *remoteslot; > >

Re: Skipping logical replication transactions on subscriber side

2022-03-15 Thread Masahiko Sawada
On Tue, Mar 15, 2022 at 7:18 PM Amit Kapila wrote: > > On Tue, Mar 15, 2022 at 11:43 AM Masahiko Sawada > wrote: > > > > 6. > @@ -1583,7 +1649,8 @@ apply_handle_insert(StringInfo s) > TupleTableSlot *remoteslot; > MemoryContext oldctx; > > - if

RE: Skipping logical replication transactions on subscriber side

2022-03-15 Thread osumi.takami...@fujitsu.com
On Tuesday, March 15, 2022 3:13 PM Masahiko Sawada wrote: > I've attached an updated version patch. A couple of minor comments on v14. (1) apply_handle_commit_internal + if (is_skipping_changes()) + { + stop_skipping_changes(); + + /* +

Re: Skipping logical replication transactions on subscriber side

2022-03-15 Thread Amit Kapila
On Tue, Mar 15, 2022 at 11:43 AM Masahiko Sawada wrote: > > I've attached an updated version patch. > Review: === 1. +++ b/doc/src/sgml/logical-replication.sgml @@ -366,15 +366,19 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER transaction, the

Re: Skipping logical replication transactions on subscriber side

2022-03-15 Thread Masahiko Sawada
Hi, On Fri, Mar 11, 2022 at 8:37 PM osumi.takami...@fujitsu.com wrote: > > On Friday, March 11, 2022 5:20 PM Masahiko Sawada > wrote: > > I've attached an updated version patch. This patch can be applied on top of > > the > > latest disable_on_error patch[1]. > Hi, thank you for the patch.

Re: Skipping logical replication transactions on subscriber side

2022-03-14 Thread Masahiko Sawada
On Mon, Mar 14, 2022 at 6:50 PM shiy.f...@fujitsu.com wrote: > > On Fri, Mar 11, 2022 4:20 PM Masahiko Sawada wrote: > > > > I've attached an updated version patch. This patch can be applied on > > top of the latest disable_on_error patch[1]. > > > > Thanks for your patch. Here are some comments

RE: Skipping logical replication transactions on subscriber side

2022-03-14 Thread osumi.takami...@fujitsu.com
On Friday, March 11, 2022 5:20 PM Masahiko Sawada wrote: > I've attached an updated version patch. This patch can be applied on top of > the > latest disable_on_error patch[1]. Hi, few extra comments on v13. (1) src/backend/replication/logical/worker.c With regard to

RE: Skipping logical replication transactions on subscriber side

2022-03-14 Thread shiy.f...@fujitsu.com
On Fri, Mar 11, 2022 4:20 PM Masahiko Sawada wrote: > > I've attached an updated version patch. This patch can be applied on > top of the latest disable_on_error patch[1]. > Thanks for your patch. Here are some comments for the v13 patch. 1. doc/src/sgml/ref/alter_subscription.sgml +

RE: Skipping logical replication transactions on subscriber side

2022-03-11 Thread osumi.takami...@fujitsu.com
On Friday, March 11, 2022 5:20 PM Masahiko Sawada wrote: > I've attached an updated version patch. This patch can be applied on top of > the > latest disable_on_error patch[1]. Hi, thank you for the patch. I'll share my review comments on v13. (a) src/backend/commands/subscriptioncmds.c @@

Re: Skipping logical replication transactions on subscriber side

2022-03-11 Thread Masahiko Sawada
On Thu, Mar 10, 2022 at 9:02 PM Amit Kapila wrote: > > On Tue, Mar 1, 2022 at 8:31 PM Masahiko Sawada wrote: > > > > I've attached an updated patch along with two patches for cfbot tests > > since the main patch (0003) depends on the other two patches. Both > > 0001 and 0002 patches are the same

Re: Skipping logical replication transactions on subscriber side

2022-03-10 Thread Masahiko Sawada
On Thu, Mar 10, 2022 at 2:10 PM osumi.takami...@fujitsu.com wrote: > > On Wednesday, March 2, 2022 12:01 AM Masahiko Sawada > wrote: > > I've attached an updated patch along with two patches for cfbot tests since > > the > > main patch (0003) depends on the other two patches. Both > > 0001 and

Re: Skipping logical replication transactions on subscriber side

2022-03-10 Thread Amit Kapila
On Tue, Mar 1, 2022 at 8:31 PM Masahiko Sawada wrote: > > I've attached an updated patch along with two patches for cfbot tests > since the main patch (0003) depends on the other two patches. Both > 0001 and 0002 patches are the same ones I attached on another > thread[2]. > Few comments on

RE: Skipping logical replication transactions on subscriber side

2022-03-09 Thread osumi.takami...@fujitsu.com
On Wednesday, March 2, 2022 12:01 AM Masahiko Sawada wrote: > I've attached an updated patch along with two patches for cfbot tests since > the > main patch (0003) depends on the other two patches. Both > 0001 and 0002 patches are the same ones I attached on another thread[2]. Hi, few comments

Re: Skipping logical replication transactions on subscriber side

2022-03-03 Thread Amit Kapila
On Tue, Mar 1, 2022 at 8:31 PM Masahiko Sawada wrote: > > I’ve considered a plan for the skipping logical replication > transaction feature toward PG15. Several ideas and patches have been > proposed here and another related thread[1][2] for the skipping > logical replication transaction feature

Re: Skipping logical replication transactions on subscriber side

2022-03-01 Thread Masahiko Sawada
On Tue, Feb 15, 2022 at 7:35 PM Peter Eisentraut wrote: > > On 14.02.22 10:16, Amit Kapila wrote: > > I think exposing LSN is a better approach as it doesn't have the > > dangers of wraparound. And, I think users can use it with the existing > > function pg_replication_origin_advance() which will

Re: Skipping logical replication transactions on subscriber side

2022-02-15 Thread Peter Eisentraut
On 14.02.22 10:16, Amit Kapila wrote: I think exposing LSN is a better approach as it doesn't have the dangers of wraparound. And, I think users can use it with the existing function pg_replication_origin_advance() which will save us from adding additional code for this feature. We can

Re: Skipping logical replication transactions on subscriber side

2022-02-14 Thread Amit Kapila
On Fri, Feb 11, 2022 at 7:40 AM Masahiko Sawada wrote: > > On Thu, Jan 27, 2022 at 10:42 PM Peter Eisentraut > wrote: > > > > On 26.01.22 05:05, Masahiko Sawada wrote: > > >> I think it is okay to clear after the first successful application of > > >> any transaction. What I was not sure was

Re: Skipping logical replication transactions on subscriber side

2022-02-10 Thread Masahiko Sawada
On Thu, Jan 27, 2022 at 10:42 PM Peter Eisentraut wrote: > > On 26.01.22 05:05, Masahiko Sawada wrote: > >> I think it is okay to clear after the first successful application of > >> any transaction. What I was not sure was about the idea of giving > >> WARNING/ERROR if the first xact to be

Re: Skipping logical replication transactions on subscriber side

2022-01-27 Thread Peter Eisentraut
On 26.01.22 05:05, Masahiko Sawada wrote: I think it is okay to clear after the first successful application of any transaction. What I was not sure was about the idea of giving WARNING/ERROR if the first xact to be applied is not the same as skip_xid. Do you prefer not to do anything in this

Re: Skipping logical replication transactions on subscriber side

2022-01-26 Thread Masahiko Sawada
On Wed, Jan 26, 2022 at 8:02 PM Amit Kapila wrote: > > On Wed, Jan 26, 2022 at 12:51 PM Masahiko Sawada > wrote: > > > > On Wed, Jan 26, 2022 at 1:43 PM David G. Johnston > > wrote: > > > > > > We probably should just provide an option for the user to specify > > > "subrelid". If null, only

Re: Skipping logical replication transactions on subscriber side

2022-01-26 Thread Amit Kapila
On Wed, Jan 26, 2022 at 12:51 PM Masahiko Sawada wrote: > > On Wed, Jan 26, 2022 at 1:43 PM David G. Johnston > wrote: > > > > We probably should just provide an option for the user to specify > > "subrelid". If null, only the main apply worker will skip the given xid, > > otherwise only the

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread Masahiko Sawada
On Wed, Jan 26, 2022 at 1:43 PM David G. Johnston wrote: > > On Tue, Jan 25, 2022 at 9:16 PM Amit Kapila wrote: >> >> On Wed, Jan 26, 2022 at 9:36 AM Masahiko Sawada >> wrote: >> > On Wed, Jan 26, 2022 at 12:54 PM Amit Kapila >> > wrote: >> >> > > > >> > > > Probably, we also need to

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread David G. Johnston
On Tue, Jan 25, 2022 at 9:16 PM Amit Kapila wrote: > On Wed, Jan 26, 2022 at 9:36 AM Masahiko Sawada > wrote: > > On Wed, Jan 26, 2022 at 12:54 PM Amit Kapila > wrote: > > > > > > > > > Probably, we also need to consider the case where the tablesync > worker > > > > entered an error loop and

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread Amit Kapila
On Wed, Jan 26, 2022 at 9:36 AM Masahiko Sawada wrote: > > On Wed, Jan 26, 2022 at 12:54 PM Amit Kapila wrote: > > > > > > I think it is okay to clear after the first successful application of > > any transaction. What I was not sure was about the idea of giving > > WARNING/ERROR if the first

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread Masahiko Sawada
On Wed, Jan 26, 2022 at 7:05 AM David G. Johnston wrote: > > On Tue, Jan 25, 2022 at 8:33 AM Masahiko Sawada wrote: >> >> Given that we cannot use rely on the pg_stat_subscription_workers view >> for this purpose, we would need either a new sub-system that tracks >> each logical replication

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread Masahiko Sawada
On Wed, Jan 26, 2022 at 12:54 PM Amit Kapila wrote: > > On Wed, Jan 26, 2022 at 8:55 AM Masahiko Sawada wrote: > > > > On Wed, Jan 26, 2022 at 11:51 AM Masahiko Sawada > > wrote: > > > > > > On Wed, Jan 26, 2022 at 11:28 AM Amit Kapila > > > wrote: > > > > > > > > IIUC, the proposal is to

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread Amit Kapila
On Wed, Jan 26, 2022 at 8:55 AM Masahiko Sawada wrote: > > On Wed, Jan 26, 2022 at 11:51 AM Masahiko Sawada > wrote: > > > > On Wed, Jan 26, 2022 at 11:28 AM Amit Kapila > > wrote: > > > > > > IIUC, the proposal is to compare the skip_xid with the very > > > transaction the apply worker

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread Masahiko Sawada
On Wed, Jan 26, 2022 at 11:51 AM Masahiko Sawada wrote: > > On Wed, Jan 26, 2022 at 11:28 AM Amit Kapila wrote: > > > > On Tue, Jan 25, 2022 at 8:39 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Jan 25, 2022 at 11:58 PM David G. Johnston > > > wrote: > > > > > > > > On Tue, Jan 25, 2022

  1   2   3   4   5   6   >