Re: Replication slot stats misgivings

2021-03-31 Thread vignesh C
On Tue, Mar 30, 2021 at 11:00 AM Andres Freund wrote: > > Hi, > > On 2021-03-30 10:13:29 +0530, vignesh C wrote: > > On Tue, Mar 30, 2021 at 6:28 AM Andres Freund wrote: > > > Any chance you could write a tap test exercising a few of these cases? > &g

Re: extra semicolon in postgres_fdw test cases

2021-03-30 Thread vignesh C
On Tue, Mar 30, 2021 at 3:21 PM Suraj Kharage wrote: > > Hi, > > Noticed that an extra semicolon in a couple of test cases related to > postgres_fdw. Removed that in the attached patch. It can be backported till > v11 where we added those test cases. > Thanks for identifying this, the changes

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-29 Thread vignesh C
On Tue, Mar 30, 2021 at 5:34 AM Peter Smith wrote: > > Please find attached the latest patch set v68* > > Differences from v67* are: > > * Rebased to HEAD @ today. > > * v68 fixes an issue reported by Vignesh [1] where a scenario was > found which still was able to cause a generated GID clash.

Re: Replication slot stats misgivings

2021-03-29 Thread vignesh C
On Tue, Mar 30, 2021 at 6:28 AM Andres Freund wrote: > > Hi, > > On 2021-03-26 07:58:58 +0530, Amit Kapila wrote: > > On Fri, Mar 26, 2021 at 1:17 AM Andres Freund wrote: > > > I suggest we wait doing anything about this until we know if the shared > > > stats patch gets in or not (I'd give it

Re: [PATCH] Provide more information to filter_prepare

2021-03-29 Thread vignesh C
On Mon, Mar 29, 2021 at 5:40 PM Markus Wanner wrote: > > On 29.03.21 14:00, vignesh C wrote: > > Have you intentionally not > > written any tests as it will be difficult to predict the xid. I just > > wanted to confirm my understanding. > > Yeah, that's t

Re: [PATCH] Provide more information to filter_prepare

2021-03-29 Thread vignesh C
On Mon, Mar 29, 2021 at 4:46 PM Markus Wanner wrote: > > On 29.03.21 13:04, vignesh C wrote: > > The above content looks sufficient to me. > > Good, thanks. Based on that, I'm adding v7 of the patch. > Thanks for the updated patch. @@ -440,7 +441,8 @@ pg_decode_

Re: [PATCH] Provide more information to filter_prepare

2021-03-29 Thread vignesh C
On Mon, Mar 29, 2021 at 4:22 PM Markus Wanner wrote: > > On 29.03.21 12:18, vignesh C wrote: > > But in prepare_filter_cb callback, by stating "other systems ..." it is > > not very clear who will change the GID. Are we referring to > > publisher/subscriber deco

Re: [PATCH] Provide more information to filter_prepare

2021-03-29 Thread vignesh C
On Mon, Mar 29, 2021 at 3:30 PM Markus Wanner < markus.wan...@enterprisedb.com> wrote: > > On 29.03.21 11:53, Amit Kapila wrote: > > Okay, but just in the previous sentence ("However, reuse of the same > > gid for example by a downstream node using > > multiple subscriptions may lead to it not

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-29 Thread vignesh C
On Sun, Mar 21, 2021 at 1:07 PM Amit Kapila wrote: > > On Sat, Mar 20, 2021 at 10:09 AM Ajin Cherian wrote: > > > > On Sat, Mar 20, 2021 at 1:35 AM Amit Kapila wrote: > >> > >> On Fri, Mar 19, 2021 at 5:03 AM Ajin Cherian wrote: > >> > > >> > Missed the patch - 0001, resending. > >> > > >> >

Re: Logical Replication vs. 2PC

2021-03-18 Thread vignesh C
On Thu, Mar 18, 2021 at 3:16 PM Amit Kapila wrote: > > While reviewing/testing subscriber-side work for $SUBJECT [1], I > noticed a problem that seems to need a broader discussion, so started > this thread. We can get prepare for the same GID more than once for > the cases where we have defined

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-16 Thread vignesh C
On Tue, Mar 16, 2021 at 7:22 PM Amit Kapila wrote: > > On Tue, Mar 16, 2021 at 6:22 PM vignesh C wrote: > > > > On Mon, Mar 15, 2021 at 6:14 PM Ajin Cherian wrote: > > > > > > On Mon, Mar 15, 2021 at 2:04 PM Amit Kapila > > > wrote: > > &

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-16 Thread vignesh C
On Mon, Mar 15, 2021 at 6:14 PM Ajin Cherian wrote: > > On Mon, Mar 15, 2021 at 2:04 PM Amit Kapila wrote: >> >> I think something on these lines should be much >> easier than the spool-file implementation unless we see any problem >> with this idea. >> > > Here's a new patch-set that implements

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-16 Thread vignesh C
On Mon, Mar 15, 2021 at 6:14 PM Ajin Cherian wrote: > > On Mon, Mar 15, 2021 at 2:04 PM Amit Kapila wrote: >> >> I think something on these lines should be much >> easier than the spool-file implementation unless we see any problem >> with this idea. >> > > Here's a new patch-set that implements

Re: subscriptionCheck failures

2021-03-16 Thread vignesh C
On Tue, Mar 16, 2021 at 12:29 PM Amit Kapila wrote: > > On Tue, Mar 16, 2021 at 9:00 AM Amit Kapila wrote: > > > > On Mon, Mar 15, 2021 at 6:00 PM Thomas Munro wrote: > > > > > > Hi, > > > > > > This seems to be a new low frequency failure, I didn't see it mentioned > > > already: > > > > > >

Re: locking [user] catalog tables vs 2pc vs logical rep

2021-03-15 Thread vignesh C
On Tue, Feb 23, 2021 at 3:59 AM Andres Freund wrote: > > Hi, > > The 2pc decoding added in > > commit a271a1b50e9bec07e2ef3a05e38e7285113e4ce6 > Author: Amit Kapila > Date: 2021-01-04 08:34:50 +0530 > > Allow decoding at prepare time in ReorderBuffer. > > has a deadlock danger when used in

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-12 Thread vignesh C
On Fri, Mar 12, 2021 at 2:29 PM Peter Smith wrote: > > On Fri, Mar 12, 2021 at 4:07 PM vignesh C wrote: > > Hi Vignesh, > > Thanks for the review comments. > > But can you please resend it with each feedback enumerated as 1. 2. > 3., or have some other clear

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-11 Thread vignesh C
On Thu, Mar 11, 2021 at 7:20 AM Peter Smith wrote: > > On Thu, Mar 11, 2021 at 12:46 PM Peter Smith wrote: > > > > Please find attached the latest patch set v57* > > > > Differences from v56* are: > > > > * Rebased to HEAD @ today > > > > * Addresses the following feedback issues: > > > > (24)

Do we support upgrade of logical replication?

2021-03-10 Thread vignesh C
Hi, I was reviewing logical decoding of two-phase transactions feature, while reviewing the feature I was checking if there is any impact on publisher/subscriber upgrade. I checked the existing pg_upgrade behaviour with logical replication. I made a logical replication data instance with

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-09 Thread vignesh C
On Tue, Mar 9, 2021 at 10:46 AM Peter Smith wrote: > > Please find attached the latest patch set v54* > > Differences from v53* are: > > * Rebased to HEAD @ today > > * Addresses some recent feedback issues for patch 0001 > > Feedback from Amit @ 7/March [ak] > - (36) Fixed. Comment about the psf

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-08 Thread vignesh C
On Tue, Mar 9, 2021 at 11:01 AM Amit Kapila wrote: > > On Mon, Mar 8, 2021 at 8:09 PM vignesh C wrote: > > > > On Mon, Mar 8, 2021 at 6:25 PM Amit Kapila wrote: > > > > > > > I think in case of two_phase option, replicatedPtr and sentPtr never > &g

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-08 Thread vignesh C
On Tue, Mar 9, 2021 at 9:14 AM Peter Smith wrote: > > On Mon, Mar 8, 2021 at 4:58 PM vignesh C wrote: > > > > LOGICAL_REP_MSG_TYPE = 'Y', > > + LOGICAL_REP_MSG_BEGIN_PREPARE = 'b', > > + LOGICAL_REP_MSG_PREPARE = 'P', > > +

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-08 Thread vignesh C
On Mon, Mar 8, 2021 at 6:25 PM Amit Kapila wrote: > > On Mon, Mar 8, 2021 at 4:20 PM vignesh C wrote: > > > > On Mon, Mar 8, 2021 at 11:30 AM Ajin Cherian wrote: > > > > > > On Fri, Mar 5, 2021 at 9:25 PM vignesh C wrote: > > > > > > >

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-08 Thread vignesh C
On Mon, Mar 8, 2021 at 11:30 AM Ajin Cherian wrote: > > On Fri, Mar 5, 2021 at 9:25 PM vignesh C wrote: > > > Created new patch v53: Thanks for the updated patch. I had noticed one issue, publisher does not get stopped normally in the following case: # Publisher steps psql

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-07 Thread vignesh C
On Mon, Mar 8, 2021 at 7:17 AM Peter Smith wrote: > > Please find attached the latest patch set v52* > Few comments: +logicalrep_read_begin_prepare(StringInfo in, LogicalRepBeginPrepareData *begin_data) +{ + /* read fields */ + begin_data->final_lsn = pq_getmsgint64(in); + if

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-05 Thread vignesh C
On Fri, Mar 5, 2021 at 12:21 PM Ajin Cherian wrote: > > On Thu, Mar 4, 2021 at 9:53 PM Peter Smith wrote: > > > [05a] Now syncing the psf file at prepare time > > The patch v46-0008 does not handle spooling of streaming prepare if > the Subscription is configured for both two-phase and

Re: Libpq support to connect to standby server as priority

2021-03-02 Thread vignesh C
On Wed, Mar 3, 2021 at 7:37 AM Tom Lane wrote: > > Greg Nancarrow writes: > > I've marked this as "Ready for Committer". > > I've pushed this after whacking it around a fair amount. A lot of > that was cosmetic, but one thing that wasn't is that I got rid of the > proposed "which_primary_host"

Buildfarm failure in crake

2021-03-02 Thread vignesh C
Hi, I noticed there is buildfarm failure in crake, it fails with the following error: Mar 02 21:22:56 ./src/test/recovery/t/001_stream_rep.pl: Variable declared in conditional statement at line 88, column 2. Declare variables outside of the condition.

Re: repeated decoding of prepared transactions

2021-03-01 Thread vignesh C
On Tue, Mar 2, 2021 at 9:33 AM Amit Kapila wrote: > > On Tue, Mar 2, 2021 at 8:20 AM vignesh C wrote: > > > > > > I have a minor comment regarding the below: > > + > > + > > + two_phase bool > > + > > + > &g

Re: repeated decoding of prepared transactions

2021-03-01 Thread vignesh C
On Tue, Mar 2, 2021 at 6:37 AM Ajin Cherian wrote: > > On Mon, Mar 1, 2021 at 8:08 PM Amit Kapila wrote: > > > Few minor comments on 0002 patch > > = > > 1. > > ctx->streaming &= enable_streaming; > > - ctx->twophase &= enable_twophase; > > + > > } > > > > Spurious

Re: repeated decoding of prepared transactions

2021-02-28 Thread vignesh C
On Mon, Mar 1, 2021 at 7:23 AM Ajin Cherian wrote: > > On Sat, Feb 27, 2021 at 11:06 PM Amit Kapila wrote: > > > Few comments on 0002 patch: > > = > > 1. > > + > > + /* > > + * Disable two-phase here, it will be set in the core if it was > > + * enabled whole creating the

Re: repeated decoding of prepared transactions

2021-02-27 Thread vignesh C
On Sat, Feb 27, 2021 at 5:36 PM Amit Kapila wrote: > > On Sat, Feb 27, 2021 at 11:38 AM Amit Kapila wrote: > > > > On Fri, Feb 26, 2021 at 4:13 PM Ajin Cherian wrote: > > > > > > On Fri, Feb 26, 2021 at 7:47 PM Ajin Cherian wrote: > > > > > > > I've updated snapshot_was_exported_at_ member to

Re: repeated decoding of prepared transactions

2021-02-27 Thread vignesh C
On Sat, Feb 27, 2021 at 8:29 AM Amit Kapila wrote: > > On Fri, Feb 26, 2021 at 7:26 PM vignesh C wrote: > > > > On Fri, Feb 26, 2021 at 4:13 PM Ajin Cherian wrote: > > > > > > On Fri, Feb 26, 2021 at 7:47 PM Ajin Cherian wrote: > > > > >

Re: repeated decoding of prepared transactions

2021-02-26 Thread vignesh C
On Fri, Feb 26, 2021 at 4:13 PM Ajin Cherian wrote: > > On Fri, Feb 26, 2021 at 7:47 PM Ajin Cherian wrote: > > > I've updated snapshot_was_exported_at_ member to pg_replication_slots as > > well. > > Do have a look and let me know if there are any comments. > > Update with both patches.

Re: repeated decoding of prepared transactions

2021-02-25 Thread vignesh C
On Wed, Feb 24, 2021 at 5:06 PM Ajin Cherian wrote: > > On Wed, Feb 24, 2021 at 4:48 PM Ajin Cherian wrote: > > > I plan to split this into two patches next. But do review and let me > > know if you have any comments. > > Attaching an updated patch-set with the changes for >

Re: logical replication seems broken

2021-02-15 Thread vignesh C
On Mon, Feb 15, 2021 at 6:14 PM wrote: > > > > On 2021.02.15. 12:31 Amit Kapila wrote: > > On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > > I compiled just now a binary from HEAD, an

Re: logical replication seems broken

2021-02-15 Thread vignesh C
On Mon, Feb 15, 2021 at 5:02 PM Amit Kapila wrote: > > On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > > > > > > > I compiled just now a binary from HEAD, and a binary from HEAD

Re: logical replication seems broken

2021-02-14 Thread vignesh C
On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > On 02/13/2021 11:49 AM Amit Kapila wrote: > > > > On Fri, Feb 12, 2021 at 10:00 PM wrote: > > > > > > > On 02/12/2021 1:51 PM Amit Kapila wrote: > > > > > > > > On Fri, Feb 12, 2021 at 6:04 PM Erik Rijkers wrote: > > > > > > > > > > I

Re: Libpq support to connect to standby server as priority

2021-02-11 Thread vignesh C
On Fri, Feb 12, 2021 at 7:07 AM Greg Nancarrow wrote: > > On Wed, Feb 10, 2021 at 5:09 PM vignesh C wrote: > > > > Modified. > > These comments are handled in v22 patch posted in my earlier mail. > > > > Thanks, just one minor thing I missed in doc/src/sgml/

Re: Libpq support to connect to standby server as priority

2021-02-09 Thread vignesh C
Thanks for the comments Greg, please find my comments inline below. On Tue, Feb 9, 2021 at 2:27 PM Greg Nancarrow wrote: > > On Mon, Feb 8, 2021 at 8:17 PM vignesh C wrote: > > > > > > > > I think what we want to do is mark default_transaction_read_onl

Re: Libpq support to connect to standby server as priority

2021-02-09 Thread vignesh C
On Tue, Feb 9, 2021 at 5:47 AM Greg Nancarrow wrote: > > On Mon, Feb 8, 2021 at 8:17 PM vignesh C wrote: > > > > > > > > I think what we want to do is mark default_transaction_read_only as > > > GUC_REPORT, instead. That will give a reliable report of what

Re: Libpq support to connect to standby server as priority

2021-02-08 Thread vignesh C
changes suggested and rebased it with the head code. Attached v21 patch which has the changes for the same. Thoughts? Regards, Vignesh From 9d85abfe1e4b43d67ee746891830abe53077c0e7 Mon Sep 17 00:00:00 2001 From: Vignesh C Date: Mon, 8 Feb 2021 11:23:31 +0530 Subject: [PATCH v21] Enhance the libpq "target_sessio

Re: Printing backtrace of postgres processes

2021-02-03 Thread vignesh C
On Wed, Feb 3, 2021 at 1:00 PM Tom Lane wrote: > > vignesh C writes: > > On Mon, Feb 1, 2021 at 11:04 AM Bharath Rupireddy > > wrote: > >> Are these superuser and permission checks enough from a security > >> standpoint that we don't expose some se

Re: Printing backtrace of postgres processes

2021-02-02 Thread vignesh C
On Mon, Feb 1, 2021 at 11:04 AM Bharath Rupireddy wrote: > > On Mon, Feb 1, 2021 at 6:14 AM Bharath Rupireddy > wrote: > > On Fri, Jan 29, 2021 at 7:10 PM vignesh C wrote: > > > > 4) How about following > > > > + errmsg(

Re: Printing backtrace of postgres processes

2021-02-02 Thread vignesh C
Thanks Bharath for your comments. On Mon, Feb 1, 2021 at 6:14 AM Bharath Rupireddy wrote: > > On Fri, Jan 29, 2021 at 7:10 PM vignesh C wrote: > > > 4) How about following > > > + errmsg("must be a superuser to print backtrace > > >

Re: Added schema level support for publication.

2021-01-31 Thread vignesh C
On Fri, Jan 22, 2021 at 10:01 AM vignesh C wrote: > > Thanks Rahila for your comments. Please find my thoughts below: > > On Wed, Jan 20, 2021 at 6:27 PM Rahila Syed wrote: > > > > Hi Vignesh, > > > >> > >> I have handled the above scenario(drop sc

Re: Printing backtrace of postgres processes

2021-01-29 Thread vignesh C
Thanks Bharath for your review comments. Please find my comments inline below. On Thu, Jan 28, 2021 at 7:40 PM Bharath Rupireddy wrote: > > On Thu, Jan 28, 2021 at 5:22 PM vignesh C wrote: > > Thanks for the comments, I have fixed and attached an updated patch > > with the

Re: Printing backtrace of postgres processes

2021-01-28 Thread vignesh C
On Wed, Jan 27, 2021 at 10:40 PM Andres Freund wrote: > > Hi, > > On 2021-01-27 19:05:16 +0530, vignesh C wrote: > > > /* > > + * LogBackTrace > > + * > > + * Get the backtrace and log the backtrace to log file. > > + */ > >

Re: Printing backtrace of postgres processes

2021-01-27 Thread vignesh C
tions. It includes the following fixes: 1) Removal of support to get callstack of all postgres process, user can get only one process callstack. 2) Update the documentation. 3) Added necessary checks for pg_print_callstack similar to pg_terminate_backend. 4) Changed LOG to LOG_SERVER_ONLY. Though

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread vignesh C
On Mon, Jan 25, 2021 at 3:07 PM Dilip Kumar wrote: > > On Mon, Jan 25, 2021 at 2:48 PM Bharath Rupireddy > wrote: > > > > On Mon, Jan 25, 2021 at 2:42 PM Dilip Kumar wrote: > > > > > > On Mon, Jan 25, 2021 at 1:10 PM vignesh C wrote: > > > &g

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread vignesh C
On Mon, Jan 25, 2021 at 5:18 PM japin wrote: > > > On Mon, 25 Jan 2021 at 17:18, Bharath Rupireddy > wrote: > > On Mon, Jan 25, 2021 at 2:42 PM Dilip Kumar wrote: > >> > >> On Mon, Jan 25, 2021 at 1:10 PM vignesh C wrote: > >> > > >

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-24 Thread vignesh C
On Thu, Jan 21, 2021 at 10:21 PM Bharath Rupireddy wrote: > > On Thu, Jan 21, 2021 at 6:56 PM vignesh C wrote: > > > > Hi, > > > > Creating/altering subscription is successful when we specify a > > publication which does not exist in the publisher.

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-22 Thread vignesh C
On Fri, Jan 22, 2021 at 10:14 AM japin wrote: > > > On Fri, 22 Jan 2021 at 00:51, Bharath Rupireddy > wrote: > > On Thu, Jan 21, 2021 at 6:56 PM vignesh C wrote: > >> > >> Hi, > >> > >> Creating/altering subscription is successful w

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-22 Thread vignesh C
On Fri, Jan 22, 2021 at 12:14 PM Bharath Rupireddy wrote: > > On Fri, Jan 22, 2021 at 10:14 AM japin wrote: > > > 2) Can't we know whether the publications exist on the publisher with > > > the existing (or modifying it a bit if required) query in > > > fetch_table_list(), so that we can avoid

Re: Added schema level support for publication.

2021-01-21 Thread vignesh C
Thanks Rahila for your comments. Please find my thoughts below: On Wed, Jan 20, 2021 at 6:27 PM Rahila Syed wrote: > > Hi Vignesh, > >> >> I have handled the above scenario(drop schema should automatically >> remove the schema entry from publication schema relation) & addition >> of tests in the

Identify missing publications from publisher while create/alter subscription.

2021-01-21 Thread vignesh C
From 2e7a6e41f789f7f1717058e9c78441ae8d5faf9e Mon Sep 17 00:00:00 2001 From: Vignesh C Date: Thu, 21 Jan 2021 18:38:54 +0530 Subject: [PATCH v1] Identify missing publications from publisher while create/alter subscription. Creating/altering subscription is successful when we specify a publication

Re: Printing backtrace of postgres processes

2021-01-20 Thread vignesh C
On Wed, Jan 20, 2021 at 2:52 AM Tom Lane wrote: > > Robert Haas writes: > > On Tue, Jan 19, 2021 at 12:50 PM Tom Lane wrote: > >> I think it's got security hazards as well. If we restricted the > >> feature to cause a trace of only one process at a time, and required > >> that process to be

Re: Printing backtrace of postgres processes

2021-01-19 Thread vignesh C
On Sun, Jan 17, 2021 at 10:26 PM vignesh C wrote: > > On Sat, Jan 16, 2021 at 11:10 PM Andres Freund wrote: > > > > Hi, > > > > On Sat, Jan 16, 2021, at 09:34, vignesh C wrote: > > > On Sat, Jan 16, 2021 at 1:40 AM Andres Freund wrote: > > &g

Re: Added schema level support for publication.

2021-01-19 Thread vignesh C
On Sat, Jan 9, 2021 at 8:08 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > > > As Amit pointed out earlier, the behaviour when schema dropped, I > think we should also consider when schema is altered, say altered to a > different name, maybe we should change that in the

Re: Added schema level support for publication.

2021-01-18 Thread vignesh C
On Sat, Jan 9, 2021 at 5:21 PM vignesh C wrote: > > On Fri, Jan 8, 2021 at 4:32 PM Amit Kapila wrote: > > > > On Thu, Jan 7, 2021 at 10:03 PM vignesh C wrote: > > > > > > This feature adds schema option while creating publication. Users will > > >

Re: Printing backtrace of postgres processes

2021-01-17 Thread vignesh C
On Sat, Jan 16, 2021 at 11:10 PM Andres Freund wrote: > > Hi, > > On Sat, Jan 16, 2021, at 09:34, vignesh C wrote: > > On Sat, Jan 16, 2021 at 1:40 AM Andres Freund wrote: > > > > > > On 2021-01-15 09:53:05 +0100, Peter Eisentraut wrote: > > > > O

Re: Printing backtrace of postgres processes

2021-01-16 Thread vignesh C
On Sat, Jan 16, 2021 at 1:40 AM Andres Freund wrote: > > On 2021-01-15 09:53:05 +0100, Peter Eisentraut wrote: > > On 2020-12-08 10:38, vignesh C wrote: > > > I have implemented printing of backtrace based on handling it in > > > CHECK_FOR_INTERRUPTS. This patch als

Re: Added schema level support for publication.

2021-01-15 Thread vignesh C
Thanks Rahila for your comments, please find my thoughts below. On Tue, Jan 12, 2021 at 5:16 PM Rahila Syed wrote: > > Hi Vignesh, > > I had a look at the patch, please consider following comments. > > On Thu, Jan 7, 2021 at 10:03 PM vignesh C wrote: >> >> Hi,

Re: Added schema level support for publication.

2021-01-12 Thread vignesh C
On Mon, Jan 11, 2021 at 11:45 AM Bharath Rupireddy wrote: > > On Sun, Jan 10, 2021 at 11:21 PM vignesh C wrote: > > On Sat, Jan 9, 2021 at 8:08 PM Bharath Rupireddy > > wrote: > > > I think this feature can be useful, in case a user has a lot of tables > > >

Re: Added schema level support for publication.

2021-01-10 Thread vignesh C
On Sat, Jan 9, 2021 at 8:14 PM Bharath Rupireddy wrote: > > One more point - if the publication is created for a schema with no or > some initial tables, will all the future tables that may get added to > the schema will be replicated too? > I agree on this, when a relation is added to the

Re: Added schema level support for publication.

2021-01-10 Thread vignesh C
Thanks for your comments Bharath, please find my opinion below. On Sat, Jan 9, 2021 at 8:08 PM Bharath Rupireddy wrote: > I think this feature can be useful, in case a user has a lot of tables > to publish inside a schema. Having said that, I wonder if this feature > mandates users to create the

Re: Added schema level support for publication.

2021-01-09 Thread vignesh C
On Fri, Jan 8, 2021 at 4:32 PM Amit Kapila wrote: > > On Thu, Jan 7, 2021 at 10:03 PM vignesh C wrote: > > > > This feature adds schema option while creating publication. Users will > > be able to specify one or more schemas while creating publication, > > when th

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-08 Thread vignesh C
On Fri, Dec 11, 2020 at 4:30 PM Greg Nancarrow wrote: > > Posting an updated set of patches to address recent feedback: > > - Removed conditional-locking code used in parallel-safety checking > code (Tsunakawa-san feedback). It turns out that for the problem test > case, no parallel-safety

Added schema level support for publication.

2021-01-07 Thread vignesh C
production_quarterly_publication DROP SCHEMA production_july; Attached is a POC patch for the same. I felt this feature would be quite useful. Thoughts? Regards, Vignesh EnterpriseDB: http://www.enterprisedb.com From f9b5134182229f718bbc1a9162b6043f879a6410 Mon Sep 17 00:00:00 2001 From: Vignesh C Date: Thu, 7 Jan 2021 11:38

Re: Parallel Inserts in CREATE TABLE AS

2021-01-04 Thread vignesh C
On Mon, Jan 4, 2021 at 3:07 PM Bharath Rupireddy wrote: > > On Wed, Dec 30, 2020 at 5:28 PM vignesh C wrote: > > Few comments: > > - /* > > -* To allow parallel inserts, we need to ensure that they are safe > > to be > > -* performed in

Re: Added missing copy related data structures to typedefs.list

2021-01-04 Thread vignesh C
On Wed, Dec 30, 2020 at 7:10 PM Amit Kapila wrote: > > On Sat, Dec 26, 2020 at 9:16 PM vignesh C wrote: > > > > On Thu, Dec 17, 2020 at 4:28 AM Bruce Momjian wrote: > > > > > > On Mon, Dec 7, 2020 at 01:56:50PM +0530, vignesh C wrote: > > > > Hi,

Re: Parallel Inserts in CREATE TABLE AS

2020-12-30 Thread vignesh C
On Wed, Dec 30, 2020 at 9:25 AM Bharath Rupireddy wrote: > > On Wed, Dec 30, 2020 at 5:22 AM Zhihong Yu wrote: > > w.r.t. v17-0004-Enable-CTAS-Parallel-Inserts-For-Append.patch > > > > + * Push the dest receiver to Gather node when it is either at the top of > > the > > + * plan or under top

Re: Parallel Inserts in CREATE TABLE AS

2020-12-30 Thread vignesh C
On Wed, Dec 30, 2020 at 10:47 AM Bharath Rupireddy wrote: > > On Wed, Dec 30, 2020 at 10:32 AM Dilip Kumar wrote: > > I have completed reviewing 0001, I don't have more comments, just one > > question. Soon I will review the remaining patches. > > Thanks. > > > +/* If parallel inserts are

Re: Parallel copy

2020-12-28 Thread vignesh C
Attached is a patch that was used for the same. The patch is written on top of the parallel copy patch. The design Amit, Andres & myself voted for that is the leader identifying the line bound design and sharing it in shared memory is performing better. Regards, Vignesh EnterpriseDB: http:

Re: Parallel Inserts in CREATE TABLE AS

2020-12-27 Thread vignesh C
On Sun, Dec 27, 2020 at 2:28 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > > On Sat, Dec 26, 2020 at 9:20 PM vignesh C wrote: > > +-- parallel inserts must occur > > +select explain_pictas( > > +'create table parallel_write as sele

Re: Parallel Inserts in CREATE TABLE AS

2020-12-26 Thread vignesh C
On Thu, Dec 24, 2020 at 1:07 PM Bharath Rupireddy wrote: > > On Thu, Dec 24, 2020 at 10:25 AM vignesh C wrote: > > You could change intoclause_len = strlen(intoclausestr) to > > strlen(intoclausestr) + 1 and use intoclause_len in the remaining > > places. We can avoid th

Re: Parallel copy

2020-12-26 Thread vignesh C
On Wed, Dec 23, 2020 at 3:05 PM Hou, Zhijie wrote: > > Hi > > > Yes this optimization can be done, I will handle this in the next patch > > set. > > > > I have a suggestion for the parallel safety-check. > > As designed, The leader does not participate in the insertion of data. > If User use

Re: Added missing copy related data structures to typedefs.list

2020-12-26 Thread vignesh C
On Thu, Dec 17, 2020 at 4:28 AM Bruce Momjian wrote: > > On Mon, Dec 7, 2020 at 01:56:50PM +0530, vignesh C wrote: > > Hi, > > > > Added missing copy related data structures to typedefs.list, these > > data structures were added while copy files were split during

Re: Parallel Inserts in CREATE TABLE AS

2020-12-24 Thread vignesh C
On Thu, Dec 24, 2020 at 11:29 AM Amit Kapila wrote: > > On Thu, Dec 24, 2020 at 10:25 AM vignesh C wrote: > > > > On Tue, Dec 22, 2020 at 2:16 PM Bharath Rupireddy > > wrote: > > > > > > On Tue, Dec 22, 2020 at 12:32 PM Bharath Rupireddy > >

Re: Parallel Inserts in CREATE TABLE AS

2020-12-23 Thread vignesh C
On Tue, Dec 22, 2020 at 2:16 PM Bharath Rupireddy wrote: > > On Tue, Dec 22, 2020 at 12:32 PM Bharath Rupireddy > Attaching v14 patch set that has above changes. Please consider this > for further review. > Few comments: In the below case, should create be above Gather? postgres=# explain

Re: Parallel INSERT (INTO ... SELECT ...)

2020-12-09 Thread vignesh C
On Wed, Dec 9, 2020 at 10:11 AM Greg Nancarrow wrote: > > On Wed, Dec 9, 2020 at 1:35 AM vignesh C wrote: > > > > Most of the code present in > > v9-0001-Enable-parallel-SELECT-for-INSERT-INTO-.-SELECT.patch is > > applicable for parallel copy patch also. The pa

Re: Parallel copy

2020-12-09 Thread vignesh C
On Mon, Dec 7, 2020 at 3:00 PM Hou, Zhijie wrote: > > > Attached v11 patch has the fix for this, it also includes the changes to > > rebase on top of head. > > Thanks for the explanation. > > I think there is still chances we can know the size. > > +* line_size will be set. Read

Re: Parallel INSERT (INTO ... SELECT ...)

2020-12-08 Thread vignesh C
On Mon, Dec 7, 2020 at 2:35 PM Greg Nancarrow wrote: > > On Fri, Nov 20, 2020 at 7:44 PM Greg Nancarrow wrote: > > > > Posting an updated set of patches, with some additional testing and > > documentation updates, and including the latest version of the > > Parallel Insert patch. > > Any

Re: Printing backtrace of postgres processes

2020-12-08 Thread vignesh C
On Tue, Dec 1, 2020 at 2:15 PM vignesh C wrote: > > On Tue, Dec 1, 2020 at 9:31 AM Tom Lane wrote: > > > > Andres Freund writes: > > > It should be quite doable to emit such backtraces directly to stderr, > > > instead of using appendStringInfoString()/elog(

Added missing copy related data structures to typedefs.list

2020-12-07 Thread vignesh C
EnterpriseDB: http://www.enterprisedb.com From d656c2f0bfbf68f5ceb98a0eb205e5e77f21602f Mon Sep 17 00:00:00 2001 From: Vignesh C Date: Mon, 7 Dec 2020 13:48:31 +0530 Subject: [PATCH] Added missing copy related data structures. Added missing copy related data structures to typedefs.list, these data

Re: Printing backtrace of postgres processes

2020-12-01 Thread vignesh C
On Tue, Dec 1, 2020 at 9:31 AM Tom Lane wrote: > > Andres Freund writes: > > It should be quite doable to emit such backtraces directly to stderr, > > instead of using appendStringInfoString()/elog(). > > No, please no. > > (1) On lots of logging setups (think syslog), anything that goes to >

Re: Printing backtrace of postgres processes

2020-11-29 Thread vignesh C
On Sun, Nov 22, 2020 at 11:55 AM Tom Lane wrote: > > vignesh C writes: > > The idea here is to implement & expose pg_print_callstack function, > > internally what this function does is, the connected backend will send > > SIGUSR1 signal by setting PMSIGNAL_BACK

Printing backtrace of postgres processes

2020-11-21 Thread vignesh C
the same. Thoughts? Regards, Vignesh EnterpriseDB: http://www.enterprisedb.com From c1006110bdeac2135d1c8e9220f65d50cd49ab63 Mon Sep 17 00:00:00 2001 From: Vignesh C Date: Sun, 22 Nov 2020 05:58:24 +0530 Subject: [PATCH] Print backtrace of postgres process that are part of this instance. The i

Re: Parallel copy

2020-11-18 Thread vignesh C
On Sat, Nov 7, 2020 at 7:01 PM vignesh C wrote: > > On Thu, Nov 5, 2020 at 6:33 PM Hou, Zhijie wrote: > > > > Hi > > > > > > > > my $bytes = $ARGV[0]; > > > for(my $i = 0; $i < $bytes; $i+=8){ > > > print "longdata&qu

Re: Parallel copy

2020-11-18 Thread vignesh C
On Sat, Oct 31, 2020 at 2:07 AM Tomas Vondra wrote: > > Hi, > > I've done a bit more testing today, and I think the parsing is busted in > some way. Consider this: > > test=# create extension random; > CREATE EXTENSION > > test=# create table t (a text); > CREATE TABLE > >

Re: Parallel copy

2020-11-18 Thread vignesh C
On Fri, Nov 13, 2020 at 2:25 PM Amit Kapila wrote: > > On Wed, Nov 11, 2020 at 10:42 PM vignesh C wrote: > > > > On Tue, Nov 10, 2020 at 7:27 PM Amit Kapila wrote: > > > > > > On Tue, Nov 10, 2020 at 7:12 PM vignesh C wrote: > > > > &g

Re: Parallel copy

2020-11-18 Thread vignesh C
On Thu, Oct 29, 2020 at 2:26 PM Daniel Westermann (DWE) wrote: > > On 27/10/2020 15:36, vignesh C wrote: > >> Attached v9 patches have the fixes for the above comments. > > >I did some testing: > > I did some testing as well and have a cosmetic remark: > > post

Re: Parallel copy

2020-11-18 Thread vignesh C
On Wed, Oct 28, 2020 at 5:36 PM Hou, Zhijie wrote: > > Hi > > I found some issue in v9-0002 > > 1. > + > + elog(DEBUG1, "[Worker] Processing - line position:%d, block:%d, unprocessed lines:%d, offset:%d, line size:%d", > +write_pos, lineInfo->first_block, > +

Re: Parallel copy

2020-11-18 Thread vignesh C
On Thu, Oct 29, 2020 at 2:20 PM Heikki Linnakangas wrote: > > On 27/10/2020 15:36, vignesh C wrote: > > Attached v9 patches have the fixes for the above comments. > > I did some testing: > > /tmp/longdata.pl: > > #!/usr/bin/perl > #

Re: Parallel copy

2020-11-11 Thread vignesh C
On Tue, Nov 10, 2020 at 7:27 PM Amit Kapila wrote: > > On Tue, Nov 10, 2020 at 7:12 PM vignesh C wrote: > > > > On Tue, Nov 3, 2020 at 2:28 PM Amit Kapila wrote: > > > > > > > I have worked to provide a patch for the parallel safety checks. It > &

Re: Parallel copy

2020-11-10 Thread vignesh C
llowed, for example need to identify whether table > or one of its partitions has any constraint/expression which is > parallel-unsafe. > I have worked to provide a patch for the parallel safety checks. It checks if parallely copy can be performed, Parallel copy cannot be performed for the fo

Re: Parallel copy

2020-11-07 Thread vignesh C
On Thu, Nov 5, 2020 at 6:33 PM Hou, Zhijie wrote: > > Hi > > > > > my $bytes = $ARGV[0]; > > for(my $i = 0; $i < $bytes; $i+=8){ > > print "longdata"; > > } > > print "\n"; > > > > > > postgres=# copy longdata from program 'perl /tmp/longdata.pl 1' > > with (parallel 2); > >

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-11-06 Thread vignesh C
> after > > feature freeze. > > > > Thanks. > > On Tue, Nov 3, 2020 at 12:49 PM vignesh C wrote: > > > > Thanks for the explanation, I have attached a v5 patch with the > > changes where the translation should not have any problem. > > > > I

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-11-02 Thread vignesh C
t follow the grammar rules. You will have "replicação > conexão autorizada" instead of "conexão de replicação autorizada". The former > isn't grammatically correct. Avoid splitting sentences that are translated. > Thanks for the explanation, I have attached a v5 pat

Re: Parallel INSERT (INTO ... SELECT ...)

2020-11-02 Thread vignesh C
> > See attached patches. > Thanks for providing the patches. I had reviewed v6-0001-Enable-parallel-SELECT-for-INSERT-INTO-.-SELECT.patch, please find my comments: -> commandType is not used, we can remove it. + * Prepare for entering parallel mode by assigning a FullTransactionId, to be + *

Re: Split copy.c

2020-11-02 Thread vignesh C
On Mon, Nov 2, 2020 at 2:33 PM Heikki Linnakangas wrote: > > While looking at the parallel copy patches, it started to annoy me how > large copy.c is. It confuses my little head. (Ok, it's annoyed me many > times in the past, but I haven't done anything about it.) > +1 for having copy from &

<    8   9   10   11   12   13   14   15   16   >