On Wed, Jul 3, 2024 at 5:06 PM Dilip Kumar wrote:
>
> On Wed, Jul 3, 2024 at 4:48 PM Amit Kapila wrote:
> >
> > On Wed, Jul 3, 2024 at 4:04 PM Dilip Kumar wrote:
> > >
> > >
> > > What I am saying is that if we are not applying the whole transa
ates that this call originated
* from AlterSubscription.
*/
if (!IsSet(supported_opts, SUBOPT_TWOPHASE_COMMIT))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized subscription parameter: \"%s\"", defel->defname)));
This part of the code must either be removed or converted to an assert.
3. The tests added in 099_twophase_added.pl should be part of 021_twophase.pl
--
With Regards,
Amit Kapila.
if the subscription is
+ * disabled.
+ */
+ logicalrep_workers_stop(subid);
In which case the workers will survive when the subscription is disabled?
--
With Regards,
Amit Kapila.
On Wed, Jul 3, 2024 at 4:04 PM Dilip Kumar wrote:
>
> On Wed, Jul 3, 2024 at 3:35 PM Amit Kapila wrote:
> >
> > On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar wrote:
> > >
> > > On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila
> > > wrote:
> > >
ent was added
> in 0002 to clarify why we must wait. For 0003, a comment was added and
> setting for standby was reverted.
>
Pushed 0002 and 0003. Let's wait for a discussion on 0001.
--
With Regards,
Amit Kapila.
On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar wrote:
>
> On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila wrote:
> >
> > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote:
>
> > But waiting after applying the operations and before applying the
> > commit would mean th
lel workers, we might not know the commit_ts of those
> transactions since they may not have been committed yet. One simple
> option could be to prevent parallel workers from applying in-progress
> transactions when CDR is set up. Instead, we could let these
> transactions spill to files and only apply them once we receive the
> commit record.
>
Agreed, we should do it as you have suggested and document it.
--
With Regards,
Amit Kapila.
p and node_s. I think node_p is
> enough, but did
> like that just in case. This fixes a second failure.
>
Disabling on the primary node should be sufficient. Let's do the
minimum required to stabilize this test.
--
With Regards,
Amit Kapila.
On Wed, Jul 3, 2024 at 11:27 AM Amit Kapila wrote:
>
> > Do you have any other idea?
> >
>
> The other idea could be that we use the minimum restart_lsn of all the
> slots created by this tool as a consistent_lsn. We can probably get
> that value by using pg_get_replic
hear from others.
> Do you have any other idea?
>
The other idea could be that we use the minimum restart_lsn of all the
slots created by this tool as a consistent_lsn. We can probably get
that value by using pg_get_replication_slots() but this idea needs
further evaluation as to whether it will lead to a consistent
subscriber.
--
With Regards,
Amit Kapila.
which could reduce much
> execution
> time on my environment.
>
This sounds like an ugly hack to me and don't know if we can use it.
The ideal way to fix this is to get the start_lsn from the
create_logical_slot functionality or have some parameter like
recover_target_end_lsn but I don't know if this is a good time to
extend such a functionality.
--
With Regards,
Amit Kapila.
New patch set did that.
> >
> > > You can create a dummy subscription on node_p
> > > and do a test similar to what we are doing in "# Create failover slot
> > > to test its removal".
> >
> > Your approach looks better than mine. I followed the approach.
>
> Hi Kuroda-san,
>
> I tested the patches on linux and windows and I confirm that it
> successfully fixes the issue [1].
>
Thanks for the verification. I have pushed the patch.
--
With Regards,
Amit Kapila.
On Mon, Jul 1, 2024 at 6:01 PM Zhijie Hou (Fujitsu)
wrote:
>
> Thanks! I am attaching another patch to modify the release note as discussed.
>
Pushed.
--
With Regards,
Amit Kapila.
ine. I followed the approach.
>
LGTM.
--
With Regards,
Amit Kapila.
On Thu, Jun 27, 2024 at 7:14 AM Masahiko Sawada wrote:
>
> On Wed, Jun 26, 2024 at 6:15 PM Zhijie Hou (Fujitsu)
> wrote:
>
> Thank you for updating the patch. The v2 patch looks good to me.
>
Pushed.
--
With Regards,
Amit Kapila.
CTION 'conninfo'
PUBLICATION publication_name [, ...] CONFLICT RESOLVER
'conflict_resolver' FOR 'conflict_type';
> And another question is: a
> table-level resolver setting is precedent over all subscriber-level
> resolver settings in the database?
>
Yes.
--
With Regards,
Amit Kapila.
ed.
> - discard DELETE, because the commit timestamp of UPDATE is newer
> than DELETE' one.ard DELETE, because the commit timestamp of UPDATE is
> newer than DELETE' one.
>
> As a result, both nodes have the new version row.
>
Right, it seems to me that we should implement 'latest_time_wins' if
we want consistency in such cases.
[1] -
https://www.postgresql.org/message-id/CAA4eK1Lj-PWrP789KnKxZydisHajd38rSihWXO8MVBLDwxG1Kg%40mail.gmail.com
--
With Regards,
Amit Kapila.
only heap scans. We normally get benefits of parallelism because of
using multiple CPUs but parallelizing scans (I/O) shouldn't give much
benefits. Is it possible that you are seeing benefits because most of
the data is either in shared_buffers or in memory? We can probably try
vacuuming tables by restarting the nodes to ensure the data is not in
memory.
--
With Regards,
Amit Kapila.
dd a new test to one of the existing tests in
040_pg_createsubscriber. You can create a dummy subscription on node_p
and do a test similar to what we are doing in "# Create failover slot
to test its removal".
--
With Regards,
Amit Kapila.
v3-0001-pg_createsubscriber-Drop-pre-existing-subscriptio.patch
Description: Binary data
On Fri, Jun 28, 2024 at 12:55 PM Peter Smith wrote:
>
> On Fri, Jun 28, 2024 at 4:18 PM Amit Kapila wrote:
> >
> > On Fri, Jun 28, 2024 at 5:15 AM Peter Smith wrote:
> > >
> > > On Thu, Jun 27, 2024 at 3:44 PM Michael Paquier
> > > wrote:
>
WalSndWaitForWal().
The comment seems to suggests the same: "Make sure we have enough WAL
available before retrieving the current timeline. .."
--
With Regards,
Amit Kapila.
oc.
> Also, this executable warns if the max_prepared_transactions != 0 on the
> publisher.
>
I have slightly adjusted the doc update and warning message for the
0002 patch. Please see attached and let me know what do you think.
--
With Regards,
Amit Kapila.
v3-0001-pg_createsubscriber-Warn-the-two-phase-is-disable.patch
Description: Binary data
On Wed, Jun 26, 2024 at 4:57 PM Tomas Vondra
wrote:
>
> On 6/25/24 07:04, Amit Kapila wrote:
> > On Mon, Jun 24, 2024 at 8:06 PM Tomas Vondra
> > wrote:
> >>
> >> On 6/24/24 12:54, Amit Kapila wrote:
> >>> ...
> >>>>
> >>&
lease-17.sgml according to the comment from Amit.
> >
>
> Thanks! LGTM.
>
As per my reading of this thread, we have an agreement on changing the
GUC name standby_slot_names to synchronized_standby_slots. I'll wait
for a day and push the change unless someone thinks otherwise.
--
With Regards,
Amit Kapila.
such that any subscription not
existing in that list will be dropped.
Shouldn't this be an open item for PG17?
--
With Regards,
Amit Kapila.
On Tue, Jun 25, 2024 at 3:39 PM shveta malik wrote:
>
> On Tue, Jun 25, 2024 at 3:12 PM Amit Kapila wrote:
> >
> > On Mon, Jun 24, 2024 at 1:47 PM shveta malik wrote:
> > >
> > > On Thu, Jun 20, 2024 at 6:41 PM Amit Kapila
> > > wrote:
> > &
On Wed, Jun 26, 2024 at 10:19 AM Bertrand Drouvot
wrote:
>
>
> 2
>
> Should we rename StandbySlotNamesConfigData too?
>
How about SyncStandbySlotsConfigData?
> 3
>
> Should we rename SlotExistsInStandbySlotNames too?
>
Similarly SlotExistsInSyncStandbySlots?
> 4
>
> Should we re
e = on) is supported, users can change it
> after running pg_createsubscriber. The other option is to add a command-line
> option to enable or disable it.
>
Yeah, it is a good idea to add a new option for two_phase but that
should be done in the next version. For now, I suggest updating the
docs and probably raising a warning (if max_prepared_transactions !=
0) as suggested by Noah. This WARNING is useful because one could
expect that setting max_prepared_transactions != 0 means apply will
happen at prepare time after the subscriber is created by this tool.
The WARNING will be useful even if we support two_phase option as the
user may have set the non-zero value of max_prepared_transactions but
didn't use two_phase option.
--
With Regards,
Amit Kapila.
On Mon, Jun 24, 2024 at 1:47 PM shveta malik wrote:
>
> On Thu, Jun 20, 2024 at 6:41 PM Amit Kapila wrote:
> >
> > >> In the second patch, we can implement simple built-in resolution
> > >> strategies like apply and skip (which can be named as remote_apply a
On Tue, Jun 25, 2024 at 12:30 PM Masahiko Sawada wrote:
>
> On Tue, Jun 25, 2024 at 1:54 PM Amit Kapila wrote:
> >
>
> > So, my
> > preference is in order as follows: synchronized_standby_slots,
> > wait_for_standby_slots, logical_replication_wait_slots,
> >
On Tue, Jun 25, 2024 at 3:38 AM Noah Misch wrote:
>
> On Mon, Jun 24, 2024 at 05:20:21PM +0530, Amit Kapila wrote:
> > On Sun, Jun 23, 2024 at 11:52 AM Noah Misch wrote:
> > >
> > > > +static void
> > > > +create_publicatio
On Mon, Jun 24, 2024 at 8:06 PM Tomas Vondra
wrote:
>
> On 6/24/24 12:54, Amit Kapila wrote:
> > ...
> >>
> >>>> I'm not sure there are any cases where using SRE instead of AE would
> >>>> cause
> >>>> problems for
that is more closely associated with physical standby's and this list
contains physical slots corresponding to physical standby's. So, my
preference is in order as follows: synchronized_standby_slots,
wait_for_standby_slots, logical_replication_wait_slots,
logical_replication_synchronous_slots, and
logical_replication_synchronous_standby_slots.
--
With Regards,
Amit Kapila.
ns because we use the old cache state. I am missing your
point about the race condition mentioned in the thread you quoted with
snapbuild.c. Can you please elaborate a bit more?
--
With Regards,
Amit Kapila.
from a logical decoding POV,
> it'd suffice to increase the lock level to just SRE instead of AE.
>
> Since I don't see how it'd be correct otherwise, it's kind of a moot question.
>
We lost track of this thread and the bug is still open. IIUC, the
conclusion is to use SRE in OpenTableList() to fix the reported issue.
Andres, Tomas, please let me know if my understanding is wrong,
otherwise, let's proceed and fix this issue.
--
With Regards,
Amit Kapila.
ransactions? If so, the documentation should say that, and the tool likely
> should warn on startup if max_prepared_transactions != 0.
>
The other point to note in this regard is that if we don't support
two_phase in the beginning during subscription/slot setup, users won't
be able to change it as we don't yet support changing it via alter
subscription (though the patch to alter two_pc is proposed for PG18).
--
With Regards,
Amit Kapila.
veta Malik)
> >
> > it seems like the name ought to have some connection to
> > synchronization. Perhaps something like "synchronized_standby_slots"?
>
> IMHO that might be a bit too close to synchronous_standby_names.
>
Right, but better than the current one.
names are used by
logical WAL senders such that they will send decoded changes to
plugins only after the specified replication slots confirm receiving
WAL. So, combining them doesn't sound advisable.
--
With Regards,
Amit Kapila.
ur of skipping
> generated columns is the same for all of them.
>
> Note that CREATE PUBLICATION column lists did not exist until PG15, so
> a modified patch is needed for the versions before that.
>
Pushed.
--
With Regards,
Amit Kapila.
s not sufficient. We have to point out the exact the end of the
> record.
>
You have a point but if this theory is correct why we are not able to
reproduce the issue after this patch? Also, how to get the WAL
location up to which we need to flush? Is XLogCtlData->logInsertResult
the one we are looking for?
--
With Regards,
Amit Kapila.
On Wed, Jun 19, 2024 at 8:33 PM vignesh C wrote:
>
> On Tue, 18 Jun 2024 at 16:10, Amit Kapila wrote:
> >
> >
> > Agreed and I am not sure which is better because there is a value in
> > keeping the state name the same for both sequences and tables. We
> > pro
On Thu, Jun 20, 2024 at 5:06 PM Ashutosh Bapat
wrote:
>
> On Thu, Jun 20, 2024 at 3:21 PM Amit Kapila wrote:
>>
>> On Wed, Jun 19, 2024 at 2:51 PM Ashutosh Bapat
>> wrote:
>> >
>> > On Wed, Jun 19, 2024 at 12:03 PM Amit Kapila
>> > wrote:
ndicates the
action we need to take once the clock skew is detected). There could
be multiple ways to provide these parameters, one is providing them as
GUCs, and another at the subscription or the table level. I am
thinking whether users would only like to care about a table or set of
tables or they would like to set such variables at the system level.
We already have an SKIP option (that allows us to skip the
transactions till a particular LSN) at the subscription level, so I am
wondering if there is a sense to provide these new parameters related
to conflict resolution also at the same level?
--
With Regards,
Amit Kapila.
On Wed, Jun 19, 2024 at 2:51 PM Ashutosh Bapat
wrote:
>
> On Wed, Jun 19, 2024 at 12:03 PM Amit Kapila wrote:
>>
>> > I doubt that it would be that simple. The application will have to
>> > intervene and tell one of the employees that their reservation has faile
On Thu, Jun 20, 2024 at 2:35 AM Euler Taveira wrote:
>
> I will open new threads soon if you don't.
>
Okay, thanks. I'll wait for you to start new threads and then we can
discuss the respective problems in those threads.
--
With Regards,
Amit Kapila.
On Tue, Jun 11, 2024 at 3:12 PM Ashutosh Bapat
wrote:
>
> On Sat, Jun 8, 2024 at 3:52 PM Amit Kapila wrote:
>>
>> On Fri, Jun 7, 2024 at 5:39 PM Ashutosh Bapat
>> wrote:
>> >
>> > On Thu, Jun 6, 2024 at 5:16 PM Nisha Moond
>> > wrote:
>&
gt; same key is reported as "duplicate key value violates unique constraint
> > "%s"",
> > which could confuse users.
>
> Do you mean it's ok to add a pre-check before applying the INSERT, which will
> verify if the remote tuple violates any unique constraints, and if it violates
> then we log a conflict message ? I thought about this but was slightly
> worried about the extra cost it would bring. OTOH, if we think it's
> acceptable,
> we could do that since the cost is there only when detect_conflict is enabled.
>
> I also thought of logging such a conflict message in pg_catch(), but I think
> we
> lack some necessary info(relation, index name, column name) at the catch
> block.
>
Can't we use/extend existing 'apply_error_callback_arg' for this purpose?
--
With Regards,
Amit Kapila.
nchronous_standby_names' as per your patch's idea unless
'standby_slot_names' is specified? I don't know if there is any value
in setting the 'failover' option for a slot without specifying
'standby_slot_names', so was wondering if we can additionally tie it
to 'synchronous_standby_names'. Any better ideas?
--
With Regards,
Amit Kapila.
On Wed, Jun 19, 2024 at 6:46 AM Peter Smith wrote:
>
> On Tue, Jun 18, 2024 at 9:40 PM Amit Kapila wrote:
> >
> > On Tue, Jun 18, 2024 at 12:11 PM Peter Smith wrote:
> > >
> > > While reviewing another thread that proposes to include "generated
> >
On Tue, Jun 18, 2024 at 12:13 PM Peter Eisentraut wrote:
>
> On 18.06.24 05:59, Amit Kapila wrote:
> >> 1. After promotion, the pre-existing replication objects should be
> >> removed (either optionally or always), otherwise, it can lead to a new
> >> subscr
On Tue, Jun 18, 2024 at 12:41 PM Euler Taveira wrote:
>
> On Tue, Jun 18, 2024, at 12:59 AM, Amit Kapila wrote:
>
> On Mon, May 20, 2024 at 12:12 PM Amit Kapila wrote:
> >
> > On Sun, May 19, 2024 at 11:20 PM Euler Taveira wrote:
> > >
> > > On Sun
on 29.7)
> * e.g.4 in the "CREATE PUBLICATION" reference page
>
> For now, I have provided just a simple patch for the "Generated
> Columns" section [3]. Perhaps it is enough.
>
Can we try to clarify if their corresponding values are replicated?
--
With Regards,
Amit Kapila.
On Tue, Jun 18, 2024 at 7:30 AM Masahiko Sawada wrote:
>
> On Fri, Jun 14, 2024 at 4:04 PM Amit Kapila wrote:
> >
> > On Thu, Jun 13, 2024 at 6:14 PM Masahiko Sawada
> > wrote:
> > >
> >
> > > >
> > > > Now, say we don't wan
On Tue, Jun 18, 2024 at 1:18 PM Dilip Kumar wrote:
>
> On Tue, Jun 18, 2024 at 12:11 PM Amit Kapila wrote:
> >
> > On Tue, Jun 18, 2024 at 11:54 AM Dilip Kumar wrote:
> > >
> > > On Mon, Jun 17, 2024 at 8:51 PM Robert Haas wrote:
> > > >
>
On Tue, Jun 18, 2024 at 11:54 AM Dilip Kumar wrote:
>
> On Mon, Jun 17, 2024 at 8:51 PM Robert Haas wrote:
> >
> > On Mon, Jun 17, 2024 at 1:42 AM Amit Kapila wrote:
> > > The difference w.r.t the existing mechanisms for holding deleted data
> > > is that w
On Mon, May 20, 2024 at 12:12 PM Amit Kapila wrote:
>
> On Sun, May 19, 2024 at 11:20 PM Euler Taveira wrote:
> >
> > On Sun, May 19, 2024, at 2:30 PM, Tom Lane wrote:
> >
> > I'm fairly disturbed about the readiness of pg_createsubscriber.
> > T
n we raise an ERROR. It is not clear to me that
putting bigger effort into clock skew is worth especially when other
systems providing CDR feature (like Ora or BDR) for decades have not
done anything like vector clocks. It is possible that this is less of
a problem w.r.t CDR and just detecting the anomaly in clock skew is
good enough.
[1] -
https://www.postgresql.org/message-id/flat/CAEepm%3D1iiEzCVLD%3DRoBgtZSyEY1CR-Et7fRc9prCZ9MuTz3pWg%40mail.gmail.com
--
With Regards,
Amit Kapila.
On Mon, Jun 17, 2024 at 4:18 AM Tomas Vondra
wrote:
>
> On 6/14/24 13:29, Amit Kapila wrote:
> > On Fri, Jun 14, 2024 at 12:10 AM Robert Haas wrote:
> >>
> >> On Thu, May 23, 2024 at 2:37 AM shveta malik
> >> wrote:
> >>> c) update_de
ostgresql.org/message-id/CAJpy0uCov4JfZJeOvY0O21_gk9bcgNUDp4jf8%2BBbMp%2BEAv8cVQ%40mail.gmail.com
--
With Regards,
Amit Kapila.
On Thu, Jun 13, 2024 at 11:18 PM Jonathan S. Katz wrote:
>
> On 6/13/24 7:28 AM, Amit Kapila wrote:
> >>
> >> I feel that we should log all types of conflict in an uniform way. For
> >> example, with detect_conflict being true, the update_differ conflict
> >
On Thu, Jun 13, 2024 at 7:00 PM Alvaro Herrera wrote:
>
> On 2024-Jun-07, Tomas Vondra wrote:
>
> > On 6/3/24 09:30, Amit Kapila wrote:
> > > On Sat, May 25, 2024 at 2:39 AM Tomas Vondra
> > > wrote:
>
> > >> How is this going to deal with the fa
On Fri, Jun 14, 2024 at 5:16 AM Michael Paquier wrote:
>
> On Thu, Jun 13, 2024 at 03:36:05PM +0530, Amit Kapila wrote:
> > Fair enough. However, this raises the question Dilip and Vignesh are
> > discussing whether we need a new relfilenode for sequence update even
> >
On Thu, Jun 13, 2024 at 6:14 PM Masahiko Sawada wrote:
>
> On Thu, Jun 13, 2024 at 7:06 PM Amit Kapila wrote:
> >
> > On Thu, Jun 13, 2024 at 1:09 PM Masahiko Sawada
> > wrote:
> > >
> > > On Wed, Jun 12, 2024 at 6:59 PM Amit Kapila
> > > w
On Thu, Jun 13, 2024 at 1:09 PM Masahiko Sawada wrote:
>
> On Wed, Jun 12, 2024 at 6:59 PM Amit Kapila wrote:
> >
> >
> > Yeah, starting with a single worker sounds good for now. Do you think
> > we should sync all the sequences in a single transaction or have
licate key
> value violates unique constraint "%s"", which could confuse users.
> Ideally, I think that we log such conflict detection details (table
> name, column name, conflict type, etc) to somewhere (e.g. a table or
> server logs) so that the users can resolve them manually.
>
On Wed, Jun 12, 2024 at 6:43 AM Kyotaro Horiguchi
wrote:
>
> At Tue, 11 Jun 2024 14:27:28 +0530, Amit Kapila
> wrote in
> > On Tue, Jun 11, 2024 at 12:34 PM Kyotaro Horiguchi
> > wrote:
> > >
> > > At Tue, 11 Jun 2024 11:32:12 +0530, Amit Kapila
> &
the
sequence state in pg_subscription_rel and sequence values can be
rolled back together. However, it is unclear whether we need to create
a new relfilenode while copying existing sequences (say during ALTER
SUBSCRIPTION .. REFRESH PUBLICATION SEQUENCES, or whatever command we
decide)? Probably the answer lies in how we want to implement this
command. If we want to copy all sequence values during the command
itself then it is probably okay but if we want to handover this task
to the sequence-sync worker then we need some state management and a
new relfilenode so that on error both state and sequence values are
rolled back.
--
With Regards,
Amit Kapila.
On Mon, Jun 10, 2024 at 5:12 PM Tomas Vondra
wrote:
>
> On 6/10/24 10:54, Amit Kapila wrote:
> > On Fri, Jun 7, 2024 at 6:08 PM Tomas Vondra
> > wrote:
> >>
> >> On 5/27/24 07:48, shveta malik wrote:
> >>> On Sat, May 25, 2024 at 2:39 A
ern is this removes an
> useful assert, protecting us against certain type of bugs. And now we'll
> just happily ignore them. Is that a good idea?
>
The assert was added in this release. I was also having the same
understanding as yours which is why I added it. However, the case
presented by Vignesh has revealed that I was wrong.
--
With Regards,
Amit Kapila.
On Tue, Jun 11, 2024 at 12:34 PM Kyotaro Horiguchi
wrote:
>
> At Tue, 11 Jun 2024 11:32:12 +0530, Amit Kapila
> wrote in
> > Sorry, it is not clear to me why we failed to flush the last
> > continuation record in logical walsender? I see that we try to flush
> &
t;
Thanks, I have pushed the fix.
--
With Regards,
Amit Kapila.
, it is not clear to me why we failed to flush the last
continuation record in logical walsender? I see that we try to flush
the WAL after receiving got_STOPPING in WalSndWaitForWal(), why is
that not sufficient?
--
With Regards,
Amit Kapila.
On Tue, Feb 20, 2024 at 12:35 PM vignesh C wrote:
>
> On Sat, 17 Feb 2024 at 12:03, Amit Kapila wrote:
> >
> >
> > @@ -1839,7 +1839,8 @@ LogicalConfirmReceivedLocation(XLogRecPtr lsn)
> >
> > SpinLockAcquire(&MyReplicationSlot->mutex);
> >
&g
On Mon, Jun 10, 2024 at 12:43 PM Masahiko Sawada wrote:
>
> On Mon, Jun 10, 2024 at 3:14 PM Masahiko Sawada wrote:
> >
> > On Fri, Jun 7, 2024 at 7:30 PM Amit Kapila wrote:
> > >
> > >
> > > Are you imagining the behavior for sequences associate
(T1 < T2).
>
> However, I doubt this is sufficient to declare victory. It's more likely
> that there still are problems, but the examples are likely more complex
> (changes to multiple tables, etc.).
>
Fair enough, I think we need to analyze this more to find actual
problems or in some way try to prove that there is no problem.
> I vaguely remember there were more issues with timestamp inversion, but
> those might have been related to parallel apply etc.
>
Okay, so considering there are problems due to timestamp inversion, I
think the solution to that problem would probably be somehow
generating commit LSN and timestamp in order. I don't have a solution
at this stage but will think more both on the actual problem and
solution. In the meantime, if you get a chance to refer to the place
where you have seen such a problem please try to share the same with
us. It would be helpful.
--
With Regards,
Amit Kapila.
uld that introduce an inconsistency between r1 and r2?
>
As per my understanding, this shouldn't be an inconsistency. Won't it
be true even when the transactions are performed on a single node with
the same timing?
--
With Regards,
Amit Kapila.
On Fri, Jun 7, 2024 at 7:55 AM Masahiko Sawada wrote:
>
> On Thu, Jun 6, 2024 at 6:40 PM Amit Kapila wrote:
> >
> > On Thu, Jun 6, 2024 at 11:10 AM Masahiko Sawada
> > wrote:
> > >
> > > On Wed, Jun 5, 2024 at 9:30 PM Amit Kapila
> > > w
lying it to all subscribers. It makes more sense for the
> subscriber to control this, especially when we are planning to
> compress the data sent downstream.
>
Yes, that makes sense. However, we then need to provide this option
via SQL APIs as well for other plugins.
--
With Regards,
Amit Kapila.
On Thu, Jun 6, 2024 at 7:54 PM Alvaro Herrera wrote:
>
> On 2024-Jun-06, Amit Kapila wrote:
>
> > On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote:
> > >
> > > When the content of a large transaction (size exceeding
> > > logical_decoding_wo
--
With Regards,
Amit Kapila.
On Thu, Jun 6, 2024 at 6:22 PM Julien Tachoires wrote:
>
> Le jeu. 6 juin 2024 à 04:13, Amit Kapila a écrit :
> >
> > On Thu, Jun 6, 2024 at 4:28 PM Julien Tachoires wrote:
> > >
> > > When the content of a large transaction (size exceeding
> >
oding_work_mem'?
>
> 2. Do we want a GUC to switch compression on/off?
>
It depends on the overhead of decoding. Did you try to measure the
decoding overhead of decompression when reading compressed files?
--
With Regards,
Amit Kapila.
On Thu, Jun 6, 2024 at 3:44 PM Ashutosh Bapat
wrote:
>
> On Thu, Jun 6, 2024 at 9:22 AM Amit Kapila wrote:
>>
>> On Wed, Jun 5, 2024 at 6:01 PM Ashutosh Bapat
>> wrote:
>> >
>> > On Wed, Jun 5, 2024 at 8:45 AM Amit Kapila wrote:
>> >>
On Wed, Jun 5, 2024 at 7:29 PM Dilip Kumar wrote:
>
> On Tue, Jun 4, 2024 at 9:37 AM Amit Kapila wrote:
> >
> > Can you share the use case of "earliest_timestamp_wins" resolution
> > method? It seems after the initial update on the local node, it will
> &
On Thu, Jun 6, 2024 at 11:10 AM Masahiko Sawada wrote:
>
> On Wed, Jun 5, 2024 at 9:30 PM Amit Kapila wrote:
> >
>
> > To achieve this, we can allow sequences to be copied during
> > the initial CREATE SUBSCRIPTION command similar to what we do for
> > table
On Thu, Jun 6, 2024 at 9:32 AM Masahiko Sawada wrote:
>
> On Wed, Jun 5, 2024 at 12:43 PM Amit Kapila wrote:
> >
> > On Tue, Jun 4, 2024 at 8:56 PM Yogesh Sharma
> > wrote:
> > >
> > > On 6/4/24 06:57, Amit Kapila wrote:
> > >
> > >
On Wed, Jun 5, 2024 at 3:17 PM Bharath Rupireddy
wrote:
>
> On Tue, Jun 4, 2024 at 5:40 PM Amit Kapila wrote:
> >
> > Even if we decode it periodically (say each time we decode the
> > checkpoint record) then also we need to send the entire set of
> > sequences at
On Wed, Jun 5, 2024 at 6:01 PM Ashutosh Bapat
wrote:
>
> On Wed, Jun 5, 2024 at 8:45 AM Amit Kapila wrote:
>>
>> How about periodically sending this information?
>> >
>>
>> Now, if we want to support some sort of failover then probably this
>>
On Wed, Jun 5, 2024 at 12:51 PM Peter Eisentraut wrote:
>
> On 04.06.24 12:57, Amit Kapila wrote:
> > 2. Provide a command say Alter Subscription ... Replicate Sequences
> > (or something like that) which users can perform before shutdown of
> > the publisher node during
On Wed, Jun 5, 2024 at 9:13 AM Amit Kapila wrote:
>
> On Tue, Jun 4, 2024 at 8:56 PM Yogesh Sharma
> wrote:
> >
> > On 6/4/24 06:57, Amit Kapila wrote:
> >
> > > 2. Provide a command say Alter Subscription ... Replicate Sequences
> > > (or someth
is better to encourage
users to use standby_slot_names to ensure the same. If at all we get
such use cases and or requirements then we can add such additional
steps after understanding the user's requirements. For now, we should
remove these additional steps.
--
With Regards,
Amit Kapila.
On Tue, Jun 4, 2024 at 8:56 PM Yogesh Sharma
wrote:
>
> On 6/4/24 06:57, Amit Kapila wrote:
>
> > 2. Provide a command say Alter Subscription ... Replicate Sequences
> > (or something like that) which users can perform before shutdown of
> > the publisher node duri
On Tue, Jun 4, 2024 at 7:40 PM Ashutosh Bapat
wrote:
>
> On Tue, Jun 4, 2024 at 4:27 PM Amit Kapila wrote:
>>
>>
>> 3. Replicate published sequences via walsender at the time of shutdown
>> or incrementally while decoding checkpoint record. The two ways to
>
On Tue, Jun 4, 2024 at 4:53 PM Bharath Rupireddy
wrote:
>
> On Tue, Jun 4, 2024 at 4:27 PM Amit Kapila wrote:
> >
> > 3. Replicate published sequences via walsender at the time of shutdown
> > or incrementally while decoding checkpoint record. The two ways to
> > a
%3DEwZTF5z9_xO9Ms4wsqD88Q%40mail.gmail.com
[3] -
https://www.postgresql.org/message-id/CAA4eK1KR4%3DyALKP0pOdVkqUwoUqD_v7oU3HzY-w0R_EBvgHL2w%40mail.gmail.com
[4] -
https://www.postgresql.org/message-id/12822961-b7de-9d59-dd27-2e3dc3980c7e%40enterprisedb.com
--
With Regards,
Amit Kapila.
gt; >
> > Also, doesn't all this whole design ignore the concurrency between
> > publishers? Isn't this problematic considering the commit timestamps may
> > go backwards (for a given publisher), which means the conflict
> > resolution is not deterministic (as it depends on how exactly it
> > interleaves)?
> >
I am not able to imagine the cases you are worried about. Can you
please be specific? Is it similar to the case I described in
yesterday's email [1]?
[1] -
https://www.postgresql.org/message-id/CAA4eK1JTMiBOoGqkt%3DaLPLU8Rs45ihbLhXaGHsz8XC76%2BOG3%2BQ%40mail.gmail.com
--
With Regards,
Amit Kapila.
nflict and as per the resolution method
(last_update_wins) we need to retain Update-1.
On further thinking, the operations on node-1 PN-1 as defined above
seem impossible because one of the Updates needs to wait for the other
to write a commit record. So the commits may happen with LSN1 < LSN2
but with T1 > T2 but they can't be on the same row due to locks. So,
the order of apply should still be consistent. Am, I missing
something?
--
With Regards,
Amit Kapila.
need to add WAL read I/O stats also to
> pg_stat_wal. Then, we can also add WALReadFromBuffers stats
> hits/misses there. With this approach, pg_stat_wal can be a one-stop
> view for all the WAL related stats.
>
If possible, let's have all the I/O stats (even for WAL) in
pg_stat_io. Can't we show the WAL data we get from buffers in the hits
column and then have read_bytes or something like that to know the
amount of data read?
--
With Regards,
Amit Kapila.
is a good idea to do something about this. It makes sense to
start with something simple and see how it works. I think this can
also help us whether we need to chase a particular BF failure
immediately after committing.
--
With Regards,
Amit Kapila.
On Thu, May 23, 2024 at 8:43 PM Euler Taveira wrote:
>
> On Thu, May 23, 2024, at 5:54 AM, Amit Kapila wrote:
>
>
> Why in the first place do we need to ensure that primary_slot_name is
> active on the primary? You mentioned something related to WAL
> retention but I d
201 - 300 of 2727 matches
Mail list logo
