00k tuples (normal.sql):
* without spill
HEAD: 13235.136 ms
v3 patch: 14320.082 ms
v4 patch: 13300.665 ms
* with spill
HEAD: 22970.204 ms
v3 patch: 23625.649 ms
v4 patch: 23304.366
workload-2, decode one transaction with 100k subtransaction (many-subtxn.sql):
* without spill
HEAD: 345.718 ms
v3
ame,
"primary_slot_name"));
+
I think that the detail message is not appropriate since the
primary_slot_name could actually be a valid name. I think we can
rephrase it to something like "The replication slot %s specified by %s
does not exist on the primary server".
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Feb 8, 2024 at 4:06 PM Peter Eisentraut wrote:
>
> On 08.02.24 05:05, Masahiko Sawada wrote:
> > On Thu, Feb 8, 2024 at 3:11 AM Peter Eisentraut
> > wrote:
> >>
> >> The way src/test/modules/xid_wraparound/meson.build is written, it
> &
ng test modules such as
> src/test/modules/test_rbtree/meson.build for examples.
>
Good catch, thanks.
I've attached the patch to fix it. Does it make sense?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
0001-Prevent-installation-of-xid_wraparound-test-durin
On Tue, Feb 6, 2024 at 8:21 PM Amit Kapila wrote:
>
> On Tue, Feb 6, 2024 at 3:33 PM Amit Kapila wrote:
> >
> > On Tue, Feb 6, 2024 at 1:09 PM Masahiko Sawada
> > wrote:
> > >
> > > On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila
> > > wrote:
>
On Tue, Feb 6, 2024 at 3:19 PM Amit Kapila wrote:
>
> On Mon, Feb 5, 2024 at 7:56 PM Masahiko Sawada wrote:
> >
> > ---
> > Since Two processes (e.g. the slotsync worker and
> > pg_sync_replication_slots()) concurrently fetch and update the slot
> > informat
On Fri, Feb 2, 2024 at 5:16 PM Masahiko Sawada wrote:
>
> On Fri, Feb 2, 2024 at 1:59 PM Shubham Khanna
> wrote:
> >
> > On Fri, Jan 26, 2024 at 2:07 PM Masahiko Sawada
> > wrote:
> > >
> > > On Wed, Dec 20, 2023 at 12:11 PM Amit Kapila
> >
On Fri, Feb 2, 2024 at 8:47 PM John Naylor wrote:
>
> On Wed, Jan 31, 2024 at 12:50 PM Masahiko Sawada
> wrote:
> > I've attached the new patch set (v56). I've squashed previous updates
> > and addressed review comments on v55 in separate patches. Here are the
> > u
we use CONCAT function, we can replace '||' with ','.
---
+ Confirm that the standby server is not lagging behind the subscribers.
+ This step can be skipped if
+ standby_slot_names
+ has been correctly configured.
How can the user confirm if standby_slot_names is correctly configured?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
g',
>dir_include_server / 'catalog',
>dir_include_server / 'catalog',
> ]
>
>
Thank you for reporting the issue and the patch.
I've confirmed this patch fixes the issue. But I don't have enough
knowledge of meson to assess this fix.
Peter, could you check this fix as
On Fri, Feb 2, 2024 at 1:59 PM Shubham Khanna
wrote:
>
> On Fri, Jan 26, 2024 at 2:07 PM Masahiko Sawada wrote:
> >
> > On Wed, Dec 20, 2023 at 12:11 PM Amit Kapila
> > wrote:
> > >
> > > On Wed, Dec 20, 2023 at 6:49 AM Masahiko Sawada
> > &g
Hi,
On Wed, Jan 31, 2024 at 5:32 PM vignesh C wrote:
>
> On Tue, 30 Jan 2024 at 13:37, Masahiko Sawada wrote:
> >
> > On Fri, Jan 26, 2024 at 5:36 PM Masahiko Sawada
> > wrote:
> > >
> > > On Wed, Dec 20, 2023 at 12:11 PM Amit Kapila
> > >
ted
> 2022: 58 committed
> 2021: 56 committed
> 2020: 49 committed
>
> A special thanks to the reviewers/committers who spent tireless
> effort in moving the patches forward.
>
Thank you for all the hard work, Vignesh!
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
small amount of
> changes, the performance may be degraded, right? Do you have a result in sucha
> a case?
I've run a benchmark test that I shared before[1]. Here are results of
decoding a transaction that has 1M subtransaction each of which has 1
INSERT:
HEAD:
1810.192 ms
HEAD w/ patch:
2001.094 ms
I set a large enough value to logical_decoding_work_mem not to evict
any transactions. I can see about about 10% performance regression in
this case.
Regards,
[1]
https://www.postgresql.org/message-id/CAD21AoAfKTgrBrLq96GcTv9d6k97zaQcDM-rxfKEt4GSe0qnaQ%40mail.gmail.com
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Feb 2, 2024 at 1:58 PM Amit Kapila wrote:
>
> On Fri, Feb 2, 2024 at 6:46 AM Masahiko Sawada wrote:
> >
> > On Thu, Feb 1, 2024 at 12:51 PM Amit Kapila wrote:
> > >
> > >
> > > > BTW I've tested the following switch/fail-back scenario but
On Thu, Feb 1, 2024 at 12:51 PM Amit Kapila wrote:
>
> On Wed, Jan 31, 2024 at 9:20 PM Masahiko Sawada wrote:
> >
> > On Wed, Jan 31, 2024 at 7:42 PM Amit Kapila wrote:
> > >
> > >
> > > Considering my previous where we don't want to restart
On Wed, Jan 31, 2024 at 7:42 PM Amit Kapila wrote:
>
> On Wed, Jan 31, 2024 at 2:02 PM Masahiko Sawada wrote:
> >
> > Thank you for updating the patches. As for the slotsync worker patch,
> > is there any reason why 0001, 0002, and 0004 patches are still
> > separat
replication slot "s" is
active for PID 1103935
2024-01-31 17:25:21.750 JST [1103933] CONTEXT: WAL redo at 0/3020D20
for Database/DROP: dir 1663/16384
2024-01-31 17:25:21.751 JST [1103930] LOG: startup process (PID
1103933) exited with exit code 1
It seems that because the slo
On Tue, Jan 30, 2024 at 7:20 PM John Naylor wrote:
>
> On Tue, Jan 30, 2024 at 7:56 AM Masahiko Sawada wrote:
> >
> > On Mon, Jan 29, 2024 at 8:48 PM John Naylor wrote:
>
> > > I meant the macro could probably be
> > >
> > > Max(SLAB_DEFAULT_BLO
On Tue, Jan 30, 2024 at 4:43 PM Bharath Rupireddy
wrote:
>
> On Tue, Jan 30, 2024 at 6:24 AM Masahiko Sawada wrote:
> >
> > Hi,
> >
> > While working on radix tree patch[1], John Naylor found that dsa.c
> > doesn't already use shared locks even in dsa_dump
On Fri, Jan 26, 2024 at 5:36 PM Masahiko Sawada wrote:
>
> On Wed, Dec 20, 2023 at 12:11 PM Amit Kapila wrote:
> >
> > On Wed, Dec 20, 2023 at 6:49 AM Masahiko Sawada
> > wrote:
> > >
> > > On Tue, Dec 19, 2023 at 8:02 PM Amit Kapila
> > >
t | N attributes| Attr1's format |...| AttrN's format |
+ * ++++++ +++
+ * 0: text 0: text 0: text
+ * 1: binary1: binary 1: binary
+ */
I think this kind of diagram could be missed from being updated when
we update the CopyOutResponse format. It's better to refer to the
documentation instead.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Mon, Jan 29, 2024 at 8:48 PM John Naylor wrote:
>
> On Mon, Jan 29, 2024 at 2:29 PM Masahiko Sawada wrote:
>
> > > > +/*
> > > > + * Calculate the slab blocksize so that we can allocate at least 32
> > > > chunks
> > > > + * from th
-TRD62BTwom8iLXLOWMsHkkwFi%3Drzg%40mail.gmail.com
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Jan 26, 2024 at 11:05 PM Masahiko Sawada wrote:
>
> On Wed, Jan 24, 2024 at 3:42 PM John Naylor wrote:
> >
> > On Tue, Jan 23, 2024 at 10:58 AM Masahiko Sawada
> > wrote:
> > >
> > > The new patches probably need to be polished but the V
On Mon, Jan 29, 2024 at 12:10 PM Junwang Zhao wrote:
>
> On Mon, Jan 29, 2024 at 10:42 AM Masahiko Sawada
> wrote:
> >
> > On Fri, Jan 26, 2024 at 6:02 PM Junwang Zhao wrote:
> > >
> > > On Fri, Jan 26, 2024 at 4:55 PM Sutou Kouhei wrote:
> > >
false if the passed option is not
supported. If one of the builtin formats is specified and the rest
options list has at least one option, we raise "option %s not
recognized" error. IOW it's the core's responsibility to ranse the
"option %s not recognized" error, which is in order to raise a
consistent error message. Also, I think the core should check the
redundant options including bultiin and custom options.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Jan 26, 2024 at 4:04 PM Masahiko Sawada wrote:
>
> On Thu, Jan 25, 2024 at 10:17 PM Aleksander Alekseev
> wrote:
> >
> > Hi,
> >
> > > > Here is the corrected patch.
> > >
> > > Thank you for updating the p
On Wed, Jan 24, 2024 at 3:42 PM John Naylor wrote:
>
> On Tue, Jan 23, 2024 at 10:58 AM Masahiko Sawada
> wrote:
> >
> > The new patches probably need to be polished but the VacDeadItemInfo
> > idea looks good to me.
>
> That idea looks good to me, too. S
On Wed, Dec 20, 2023 at 12:11 PM Amit Kapila wrote:
>
> On Wed, Dec 20, 2023 at 6:49 AM Masahiko Sawada wrote:
> >
> > On Tue, Dec 19, 2023 at 8:02 PM Amit Kapila wrote:
> > >
> > > On Tue, Dec 19, 2023 at 8:31 AM Masahiko Sawada
> > > wrote:
>
-pfree(tuple);
> > -}
> > -
> >
> > Why does ReorderBufferReturnTupleBuf need to be moved from
> > reorderbuffer.c to reorderbuffer.h? It seems not related to this
> > refactoring patch so I think we should do it in a separate patch if we
> > really want it. I'm not sure it's necessary, though.
>
> OK, fixed.
Thank you for updating the patch. It looks good to me. I'm going to
push it next week, barring any objections.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Jan 26, 2024 at 2:40 PM Yugo NAGATA wrote:
>
> On Fri, 26 Jan 2024 13:59:09 +0900
> Masahiko Sawada wrote:
>
> > On Fri, Jan 26, 2024 at 11:28 AM Yugo NAGATA wrote:
> > >
> > > Hi,
> > >
> > > I found that the documentation of COPY
+This should not lead to problems in the event of a COPY
How about the followings for consistency with the description of the
ON_ERROR option?
COPY stops operation at the first error if the stop value is specified
to the ON_ERROR option. This should not lead to ...
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
er
> read.
> Lines 3806, 3836, 3854, 3889, 3910
>
> I can create patch and post it to this thread or a new one if deemed
> worthwhile.
I'm not sure these changes are really beneficial. They contribute to
improving neither readability and performance IMO.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Jan 25, 2024 at 11:25 AM torikoshia wrote:
>
> On 2024-01-24 17:05, Masahiko Sawada wrote:
> > On Tue, Jan 23, 2024 at 1:02 AM torikoshia
> > wrote:
> >>
> >> On 2024-01-17 14:47, Masahiko Sawada wrote:
> >> > On Wed, Jan 17, 2024 at 2:22
On Thu, Jan 25, 2024 at 1:53 PM Michael Paquier wrote:
>
> On Thu, Jan 25, 2024 at 01:36:03PM +0900, Masahiko Sawada wrote:
> > Hmm I can see a similar trend that Suto-san had; the binary format got
> > slightly faster whereas both text and csv format has small regression
&g
if (cstate->opts.binary)" branches from the original
CopyOneRowTo(). I've experimented with a similar optimization for csv
and text format; have different callbacks for text and csv format and
remove "if (cstate->opts.csv_mode)" branches. I've attached a patch
for that. Here are results:
HEAD w/ 0001 patch + remove branches:
binary 2824.502 ms
text 2715.264 ms
csv 2803.381 ms
The numbers look better now. I'm not sure these are within a noise
range but it might be worth considering having different callbacks for
text and csv formats.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
add_callback_for_csv_format.patch
Description: Binary data
oved from
reorderbuffer.c to reorderbuffer.h? It seems not related to this
refactoring patch so I think we should do it in a separate patch if we
really want it. I'm not sure it's necessary, though.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Tue, Jan 23, 2024 at 1:02 AM torikoshia wrote:
>
> On 2024-01-17 14:47, Masahiko Sawada wrote:
> > On Wed, Jan 17, 2024 at 2:22 PM torikoshia
> > wrote:
> >>
> >> Hi,
> >>
> >> 132de9968840c introduced SAVE_ERROR_TO option to COPY and en
On Wed, Jan 24, 2024 at 2:43 PM Amit Kapila wrote:
>
> On Wed, Jan 24, 2024 at 10:41 AM Masahiko Sawada
> wrote:
> >
> > On Mon, Jan 22, 2024 at 3:58 PM Amit Kapila wrote:
> > >
> > > Can we think of using GetStandbyFlushRecPtr()? We probably need to
&g
On Mon, Jan 22, 2024 at 3:58 PM Amit Kapila wrote:
>
> On Fri, Jan 19, 2024 at 3:55 PM shveta malik wrote:
> >
> > On Fri, Jan 19, 2024 at 10:35 AM Masahiko Sawada
> > wrote:
> > >
> > >
> > > Thank you for updating the patch. I have some co
On Tue, Jan 23, 2024 at 12:58 PM Masahiko Sawada wrote:
>
> On Mon, Jan 22, 2024 at 5:18 PM John Naylor wrote:
> >
> > On Mon, Jan 22, 2024 at 2:24 PM Masahiko Sawada
> > wrote:
> > >
> > > For the next version patch, I'll work on this idea an
On Mon, Jan 22, 2024 at 5:18 PM John Naylor wrote:
>
> On Mon, Jan 22, 2024 at 2:24 PM Masahiko Sawada wrote:
> >
> > For the next version patch, I'll work on this idea and try to clean up
> > locking stuff both in tidstore and radix tree. Or if you're already
> > w
On Mon, Jan 22, 2024 at 9:26 PM Amit Kapila wrote:
>
> On Mon, Jan 22, 2024 at 5:28 PM Masahiko Sawada wrote:
> >
> > On Sat, Jan 20, 2024 at 7:44 PM Amit Kapila wrote:
> > >
> > > On Sat, Jan 20, 2024 at 10:52 AM Dilip Kumar
> > > wrote:
> &g
bout filtering slots on the standby side? That is, for
example, the LIST_SLOT command returns all slots and the slotsync
worker filters out non-failover slots. Also such command could
potentially be used also in client tools like pg_basebackup,
pg_receivewal, and pg_recvlogical to list the available replication
slots to specify.
> > > Considering all this it seems that for now probably extending
> > > replication commands can simplify a few things like mentioned above
> > > but using SQL's with db-connection is more extendable.
> >
Agreed.
Having said that, considering Amit, Bertrand, and Dilip already agreed
with the current design (using SQL's with db-connection), I might be
worrying too much. So we can probably go with the current design and
improve it if we find some problems.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Mon, Jan 22, 2024 at 2:36 PM John Naylor wrote:
>
> On Mon, Jan 22, 2024 at 10:28 AM Masahiko Sawada
> wrote:
> >
> > On further thought, as you pointed out before, "num_tids" should not
> > be in tidstore in terms of integration with tidbitmap.c, bec
of polish, but
> wanted to get it out there anyway.
Cool.
I'll merge these patches in the next version v54 patch set.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Jan 19, 2024 at 6:48 PM John Naylor wrote:
>
> On Fri, Jan 19, 2024 at 2:26 PM Masahiko Sawada wrote:
> >
> > On Thu, Jan 18, 2024 at 1:30 PM John Naylor wrote:
> > > I'm not quite sure what the point of "num_items" is anymore, because
On Thu, Jan 18, 2024 at 6:54 PM Amit Kapila wrote:
>
> On Thu, Jan 18, 2024 at 11:15 AM Peter Smith wrote:
> >
> > On Thu, Jan 18, 2024 at 12:55 PM Masahiko Sawada
> > wrote:
> > >
> > ...
> > >
> > > Although we can improve it to han
item:
{
$$ = makeDefElem("encoding", (Node *) makeString($2), @1);
}
+ | FORCE ARRAY
+ {
+ $$ = makeDefElem("force_array", (Node *)
makeBoolean(true), @1);
+ }
;
I believe we don't need to support new options in old-style syntax.
---
@@ -3469,6 +3477,10 @@ copy_generic_opt_elem:
{
$$ = makeDefElem($1, $2, @1);
}
+ | FORMAT_LA copy_generic_opt_arg
+ {
+ $$ = makeDefElem("format", $2, @1);
+ }
;
I think it's not necessary. "format" option is already handled in
copy_generic_opt_elem.
---
+/* need delimiter to start next json array element */
+static bool json_row_delim_needed = false;
I think it's cleaner to include json_row_delim_needed into CopyToStateData.
---
Splitting the patch into two patches: add json format and add
force_array option would make reviews easy.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Jan 18, 2024 at 1:30 PM John Naylor wrote:
>
> On Thu, Jan 18, 2024 at 8:31 AM Masahiko Sawada wrote:
> > It seems we cannot make this work nicely. IIUC VacDeadItems is
> > allocated in DSM and TidStore is embedded there. However,
> > dead_items->items.area is
" LSN %X/%X for slot \"%s\" is ahead of the
standby position %X/%X",
+ LSN_FORMAT_ARGS(remote_slot->confirmed_lsn),
+ remote_slot->name,
+ LSN_FORMAT_ARGS(latestWalEnd));
+}
IIUC GetWalRcvLatestWalEnd () returns walrcv->latestWalEnd, which is
typically the primary server's flush position and doesn't mean the LSN
where the walreceiver received/flushed up to. Does it really happen
that the slot's confirmed_flush_lsn is higher than the primary's flush
lsn?
---
After dropping a database on the primary, I got the following LOG (PID
2978463 is the slotsync worker on the standby):
LOG: still waiting for backend with PID 2978463 to accept ProcSignalBarrier
CONTEXT: WAL redo at 0/301CE00 for Database/DROP: dir 1663/16384
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Jan 18, 2024 at 4:59 PM Alexander Korotkov wrote:
>
> On Thu, Jan 18, 2024 at 4:16 AM torikoshia wrote:
> > On 2024-01-18 10:10, jian he wrote:
> > > On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada
> > > wrote:
> > >> On Thu, Jan 18, 2024 at
se too, I'm not sure it's
a bug. The doc says[1]:
Specifies whether the subscription should be automatically disabled if
any errors are detected by subscription workers during data
replication from the publisher.
When an apply worker is trying to establish a connection, it's not
replicating data fro
On Wed, Jan 17, 2024 at 11:37 AM John Naylor wrote:
>
> On Wed, Jan 17, 2024 at 8:39 AM Masahiko Sawada wrote:
> >
> > On Wed, Jan 17, 2024 at 9:20 AM John Naylor wrote:
> > >
> > > On Tue, Jan 16, 2024 at 1:18 PM Masahiko Sawada
> > > wrote:
context seems impossible.
I've been thinking we can add more values to this option to log errors
not only to the server logs but also to the error table (not sure
details but I imagined an error table is created for each table on
error), without an additional option for the destination name. The
values would be like error_action {error|ignore|save-logs|save-table}.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
>
>
> What do you think?
+1
The patch is pretty simple. Here is a comment:
+ (if SAVE_ERROR_TO is specified, otherwise zero).
+
+
To be precise, this counter only advances when a value other than
'ERROR' is specified to SAVE_ERROR_TO option.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Wed, Jan 17, 2024 at 9:20 AM John Naylor wrote:
>
> On Tue, Jan 16, 2024 at 1:18 PM Masahiko Sawada wrote:
> > Just changing "items" to be the local tidstore struct could make the
> > code tricky a bit, since max_bytes and num_items are on the shared
> &g
On Tue, Jan 16, 2024 at 6:40 PM shveta malik wrote:
>
> On Tue, Jan 16, 2024 at 12:59 PM Masahiko Sawada
> wrote:
> >
> > On Tue, Jan 16, 2024 at 1:07 PM Amit Kapila wrote:
> > >
> > > On Tue, Jan 16, 2024 at 9:03 AM shveta malik
> > > wrote:
of the approaches but I still feel that when we
> > have a standard way of doing things (bgworker) we should not keep
> > adding code to do things in a special way unless there is a strong
> > reason to do so. Now we need to decide if 'enable_syncslot' being
> > PGC_POSTMASTER is a strong reason to go the non-standard way?
> >
>
> Agreed and as said earlier I think it is better to make it a
> PGC_SIGHUP. Also, not sure we can say it is a non-standard way as
> already autovacuum launcher is handled in the same way. One more minor
> thing is it will save us for having a new bgworker state
> BgWorkerStart_ConsistentState_HotStandby as introduced by this patch.
Why do we need to add a new BgWorkerStart_ConsistentState_HotStandby
for the slotsync worker? Isn't it sufficient that the slotsync worker
exits if not in hot standby mode?
Is there any technical difficulty or obstacle to make the slotsync
worker start using bgworker after reloading the config file?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Sun, Jan 14, 2024 at 10:43 PM John Naylor wrote:
>
> On Fri, Jan 12, 2024 at 3:49 PM Masahiko Sawada wrote:
> >
> > On Thu, Jan 11, 2024 at 9:28 AM Masahiko Sawada
> > wrote:
> > > So I agree to remove both max_bytes and num_items from the control
> >
On Thu, Jan 11, 2024 at 10:24 AM Sutou Kouhei wrote:
>
> Hi,
>
> In
> "Re: Make COPY format extendable: Extract COPY TO format implementations"
> on Wed, 10 Jan 2024 16:53:48 +0900,
> Masahiko Sawada wrote:
>
> >> Interesting. But I feel tha
On Mon, Jan 15, 2024 at 8:21 AM Alexander Korotkov wrote:
>
> On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada
> wrote:
> > Thank you for updating the patch. Here are two comments:
> >
> > ---
> > + if (cstate->opts.save_error_to != COPY_SAVE_ERROR_
-- test missing data: should fail
+COPY check_ign_err FROM STDIN WITH (save_error_to none);
+1 {1}
+\.
We might want to cover the extra data cases too.
Regards,
[1]
https://www.postgresql.org/message-id/CACJufxEkkqnozdnvNMGxVAA94KZaCPkYw_Cx4JKG9ueNaZma_A%40mail.gmail.com
[2]
https://www.postgresql.org/message-id/3d0b349ddbd4ae5f605f77b491697158%40oss.nttdata.com
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
trand, Dilip, Sawada-San, and others, please share your opinion on
> this problem as I think it is important to handle this race condition.
Is there any good use case of copying a failover slot in the first
place? If it's not a normal use case and we can probably live without
it, why not always disable failover during the copy? FYI we always
disable two_phase on copied slots. It seems to me that copying a
failover slot could lead to problems, as long as we synchronize slots
based on their names. IIUC without the copy, this pass should never
happen.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Jan 11, 2024 at 9:28 AM Masahiko Sawada wrote:
>
> On Mon, Jan 8, 2024 at 8:35 PM John Naylor wrote:
> >
> > On Wed, Jan 3, 2024 at 9:10 PM John Naylor wrote:
> > >
> > > On Tue, Jan 2, 2024 at 8:01 PM Masahiko Sawada
> > > wrote:
On Mon, Jan 8, 2024 at 8:35 PM John Naylor wrote:
>
> On Wed, Jan 3, 2024 at 9:10 PM John Naylor wrote:
> >
> > On Tue, Jan 2, 2024 at 8:01 PM Masahiko Sawada
> > wrote:
> >
> > > I agree that we expose RT_LOCK_* functions and have t
On Wed, Jan 10, 2024 at 3:40 PM Sutou Kouhei wrote:
>
> Hi,
>
> In
> "Re: Make COPY format extendable: Extract COPY TO format implementations"
> on Wed, 10 Jan 2024 15:33:22 +0900,
> Masahiko Sawada wrote:
>
> >> We can use the
On Tue, Jan 9, 2024 at 11:36 PM torikoshia wrote:
>
> On Tue, Dec 19, 2023 at 10:14 AM Masahiko Sawada
> wrote:
> > If we want only such a feature we need to implement it together (the
> > patch could be split, though). But if some parts of the feature are
> > us
dea but the fourth idea is to provide a convenient macro
to make it easy to construct the CopyFormatRoutine. For example,
#define COPYTO_ROUTINE(...) (Node *) &(CopyToFormatRoutine) {__VA_ARGS__}
static const CopyFormatRoutine testfmt_copyto_handler = {
.type = T_CopyFormatRoutine,
.is_from = true,
.routine = COPYTO_ROUTINE (
.start_fn = testfmt_copyto_start,
.onerow_fn = testfmt_copyto_onerow,
.end_fn = testfmt_copyto_end
)
};
Datum
copy_testfmt_handler(PG_FUNCTION_ARGS)
{
PG_RETURN_POINTER(& testfmt_copyto_handler);
}
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Tue, Jan 9, 2024 at 8:19 PM John Naylor wrote:
>
> On Tue, Jan 9, 2024 at 9:40 AM Masahiko Sawada wrote:
> > In addition, I've made some changes and cleanups:
>
> These look good to me, although I have not tried dumping a node in a while.
>
> > 0011 - simplify the r
On Wed, Jan 3, 2024 at 11:10 PM John Naylor wrote:
>
> On Tue, Jan 2, 2024 at 8:01 PM Masahiko Sawada wrote:
>
> > I agree that we expose RT_LOCK_* functions and have tidstore use them,
> > but am not sure the if (TidStoreIsShared(ts) LWLockAcquire(..., ...)"
> &g
On Wed, Dec 27, 2023 at 12:08 PM John Naylor wrote:
>
> On Tue, Dec 26, 2023 at 12:43 PM Masahiko Sawada
> wrote:
> >
> > On Thu, Dec 21, 2023 at 4:41 PM John Naylor wrote:
>
> > > +TidStoreSetBlockOffsets(TidStore *ts, BlockNumber blkno, OffsetNumber
> &
On Wed, Dec 27, 2023 at 7:13 PM shveta malik wrote:
>
> On Wed, Dec 27, 2023 at 11:36 AM Masahiko Sawada
> wrote:
> >
> > Hi,
> >
> > Thank you for working on this.
> >
> > On Tue, Dec 26, 2023 at 9:27 PM shveta malik wrote:
> > >
>
On Wed, Dec 27, 2023 at 7:43 PM Amit Kapila wrote:
>
> On Wed, Dec 27, 2023 at 11:36 AM Masahiko Sawada
> wrote:
> >
> > On Tue, Dec 26, 2023 at 9:27 PM shveta malik wrote:
> > >
> > > I would like to revisit the current dependency of slotsync w
ers
> DETAIL: There is 1 other session using the database.
>
> But once the slot-sync worker or standby goes down, user can always
> drop this and next time slot-sync worker may not be able to come up.
>
Other random ideas for discussion are:
3) The slotsync worker uses primary_conninfo but also uses a new GUC
parameter, say slot_sync_dbname, to specify the database to connect.
The slot_sync_dbname overwrites the dbname if primary_conninfo also
specifies it. If both don't have a dbname, raise an error.
4) The slotsync worker uses a new GUC parameter, say
slot_sync_conninfo, to specify the connection string to the primary
aside from primary_conninfo. And pg_basebackup -R generates
slot_sync_conninfo as well if required (new option required).
BTW given that the slotsync worker executes only normal SQL queries,
is there any reason why it uses a replication connection? It's
slightly odd to me that the pg_stat_replication view shows one entry
that remains in the "startup" state.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Dec 21, 2023 at 4:41 PM John Naylor wrote:
>
> On Thu, Dec 21, 2023 at 8:33 AM Masahiko Sawada wrote:
> >
> > I found the following comment and wanted to discuss:
> >
> > // this might be better as "iterate over nodes", plus a callback to
> >
On Thu, Dec 21, 2023 at 6:35 PM Sutou Kouhei wrote:
>
> Hi,
>
> In
> "Re: Make COPY format extendable: Extract COPY TO format implementations"
> on Mon, 11 Dec 2023 23:31:29 +0900,
> Masahiko Sawada wrote:
>
> > I've sketched the above idea includ
binary.
>
> > 2. Do we need more tries for design discussion for the first
> >implementation? If we need, what should we try?
>
> A makeNode() is used with an allocation in the current memory context
> in the function returning the handler. I would have assume that this
> stuff returns a handler as a const struct like table AMs.
+1
The example I mentioned above does that.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Dec 21, 2023 at 10:19 AM John Naylor wrote:
>
> On Wed, Dec 20, 2023 at 6:36 PM Masahiko Sawada wrote:
> >
> > I've updated the new patch set that incorporated comments I got so
> > far. 0007, 0008, and 0012 patches are updates from the v45 patch set.
>
On Thu, Dec 21, 2023 at 10:13 AM Masahiko Sawada wrote:
>
> On Thu, Dec 21, 2023 at 9:04 AM Michael Paquier wrote:
> >
> > On Wed, Dec 20, 2023 at 10:43:30AM +0900, Masahiko Sawada wrote:
> > > Thank you for updating the patch. The v2 patch looks good to me. I'll
On Thu, Dec 21, 2023 at 9:04 AM Michael Paquier wrote:
>
> On Wed, Dec 20, 2023 at 10:43:30AM +0900, Masahiko Sawada wrote:
> > Thank you for updating the patch. The v2 patch looks good to me. I'll
> > push it, barring any objections.
>
> This is capturing the eight recor
On Wed, Dec 20, 2023 at 1:07 PM jian he wrote:
>
> On Tue, Dec 19, 2023 at 9:14 AM Masahiko Sawada wrote:
> >
> >
> > The error table hub idea is still unclear to me. I assume that there
> > are error tables at least on each database. And an error table can
>
On Tue, Dec 19, 2023 at 4:37 PM John Naylor wrote:
>
> On Tue, Dec 19, 2023 at 12:37 PM Masahiko Sawada
> wrote:
> >
> > On Mon, Dec 18, 2023 at 3:41 PM John Naylor wrote:
> > > Let's do it in just one place. In TidStoreCreate(), do
> > >
> &
On Tue, Dec 19, 2023 at 6:27 PM Drouvot, Bertrand
wrote:
>
> Hi,
>
> On 12/19/23 9:00 AM, Masahiko Sawada wrote:
> > On Tue, Dec 12, 2023 at 6:15 PM Michael Paquier wrote:
> >>
> >> On Tue, Dec 12, 2023 at 09:23:46AM +0100, Drouvot, Bertrand wrote:
>
On Tue, Dec 19, 2023 at 8:02 PM Amit Kapila wrote:
>
> On Tue, Dec 19, 2023 at 8:31 AM Masahiko Sawada wrote:
> >
> > On Sun, Dec 17, 2023 at 11:40 AM Amit Kapila
> > wrote:
> > >
> > >
> > > The individual transactions shouldn't cross
>
xlrec->is_primary_bucket_page ? 'T' : 'F');
Is it probably worth considering such formats? I prefer to always
print the field name like the current patch and hashdesc.c since it's
easier to parse it. But I'm fine with either way to show the field
value.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Hi,
On Wed, Mar 22, 2023 at 12:15 AM Masahiko Sawada wrote:
>
> Hi all,
>
> I started this new thread from another thread[1] where we're
> discussing a new storage for TIDs, TidStore, since we found a
> difficulty about the memory usage limit for TidStores on DSA.
>
>
On Mon, Dec 18, 2023 at 3:41 PM John Naylor wrote:
>
> On Fri, Dec 15, 2023 at 3:15 PM Masahiko Sawada wrote:
> >
> > On Fri, Dec 15, 2023 at 10:30 AM John Naylor
> > wrote:
>
> > > parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes,
On Sun, Dec 17, 2023 at 11:40 AM Amit Kapila wrote:
>
> On Fri, Dec 15, 2023 at 11:29 AM Masahiko Sawada
> wrote:
> >
> > On Fri, Dec 15, 2023 at 12:37 PM Amit Kapila
> > wrote:
> > >
> > > On Wed, Dec 13, 2023 at 6:01 AM Masahiko Sawada
> >
bles at least on each database. And an error table can
have error data that happened during COPY FROM, including malformed
lines. Do the error tables grow without bounds and the users have to
delete rows at some point? If so, who can do that? How can we achieve
that the users can see only errored rows they generated? And the issue
with logical replication also needs to be resolved. Anyway, if we go
this direction, we need to discuss the overall design.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Mon, Dec 18, 2023 at 9:16 AM jian he wrote:
>
> On Fri, Dec 15, 2023 at 4:49 AM Masahiko Sawada wrote:
> >
> > Hi,
> >
> > I've read this thread and the latest patch. IIUC with SAVE_ERROR
> > option, COPY FROM creates an error table for the target ta
On Sat, Dec 16, 2023 at 1:36 AM Euler Taveira wrote:
>
> On Fri, Dec 15, 2023, at 9:11 AM, Masahiko Sawada wrote:
>
>
> I assume you mean to add ReorderBufferTXN entries to the binaryheap
> and then build it by comparing their sizes (i.e. txn->size). But
> ReorderBuffe
On Fri, Dec 15, 2023 at 2:59 PM Masahiko Sawada wrote:
>
> On Fri, Dec 15, 2023 at 12:37 PM Amit Kapila wrote:
> >
> > On Wed, Dec 13, 2023 at 6:01 AM Masahiko Sawada
> > wrote:
> > >
> > > On Tue, Dec 12, 2023 at 1:33 PM Dilip Kumar wrote:
>
On Fri, Dec 15, 2023 at 7:10 PM Alvaro Herrera wrote:
>
> On 2023-Dec-12, Masahiko Sawada wrote:
>
> > To deal with this problem, I initially thought of the idea (a)
> > mentioned in the comment; use a binary heap to maintain the
> > transactions sorted by the am
On Fri, Dec 15, 2023 at 10:30 AM John Naylor wrote:
>
> On Thu, Dec 14, 2023 at 7:22 AM Masahiko Sawada wrote:
> > In v45, 0001 - 0006 are from earlier versions but I've merged previous
> > updates. So the radix tree now has RT_SET() and RT_FIND() but not
> > RT_GET
On Fri, Dec 15, 2023 at 12:37 PM Amit Kapila wrote:
>
> On Wed, Dec 13, 2023 at 6:01 AM Masahiko Sawada wrote:
> >
> > On Tue, Dec 12, 2023 at 1:33 PM Dilip Kumar wrote:
> > >
> > > On Tue, Dec 12, 2023 at 9:01 AM Masahiko Sawada
> > > wrote:
>
On Fri, Dec 15, 2023 at 9:53 AM Sutou Kouhei wrote:
>
> Hi,
>
> In
> "Re: Make COPY format extendable: Extract COPY TO format implementations"
> on Fri, 15 Dec 2023 05:19:43 +0900,
> Masahiko Sawada wrote:
>
> > To avoid collisions, extensions can b
tolerate errors infinitely and log errors to /dev/null. The user
cannot see the error details but I guess it could still help some
cases as Andres mentioned[1] (it might be a good idea to send the
number of rows successfully loaded in a NOTICE message if some rows
could not be loaded). Then with
On Thu, Dec 14, 2023 at 6:44 PM Sutou Kouhei wrote:
>
> Hi,
>
> In
> "Re: Make COPY format extendable: Extract COPY TO format implementations"
> on Mon, 11 Dec 2023 10:57:15 +0900,
> Masahiko Sawada wrote:
>
> > IIUC we cannot create two sam
201 - 300 of 2813 matches
Mail list logo
