Re: Global snapshots

Patch in the previous letter is full of faulties. Please, use new version. Also, here we fixed the problem with loosing CSN value in a parallel worker (TAP test 003_parallel_safe.pl). Thanks for a.pyhalov for the problem detection and a bugfix. -- regards, Andrey Lepikhov Postgres

Re: Global snapshots

Next version of CSN implementation in snapshots to achieve a proper snapshot isolation in the case of a cross-instance distributed transaction. -- regards, Andrey Lepikhov Postgres Professional >From bbb7dd1d7621c091f11e697d3d894fe7a36918a6 Mon Sep 17 00:00:00 2001 From: Andrey Lepikhov Date:

RE: Global snapshots

From: Andrey V. Lepikhov > Current state of the patch set rebased on master, 5aed6a1fc2. > > It is development version. Here some problems with visibility still detected > in > two tests: > 1. CSN Snapshot module - TAP test on time skew. > 2. Clock SI implementation - TAP test on emulation of

RE: Global snapshots

From: Andrey V. Lepikhov > After researching in depth, I think this is the real problem. > My idea was that we are not using real clocks, we only use clock ticks to > measure time intervals. It can also be interpreted as a kind of clock. Yes, patent claims tend to be written to cover broad

Re: Global snapshots

On 1/1/21 8:14 AM, tsunakawa.ta...@fujitsu.com wrote: -- 11. A method comprising: receiving information relating to a distributed database transaction operating on data in data stores associated with respective participating nodes associated with

RE: Global snapshots

Hello, Andrey-san, all, Based on the request at HighGo's sharding meeting, I'm re-sending the information on Commitment Ordering that could be used for global visibility. Their patents have already expired. -- Have anyone examined the

Re: Global snapshots

On 2021/01/01 12:14, tsunakawa.ta...@fujitsu.com wrote: Hello, Fujii-san and I discussed how to move the scale-out development forward. We are both worried that Clock-SI is (highly?) likely to infringe the said Microsoft's patent. So we agreed we are going to investigate the Clock-SI

RE: Global snapshots

Hello, Fujii-san and I discussed how to move the scale-out development forward. We are both worried that Clock-SI is (highly?) likely to infringe the said Microsoft's patent. So we agreed we are going to investigate the Clock-SI and the patent, and if we have to conclude that we cannot

RE: Global snapshots

Fujii-san, Sawada-san, all, From: Fujii Masao > Yeah, so if we need to guarantee the transaction linearizability even > in distributed env (probably this is yes. Right?), using only Clock-SI > is not enough. We would need to implement something more > in addition to Clock-SI or adopt the

Re: Global snapshots

On 2020/10/23 11:58, Masahiko Sawada wrote: On Thu, 15 Oct 2020 at 01:41, Fujii Masao wrote: On 2020/09/17 15:56, Amit Kapila wrote: On Thu, Sep 10, 2020 at 4:20 PM Fujii Masao wrote: One alternative is to add only hooks into PostgreSQL core so that we can implement the global

Re: Global snapshots

On Thu, 15 Oct 2020 at 01:41, Fujii Masao wrote: > > > > On 2020/09/17 15:56, Amit Kapila wrote: > > On Thu, Sep 10, 2020 at 4:20 PM Fujii Masao > > wrote: > >> > One alternative is to add only hooks into PostgreSQL core so that we can > implement the global transaction management

Re: Global snapshots

On 2020/09/17 15:56, Amit Kapila wrote: On Thu, Sep 10, 2020 at 4:20 PM Fujii Masao wrote: One alternative is to add only hooks into PostgreSQL core so that we can implement the global transaction management outside. This idea was discussed before as the title "eXtensible Transaction

RE: Global snapshots

Hi Andrey san, all, From: tsunakawa.ta...@fujitsu.com > And please wait. As below, the patent holder just says that Clock-SI is not > based on the patent and an independent development. He doesn't say > Clock-SI does not overlap with the patent or implementing Clock-SI does not > infringe on

RE: Global snapshots

From: Andrey Lepikhov > Thank you for this work! > As I can see, main development difficulties placed in other areas: CSN, > resolver, > global deadlocks, 2PC commit... I'm not lawyer too. But if we get remarks from > the patent holders, we can rewrite our Clock-SI implementation. Yeah, I

Re: Global snapshots

22.09.2020 03:47, tsunakawa.ta...@fujitsu.com пишет: Does this make sense from your viewpoint, and can we think that we can use Clock-SI without infrindging on the patent? According to the patent holder, the difference between Clock-SI and the patent seems to be fewer than the

RE: Global snapshots

Hi Andrey-san, all, From: Andrey V. Lepikhov > On 7/27/20 11:22 AM, tsunakawa.ta...@fujitsu.com wrote: > > Could you take a look at this patent? I'm afraid this is the Clock-SI for > > MVCC. > Microsoft holds this until 2031. I couldn't find this with the keyword > "Clock-SI."" > > > > > >

Re: Global snapshots

On 2020-09-18 00:54, Bruce Momjian wrote: On Tue, Sep 8, 2020 at 01:36:16PM +0300, Alexey Kondratov wrote: Thank you for the link! After a quick look on the Sawada-san's patch set I think that there are two major differences: 1. There is a built-in foreign xacts resolver in the [1], which

Re: Global snapshots

On Tue, Sep 8, 2020 at 01:36:16PM +0300, Alexey Kondratov wrote: > Thank you for the link! > > After a quick look on the Sawada-san's patch set I think that there are two > major differences: > > 1. There is a built-in foreign xacts resolver in the [1], which should be > much more convenient

Re: Global snapshots

On Thu, Sep 10, 2020 at 4:20 PM Fujii Masao wrote: > > >> One alternative is to add only hooks into PostgreSQL core so that we can > >> implement the global transaction management outside. This idea was > >> discussed before as the title "eXtensible Transaction Manager API". > > > > Yeah, I read

Re: Global snapshots

On 2020-09-09 20:29, Fujii Masao wrote: On 2020/09/09 2:00, Alexey Kondratov wrote: According to the Sawada-san's v25 0002 the logic is pretty much the same there: +2. Pre-Commit phase (1st phase of two-phase commit) +3. Commit locally +Once we've prepared all of them, commit the

Re: Global snapshots

On 2020/09/10 18:01, tsunakawa.ta...@fujitsu.com wrote: From: Fujii Masao But I'm concerned about that it's really hard to say there is no patent risk around that. I'm not sure who can judge there is no patent risk, in the community. Maybe no one? Anyway, I was thinking that Google Spanner,

RE: Global snapshots

From: Fujii Masao > But I'm concerned about that it's really hard to say there is no patent risk > around that. I'm not sure who can judge there is no patent risk, > in the community. Maybe no one? Anyway, I was thinking that Google Spanner, > YugabyteDB, etc use the global transaction approach

Re: Global snapshots

On 2020/09/10 10:38, tsunakawa.ta...@fujitsu.com wrote: Hi Andrey san, From: Andrey V. Lepikhov > > From: tsunakawa.ta...@fujitsu.com While Clock-SI seems to be considered the best promising for global Could you take a look at this patent? I'm afraid this is the Clock-SI for MVCC.

RE: Global snapshots

Hi Andrey san, From: Andrey V. Lepikhov > > From: tsunakawa.ta...@fujitsu.com > >> While Clock-SI seems to be considered the best promising for global > >>> > Could you take a look at this patent? I'm afraid this is the Clock-SI > >>> > for MVCC. > Microsoft holds this until 2031. I couldn't

Re: Global snapshots

On 2020/09/09 2:00, Alexey Kondratov wrote: On 2020-09-08 14:48, Fujii Masao wrote: On 2020/09/08 19:36, Alexey Kondratov wrote: On 2020-09-08 05:49, Fujii Masao wrote: On 2020/09/05 3:31, Alexey Kondratov wrote: Attached is a patch, which implements a plain 2PC in the postgres_fdw and

Re: Global snapshots

On 2020-09-09 08:35, Masahiko Sawada wrote: On Wed, 9 Sep 2020 at 02:00, Alexey Kondratov wrote: On 2020-09-08 14:48, Fujii Masao wrote: > > IIUC, yes, the information required for automatic resolution is > WAL-logged and the standby tries to resolve those orphan transactions > from WAL after

Re: Global snapshots

On Wed, 9 Sep 2020 at 02:00, Alexey Kondratov wrote: > > On 2020-09-08 14:48, Fujii Masao wrote: > > On 2020/09/08 19:36, Alexey Kondratov wrote: > >> On 2020-09-08 05:49, Fujii Masao wrote: > >>> On 2020/09/05 3:31, Alexey Kondratov wrote: > > Attached is a patch, which implements a

Re: Global snapshots

On 2020-09-08 14:48, Fujii Masao wrote: On 2020/09/08 19:36, Alexey Kondratov wrote: On 2020-09-08 05:49, Fujii Masao wrote: On 2020/09/05 3:31, Alexey Kondratov wrote: Attached is a patch, which implements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'. Also it

Re: Global snapshots

On 2020/09/08 19:36, Alexey Kondratov wrote: On 2020-09-08 05:49, Fujii Masao wrote: On 2020/09/05 3:31, Alexey Kondratov wrote: Attached is a patch, which implements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'. Also it solves these errors handling issues

Re: Global snapshots

On 2020-09-08 05:49, Fujii Masao wrote: On 2020/09/05 3:31, Alexey Kondratov wrote: Attached is a patch, which implements a plain 2PC in the postgres_fdw and adds a GUC 'postgres_fdw.use_twophase'. Also it solves these errors handling issues above and tries to add proper comments

Re: Global snapshots

, IIUC atomicity of the distributed transaction in the postgres_fdw is achieved by the usage of 2PC. I think that this postgres_fdw 2PC support should be separated from global snapshots. Agreed. It could be useful to have such atomic distributed transactions even without a proper visibility

Re: Global snapshots

transaction in the postgres_fdw is achieved by the usage of 2PC. I think that this postgres_fdw 2PC support should be separated from global snapshots. It could be useful to have such atomic distributed transactions even without a proper visibility, which is guaranteed by the global snapshot. Especially

Re: Global snapshots

On 7/27/20 11:22 AM, tsunakawa.ta...@fujitsu.com wrote: Hi Andrey san, Movead san, From: tsunakawa.ta...@fujitsu.com While Clock-SI seems to be considered the best promising for global serializability here, * Why does Clock-SI gets so much attention? How did Clock-SI become the only

RE: Global snapshots

Hi Andrey san, Movead san, From: tsunakawa.ta...@fujitsu.com > While Clock-SI seems to be considered the best promising for global > serializability here, > > * Why does Clock-SI gets so much attention? How did Clock-SI become the > only choice? > > * Clock-SI was devised in Microsoft

Re: Global snapshots

On Mon, 13 Jul 2020 at 20:18, Amit Kapila wrote: > > On Fri, Jul 10, 2020 at 8:46 AM Masahiko Sawada > wrote: > > > > On Wed, 8 Jul 2020 at 21:35, Amit Kapila wrote: > > > > > > > > > Cool. While studying, if you can try to think whether this approach is > > > different from the global

RE: Global snapshots

Hello, While I'm thinking of the following issues of the current approach Andrey raised, I'm getting puzzled and can't help asking certain things. Please forgive me if I'm missing some discussions in the past. > 1. Dependency on clocks synchronization > 2. Needs guarantees of monotonically

Re: Global snapshots

On Fri, Jul 10, 2020 at 8:46 AM Masahiko Sawada wrote: > > On Wed, 8 Jul 2020 at 21:35, Amit Kapila wrote: > > > > > > Cool. While studying, if you can try to think whether this approach is > > different from the global coordinator based approach then it would be > > great. Here is my initial

Re: Global snapshots

On Wed, 8 Jul 2020 at 21:35, Amit Kapila wrote: > > On Wed, Jul 8, 2020 at 11:16 AM Masahiko Sawada > wrote: > > > > On Tue, 7 Jul 2020 at 15:40, Amit Kapila wrote: > > > > > > > > > Okay, but isn't there some advantage with this approach (manage 2PC at > > > postgres_fdw level) as well which

Re: Global snapshots

On Wed, Jul 8, 2020 at 11:16 AM Masahiko Sawada wrote: > > On Tue, 7 Jul 2020 at 15:40, Amit Kapila wrote: > > > > > > Okay, but isn't there some advantage with this approach (manage 2PC at > > postgres_fdw level) as well which is that any node will be capable of > > handling global transactions

Re: Global snapshots

rote: > > > > > > >Also, can you let us know if this > > > > > supports 2PC in some way and if so how is it different from what the > > > > > other thread on the same topic [1] is trying to achieve? > > > > Yes, the patch '0003-postgr

Re: Global snapshots

2PC in some way and if so how is it different from what the > > > > other thread on the same topic [1] is trying to achieve? > > > Yes, the patch '0003-postgres_fdw-support-for-global-snapshots' contains > > > 2PC machinery. Now I'd not judge which approach

Re: Global snapshots

; distributed transaction manager. > > > > When you say "single distributed transaction manager" do you mean > something like pg_dtm which is inspired by Postgres-XL? > > >Also, can you let us know if this > > > supports 2PC in some way and if so how is i

Re: Global snapshots

On Mon, Jun 22, 2020 at 8:36 PM Bruce Momjian wrote: > > On Sat, Jun 20, 2020 at 05:51:21PM +0530, Amit Kapila wrote: > > I feel if Sawada-San or someone involved in another patch also once > > studies this approach and try to come up with some form of comparison > > then we might be able to make

Re: Global snapshots

On Sat, Jun 20, 2020 at 05:51:21PM +0530, Amit Kapila wrote: > I feel if Sawada-San or someone involved in another patch also once > studies this approach and try to come up with some form of comparison > then we might be able to make better decision. It is possible that > there are few good

Re: Global snapshots

On Sat, Jun 20, 2020 at 05:54:18PM +0530, Amit Kapila wrote: > On Fri, Jun 19, 2020 at 6:33 PM Bruce Momjian wrote: > > > > On Fri, Jun 19, 2020 at 05:03:20PM +0800, movead...@highgo.ca wrote: > > > > > > >> would like to know if the patch related to CSN based snapshot [2] is a > > > >> precursor

Re: Global snapshots

On Sat, Jun 20, 2020 at 5:51 PM Amit Kapila wrote: > > > So, is anyone working on improving these parts of the patch. AFAICS > from what Bruce has shared [1], > oops, forgot to share the link [1] - https://wiki.postgresql.org/wiki/WIP_PostgreSQL_Sharding -- With Regards, Amit Kapila.

Re: Global snapshots

On Fri, Jun 19, 2020 at 6:33 PM Bruce Momjian wrote: > > On Fri, Jun 19, 2020 at 05:03:20PM +0800, movead...@highgo.ca wrote: > > > > >> would like to know if the patch related to CSN based snapshot [2] is a > > >> precursor for this, if not, then is it any way related to this patch > > >>

Re: Global snapshots

is > > supports 2PC in some way and if so how is it different from what the > > other thread on the same topic [1] is trying to achieve? > Yes, the patch '0003-postgres_fdw-support-for-global-snapshots' contains > 2PC machinery. Now I'd not judge which approach is better. > Yeah

Re: Global snapshots

On Fri, Jun 19, 2020 at 05:03:20PM +0800, movead...@highgo.ca wrote: > > >> would like to know if the patch related to CSN based snapshot [2] is a > >> precursor for this, if not, then is it any way related to this patch > >> because I see the latest reply on that thread [2] which says it is an >

Re: Global snapshots

>> would like to know if the patch related to CSN based snapshot [2] is a >> precursor for this, if not, then is it any way related to this patch >> because I see the latest reply on that thread [2] which says it is an >> infrastructure of sharding feature but I don't understand completely >>

Re: Global snapshots

let us know if this supports 2PC in some way and if so how is it different from what the other thread on the same topic [1] is trying to achieve? Yes, the patch '0003-postgres_fdw-support-for-global-snapshots' contains 2PC machinery. Now I'd not judge which approach is better. Also, I would like

Re: Global snapshots

On Wed, Jun 10, 2020 at 8:36 AM Andrey V. Lepikhov wrote: > > > On 09.06.2020 11:41, Fujii Masao wrote: > > > > > > The patches seem not to be registered in CommitFest yet. > > Are you planning to do that? > Not now. It is a sharding-related feature. I'm not sure that this > approach is fully

Re: Global snapshots

WLOCKS, LWTRANCHE_COMMITTS_BUFFER, LWTRANCHE_SUBTRANS_BUFFER, + LWTRANCHE_GLOBAL_CSN_LOG_BUFFERS, LWTRANCHE_MULTIXACTOFFSET_BUFFER, LWTRANCHE_MULTIXACTMEMBER_BUFFER, LWTRANCHE_NOTIFY_BUFFER, diff --git a/src/include/utils/snapshot.h b/src/include/utils/snapshot.h index 4796

Re: Global snapshots

On 2020/05/12 19:24, Andrey Lepikhov wrote: Rebased onto current master (fb544735f1). Thanks for the patches! These patches are no longer applied cleanly and caused the compilation failure. So could you rebase and update them? The patches seem not to be registered in CommitFest yet. Are

Re: Global snapshots

MXACTOFFSET_BUFFERS, LWTRANCHE_MXACTMEMBER_BUFFERS, LWTRANCHE_ASYNC_BUFFERS, diff --git a/src/include/utils/snapshot.h b/src/include/utils/snapshot.h index 4796edb63a..57d2dfaa67 100644 --- a/src/include/utils/snapshot.h +++ b/src/include/utils/snapshot.h @@ -20,6 +20,9 @@ #include "storag

Re: Global snapshots

Hi! > 30 нояб. 2018 г., в 18:00, Stas Kelvich написал(а): > > > <0001-GlobalCSNLog-SLRU-v3.patch><0002-Global-snapshots-v3.patch><0003-postgres_fdw-support-for-global-snapshots-v3.patch> In spite of recent backup discussions I realized that we need to backup

Re: Global snapshots

> On 31 Jan 2019, at 18:42, Andres Freund wrote: > > Hi, > > On 2018-11-30 16:00:17 +0300, Stas Kelvich wrote: >>> On 29 Nov 2018, at 18:21, Dmitry Dolgov <9erthali...@gmail.com> wrote: >>> Is there any resulting patch where the ideas how to implement this are >>> outlined? >> >> Not yet.

Re: Global snapshots

Hi, On 2018-11-30 16:00:17 +0300, Stas Kelvich wrote: > > On 29 Nov 2018, at 18:21, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Is there any resulting patch where the ideas how to implement this are > > outlined? > > Not yet. I’m going to continue work on this in January. And probably try

Re: Global snapshots

> On Wed, Jul 25, 2018 at 1:35 PM Arseny Sher wrote: > > Hello, > > I have looked through the patches and found them pretty accurate. I'd > fixed a lot of small issues here and there; updated patchset is > attached. Hi, Thank you for working on this patch. Unfortunately, the patch has some

Re: Global snapshots

on > time of InDoubt state fix the issue? I understand that aborting > automatically is kind of dangerous... Sure, this is just a generalization of basic deadlock problem to distributed system. To deal with it, someone must periodically collect locks across the nodes, build graph, check for

Re: Global snapshots

Hi! I want to review this patch set. Though I understand that it probably will be quite long process. I like the idea that with this patch set universally all postgres instances are bound into single distributed DB, even if they never heard about each other before :) This is just amazing. Or

Re: Global snapshots

git a/src/include/utils/snapshot.h b/src/include/utils/snapshot.h index a8a5a8f4c0..318d41e6f7 100644 --- a/src/include/utils/snapshot.h +++ b/src/include/utils/snapshot.h @@ -24,6 +24,9 @@ typedef struct SnapshotData *Snapshot; #define InvalidSnapshot ((Snapshot) NULL) +typedef uint64 G

Re: Global snapshots

> On 15 May 2018, at 15:53, Robert Haas <robertmh...@gmail.com> wrote: > > Actually, I think if we're going to pursue that approach, we ought to > back off a bit from thinking about global snapshots and think about > what kind of general mechanism we want. For example, ma

Re: Global snapshots

t that thing in general. It means we don't want yours. :-) Actually, I think if we're going to pursue that approach, we ought to back off a bit from thinking about global snapshots and think about what kind of general mechanism we want. For example, maybe you can imagine it like a message bus

Re: Global snapshots

ailure. > With this patch, can we start a remote transaction at READ COMMITTED > with imported a global snapshot if the local transaction started at > READ COMMITTED? In theory it is possible, one just need to send new snapshot before each statement. With some amount of careful work it is poss

Re: Global snapshots

> On 9 May 2018, at 17:51, Robert Haas wrote: > > Ouch. That's not so bad at READ COMMITTED, but at higher isolation > levels failure becomes extremely likely. Any multi-statement > transaction that lasts longer than global_snapshot_defer_time is > pretty much doomed.

Re: Global snapshots

On Fri, May 4, 2018 at 2:11 AM, Stas Kelvich wrote: > > >> On 3 May 2018, at 18:28, Masahiko Sawada wrote: >> >> On Wed, May 2, 2018 at 1:27 AM, Stas Kelvich >> wrote: >>> 1) To achieve commit atomicity of different

Re: Global snapshots

s written by one transaction and read by another, but if there's a portion of the data that is only read and not written, or at least not written by any transactions that were active around the same time, then it's fine. While the details aren't really clear to me, I'm inclined to think that any so

Re: Global snapshots

> On 7 May 2018, at 20:04, Robert Haas wrote: > > But what happens if a transaction starts on node A at time T0 but > first touches node B at a much later time T1, such that T1 - T0 > > global_snapshot_defer_time? > Such transaction will get "global snapshot too old"

Re: Global snapshots

On Sun, May 6, 2018 at 6:22 AM, Stas Kelvich wrote: > Also each second GetSnapshotData writes globalxmin (as it was before > procArray->global_snapshot_xmin was taken into account) into a circle > buffer with a size equal to global_snapshot_defer_time value. That more >

Re: Global snapshots

> On 4 May 2018, at 22:09, Robert Haas wrote: > > So, is the idea that we'll definitely find out about any remote > transactions within 30 seconds, and then after we know about remote > transactions, we'll hold back OldestXmin some other way? Yes, kind of. There is a

Re: Global snapshots

On Tue, May 1, 2018 at 5:02 PM, Stas Kelvich wrote: > Yes, that totally possible. On both systems you need: Cool. > * set track_global_snapshots='on' -- this will start writing each > transaction commit sequence number to SRLU. > * set global_snapshot_defer_time to

Re: Global snapshots

> On 3 May 2018, at 18:28, Masahiko Sawada wrote: > > On Wed, May 2, 2018 at 1:27 AM, Stas Kelvich wrote: >> 1) To achieve commit atomicity of different nodes intermediate step is >> introduced: at first running transaction is marked as

Re: Global snapshots

On Wed, May 2, 2018 at 1:27 AM, Stas Kelvich wrote: > 1) To achieve commit atomicity of different nodes intermediate step is >introduced: at first running transaction is marked as InDoubt on all nodes, >and only after that each node commit it and stamps with a

Re: Global snapshots

> On 2 May 2018, at 05:58, Peter Eisentraut > wrote: > > On 5/1/18 12:27, Stas Kelvich wrote: >> Clock-SI is described in [5] and here I provide a small overview, which >> supposedly should be enough to catch the idea. Assume that each node runs >> Commit >>

Re: Global snapshots

On 5/1/18 12:27, Stas Kelvich wrote: > Clock-SI is described in [5] and here I provide a small overview, which > supposedly should be enough to catch the idea. Assume that each node runs > Commit > Sequence Number (CSN) based visibility: database tracks one counter for each > transaction start

Re: Global snapshots

> On 1 May 2018, at 22:43, Robert Haas wrote: > > I'm concerned about the provisioning aspect of this problem. Suppose > I have two existing database systems with, perhaps, wildly different > XID counters. On a certain date, I want to start using this system. Yes,

Re: Global snapshots

On Tue, May 1, 2018 at 12:27 PM, Stas Kelvich wrote: > Here proposed a set of patches that allow achieving proper snapshot isolation > semantics in the case of cross-node transactions. Provided infrastructure to > synchronize snapshots between different Postgres