Re: Discuss for integrate with saga and sharding-sphere
OK, I'm very interesting with the revert SQL generation and keep me update
please.
2018-06-07 11:19 GMT+08:00 Willem Jiang :
> I guess the revert SQL work in just on the roadmap.
> But shading-sphere can analyze the SQL do the shading work.
>
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Thu, Jun 7, 2018 at 11:11 AM, Zheng Feng wrote:
>
> > Can you point out to me the codes related to the SQL convert in the
> > sharding-sphere ?
> >
> > 2018-06-07 8:10 GMT+08:00 Willem Jiang :
> >
> > > Hi，
> > >
> > > Current Saga implementation cannot guarantee the data isolation. I
> guess
> > > we need to leverage the DB to provide the version of data, and revert
> it
> > > to the states before the transaction.
> > >
> > > Maybe we need hold T2 before the T1 is finished, otherwise it really
> hard
> > > to tell if the original states is x=1 or x=0.
> > >
> > > It's kind of trade off, If we want to strong consistent we have to use
> > > lock to do it, otherwise we need do introduce the semantic lock from
> the
> > > application such as the credit limits.
> > >
> > > Let's keep up the discussion and try to find a good way to solve the
> > > distribution transaction problems.
> > >
> > >
> > >
> > > Willem Jiang
> > >
> > > Twitter: willemjiang
> > > Weibo: 姜宁willem
> > >
> > > On Thu, Jun 7, 2018 at 12:33 AM, Zheng Feng wrote:
> > >
> > > > Hi Zhang Liang,
> > > >
> > > > It could be great to integrate the saga and the sharding-sphere. I'm
> > > trying
> > > > to understand the SQL convert and the data snapshot.
> > > > So I suppose that we have the simple data set just like {id = 1, x =
> 0}
> > > and
> > > > we have the updateX service to update the X
> > > >
> > > > func updateX ( int x, int i ) {
> > > > executeSQL("UPDATE x = $x WHERE id = $i", x, i);
> > > > }
> > > >
> > > > T1: T2:
> > > > saga_start(); saga_start();
> > > > updateX(1, 1);updateX(2, 1);
> > > > call_other_service(); call_other_service();
> > > > saga_end();saga_end();
> > > >
> > > > So after the T1 runs, the snapshot data should be {id = 1, x = 1} and
> > the
> > > > convertSQL should be "UPDATE x = 0 WHERE id = 1", is it right ?
> > > > Now we have the other T2 to run after T1, the snapshot data should be
> > > {id =
> > > > 1, x = 2} and convertSQL should be "UPDATE x= 1 WHERE id = 1" ?
> > > >
> > > > If the Alpha (Saga coordinator) decides to compensate the T1 when the
> > > > call_other_service() throws exception, will we call the converSQL to
> > set
> > > x
> > > > = 0 ?
> > > >
> > > > Thanks,
> > > > Zheng Feng
> > > >
> > > > 2018-06-06 15:51 GMT+08:00 Liang Zhang :
> > > >
> > > > > Hi Willem Jiang,
> > > > >
> > > > > I fully agree to keep Omega as simple as possible.
> > > > >
> > > > > Sharding-Sphere can ask transaction id from Saga, then save SQL and
> > > > > snapshot into Sharding-Sphere local db. Saga just as a transaction
> > > engine
> > > > > to determine and notify Sharding-Sphere when to forward(retry) and
> > > > > backward(compensate).
> > > > >
> > > > > Best regrads,
> > > > >
> > > > > Zhang Liang
> > > > >
> > > > >
> > > > > On 2018/06/06 06:50:31, Willem Jiang
> wrote:
> > > > > > Hi Zhangliang，
> > > > > >
> > > > > > It's great that you are interesting about the Saga project.
> > > > > > Please check out my comments in line.
> > > > > >
> > > > > >
> > > > > > Willem Jiang
> > > > > >
> > > > > > Twitter: willemjiang
> > > > > > Weibo: 姜宁willem
> > > > > >
> > > > > > On Wed, Jun 6, 2018 at 12:21 PM, zhangli...@apache.org <
> > > > > > zhangli...@apache.org> wrote:
> > > > > >
> > > > > > > Hello servicecomb devs,
> > > > > > >
> > > > > > > I want discuss for how to integrate with saga and
> > sharding-sphere.
> > > > > > >
> > > > > > > 1. Split saga as service scope and RDB scope. Service scope
> still
> > > use
> > > > > > > annotation to associate compensation method, RDB scope can
> > generate
> > > > > > > reverted SQL and data snapshot automatically. Alpha can use
> > revert
> > > > SQL
> > > > > and
> > > > > > > previous data snapshot to compensate automatically, do not need
> > end
> > > > > users
> > > > > > > provide additional cancel method.
> > > > > > >
> > > > > >
> > > > > > Current Saga doesn't touch the DB layer, I think we need to do
> > some
> > > > work
> > > > > > on the Omega.
> > > > > > Maybe we need to add a common layer handle the common co-ordinate
> > > > events,
> > > > > > and sharding-sphere
> > > > > > could do some implementation on the data layer. In this way
> > > > > > sharding-sphere could reuse the transaction
> > > > > > co-ordinate feature here.
> > > > > >
> > > > > > BTW, for other who don't know sharding-sphere, you may check out
> > the
> > > > > > github project[1] here.
> > > > > >
> > > > > > [1] https://github.com/sharding-sphere/sharding-sphere
> > > > > >
> > > > > > >
> > > > > > > 2. Sharding-Sphere can generate reverted SQL and data
Re: Discuss for integrate with saga and sharding-sphere
I guess the revert SQL work in just on the roadmap.
But shading-sphere can analyze the SQL do the shading work.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Thu, Jun 7, 2018 at 11:11 AM, Zheng Feng wrote:
> Can you point out to me the codes related to the SQL convert in the
> sharding-sphere ?
>
> 2018-06-07 8:10 GMT+08:00 Willem Jiang :
>
> > Hi，
> >
> > Current Saga implementation cannot guarantee the data isolation. I guess
> > we need to leverage the DB to provide the version of data, and revert it
> > to the states before the transaction.
> >
> > Maybe we need hold T2 before the T1 is finished, otherwise it really hard
> > to tell if the original states is x=1 or x=0.
> >
> > It's kind of trade off, If we want to strong consistent we have to use
> > lock to do it, otherwise we need do introduce the semantic lock from the
> > application such as the credit limits.
> >
> > Let's keep up the discussion and try to find a good way to solve the
> > distribution transaction problems.
> >
> >
> >
> > Willem Jiang
> >
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> > On Thu, Jun 7, 2018 at 12:33 AM, Zheng Feng wrote:
> >
> > > Hi Zhang Liang,
> > >
> > > It could be great to integrate the saga and the sharding-sphere. I'm
> > trying
> > > to understand the SQL convert and the data snapshot.
> > > So I suppose that we have the simple data set just like {id = 1, x = 0}
> > and
> > > we have the updateX service to update the X
> > >
> > > func updateX ( int x, int i ) {
> > > executeSQL("UPDATE x = $x WHERE id = $i", x, i);
> > > }
> > >
> > > T1: T2:
> > > saga_start(); saga_start();
> > > updateX(1, 1);updateX(2, 1);
> > > call_other_service(); call_other_service();
> > > saga_end();saga_end();
> > >
> > > So after the T1 runs, the snapshot data should be {id = 1, x = 1} and
> the
> > > convertSQL should be "UPDATE x = 0 WHERE id = 1", is it right ?
> > > Now we have the other T2 to run after T1, the snapshot data should be
> > {id =
> > > 1, x = 2} and convertSQL should be "UPDATE x= 1 WHERE id = 1" ?
> > >
> > > If the Alpha (Saga coordinator) decides to compensate the T1 when the
> > > call_other_service() throws exception, will we call the converSQL to
> set
> > x
> > > = 0 ?
> > >
> > > Thanks,
> > > Zheng Feng
> > >
> > > 2018-06-06 15:51 GMT+08:00 Liang Zhang :
> > >
> > > > Hi Willem Jiang,
> > > >
> > > > I fully agree to keep Omega as simple as possible.
> > > >
> > > > Sharding-Sphere can ask transaction id from Saga, then save SQL and
> > > > snapshot into Sharding-Sphere local db. Saga just as a transaction
> > engine
> > > > to determine and notify Sharding-Sphere when to forward(retry) and
> > > > backward(compensate).
> > > >
> > > > Best regrads,
> > > >
> > > > Zhang Liang
> > > >
> > > >
> > > > On 2018/06/06 06:50:31, Willem Jiang wrote:
> > > > > Hi Zhangliang，
> > > > >
> > > > > It's great that you are interesting about the Saga project.
> > > > > Please check out my comments in line.
> > > > >
> > > > >
> > > > > Willem Jiang
> > > > >
> > > > > Twitter: willemjiang
> > > > > Weibo: 姜宁willem
> > > > >
> > > > > On Wed, Jun 6, 2018 at 12:21 PM, zhangli...@apache.org <
> > > > > zhangli...@apache.org> wrote:
> > > > >
> > > > > > Hello servicecomb devs,
> > > > > >
> > > > > > I want discuss for how to integrate with saga and
> sharding-sphere.
> > > > > >
> > > > > > 1. Split saga as service scope and RDB scope. Service scope still
> > use
> > > > > > annotation to associate compensation method, RDB scope can
> generate
> > > > > > reverted SQL and data snapshot automatically. Alpha can use
> revert
> > > SQL
> > > > and
> > > > > > previous data snapshot to compensate automatically, do not need
> end
> > > > users
> > > > > > provide additional cancel method.
> > > > > >
> > > > >
> > > > > Current Saga doesn't touch the DB layer, I think we need to do
> some
> > > work
> > > > > on the Omega.
> > > > > Maybe we need to add a common layer handle the common co-ordinate
> > > events,
> > > > > and sharding-sphere
> > > > > could do some implementation on the data layer. In this way
> > > > > sharding-sphere could reuse the transaction
> > > > > co-ordinate feature here.
> > > > >
> > > > > BTW, for other who don't know sharding-sphere, you may check out
> the
> > > > > github project[1] here.
> > > > >
> > > > > [1] https://github.com/sharding-sphere/sharding-sphere
> > > > >
> > > > > >
> > > > > > 2. Sharding-Sphere can generate reverted SQL and data snapshot:
> > > > > > INSERT SQL will revert to DELETE SQL with primary key and
> sharding
> > > key.
> > > > > > UPDATE/DELETE SQL will generate SELECT SQL and save data snapshot
> > > > first,
> > > > > > and then use UPDATE OR INSERT to compensate data.
> > > > > > Saga may need to change data structure for receive SQL and data
> > > > snapshot,
> > > > > > maybe use json format to save on payload field is a good idea.
>
Re: Discuss for integrate with saga and sharding-sphere
Can you point out to me the codes related to the SQL convert in the
sharding-sphere ?
2018-06-07 8:10 GMT+08:00 Willem Jiang :
> Hi，
>
> Current Saga implementation cannot guarantee the data isolation. I guess
> we need to leverage the DB to provide the version of data, and revert it
> to the states before the transaction.
>
> Maybe we need hold T2 before the T1 is finished, otherwise it really hard
> to tell if the original states is x=1 or x=0.
>
> It's kind of trade off, If we want to strong consistent we have to use
> lock to do it, otherwise we need do introduce the semantic lock from the
> application such as the credit limits.
>
> Let's keep up the discussion and try to find a good way to solve the
> distribution transaction problems.
>
>
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Thu, Jun 7, 2018 at 12:33 AM, Zheng Feng wrote:
>
> > Hi Zhang Liang,
> >
> > It could be great to integrate the saga and the sharding-sphere. I'm
> trying
> > to understand the SQL convert and the data snapshot.
> > So I suppose that we have the simple data set just like {id = 1, x = 0}
> and
> > we have the updateX service to update the X
> >
> > func updateX ( int x, int i ) {
> > executeSQL("UPDATE x = $x WHERE id = $i", x, i);
> > }
> >
> > T1: T2:
> > saga_start(); saga_start();
> > updateX(1, 1);updateX(2, 1);
> > call_other_service(); call_other_service();
> > saga_end();saga_end();
> >
> > So after the T1 runs, the snapshot data should be {id = 1, x = 1} and the
> > convertSQL should be "UPDATE x = 0 WHERE id = 1", is it right ?
> > Now we have the other T2 to run after T1, the snapshot data should be
> {id =
> > 1, x = 2} and convertSQL should be "UPDATE x= 1 WHERE id = 1" ?
> >
> > If the Alpha (Saga coordinator) decides to compensate the T1 when the
> > call_other_service() throws exception, will we call the converSQL to set
> x
> > = 0 ?
> >
> > Thanks,
> > Zheng Feng
> >
> > 2018-06-06 15:51 GMT+08:00 Liang Zhang :
> >
> > > Hi Willem Jiang,
> > >
> > > I fully agree to keep Omega as simple as possible.
> > >
> > > Sharding-Sphere can ask transaction id from Saga, then save SQL and
> > > snapshot into Sharding-Sphere local db. Saga just as a transaction
> engine
> > > to determine and notify Sharding-Sphere when to forward(retry) and
> > > backward(compensate).
> > >
> > > Best regrads,
> > >
> > > Zhang Liang
> > >
> > >
> > > On 2018/06/06 06:50:31, Willem Jiang wrote:
> > > > Hi Zhangliang，
> > > >
> > > > It's great that you are interesting about the Saga project.
> > > > Please check out my comments in line.
> > > >
> > > >
> > > > Willem Jiang
> > > >
> > > > Twitter: willemjiang
> > > > Weibo: 姜宁willem
> > > >
> > > > On Wed, Jun 6, 2018 at 12:21 PM, zhangli...@apache.org <
> > > > zhangli...@apache.org> wrote:
> > > >
> > > > > Hello servicecomb devs,
> > > > >
> > > > > I want discuss for how to integrate with saga and sharding-sphere.
> > > > >
> > > > > 1. Split saga as service scope and RDB scope. Service scope still
> use
> > > > > annotation to associate compensation method, RDB scope can generate
> > > > > reverted SQL and data snapshot automatically. Alpha can use revert
> > SQL
> > > and
> > > > > previous data snapshot to compensate automatically, do not need end
> > > users
> > > > > provide additional cancel method.
> > > > >
> > > >
> > > > Current Saga doesn't touch the DB layer, I think we need to do some
> > work
> > > > on the Omega.
> > > > Maybe we need to add a common layer handle the common co-ordinate
> > events,
> > > > and sharding-sphere
> > > > could do some implementation on the data layer. In this way
> > > > sharding-sphere could reuse the transaction
> > > > co-ordinate feature here.
> > > >
> > > > BTW, for other who don't know sharding-sphere, you may check out the
> > > > github project[1] here.
> > > >
> > > > [1] https://github.com/sharding-sphere/sharding-sphere
> > > >
> > > > >
> > > > > 2. Sharding-Sphere can generate reverted SQL and data snapshot:
> > > > > INSERT SQL will revert to DELETE SQL with primary key and sharding
> > key.
> > > > > UPDATE/DELETE SQL will generate SELECT SQL and save data snapshot
> > > first,
> > > > > and then use UPDATE OR INSERT to compensate data.
> > > > > Saga may need to change data structure for receive SQL and data
> > > snapshot,
> > > > > maybe use json format to save on payload field is a good idea.
> > > >
> > > >
> > > > I think we need to go through the events structure[2], but if the
> Omega
> > > > hold the reference,
> > > > Alpha just need to send the revert command to Omega.
> > > > If the Sharding-Sphere can store the SQL and revert SQL into the DB,
> > > > I don't think Alpha need to know the SQL information.
> > > >
> > > > [2]
> > > > https://github.com/apache/incubator-servicecomb-saga/blob/
> > > ce71ab73ae80bc90ba59fe9b038f134fef9426b1/omega/omega-
> > > transaction/src/main/java/
Re: Discuss for integrate with saga and sharding-sphere
Hi，
Current Saga implementation cannot guarantee the data isolation. I guess
we need to leverage the DB to provide the version of data, and revert it
to the states before the transaction.
Maybe we need hold T2 before the T1 is finished, otherwise it really hard
to tell if the original states is x=1 or x=0.
It's kind of trade off, If we want to strong consistent we have to use
lock to do it, otherwise we need do introduce the semantic lock from the
application such as the credit limits.
Let's keep up the discussion and try to find a good way to solve the
distribution transaction problems.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Thu, Jun 7, 2018 at 12:33 AM, Zheng Feng wrote:
> Hi Zhang Liang,
>
> It could be great to integrate the saga and the sharding-sphere. I'm trying
> to understand the SQL convert and the data snapshot.
> So I suppose that we have the simple data set just like {id = 1, x = 0} and
> we have the updateX service to update the X
>
> func updateX ( int x, int i ) {
> executeSQL("UPDATE x = $x WHERE id = $i", x, i);
> }
>
> T1: T2:
> saga_start(); saga_start();
> updateX(1, 1);updateX(2, 1);
> call_other_service(); call_other_service();
> saga_end();saga_end();
>
> So after the T1 runs, the snapshot data should be {id = 1, x = 1} and the
> convertSQL should be "UPDATE x = 0 WHERE id = 1", is it right ?
> Now we have the other T2 to run after T1, the snapshot data should be {id =
> 1, x = 2} and convertSQL should be "UPDATE x= 1 WHERE id = 1" ?
>
> If the Alpha (Saga coordinator) decides to compensate the T1 when the
> call_other_service() throws exception, will we call the converSQL to set x
> = 0 ?
>
> Thanks,
> Zheng Feng
>
> 2018-06-06 15:51 GMT+08:00 Liang Zhang :
>
> > Hi Willem Jiang,
> >
> > I fully agree to keep Omega as simple as possible.
> >
> > Sharding-Sphere can ask transaction id from Saga, then save SQL and
> > snapshot into Sharding-Sphere local db. Saga just as a transaction engine
> > to determine and notify Sharding-Sphere when to forward(retry) and
> > backward(compensate).
> >
> > Best regrads,
> >
> > Zhang Liang
> >
> >
> > On 2018/06/06 06:50:31, Willem Jiang wrote:
> > > Hi Zhangliang，
> > >
> > > It's great that you are interesting about the Saga project.
> > > Please check out my comments in line.
> > >
> > >
> > > Willem Jiang
> > >
> > > Twitter: willemjiang
> > > Weibo: 姜宁willem
> > >
> > > On Wed, Jun 6, 2018 at 12:21 PM, zhangli...@apache.org <
> > > zhangli...@apache.org> wrote:
> > >
> > > > Hello servicecomb devs,
> > > >
> > > > I want discuss for how to integrate with saga and sharding-sphere.
> > > >
> > > > 1. Split saga as service scope and RDB scope. Service scope still use
> > > > annotation to associate compensation method, RDB scope can generate
> > > > reverted SQL and data snapshot automatically. Alpha can use revert
> SQL
> > and
> > > > previous data snapshot to compensate automatically, do not need end
> > users
> > > > provide additional cancel method.
> > > >
> > >
> > > Current Saga doesn't touch the DB layer, I think we need to do some
> work
> > > on the Omega.
> > > Maybe we need to add a common layer handle the common co-ordinate
> events,
> > > and sharding-sphere
> > > could do some implementation on the data layer. In this way
> > > sharding-sphere could reuse the transaction
> > > co-ordinate feature here.
> > >
> > > BTW, for other who don't know sharding-sphere, you may check out the
> > > github project[1] here.
> > >
> > > [1] https://github.com/sharding-sphere/sharding-sphere
> > >
> > > >
> > > > 2. Sharding-Sphere can generate reverted SQL and data snapshot:
> > > > INSERT SQL will revert to DELETE SQL with primary key and sharding
> key.
> > > > UPDATE/DELETE SQL will generate SELECT SQL and save data snapshot
> > first,
> > > > and then use UPDATE OR INSERT to compensate data.
> > > > Saga may need to change data structure for receive SQL and data
> > snapshot,
> > > > maybe use json format to save on payload field is a good idea.
> > >
> > >
> > > I think we need to go through the events structure[2], but if the Omega
> > > hold the reference,
> > > Alpha just need to send the revert command to Omega.
> > > If the Sharding-Sphere can store the SQL and revert SQL into the DB,
> > > I don't think Alpha need to know the SQL information.
> > >
> > > [2]
> > > https://github.com/apache/incubator-servicecomb-saga/blob/
> > ce71ab73ae80bc90ba59fe9b038f134fef9426b1/omega/omega-
> > transaction/src/main/java/org/apache/servicecomb/saga/omega/
> > transaction/TxEvent.java
> > >
> > >
> > > >
> > >
> > >
> > > > 3. Sharding-Sphere use guava's eventBus to post DML event.
> > Sharding-Sphere
> > > > plan to add 2 listeners, one is just save reverted SQL and snapshot,
> > other
> > > > is for send correct info to Omega.
> > > >
> > >
> > > I think we can keep Omega as simple as possible, maybe the event with
>
Re: Discuss for integrate with saga and sharding-sphere
Hi Zhang Liang,
It could be great to integrate the saga and the sharding-sphere. I'm trying
to understand the SQL convert and the data snapshot.
So I suppose that we have the simple data set just like {id = 1, x = 0} and
we have the updateX service to update the X
func updateX ( int x, int i ) {
executeSQL("UPDATE x = $x WHERE id = $i", x, i);
}
T1: T2:
saga_start(); saga_start();
updateX(1, 1);updateX(2, 1);
call_other_service(); call_other_service();
saga_end();saga_end();
So after the T1 runs, the snapshot data should be {id = 1, x = 1} and the
convertSQL should be "UPDATE x = 0 WHERE id = 1", is it right ?
Now we have the other T2 to run after T1, the snapshot data should be {id =
1, x = 2} and convertSQL should be "UPDATE x= 1 WHERE id = 1" ?
If the Alpha (Saga coordinator) decides to compensate the T1 when the
call_other_service() throws exception, will we call the converSQL to set x
= 0 ?
Thanks,
Zheng Feng
2018-06-06 15:51 GMT+08:00 Liang Zhang :
> Hi Willem Jiang,
>
> I fully agree to keep Omega as simple as possible.
>
> Sharding-Sphere can ask transaction id from Saga, then save SQL and
> snapshot into Sharding-Sphere local db. Saga just as a transaction engine
> to determine and notify Sharding-Sphere when to forward(retry) and
> backward(compensate).
>
> Best regrads,
>
> Zhang Liang
>
>
> On 2018/06/06 06:50:31, Willem Jiang wrote:
> > Hi Zhangliang，
> >
> > It's great that you are interesting about the Saga project.
> > Please check out my comments in line.
> >
> >
> > Willem Jiang
> >
> > Twitter: willemjiang
> > Weibo: 姜宁willem
> >
> > On Wed, Jun 6, 2018 at 12:21 PM, zhangli...@apache.org <
> > zhangli...@apache.org> wrote:
> >
> > > Hello servicecomb devs,
> > >
> > > I want discuss for how to integrate with saga and sharding-sphere.
> > >
> > > 1. Split saga as service scope and RDB scope. Service scope still use
> > > annotation to associate compensation method, RDB scope can generate
> > > reverted SQL and data snapshot automatically. Alpha can use revert SQL
> and
> > > previous data snapshot to compensate automatically, do not need end
> users
> > > provide additional cancel method.
> > >
> >
> > Current Saga doesn't touch the DB layer, I think we need to do some work
> > on the Omega.
> > Maybe we need to add a common layer handle the common co-ordinate events,
> > and sharding-sphere
> > could do some implementation on the data layer. In this way
> > sharding-sphere could reuse the transaction
> > co-ordinate feature here.
> >
> > BTW, for other who don't know sharding-sphere, you may check out the
> > github project[1] here.
> >
> > [1] https://github.com/sharding-sphere/sharding-sphere
> >
> > >
> > > 2. Sharding-Sphere can generate reverted SQL and data snapshot:
> > > INSERT SQL will revert to DELETE SQL with primary key and sharding key.
> > > UPDATE/DELETE SQL will generate SELECT SQL and save data snapshot
> first,
> > > and then use UPDATE OR INSERT to compensate data.
> > > Saga may need to change data structure for receive SQL and data
> snapshot,
> > > maybe use json format to save on payload field is a good idea.
> >
> >
> > I think we need to go through the events structure[2], but if the Omega
> > hold the reference,
> > Alpha just need to send the revert command to Omega.
> > If the Sharding-Sphere can store the SQL and revert SQL into the DB,
> > I don't think Alpha need to know the SQL information.
> >
> > [2]
> > https://github.com/apache/incubator-servicecomb-saga/blob/
> ce71ab73ae80bc90ba59fe9b038f134fef9426b1/omega/omega-
> transaction/src/main/java/org/apache/servicecomb/saga/omega/
> transaction/TxEvent.java
> >
> >
> > >
> >
> >
> > > 3. Sharding-Sphere use guava's eventBus to post DML event.
> Sharding-Sphere
> > > plan to add 2 listeners, one is just save reverted SQL and snapshot,
> other
> > > is for send correct info to Omega.
> > >
> >
> > I think we can keep Omega as simple as possible, maybe the event with
> > transaction id
> > could be enough.
> >
> >
> > >
> > > 4. For version of data modification, just guarantee by end user right
> now,
> > > in future we can consider about use additional version field or shadow
> > > table to handle it.
> > >
> >
> > This is the hardest part which bother me a lot. For the Saga service, we
> > cannot guarantee
> > the data isolation. The user need to do some addition work on the
> > application layer to
> > minimize the side effect.
> > Maybe we should spend some time to check the project omid[3]
> >
> > [3]https://github.com/apache/incubator-omid/
> >
> >
> >
> > >
> > > Best regards,
> > > Zhang Liang
> > >
> >
>
Re: Discuss for integrate with saga and sharding-sphere
Hi Willem Jiang, I fully agree to keep Omega as simple as possible. Sharding-Sphere can ask transaction id from Saga, then save SQL and snapshot into Sharding-Sphere local db. Saga just as a transaction engine to determine and notify Sharding-Sphere when to forward(retry) and backward(compensate). Best regrads, Zhang Liang On 2018/06/06 06:50:31, Willem Jiang wrote: > Hi Zhangliang， > > It's great that you are interesting about the Saga project. > Please check out my comments in line. > > > Willem Jiang > > Twitter: willemjiang > Weibo: 姜宁willem > > On Wed, Jun 6, 2018 at 12:21 PM, zhangli...@apache.org < > zhangli...@apache.org> wrote: > > > Hello servicecomb devs, > > > > I want discuss for how to integrate with saga and sharding-sphere. > > > > 1. Split saga as service scope and RDB scope. Service scope still use > > annotation to associate compensation method, RDB scope can generate > > reverted SQL and data snapshot automatically. Alpha can use revert SQL and > > previous data snapshot to compensate automatically, do not need end users > > provide additional cancel method. > > > > Current Saga doesn't touch the DB layer, I think we need to do some work > on the Omega. > Maybe we need to add a common layer handle the common co-ordinate events, > and sharding-sphere > could do some implementation on the data layer. In this way > sharding-sphere could reuse the transaction > co-ordinate feature here. > > BTW, for other who don't know sharding-sphere, you may check out the > github project[1] here. > > [1] https://github.com/sharding-sphere/sharding-sphere > > > > > 2. Sharding-Sphere can generate reverted SQL and data snapshot: > > INSERT SQL will revert to DELETE SQL with primary key and sharding key. > > UPDATE/DELETE SQL will generate SELECT SQL and save data snapshot first, > > and then use UPDATE OR INSERT to compensate data. > > Saga may need to change data structure for receive SQL and data snapshot, > > maybe use json format to save on payload field is a good idea. > > > I think we need to go through the events structure[2], but if the Omega > hold the reference, > Alpha just need to send the revert command to Omega. > If the Sharding-Sphere can store the SQL and revert SQL into the DB, > I don't think Alpha need to know the SQL information. > > [2] > https://github.com/apache/incubator-servicecomb-saga/blob/ce71ab73ae80bc90ba59fe9b038f134fef9426b1/omega/omega-transaction/src/main/java/org/apache/servicecomb/saga/omega/transaction/TxEvent.java > > > > > > > > 3. Sharding-Sphere use guava's eventBus to post DML event. Sharding-Sphere > > plan to add 2 listeners, one is just save reverted SQL and snapshot, other > > is for send correct info to Omega. > > > > I think we can keep Omega as simple as possible, maybe the event with > transaction id > could be enough. > > > > > > 4. For version of data modification, just guarantee by end user right now, > > in future we can consider about use additional version field or shadow > > table to handle it. > > > > This is the hardest part which bother me a lot. For the Saga service, we > cannot guarantee > the data isolation. The user need to do some addition work on the > application layer to > minimize the side effect. > Maybe we should spend some time to check the project omid[3] > > [3]https://github.com/apache/incubator-omid/ > > > > > > > Best regards, > > Zhang Liang > > >
Re: Discuss for integrate with saga and sharding-sphere
Hi Zhangliang， It's great that you are interesting about the Saga project. Please check out my comments in line. Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Wed, Jun 6, 2018 at 12:21 PM, zhangli...@apache.org < zhangli...@apache.org> wrote: > Hello servicecomb devs, > > I want discuss for how to integrate with saga and sharding-sphere. > > 1. Split saga as service scope and RDB scope. Service scope still use > annotation to associate compensation method, RDB scope can generate > reverted SQL and data snapshot automatically. Alpha can use revert SQL and > previous data snapshot to compensate automatically, do not need end users > provide additional cancel method. > Current Saga doesn't touch the DB layer, I think we need to do some work on the Omega. Maybe we need to add a common layer handle the common co-ordinate events, and sharding-sphere could do some implementation on the data layer. In this way sharding-sphere could reuse the transaction co-ordinate feature here. BTW, for other who don't know sharding-sphere, you may check out the github project[1] here. [1] https://github.com/sharding-sphere/sharding-sphere > > 2. Sharding-Sphere can generate reverted SQL and data snapshot: > INSERT SQL will revert to DELETE SQL with primary key and sharding key. > UPDATE/DELETE SQL will generate SELECT SQL and save data snapshot first, > and then use UPDATE OR INSERT to compensate data. > Saga may need to change data structure for receive SQL and data snapshot, > maybe use json format to save on payload field is a good idea. I think we need to go through the events structure[2], but if the Omega hold the reference, Alpha just need to send the revert command to Omega. If the Sharding-Sphere can store the SQL and revert SQL into the DB, I don't think Alpha need to know the SQL information. [2] https://github.com/apache/incubator-servicecomb-saga/blob/ce71ab73ae80bc90ba59fe9b038f134fef9426b1/omega/omega-transaction/src/main/java/org/apache/servicecomb/saga/omega/transaction/TxEvent.java > > 3. Sharding-Sphere use guava's eventBus to post DML event. Sharding-Sphere > plan to add 2 listeners, one is just save reverted SQL and snapshot, other > is for send correct info to Omega. > I think we can keep Omega as simple as possible, maybe the event with transaction id could be enough. > > 4. For version of data modification, just guarantee by end user right now, > in future we can consider about use additional version field or shadow > table to handle it. > This is the hardest part which bother me a lot. For the Saga service, we cannot guarantee the data isolation. The user need to do some addition work on the application layer to minimize the side effect. Maybe we should spend some time to check the project omid[3] [3]https://github.com/apache/incubator-omid/ > > Best regards, > Zhang Liang >
Re: Discuss for integrate with saga and sharding-sphere
最近在关注这个 张总有微信么 我加您：） 分布式链 冯德军 tel：15221883040 From: zhangli...@apache.org Date: 2018-06-06 12:21 To: dev Subject: Discuss for integrate with saga and sharding-sphere Hello servicecomb devs, I want discuss for how to integrate with saga and sharding-sphere. 1. Split saga as service scope and RDB scope. Service scope still use annotation to associate compensation method, RDB scope can generate reverted SQL and data snapshot automatically. Alpha can use revert SQL and previous data snapshot to compensate automatically, do not need end users provide additional cancel method. 2. Sharding-Sphere can generate reverted SQL and data snapshot: INSERT SQL will revert to DELETE SQL with primary key and sharding key. UPDATE/DELETE SQL will generate SELECT SQL and save data snapshot first, and then use UPDATE OR INSERT to compensate data. Saga may need to change data structure for receive SQL and data snapshot, maybe use json format to save on payload field is a good idea. 3. Sharding-Sphere use guava's eventBus to post DML event. Sharding-Sphere plan to add 2 listeners, one is just save reverted SQL and snapshot, other is for send correct info to Omega. 4. For version of data modification, just guarantee by end user right now, in future we can consider about use additional version field or shadow table to handle it. Best regards, Zhang Liang
Discuss for integrate with saga and sharding-sphere
Hello servicecomb devs, I want discuss for how to integrate with saga and sharding-sphere. 1. Split saga as service scope and RDB scope. Service scope still use annotation to associate compensation method, RDB scope can generate reverted SQL and data snapshot automatically. Alpha can use revert SQL and previous data snapshot to compensate automatically, do not need end users provide additional cancel method. 2. Sharding-Sphere can generate reverted SQL and data snapshot: INSERT SQL will revert to DELETE SQL with primary key and sharding key. UPDATE/DELETE SQL will generate SELECT SQL and save data snapshot first, and then use UPDATE OR INSERT to compensate data. Saga may need to change data structure for receive SQL and data snapshot, maybe use json format to save on payload field is a good idea. 3. Sharding-Sphere use guava's eventBus to post DML event. Sharding-Sphere plan to add 2 listeners, one is just save reverted SQL and snapshot, other is for send correct info to Omega. 4. For version of data modification, just guarantee by end user right now, in future we can consider about use additional version field or shadow table to handle it. Best regards, Zhang Liang
