Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Yakov, I have couple of questions regarding tests proposal.Thx

пт, 30 июн. 2017 г. в 19:17, ALEKSEY KUZNETSOV :

> Thanks! Do you think all test scenarios results, presented in table(in
> ticket comments) , are acceptable ?
>
> пт, 30 июн. 2017 г., 18:28 Yakov Zhdanov :
>
>> Alex, I have commented in the ticket. Please take a look.
>>
>> Thanks!
>> --
>> Yakov Zhdanov, Director R
>> *GridGain Systems*
>> www.gridgain.com
>>
>> 2017-06-29 17:27 GMT+03:00 ALEKSEY KUZNETSOV :
>>
>> > I've attached HangTest. I suppose it should not hang, am i right ?
>> >
>> > чт, 29 июн. 2017 г. в 14:54, ALEKSEY KUZNETSOV <
>> alkuznetsov...@gmail.com>:
>> >
>> > > Igntrs.
>> > > Im rewieving all usages of threadId of
>> > > transaction.(IgniteTxAdapter#threadID). What is the point of usage
>> > threadId
>> > > in mvcc entry ?
>> > >
>> > > пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV <
>> alkuznetsov...@gmail.com>:
>> > >
>> > >> so what do u think on my idea?
>> > >>
>> > >> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV <
>> alkuznetsov...@gmail.com>:
>> > >>
>> > >>> sorry for misleading you. We planned to support multi-node
>> > transactions,
>> > >>> but failed.
>> > >>>
>> > >>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk <
>> > >>> alexey.goncha...@gmail.com>:
>> > >>>
>> > >>> Well, now the scenario is more clear, but it has nothing to do with
>> > >>> multiple coordinators :) Let me think a little bit about it.
>> > >>>
>> > >>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV <
>> alkuznetsov...@gmail.com
>> > >:
>> > >>>
>> > >>> > so what do u think on the issue ?
>> > >>> >
>> > >>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV <
>> > alkuznetsov...@gmail.com
>> > >>> >:
>> > >>> >
>> > >>> > > Hi ! Thanks for help. I've created ticket :
>> > >>> > > https://issues.apache.org/jira/browse/IGNITE-4887
>> > >>> > > and a commit :
>> > >>> > >
>> > >>>
>> https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
>> > >>> > 436b638e5c
>> > >>> > > We really need this feature
>> > >>> > >
>> > >>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
>> > >>> > alexey.goncha...@gmail.com
>> > >>> > > >:
>> > >>> > >
>> > >>> > > Aleksey,
>> > >>> > >
>> > >>> > > I doubt your approach works as expected. Current transaction
>> > recovery
>> > >>> > > protocol heavily relies on the originating node ID in its
>> internal
>> > >>> logic.
>> > >>> > > For example, currently a transaction will be rolled back if you
>> > want
>> > >>> to
>> > >>> > > transfer a transaction ownership to another node and original tx
>> > >>> owner
>> > >>> > > fails. An attempt to commit such a transaction on another node
>> may
>> > >>> fail
>> > >>> > > with all sorts of assertions. After transaction ownership
>> changed,
>> > >>> you
>> > >>> > need
>> > >>> > > to notify all current transaction participants about this
>> change,
>> > >>> and it
>> > >>> > > should also be done failover-safe, let alone that you did not
>> add
>> > any
>> > >>> > tests
>> > >>> > > for these cases.
>> > >>> > >
>> > >>> > > I back Denis here. Please create a ticket first and come up with
>> > >>> clear
>> > >>> > > use-cases, API and protocol changes design. It is hard to reason
>> > >>> about
>> > >>> > the
>> > >>> > > changes you've made when we do not even understand why you are
>> > making
>> > >>> > these
>> > >>> > > changes and how they are supposed to work.
>> > >>> > >
>> > >>> > > --AG
>> > >>> > >
>> > >>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV <
>> > >>> alkuznetsov...@gmail.com>:
>> > >>> > >
>> > >>> > > > So, what do u think on my idea ?
>> > >>> > > >
>> > >>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
>> > >>> > alkuznetsov...@gmail.com
>> > >>> > > >:
>> > >>> > > >
>> > >>> > > > > Hi! No, i dont have ticket for this.
>> > >>> > > > > In the ticket i have implemented methods that change
>> > transaction
>> > >>> > status
>> > >>> > > > to
>> > >>> > > > > STOP, thus letting it to commit transaction in another
>> thread.
>> > In
>> > >>> > > another
>> > >>> > > > > thread you r going to restart transaction in order to commit
>> > it.
>> > >>> > > > > The mechanism behind it is obvious : we change thread id to
>> > >>> newer one
>> > >>> > > in
>> > >>> > > > > ThreadMap, and make use of serialization of txState,
>> > transactions
>> > >>> > > itself
>> > >>> > > > to
>> > >>> > > > > transfer them into another thread.
>> > >>> > > > >
>> > >>> > > > >
>> > >>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda > >:
>> > >>> > > > >
>> > >>> > > > > Aleksey,
>> > >>> > > > >
>> > >>> > > > > Do you have a ticket for this? Could you briefly list what
>> > >>> exactly
>> > >>> > was
>> > >>> > > > > done and how the things work.
>> > >>> > > > >
>> > >>> > > > > —
>> > >>> > > > > Denis
>> > >>> > > > >
>> > >>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
>> > >>> > > > alkuznetsov...@gmail.com>
>> > >>> > > 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Thanks! Do you think all test scenarios results, presented in table(in
ticket comments) , are acceptable ?

пт, 30 июн. 2017 г., 18:28 Yakov Zhdanov :

> Alex, I have commented in the ticket. Please take a look.
>
> Thanks!
> --
> Yakov Zhdanov, Director R
> *GridGain Systems*
> www.gridgain.com
>
> 2017-06-29 17:27 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > I've attached HangTest. I suppose it should not hang, am i right ?
> >
> > чт, 29 июн. 2017 г. в 14:54, ALEKSEY KUZNETSOV  >:
> >
> > > Igntrs.
> > > Im rewieving all usages of threadId of
> > > transaction.(IgniteTxAdapter#threadID). What is the point of usage
> > threadId
> > > in mvcc entry ?
> > >
> > > пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV  >:
> > >
> > >> so what do u think on my idea?
> > >>
> > >> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV  >:
> > >>
> > >>> sorry for misleading you. We planned to support multi-node
> > transactions,
> > >>> but failed.
> > >>>
> > >>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk <
> > >>> alexey.goncha...@gmail.com>:
> > >>>
> > >>> Well, now the scenario is more clear, but it has nothing to do with
> > >>> multiple coordinators :) Let me think a little bit about it.
> > >>>
> > >>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > >>>
> > >>> > so what do u think on the issue ?
> > >>> >
> > >>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > >>> >:
> > >>> >
> > >>> > > Hi ! Thanks for help. I've created ticket :
> > >>> > > https://issues.apache.org/jira/browse/IGNITE-4887
> > >>> > > and a commit :
> > >>> > >
> > >>>
> https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
> > >>> > 436b638e5c
> > >>> > > We really need this feature
> > >>> > >
> > >>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
> > >>> > alexey.goncha...@gmail.com
> > >>> > > >:
> > >>> > >
> > >>> > > Aleksey,
> > >>> > >
> > >>> > > I doubt your approach works as expected. Current transaction
> > recovery
> > >>> > > protocol heavily relies on the originating node ID in its
> internal
> > >>> logic.
> > >>> > > For example, currently a transaction will be rolled back if you
> > want
> > >>> to
> > >>> > > transfer a transaction ownership to another node and original tx
> > >>> owner
> > >>> > > fails. An attempt to commit such a transaction on another node
> may
> > >>> fail
> > >>> > > with all sorts of assertions. After transaction ownership
> changed,
> > >>> you
> > >>> > need
> > >>> > > to notify all current transaction participants about this change,
> > >>> and it
> > >>> > > should also be done failover-safe, let alone that you did not add
> > any
> > >>> > tests
> > >>> > > for these cases.
> > >>> > >
> > >>> > > I back Denis here. Please create a ticket first and come up with
> > >>> clear
> > >>> > > use-cases, API and protocol changes design. It is hard to reason
> > >>> about
> > >>> > the
> > >>> > > changes you've made when we do not even understand why you are
> > making
> > >>> > these
> > >>> > > changes and how they are supposed to work.
> > >>> > >
> > >>> > > --AG
> > >>> > >
> > >>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV <
> > >>> alkuznetsov...@gmail.com>:
> > >>> > >
> > >>> > > > So, what do u think on my idea ?
> > >>> > > >
> > >>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
> > >>> > alkuznetsov...@gmail.com
> > >>> > > >:
> > >>> > > >
> > >>> > > > > Hi! No, i dont have ticket for this.
> > >>> > > > > In the ticket i have implemented methods that change
> > transaction
> > >>> > status
> > >>> > > > to
> > >>> > > > > STOP, thus letting it to commit transaction in another
> thread.
> > In
> > >>> > > another
> > >>> > > > > thread you r going to restart transaction in order to commit
> > it.
> > >>> > > > > The mechanism behind it is obvious : we change thread id to
> > >>> newer one
> > >>> > > in
> > >>> > > > > ThreadMap, and make use of serialization of txState,
> > transactions
> > >>> > > itself
> > >>> > > > to
> > >>> > > > > transfer them into another thread.
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda  >:
> > >>> > > > >
> > >>> > > > > Aleksey,
> > >>> > > > >
> > >>> > > > > Do you have a ticket for this? Could you briefly list what
> > >>> exactly
> > >>> > was
> > >>> > > > > done and how the things work.
> > >>> > > > >
> > >>> > > > > —
> > >>> > > > > Denis
> > >>> > > > >
> > >>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
> > >>> > > > alkuznetsov...@gmail.com>
> > >>> > > > > wrote:
> > >>> > > > > >
> > >>> > > > > > Hi, Igniters! I 've made implementation of transactions of
> > >>> > non-single
> > >>> > > > > > coordinator. Here you can start transaction in one thread
> and
> > >>> > commit
> > >>> > > it
> > >>> > > > > in
> > >>> > > > > > another thread.
> > 

Re: distributed transaction of non-single coordinator

[Yakov Zhdanov]

Alex, I have commented in the ticket. Please take a look.

Thanks!
--
Yakov Zhdanov, Director R
*GridGain Systems*
www.gridgain.com

2017-06-29 17:27 GMT+03:00 ALEKSEY KUZNETSOV :

> I've attached HangTest. I suppose it should not hang, am i right ?
>
> чт, 29 июн. 2017 г. в 14:54, ALEKSEY KUZNETSOV :
>
> > Igntrs.
> > Im rewieving all usages of threadId of
> > transaction.(IgniteTxAdapter#threadID). What is the point of usage
> threadId
> > in mvcc entry ?
> >
> > пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV :
> >
> >> so what do u think on my idea?
> >>
> >> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV :
> >>
> >>> sorry for misleading you. We planned to support multi-node
> transactions,
> >>> but failed.
> >>>
> >>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk <
> >>> alexey.goncha...@gmail.com>:
> >>>
> >>> Well, now the scenario is more clear, but it has nothing to do with
> >>> multiple coordinators :) Let me think a little bit about it.
> >>>
> >>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV  >:
> >>>
> >>> > so what do u think on the issue ?
> >>> >
> >>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> >>> >:
> >>> >
> >>> > > Hi ! Thanks for help. I've created ticket :
> >>> > > https://issues.apache.org/jira/browse/IGNITE-4887
> >>> > > and a commit :
> >>> > >
> >>> https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
> >>> > 436b638e5c
> >>> > > We really need this feature
> >>> > >
> >>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
> >>> > alexey.goncha...@gmail.com
> >>> > > >:
> >>> > >
> >>> > > Aleksey,
> >>> > >
> >>> > > I doubt your approach works as expected. Current transaction
> recovery
> >>> > > protocol heavily relies on the originating node ID in its internal
> >>> logic.
> >>> > > For example, currently a transaction will be rolled back if you
> want
> >>> to
> >>> > > transfer a transaction ownership to another node and original tx
> >>> owner
> >>> > > fails. An attempt to commit such a transaction on another node may
> >>> fail
> >>> > > with all sorts of assertions. After transaction ownership changed,
> >>> you
> >>> > need
> >>> > > to notify all current transaction participants about this change,
> >>> and it
> >>> > > should also be done failover-safe, let alone that you did not add
> any
> >>> > tests
> >>> > > for these cases.
> >>> > >
> >>> > > I back Denis here. Please create a ticket first and come up with
> >>> clear
> >>> > > use-cases, API and protocol changes design. It is hard to reason
> >>> about
> >>> > the
> >>> > > changes you've made when we do not even understand why you are
> making
> >>> > these
> >>> > > changes and how they are supposed to work.
> >>> > >
> >>> > > --AG
> >>> > >
> >>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV <
> >>> alkuznetsov...@gmail.com>:
> >>> > >
> >>> > > > So, what do u think on my idea ?
> >>> > > >
> >>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
> >>> > alkuznetsov...@gmail.com
> >>> > > >:
> >>> > > >
> >>> > > > > Hi! No, i dont have ticket for this.
> >>> > > > > In the ticket i have implemented methods that change
> transaction
> >>> > status
> >>> > > > to
> >>> > > > > STOP, thus letting it to commit transaction in another thread.
> In
> >>> > > another
> >>> > > > > thread you r going to restart transaction in order to commit
> it.
> >>> > > > > The mechanism behind it is obvious : we change thread id to
> >>> newer one
> >>> > > in
> >>> > > > > ThreadMap, and make use of serialization of txState,
> transactions
> >>> > > itself
> >>> > > > to
> >>> > > > > transfer them into another thread.
> >>> > > > >
> >>> > > > >
> >>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
> >>> > > > >
> >>> > > > > Aleksey,
> >>> > > > >
> >>> > > > > Do you have a ticket for this? Could you briefly list what
> >>> exactly
> >>> > was
> >>> > > > > done and how the things work.
> >>> > > > >
> >>> > > > > —
> >>> > > > > Denis
> >>> > > > >
> >>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
> >>> > > > alkuznetsov...@gmail.com>
> >>> > > > > wrote:
> >>> > > > > >
> >>> > > > > > Hi, Igniters! I 've made implementation of transactions of
> >>> > non-single
> >>> > > > > > coordinator. Here you can start transaction in one thread and
> >>> > commit
> >>> > > it
> >>> > > > > in
> >>> > > > > > another thread.
> >>> > > > > > Take a look on it. Give your thoughts on it.
> >>> > > > > >
> >>> > > > > >
> >>> > > > > https://github.com/voipp/ignite/pull/10/commits/
> >>> > > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> >>> > > > > >
> >>> > > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
> >>> > > sergi.vlady...@gmail.com
> >>> > > > >:
> >>> > > > > >
> >>> > > > > >> You know better, go ahead! :)
> >>> > > > > >>
> >>> > > > > >> Sergi
> >>> > > > > >>
> >>> > > > > >> 2017-03-17 16:16 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

I've attached HangTest. I suppose it should not hang, am i right ?

чт, 29 июн. 2017 г. в 14:54, ALEKSEY KUZNETSOV :

> Igntrs.
> Im rewieving all usages of threadId of
> transaction.(IgniteTxAdapter#threadID). What is the point of usage threadId
> in mvcc entry ?
>
> пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV :
>
>> so what do u think on my idea?
>>
>> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV :
>>
>>> sorry for misleading you. We planned to support multi-node transactions,
>>> but failed.
>>>
>>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk <
>>> alexey.goncha...@gmail.com>:
>>>
>>> Well, now the scenario is more clear, but it has nothing to do with
>>> multiple coordinators :) Let me think a little bit about it.
>>>
>>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV :
>>>
>>> > so what do u think on the issue ?
>>> >
>>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV >> >:
>>> >
>>> > > Hi ! Thanks for help. I've created ticket :
>>> > > https://issues.apache.org/jira/browse/IGNITE-4887
>>> > > and a commit :
>>> > >
>>> https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
>>> > 436b638e5c
>>> > > We really need this feature
>>> > >
>>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
>>> > alexey.goncha...@gmail.com
>>> > > >:
>>> > >
>>> > > Aleksey,
>>> > >
>>> > > I doubt your approach works as expected. Current transaction recovery
>>> > > protocol heavily relies on the originating node ID in its internal
>>> logic.
>>> > > For example, currently a transaction will be rolled back if you want
>>> to
>>> > > transfer a transaction ownership to another node and original tx
>>> owner
>>> > > fails. An attempt to commit such a transaction on another node may
>>> fail
>>> > > with all sorts of assertions. After transaction ownership changed,
>>> you
>>> > need
>>> > > to notify all current transaction participants about this change,
>>> and it
>>> > > should also be done failover-safe, let alone that you did not add any
>>> > tests
>>> > > for these cases.
>>> > >
>>> > > I back Denis here. Please create a ticket first and come up with
>>> clear
>>> > > use-cases, API and protocol changes design. It is hard to reason
>>> about
>>> > the
>>> > > changes you've made when we do not even understand why you are making
>>> > these
>>> > > changes and how they are supposed to work.
>>> > >
>>> > > --AG
>>> > >
>>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV <
>>> alkuznetsov...@gmail.com>:
>>> > >
>>> > > > So, what do u think on my idea ?
>>> > > >
>>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
>>> > alkuznetsov...@gmail.com
>>> > > >:
>>> > > >
>>> > > > > Hi! No, i dont have ticket for this.
>>> > > > > In the ticket i have implemented methods that change transaction
>>> > status
>>> > > > to
>>> > > > > STOP, thus letting it to commit transaction in another thread. In
>>> > > another
>>> > > > > thread you r going to restart transaction in order to commit it.
>>> > > > > The mechanism behind it is obvious : we change thread id to
>>> newer one
>>> > > in
>>> > > > > ThreadMap, and make use of serialization of txState, transactions
>>> > > itself
>>> > > > to
>>> > > > > transfer them into another thread.
>>> > > > >
>>> > > > >
>>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
>>> > > > >
>>> > > > > Aleksey,
>>> > > > >
>>> > > > > Do you have a ticket for this? Could you briefly list what
>>> exactly
>>> > was
>>> > > > > done and how the things work.
>>> > > > >
>>> > > > > —
>>> > > > > Denis
>>> > > > >
>>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
>>> > > > alkuznetsov...@gmail.com>
>>> > > > > wrote:
>>> > > > > >
>>> > > > > > Hi, Igniters! I 've made implementation of transactions of
>>> > non-single
>>> > > > > > coordinator. Here you can start transaction in one thread and
>>> > commit
>>> > > it
>>> > > > > in
>>> > > > > > another thread.
>>> > > > > > Take a look on it. Give your thoughts on it.
>>> > > > > >
>>> > > > > >
>>> > > > > https://github.com/voipp/ignite/pull/10/commits/
>>> > > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
>>> > > > > >
>>> > > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
>>> > > sergi.vlady...@gmail.com
>>> > > > >:
>>> > > > > >
>>> > > > > >> You know better, go ahead! :)
>>> > > > > >>
>>> > > > > >> Sergi
>>> > > > > >>
>>> > > > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
>>> > > > alkuznetsov...@gmail.com
>>> > > > > >:
>>> > > > > >>
>>> > > > > >>> we've discovered several problems regarding your
>>> "accumulation"
>>> > > > > >>> approach.These are
>>> > > > > >>>
>>> > > > > >>>   1. perfomance issues when transfering data from temporary
>>> cache
>>> > > to
>>> > > > > >>>   permanent one. Keep in mind big deal of concurent
>>> transactions
>>> > in
>>> > > > > >>> Service
>>> > > > > >>>   commiter
>>> > > > > >>>   2. extreme 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Igntrs.
Im rewieving all usages of threadId of
transaction.(IgniteTxAdapter#threadID). What is the point of usage threadId
in mvcc entry ?

пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV :

> so what do u think on my idea?
>
> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV :
>
>> sorry for misleading you. We planned to support multi-node transactions,
>> but failed.
>>
>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk > >:
>>
>> Well, now the scenario is more clear, but it has nothing to do with
>> multiple coordinators :) Let me think a little bit about it.
>>
>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV :
>>
>> > so what do u think on the issue ?
>> >
>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV :
>> >
>> > > Hi ! Thanks for help. I've created ticket :
>> > > https://issues.apache.org/jira/browse/IGNITE-4887
>> > > and a commit :
>> > > https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
>> > 436b638e5c
>> > > We really need this feature
>> > >
>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
>> > alexey.goncha...@gmail.com
>> > > >:
>> > >
>> > > Aleksey,
>> > >
>> > > I doubt your approach works as expected. Current transaction recovery
>> > > protocol heavily relies on the originating node ID in its internal
>> logic.
>> > > For example, currently a transaction will be rolled back if you want
>> to
>> > > transfer a transaction ownership to another node and original tx owner
>> > > fails. An attempt to commit such a transaction on another node may
>> fail
>> > > with all sorts of assertions. After transaction ownership changed, you
>> > need
>> > > to notify all current transaction participants about this change, and
>> it
>> > > should also be done failover-safe, let alone that you did not add any
>> > tests
>> > > for these cases.
>> > >
>> > > I back Denis here. Please create a ticket first and come up with clear
>> > > use-cases, API and protocol changes design. It is hard to reason about
>> > the
>> > > changes you've made when we do not even understand why you are making
>> > these
>> > > changes and how they are supposed to work.
>> > >
>> > > --AG
>> > >
>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV <
>> alkuznetsov...@gmail.com>:
>> > >
>> > > > So, what do u think on my idea ?
>> > > >
>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
>> > alkuznetsov...@gmail.com
>> > > >:
>> > > >
>> > > > > Hi! No, i dont have ticket for this.
>> > > > > In the ticket i have implemented methods that change transaction
>> > status
>> > > > to
>> > > > > STOP, thus letting it to commit transaction in another thread. In
>> > > another
>> > > > > thread you r going to restart transaction in order to commit it.
>> > > > > The mechanism behind it is obvious : we change thread id to newer
>> one
>> > > in
>> > > > > ThreadMap, and make use of serialization of txState, transactions
>> > > itself
>> > > > to
>> > > > > transfer them into another thread.
>> > > > >
>> > > > >
>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
>> > > > >
>> > > > > Aleksey,
>> > > > >
>> > > > > Do you have a ticket for this? Could you briefly list what exactly
>> > was
>> > > > > done and how the things work.
>> > > > >
>> > > > > —
>> > > > > Denis
>> > > > >
>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
>> > > > alkuznetsov...@gmail.com>
>> > > > > wrote:
>> > > > > >
>> > > > > > Hi, Igniters! I 've made implementation of transactions of
>> > non-single
>> > > > > > coordinator. Here you can start transaction in one thread and
>> > commit
>> > > it
>> > > > > in
>> > > > > > another thread.
>> > > > > > Take a look on it. Give your thoughts on it.
>> > > > > >
>> > > > > >
>> > > > > https://github.com/voipp/ignite/pull/10/commits/
>> > > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
>> > > > > >
>> > > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
>> > > sergi.vlady...@gmail.com
>> > > > >:
>> > > > > >
>> > > > > >> You know better, go ahead! :)
>> > > > > >>
>> > > > > >> Sergi
>> > > > > >>
>> > > > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
>> > > > alkuznetsov...@gmail.com
>> > > > > >:
>> > > > > >>
>> > > > > >>> we've discovered several problems regarding your
>> "accumulation"
>> > > > > >>> approach.These are
>> > > > > >>>
>> > > > > >>>   1. perfomance issues when transfering data from temporary
>> cache
>> > > to
>> > > > > >>>   permanent one. Keep in mind big deal of concurent
>> transactions
>> > in
>> > > > > >>> Service
>> > > > > >>>   commiter
>> > > > > >>>   2. extreme memory load when keeping temporary cache in
>> memory
>> > > > > >>>   3. As long as user is not acquainted with ignite, working
>> with
>> > > > cache
>> > > > > >>>   must be transparent for him. Keep this in mind. User's node
>> can
>> > > > > >> evaluate
>> > > > > >>>   logic with no transaction at all, so we should 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

so what do u think on my idea?

пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV :

> sorry for misleading you. We planned to support multi-node transactions,
> but failed.
>
> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk  >:
>
> Well, now the scenario is more clear, but it has nothing to do with
> multiple coordinators :) Let me think a little bit about it.
>
> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > so what do u think on the issue ?
> >
> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV :
> >
> > > Hi ! Thanks for help. I've created ticket :
> > > https://issues.apache.org/jira/browse/IGNITE-4887
> > > and a commit :
> > > https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
> > 436b638e5c
> > > We really need this feature
> > >
> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > > >:
> > >
> > > Aleksey,
> > >
> > > I doubt your approach works as expected. Current transaction recovery
> > > protocol heavily relies on the originating node ID in its internal
> logic.
> > > For example, currently a transaction will be rolled back if you want to
> > > transfer a transaction ownership to another node and original tx owner
> > > fails. An attempt to commit such a transaction on another node may fail
> > > with all sorts of assertions. After transaction ownership changed, you
> > need
> > > to notify all current transaction participants about this change, and
> it
> > > should also be done failover-safe, let alone that you did not add any
> > tests
> > > for these cases.
> > >
> > > I back Denis here. Please create a ticket first and come up with clear
> > > use-cases, API and protocol changes design. It is hard to reason about
> > the
> > > changes you've made when we do not even understand why you are making
> > these
> > > changes and how they are supposed to work.
> > >
> > > --AG
> > >
> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > So, what do u think on my idea ?
> > > >
> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > >
> > > > > Hi! No, i dont have ticket for this.
> > > > > In the ticket i have implemented methods that change transaction
> > status
> > > > to
> > > > > STOP, thus letting it to commit transaction in another thread. In
> > > another
> > > > > thread you r going to restart transaction in order to commit it.
> > > > > The mechanism behind it is obvious : we change thread id to newer
> one
> > > in
> > > > > ThreadMap, and make use of serialization of txState, transactions
> > > itself
> > > > to
> > > > > transfer them into another thread.
> > > > >
> > > > >
> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
> > > > >
> > > > > Aleksey,
> > > > >
> > > > > Do you have a ticket for this? Could you briefly list what exactly
> > was
> > > > > done and how the things work.
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > Hi, Igniters! I 've made implementation of transactions of
> > non-single
> > > > > > coordinator. Here you can start transaction in one thread and
> > commit
> > > it
> > > > > in
> > > > > > another thread.
> > > > > > Take a look on it. Give your thoughts on it.
> > > > > >
> > > > > >
> > > > > https://github.com/voipp/ignite/pull/10/commits/
> > > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> > > > > >
> > > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
> > > sergi.vlady...@gmail.com
> > > > >:
> > > > > >
> > > > > >> You know better, go ahead! :)
> > > > > >>
> > > > > >> Sergi
> > > > > >>
> > > > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > >>
> > > > > >>> we've discovered several problems regarding your "accumulation"
> > > > > >>> approach.These are
> > > > > >>>
> > > > > >>>   1. perfomance issues when transfering data from temporary
> cache
> > > to
> > > > > >>>   permanent one. Keep in mind big deal of concurent
> transactions
> > in
> > > > > >>> Service
> > > > > >>>   commiter
> > > > > >>>   2. extreme memory load when keeping temporary cache in memory
> > > > > >>>   3. As long as user is not acquainted with ignite, working
> with
> > > > cache
> > > > > >>>   must be transparent for him. Keep this in mind. User's node
> can
> > > > > >> evaluate
> > > > > >>>   logic with no transaction at all, so we should deal with both
> > > types
> > > > > of
> > > > > >>>   execution flow : transactional and non-transactional.Another
> > one
> > > > > >>> problem is
> > > > > >>>   transaction id support at the user node. We would have
> handled
> > > all
> > > > > >> this
> > > > > >>>   issues and many more.
> > > > > >>>   4. we cannot pessimistically lock entity.
> > > > > >>>

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

sorry for misleading you. We planned to support multi-node transactions,
but failed.

пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk :

> Well, now the scenario is more clear, but it has nothing to do with
> multiple coordinators :) Let me think a little bit about it.
>
> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > so what do u think on the issue ?
> >
> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV :
> >
> > > Hi ! Thanks for help. I've created ticket :
> > > https://issues.apache.org/jira/browse/IGNITE-4887
> > > and a commit :
> > > https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
> > 436b638e5c
> > > We really need this feature
> > >
> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > > >:
> > >
> > > Aleksey,
> > >
> > > I doubt your approach works as expected. Current transaction recovery
> > > protocol heavily relies on the originating node ID in its internal
> logic.
> > > For example, currently a transaction will be rolled back if you want to
> > > transfer a transaction ownership to another node and original tx owner
> > > fails. An attempt to commit such a transaction on another node may fail
> > > with all sorts of assertions. After transaction ownership changed, you
> > need
> > > to notify all current transaction participants about this change, and
> it
> > > should also be done failover-safe, let alone that you did not add any
> > tests
> > > for these cases.
> > >
> > > I back Denis here. Please create a ticket first and come up with clear
> > > use-cases, API and protocol changes design. It is hard to reason about
> > the
> > > changes you've made when we do not even understand why you are making
> > these
> > > changes and how they are supposed to work.
> > >
> > > --AG
> > >
> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > So, what do u think on my idea ?
> > > >
> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > >
> > > > > Hi! No, i dont have ticket for this.
> > > > > In the ticket i have implemented methods that change transaction
> > status
> > > > to
> > > > > STOP, thus letting it to commit transaction in another thread. In
> > > another
> > > > > thread you r going to restart transaction in order to commit it.
> > > > > The mechanism behind it is obvious : we change thread id to newer
> one
> > > in
> > > > > ThreadMap, and make use of serialization of txState, transactions
> > > itself
> > > > to
> > > > > transfer them into another thread.
> > > > >
> > > > >
> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
> > > > >
> > > > > Aleksey,
> > > > >
> > > > > Do you have a ticket for this? Could you briefly list what exactly
> > was
> > > > > done and how the things work.
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > Hi, Igniters! I 've made implementation of transactions of
> > non-single
> > > > > > coordinator. Here you can start transaction in one thread and
> > commit
> > > it
> > > > > in
> > > > > > another thread.
> > > > > > Take a look on it. Give your thoughts on it.
> > > > > >
> > > > > >
> > > > > https://github.com/voipp/ignite/pull/10/commits/
> > > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> > > > > >
> > > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
> > > sergi.vlady...@gmail.com
> > > > >:
> > > > > >
> > > > > >> You know better, go ahead! :)
> > > > > >>
> > > > > >> Sergi
> > > > > >>
> > > > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > >>
> > > > > >>> we've discovered several problems regarding your "accumulation"
> > > > > >>> approach.These are
> > > > > >>>
> > > > > >>>   1. perfomance issues when transfering data from temporary
> cache
> > > to
> > > > > >>>   permanent one. Keep in mind big deal of concurent
> transactions
> > in
> > > > > >>> Service
> > > > > >>>   commiter
> > > > > >>>   2. extreme memory load when keeping temporary cache in memory
> > > > > >>>   3. As long as user is not acquainted with ignite, working
> with
> > > > cache
> > > > > >>>   must be transparent for him. Keep this in mind. User's node
> can
> > > > > >> evaluate
> > > > > >>>   logic with no transaction at all, so we should deal with both
> > > types
> > > > > of
> > > > > >>>   execution flow : transactional and non-transactional.Another
> > one
> > > > > >>> problem is
> > > > > >>>   transaction id support at the user node. We would have
> handled
> > > all
> > > > > >> this
> > > > > >>>   issues and many more.
> > > > > >>>   4. we cannot pessimistically lock entity.
> > > > > >>>
> > > > > >>> As a result, we decided to move on building distributed
> > > transaction.
> > > > We
> > > > > >> put

Re: distributed transaction of non-single coordinator

[Alexey Goncharuk]

Well, now the scenario is more clear, but it has nothing to do with
multiple coordinators :) Let me think a little bit about it.

2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV :

> so what do u think on the issue ?
>
> чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV :
>
> > Hi ! Thanks for help. I've created ticket :
> > https://issues.apache.org/jira/browse/IGNITE-4887
> > and a commit :
> > https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06
> 436b638e5c
> > We really need this feature
> >
> > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> >
> > Aleksey,
> >
> > I doubt your approach works as expected. Current transaction recovery
> > protocol heavily relies on the originating node ID in its internal logic.
> > For example, currently a transaction will be rolled back if you want to
> > transfer a transaction ownership to another node and original tx owner
> > fails. An attempt to commit such a transaction on another node may fail
> > with all sorts of assertions. After transaction ownership changed, you
> need
> > to notify all current transaction participants about this change, and it
> > should also be done failover-safe, let alone that you did not add any
> tests
> > for these cases.
> >
> > I back Denis here. Please create a ticket first and come up with clear
> > use-cases, API and protocol changes design. It is hard to reason about
> the
> > changes you've made when we do not even understand why you are making
> these
> > changes and how they are supposed to work.
> >
> > --AG
> >
> > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > So, what do u think on my idea ?
> > >
> > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > >
> > > > Hi! No, i dont have ticket for this.
> > > > In the ticket i have implemented methods that change transaction
> status
> > > to
> > > > STOP, thus letting it to commit transaction in another thread. In
> > another
> > > > thread you r going to restart transaction in order to commit it.
> > > > The mechanism behind it is obvious : we change thread id to newer one
> > in
> > > > ThreadMap, and make use of serialization of txState, transactions
> > itself
> > > to
> > > > transfer them into another thread.
> > > >
> > > >
> > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
> > > >
> > > > Aleksey,
> > > >
> > > > Do you have a ticket for this? Could you briefly list what exactly
> was
> > > > done and how the things work.
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Hi, Igniters! I 've made implementation of transactions of
> non-single
> > > > > coordinator. Here you can start transaction in one thread and
> commit
> > it
> > > > in
> > > > > another thread.
> > > > > Take a look on it. Give your thoughts on it.
> > > > >
> > > > >
> > > > https://github.com/voipp/ignite/pull/10/commits/
> > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> > > > >
> > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > >:
> > > > >
> > > > >> You know better, go ahead! :)
> > > > >>
> > > > >> Sergi
> > > > >>
> > > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > >>
> > > > >>> we've discovered several problems regarding your "accumulation"
> > > > >>> approach.These are
> > > > >>>
> > > > >>>   1. perfomance issues when transfering data from temporary cache
> > to
> > > > >>>   permanent one. Keep in mind big deal of concurent transactions
> in
> > > > >>> Service
> > > > >>>   commiter
> > > > >>>   2. extreme memory load when keeping temporary cache in memory
> > > > >>>   3. As long as user is not acquainted with ignite, working with
> > > cache
> > > > >>>   must be transparent for him. Keep this in mind. User's node can
> > > > >> evaluate
> > > > >>>   logic with no transaction at all, so we should deal with both
> > types
> > > > of
> > > > >>>   execution flow : transactional and non-transactional.Another
> one
> > > > >>> problem is
> > > > >>>   transaction id support at the user node. We would have handled
> > all
> > > > >> this
> > > > >>>   issues and many more.
> > > > >>>   4. we cannot pessimistically lock entity.
> > > > >>>
> > > > >>> As a result, we decided to move on building distributed
> > transaction.
> > > We
> > > > >> put
> > > > >>> aside your "accumulation" approach until we realize how to solve
> > > > >>> difficulties above .
> > > > >>>
> > > > >>> чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin <
> > > sergi.vlady...@gmail.com
> > > > >:
> > > > >>>
> > > >  The problem "How to run millions of entities, and millions of
> > > > >> operations
> > > > >>> on
> > > >  a single Pentium3" is out of scope here. Do the math, plan
> > capacity
> > > >  

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

so what do u think on the issue ?

чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV :

> Hi ! Thanks for help. I've created ticket :
> https://issues.apache.org/jira/browse/IGNITE-4887
> and a commit :
> https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06436b638e5c
> We really need this feature
>
> чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk  >:
>
> Aleksey,
>
> I doubt your approach works as expected. Current transaction recovery
> protocol heavily relies on the originating node ID in its internal logic.
> For example, currently a transaction will be rolled back if you want to
> transfer a transaction ownership to another node and original tx owner
> fails. An attempt to commit such a transaction on another node may fail
> with all sorts of assertions. After transaction ownership changed, you need
> to notify all current transaction participants about this change, and it
> should also be done failover-safe, let alone that you did not add any tests
> for these cases.
>
> I back Denis here. Please create a ticket first and come up with clear
> use-cases, API and protocol changes design. It is hard to reason about the
> changes you've made when we do not even understand why you are making these
> changes and how they are supposed to work.
>
> --AG
>
> 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > So, what do u think on my idea ?
> >
> > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV  >:
> >
> > > Hi! No, i dont have ticket for this.
> > > In the ticket i have implemented methods that change transaction status
> > to
> > > STOP, thus letting it to commit transaction in another thread. In
> another
> > > thread you r going to restart transaction in order to commit it.
> > > The mechanism behind it is obvious : we change thread id to newer one
> in
> > > ThreadMap, and make use of serialization of txState, transactions
> itself
> > to
> > > transfer them into another thread.
> > >
> > >
> > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
> > >
> > > Aleksey,
> > >
> > > Do you have a ticket for this? Could you briefly list what exactly was
> > > done and how the things work.
> > >
> > > —
> > > Denis
> > >
> > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com>
> > > wrote:
> > > >
> > > > Hi, Igniters! I 've made implementation of transactions of non-single
> > > > coordinator. Here you can start transaction in one thread and commit
> it
> > > in
> > > > another thread.
> > > > Take a look on it. Give your thoughts on it.
> > > >
> > > >
> > > https://github.com/voipp/ignite/pull/10/commits/
> > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> > > >
> > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > >
> > > >> You know better, go ahead! :)
> > > >>
> > > >> Sergi
> > > >>
> > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > >>
> > > >>> we've discovered several problems regarding your "accumulation"
> > > >>> approach.These are
> > > >>>
> > > >>>   1. perfomance issues when transfering data from temporary cache
> to
> > > >>>   permanent one. Keep in mind big deal of concurent transactions in
> > > >>> Service
> > > >>>   commiter
> > > >>>   2. extreme memory load when keeping temporary cache in memory
> > > >>>   3. As long as user is not acquainted with ignite, working with
> > cache
> > > >>>   must be transparent for him. Keep this in mind. User's node can
> > > >> evaluate
> > > >>>   logic with no transaction at all, so we should deal with both
> types
> > > of
> > > >>>   execution flow : transactional and non-transactional.Another one
> > > >>> problem is
> > > >>>   transaction id support at the user node. We would have handled
> all
> > > >> this
> > > >>>   issues and many more.
> > > >>>   4. we cannot pessimistically lock entity.
> > > >>>
> > > >>> As a result, we decided to move on building distributed
> transaction.
> > We
> > > >> put
> > > >>> aside your "accumulation" approach until we realize how to solve
> > > >>> difficulties above .
> > > >>>
> > > >>> чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > >:
> > > >>>
> > >  The problem "How to run millions of entities, and millions of
> > > >> operations
> > > >>> on
> > >  a single Pentium3" is out of scope here. Do the math, plan
> capacity
> > >  reasonably.
> > > 
> > >  Sergi
> > > 
> > >  2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > >>> :
> > > 
> > > > hmm, If we have millions of entities, and millions of operations,
> > > >> would
> > >  not
> > > > this approache lead to memory overflow and perfomance degradation
> > > >
> > > > чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin <
> > > >> sergi.vlady...@gmail.com
> > >  :
> > > >
> > > >> 1. 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Hi ! Thanks for help. I've created ticket :
https://issues.apache.org/jira/browse/IGNITE-4887
and a commit :
https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06436b638e5c
We really need this feature

чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk :

> Aleksey,
>
> I doubt your approach works as expected. Current transaction recovery
> protocol heavily relies on the originating node ID in its internal logic.
> For example, currently a transaction will be rolled back if you want to
> transfer a transaction ownership to another node and original tx owner
> fails. An attempt to commit such a transaction on another node may fail
> with all sorts of assertions. After transaction ownership changed, you need
> to notify all current transaction participants about this change, and it
> should also be done failover-safe, let alone that you did not add any tests
> for these cases.
>
> I back Denis here. Please create a ticket first and come up with clear
> use-cases, API and protocol changes design. It is hard to reason about the
> changes you've made when we do not even understand why you are making these
> changes and how they are supposed to work.
>
> --AG
>
> 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > So, what do u think on my idea ?
> >
> > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV  >:
> >
> > > Hi! No, i dont have ticket for this.
> > > In the ticket i have implemented methods that change transaction status
> > to
> > > STOP, thus letting it to commit transaction in another thread. In
> another
> > > thread you r going to restart transaction in order to commit it.
> > > The mechanism behind it is obvious : we change thread id to newer one
> in
> > > ThreadMap, and make use of serialization of txState, transactions
> itself
> > to
> > > transfer them into another thread.
> > >
> > >
> > > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
> > >
> > > Aleksey,
> > >
> > > Do you have a ticket for this? Could you briefly list what exactly was
> > > done and how the things work.
> > >
> > > —
> > > Denis
> > >
> > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com>
> > > wrote:
> > > >
> > > > Hi, Igniters! I 've made implementation of transactions of non-single
> > > > coordinator. Here you can start transaction in one thread and commit
> it
> > > in
> > > > another thread.
> > > > Take a look on it. Give your thoughts on it.
> > > >
> > > >
> > > https://github.com/voipp/ignite/pull/10/commits/
> > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> > > >
> > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > >
> > > >> You know better, go ahead! :)
> > > >>
> > > >> Sergi
> > > >>
> > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > >>
> > > >>> we've discovered several problems regarding your "accumulation"
> > > >>> approach.These are
> > > >>>
> > > >>>   1. perfomance issues when transfering data from temporary cache
> to
> > > >>>   permanent one. Keep in mind big deal of concurent transactions in
> > > >>> Service
> > > >>>   commiter
> > > >>>   2. extreme memory load when keeping temporary cache in memory
> > > >>>   3. As long as user is not acquainted with ignite, working with
> > cache
> > > >>>   must be transparent for him. Keep this in mind. User's node can
> > > >> evaluate
> > > >>>   logic with no transaction at all, so we should deal with both
> types
> > > of
> > > >>>   execution flow : transactional and non-transactional.Another one
> > > >>> problem is
> > > >>>   transaction id support at the user node. We would have handled
> all
> > > >> this
> > > >>>   issues and many more.
> > > >>>   4. we cannot pessimistically lock entity.
> > > >>>
> > > >>> As a result, we decided to move on building distributed
> transaction.
> > We
> > > >> put
> > > >>> aside your "accumulation" approach until we realize how to solve
> > > >>> difficulties above .
> > > >>>
> > > >>> чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > >:
> > > >>>
> > >  The problem "How to run millions of entities, and millions of
> > > >> operations
> > > >>> on
> > >  a single Pentium3" is out of scope here. Do the math, plan
> capacity
> > >  reasonably.
> > > 
> > >  Sergi
> > > 
> > >  2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > >>> :
> > > 
> > > > hmm, If we have millions of entities, and millions of operations,
> > > >> would
> > >  not
> > > > this approache lead to memory overflow and perfomance degradation
> > > >
> > > > чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin <
> > > >> sergi.vlady...@gmail.com
> > >  :
> > > >
> > > >> 1. Actually you have to check versions on all the values you
> have
> > > >>> read
> > > >> during the tx.
> > > >>
> > > 

Re: distributed transaction of non-single coordinator

[Alexey Goncharuk]

Aleksey,

I doubt your approach works as expected. Current transaction recovery
protocol heavily relies on the originating node ID in its internal logic.
For example, currently a transaction will be rolled back if you want to
transfer a transaction ownership to another node and original tx owner
fails. An attempt to commit such a transaction on another node may fail
with all sorts of assertions. After transaction ownership changed, you need
to notify all current transaction participants about this change, and it
should also be done failover-safe, let alone that you did not add any tests
for these cases.

I back Denis here. Please create a ticket first and come up with clear
use-cases, API and protocol changes design. It is hard to reason about the
changes you've made when we do not even understand why you are making these
changes and how they are supposed to work.

--AG

2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV :

> So, what do u think on my idea ?
>
> ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV :
>
> > Hi! No, i dont have ticket for this.
> > In the ticket i have implemented methods that change transaction status
> to
> > STOP, thus letting it to commit transaction in another thread. In another
> > thread you r going to restart transaction in order to commit it.
> > The mechanism behind it is obvious : we change thread id to newer one in
> > ThreadMap, and make use of serialization of txState, transactions itself
> to
> > transfer them into another thread.
> >
> >
> > вт, 28 мар. 2017 г. в 20:15, Denis Magda :
> >
> > Aleksey,
> >
> > Do you have a ticket for this? Could you briefly list what exactly was
> > done and how the things work.
> >
> > —
> > Denis
> >
> > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com>
> > wrote:
> > >
> > > Hi, Igniters! I 've made implementation of transactions of non-single
> > > coordinator. Here you can start transaction in one thread and commit it
> > in
> > > another thread.
> > > Take a look on it. Give your thoughts on it.
> > >
> > >
> > https://github.com/voipp/ignite/pull/10/commits/
> 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> > >
> > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin  >:
> > >
> > >> You know better, go ahead! :)
> > >>
> > >> Sergi
> > >>
> > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > >>
> > >>> we've discovered several problems regarding your "accumulation"
> > >>> approach.These are
> > >>>
> > >>>   1. perfomance issues when transfering data from temporary cache to
> > >>>   permanent one. Keep in mind big deal of concurent transactions in
> > >>> Service
> > >>>   commiter
> > >>>   2. extreme memory load when keeping temporary cache in memory
> > >>>   3. As long as user is not acquainted with ignite, working with
> cache
> > >>>   must be transparent for him. Keep this in mind. User's node can
> > >> evaluate
> > >>>   logic with no transaction at all, so we should deal with both types
> > of
> > >>>   execution flow : transactional and non-transactional.Another one
> > >>> problem is
> > >>>   transaction id support at the user node. We would have handled all
> > >> this
> > >>>   issues and many more.
> > >>>   4. we cannot pessimistically lock entity.
> > >>>
> > >>> As a result, we decided to move on building distributed transaction.
> We
> > >> put
> > >>> aside your "accumulation" approach until we realize how to solve
> > >>> difficulties above .
> > >>>
> > >>> чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > >>>
> >  The problem "How to run millions of entities, and millions of
> > >> operations
> > >>> on
> >  a single Pentium3" is out of scope here. Do the math, plan capacity
> >  reasonably.
> > 
> >  Sergi
> > 
> >  2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > >>> :
> > 
> > > hmm, If we have millions of entities, and millions of operations,
> > >> would
> >  not
> > > this approache lead to memory overflow and perfomance degradation
> > >
> > > чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin <
> > >> sergi.vlady...@gmail.com
> >  :
> > >
> > >> 1. Actually you have to check versions on all the values you have
> > >>> read
> > >> during the tx.
> > >>
> > >> For example if we have [k1 => v1, k2 => v2] and do:
> > >>
> > >> put(k1, get(k2) + 5)
> > >>
> > >> We have to remember the version for k2. This logic can be
> > >> relatively
> > > easily
> > >> encapsulated in a framework atop of Ignite. You need to implement
> > >> one
> >  to
> > >> make all this stuff usable.
> > >>
> > >> 2. I suggest to avoid any locking here, because you easily will
> end
> > >>> up
> > > with
> > >> deadlocks. If you do not have too frequent updates for your keys,
> > >> optimistic 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

So, what do u think on my idea ?

ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV :

> Hi! No, i dont have ticket for this.
> In the ticket i have implemented methods that change transaction status to
> STOP, thus letting it to commit transaction in another thread. In another
> thread you r going to restart transaction in order to commit it.
> The mechanism behind it is obvious : we change thread id to newer one in
> ThreadMap, and make use of serialization of txState, transactions itself to
> transfer them into another thread.
>
>
> вт, 28 мар. 2017 г. в 20:15, Denis Magda :
>
> Aleksey,
>
> Do you have a ticket for this? Could you briefly list what exactly was
> done and how the things work.
>
> —
> Denis
>
> > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV 
> wrote:
> >
> > Hi, Igniters! I 've made implementation of transactions of non-single
> > coordinator. Here you can start transaction in one thread and commit it
> in
> > another thread.
> > Take a look on it. Give your thoughts on it.
> >
> >
> https://github.com/voipp/ignite/pull/10/commits/3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> >
> > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin :
> >
> >> You know better, go ahead! :)
> >>
> >> Sergi
> >>
> >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV  >:
> >>
> >>> we've discovered several problems regarding your "accumulation"
> >>> approach.These are
> >>>
> >>>   1. perfomance issues when transfering data from temporary cache to
> >>>   permanent one. Keep in mind big deal of concurent transactions in
> >>> Service
> >>>   commiter
> >>>   2. extreme memory load when keeping temporary cache in memory
> >>>   3. As long as user is not acquainted with ignite, working with cache
> >>>   must be transparent for him. Keep this in mind. User's node can
> >> evaluate
> >>>   logic with no transaction at all, so we should deal with both types
> of
> >>>   execution flow : transactional and non-transactional.Another one
> >>> problem is
> >>>   transaction id support at the user node. We would have handled all
> >> this
> >>>   issues and many more.
> >>>   4. we cannot pessimistically lock entity.
> >>>
> >>> As a result, we decided to move on building distributed transaction. We
> >> put
> >>> aside your "accumulation" approach until we realize how to solve
> >>> difficulties above .
> >>>
> >>> чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin  >:
> >>>
>  The problem "How to run millions of entities, and millions of
> >> operations
> >>> on
>  a single Pentium3" is out of scope here. Do the math, plan capacity
>  reasonably.
> 
>  Sergi
> 
>  2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> >>> :
> 
> > hmm, If we have millions of entities, and millions of operations,
> >> would
>  not
> > this approache lead to memory overflow and perfomance degradation
> >
> > чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin <
> >> sergi.vlady...@gmail.com
>  :
> >
> >> 1. Actually you have to check versions on all the values you have
> >>> read
> >> during the tx.
> >>
> >> For example if we have [k1 => v1, k2 => v2] and do:
> >>
> >> put(k1, get(k2) + 5)
> >>
> >> We have to remember the version for k2. This logic can be
> >> relatively
> > easily
> >> encapsulated in a framework atop of Ignite. You need to implement
> >> one
>  to
> >> make all this stuff usable.
> >>
> >> 2. I suggest to avoid any locking here, because you easily will end
> >>> up
> > with
> >> deadlocks. If you do not have too frequent updates for your keys,
> >> optimistic approach will work just fine.
> >>
> >> Theoretically in the Committer Service you can start a thread for
> >> the
> >> lifetime of the whole distributed transaction, take a lock on the
> >> key
> > using
> >> IgniteCache.lock(K key) before executing any Services, wait for all
> >>> the
> >> services to complete, execute optimistic commit in the same thread
>  while
> >> keeping this lock and then release it. Notice that all the Ignite
> >> transactions inside of all Services must be optimistic here to be
> >>> able
>  to
> >> read this locked key.
> >>
> >> But again I do not recommend you using this approach until you
> >> have a
> >> reliable deadlock avoidance scheme.
> >>
> >> Sergi
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> 2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV <
> >>> alkuznetsov...@gmail.com
> > :
> >>
> >>> Yeah, now i got it.
> >>> There are some doubts on this approach
> >>> 1) During optimistic commit phase, when you assure no one altered
> >>> the
> >>> original values, you must check versions of other dependent keys.
> >>> How
> >> could
> >>> we 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Hi! No, i dont have ticket for this.
In the ticket i have implemented methods that change transaction status to
STOP, thus letting it to commit transaction in another thread. In another
thread you r going to restart transaction in order to commit it.
The mechanism behind it is obvious : we change thread id to newer one in
ThreadMap, and make use of serialization of txState, transactions itself to
transfer them into another thread.


вт, 28 мар. 2017 г. в 20:15, Denis Magda :

> Aleksey,
>
> Do you have a ticket for this? Could you briefly list what exactly was
> done and how the things work.
>
> —
> Denis
>
> > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV 
> wrote:
> >
> > Hi, Igniters! I 've made implementation of transactions of non-single
> > coordinator. Here you can start transaction in one thread and commit it
> in
> > another thread.
> > Take a look on it. Give your thoughts on it.
> >
> >
> https://github.com/voipp/ignite/pull/10/commits/3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> >
> > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin :
> >
> >> You know better, go ahead! :)
> >>
> >> Sergi
> >>
> >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV  >:
> >>
> >>> we've discovered several problems regarding your "accumulation"
> >>> approach.These are
> >>>
> >>>   1. perfomance issues when transfering data from temporary cache to
> >>>   permanent one. Keep in mind big deal of concurent transactions in
> >>> Service
> >>>   commiter
> >>>   2. extreme memory load when keeping temporary cache in memory
> >>>   3. As long as user is not acquainted with ignite, working with cache
> >>>   must be transparent for him. Keep this in mind. User's node can
> >> evaluate
> >>>   logic with no transaction at all, so we should deal with both types
> of
> >>>   execution flow : transactional and non-transactional.Another one
> >>> problem is
> >>>   transaction id support at the user node. We would have handled all
> >> this
> >>>   issues and many more.
> >>>   4. we cannot pessimistically lock entity.
> >>>
> >>> As a result, we decided to move on building distributed transaction. We
> >> put
> >>> aside your "accumulation" approach until we realize how to solve
> >>> difficulties above .
> >>>
> >>> чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin  >:
> >>>
>  The problem "How to run millions of entities, and millions of
> >> operations
> >>> on
>  a single Pentium3" is out of scope here. Do the math, plan capacity
>  reasonably.
> 
>  Sergi
> 
>  2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> >>> :
> 
> > hmm, If we have millions of entities, and millions of operations,
> >> would
>  not
> > this approache lead to memory overflow and perfomance degradation
> >
> > чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin <
> >> sergi.vlady...@gmail.com
>  :
> >
> >> 1. Actually you have to check versions on all the values you have
> >>> read
> >> during the tx.
> >>
> >> For example if we have [k1 => v1, k2 => v2] and do:
> >>
> >> put(k1, get(k2) + 5)
> >>
> >> We have to remember the version for k2. This logic can be
> >> relatively
> > easily
> >> encapsulated in a framework atop of Ignite. You need to implement
> >> one
>  to
> >> make all this stuff usable.
> >>
> >> 2. I suggest to avoid any locking here, because you easily will end
> >>> up
> > with
> >> deadlocks. If you do not have too frequent updates for your keys,
> >> optimistic approach will work just fine.
> >>
> >> Theoretically in the Committer Service you can start a thread for
> >> the
> >> lifetime of the whole distributed transaction, take a lock on the
> >> key
> > using
> >> IgniteCache.lock(K key) before executing any Services, wait for all
> >>> the
> >> services to complete, execute optimistic commit in the same thread
>  while
> >> keeping this lock and then release it. Notice that all the Ignite
> >> transactions inside of all Services must be optimistic here to be
> >>> able
>  to
> >> read this locked key.
> >>
> >> But again I do not recommend you using this approach until you
> >> have a
> >> reliable deadlock avoidance scheme.
> >>
> >> Sergi
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> 2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV <
> >>> alkuznetsov...@gmail.com
> > :
> >>
> >>> Yeah, now i got it.
> >>> There are some doubts on this approach
> >>> 1) During optimistic commit phase, when you assure no one altered
> >>> the
> >>> original values, you must check versions of other dependent keys.
> >>> How
> >> could
> >>> we obtain those keys(in an automative manner, of course) ?
> >>> 2) How could we lock a key before some Service A introduce
> >> 

Re: distributed transaction of non-single coordinator

[Denis Magda]

Aleksey,

Do you have a ticket for this? Could you briefly list what exactly was done and 
how the things work.

—
Denis

> On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV  
> wrote:
> 
> Hi, Igniters! I 've made implementation of transactions of non-single
> coordinator. Here you can start transaction in one thread and commit it in
> another thread.
> Take a look on it. Give your thoughts on it.
> 
> https://github.com/voipp/ignite/pull/10/commits/3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45
> 
> пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin :
> 
>> You know better, go ahead! :)
>> 
>> Sergi
>> 
>> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV :
>> 
>>> we've discovered several problems regarding your "accumulation"
>>> approach.These are
>>> 
>>>   1. perfomance issues when transfering data from temporary cache to
>>>   permanent one. Keep in mind big deal of concurent transactions in
>>> Service
>>>   commiter
>>>   2. extreme memory load when keeping temporary cache in memory
>>>   3. As long as user is not acquainted with ignite, working with cache
>>>   must be transparent for him. Keep this in mind. User's node can
>> evaluate
>>>   logic with no transaction at all, so we should deal with both types of
>>>   execution flow : transactional and non-transactional.Another one
>>> problem is
>>>   transaction id support at the user node. We would have handled all
>> this
>>>   issues and many more.
>>>   4. we cannot pessimistically lock entity.
>>> 
>>> As a result, we decided to move on building distributed transaction. We
>> put
>>> aside your "accumulation" approach until we realize how to solve
>>> difficulties above .
>>> 
>>> чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin :
>>> 
 The problem "How to run millions of entities, and millions of
>> operations
>>> on
 a single Pentium3" is out of scope here. Do the math, plan capacity
 reasonably.
 
 Sergi
 
 2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV >> :
 
> hmm, If we have millions of entities, and millions of operations,
>> would
 not
> this approache lead to memory overflow and perfomance degradation
> 
> чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin <
>> sergi.vlady...@gmail.com
 :
> 
>> 1. Actually you have to check versions on all the values you have
>>> read
>> during the tx.
>> 
>> For example if we have [k1 => v1, k2 => v2] and do:
>> 
>> put(k1, get(k2) + 5)
>> 
>> We have to remember the version for k2. This logic can be
>> relatively
> easily
>> encapsulated in a framework atop of Ignite. You need to implement
>> one
 to
>> make all this stuff usable.
>> 
>> 2. I suggest to avoid any locking here, because you easily will end
>>> up
> with
>> deadlocks. If you do not have too frequent updates for your keys,
>> optimistic approach will work just fine.
>> 
>> Theoretically in the Committer Service you can start a thread for
>> the
>> lifetime of the whole distributed transaction, take a lock on the
>> key
> using
>> IgniteCache.lock(K key) before executing any Services, wait for all
>>> the
>> services to complete, execute optimistic commit in the same thread
 while
>> keeping this lock and then release it. Notice that all the Ignite
>> transactions inside of all Services must be optimistic here to be
>>> able
 to
>> read this locked key.
>> 
>> But again I do not recommend you using this approach until you
>> have a
>> reliable deadlock avoidance scheme.
>> 
>> Sergi
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV <
>>> alkuznetsov...@gmail.com
> :
>> 
>>> Yeah, now i got it.
>>> There are some doubts on this approach
>>> 1) During optimistic commit phase, when you assure no one altered
>>> the
>>> original values, you must check versions of other dependent keys.
>>> How
>> could
>>> we obtain those keys(in an automative manner, of course) ?
>>> 2) How could we lock a key before some Service A introduce
>> changes?
 So
> no
>>> other service is allowed to change this key-value?(sort of
 pessimistic
>>> blocking)
>>> May be you know some implementations of such approach ?
>>> 
>>> ср, 15 мар. 2017 г. в 17:54, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
>>> :
>>> 
 Thank you very much for help.  I will answer later.
 
 ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin <
> sergi.vlady...@gmail.com
>>> :
 
 All the services do not update key in place, but only generate
>>> new
> keys
 augmented by otx and store the updated value in the same cache
>> +
>> remember
 the keys and versions participating in the transaction in some
> 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Hi, Igniters! I 've made implementation of transactions of non-single
coordinator. Here you can start transaction in one thread and commit it in
another thread.
Take a look on it. Give your thoughts on it.

https://github.com/voipp/ignite/pull/10/commits/3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45

пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin :

> You know better, go ahead! :)
>
> Sergi
>
> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > we've discovered several problems regarding your "accumulation"
> > approach.These are
> >
> >1. perfomance issues when transfering data from temporary cache to
> >permanent one. Keep in mind big deal of concurent transactions in
> > Service
> >commiter
> >2. extreme memory load when keeping temporary cache in memory
> >3. As long as user is not acquainted with ignite, working with cache
> >must be transparent for him. Keep this in mind. User's node can
> evaluate
> >logic with no transaction at all, so we should deal with both types of
> >execution flow : transactional and non-transactional.Another one
> > problem is
> >transaction id support at the user node. We would have handled all
> this
> >issues and many more.
> >4. we cannot pessimistically lock entity.
> >
> > As a result, we decided to move on building distributed transaction. We
> put
> > aside your "accumulation" approach until we realize how to solve
> > difficulties above .
> >
> > чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin :
> >
> > > The problem "How to run millions of entities, and millions of
> operations
> > on
> > > a single Pentium3" is out of scope here. Do the math, plan capacity
> > > reasonably.
> > >
> > > Sergi
> > >
> > > 2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > hmm, If we have millions of entities, and millions of operations,
> would
> > > not
> > > > this approache lead to memory overflow and perfomance degradation
> > > >
> > > > чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > >
> > > > > 1. Actually you have to check versions on all the values you have
> > read
> > > > > during the tx.
> > > > >
> > > > > For example if we have [k1 => v1, k2 => v2] and do:
> > > > >
> > > > > put(k1, get(k2) + 5)
> > > > >
> > > > > We have to remember the version for k2. This logic can be
> relatively
> > > > easily
> > > > > encapsulated in a framework atop of Ignite. You need to implement
> one
> > > to
> > > > > make all this stuff usable.
> > > > >
> > > > > 2. I suggest to avoid any locking here, because you easily will end
> > up
> > > > with
> > > > > deadlocks. If you do not have too frequent updates for your keys,
> > > > > optimistic approach will work just fine.
> > > > >
> > > > > Theoretically in the Committer Service you can start a thread for
> the
> > > > > lifetime of the whole distributed transaction, take a lock on the
> key
> > > > using
> > > > > IgniteCache.lock(K key) before executing any Services, wait for all
> > the
> > > > > services to complete, execute optimistic commit in the same thread
> > > while
> > > > > keeping this lock and then release it. Notice that all the Ignite
> > > > > transactions inside of all Services must be optimistic here to be
> > able
> > > to
> > > > > read this locked key.
> > > > >
> > > > > But again I do not recommend you using this approach until you
> have a
> > > > > reliable deadlock avoidance scheme.
> > > > >
> > > > > Sergi
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > 2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > > >
> > > > > > Yeah, now i got it.
> > > > > > There are some doubts on this approach
> > > > > > 1) During optimistic commit phase, when you assure no one altered
> > the
> > > > > > original values, you must check versions of other dependent keys.
> > How
> > > > > could
> > > > > > we obtain those keys(in an automative manner, of course) ?
> > > > > > 2) How could we lock a key before some Service A introduce
> changes?
> > > So
> > > > no
> > > > > > other service is allowed to change this key-value?(sort of
> > > pessimistic
> > > > > > blocking)
> > > > > > May be you know some implementations of such approach ?
> > > > > >
> > > > > > ср, 15 мар. 2017 г. в 17:54, ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > >
> > > > > > >  Thank you very much for help.  I will answer later.
> > > > > > >
> > > > > > > ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin <
> > > > sergi.vlady...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > All the services do not update key in place, but only generate
> > new
> > > > keys
> > > > > > > augmented by otx and store the updated value in the same cache
> +
> > > > > remember
> > > > > > > the keys and versions participating in the transaction in some
> > > > separate
> > > > > > > 

Re: distributed transaction of non-single coordinator

[Sergi Vladykin]

You know better, go ahead! :)

Sergi

2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV :

> we've discovered several problems regarding your "accumulation"
> approach.These are
>
>1. perfomance issues when transfering data from temporary cache to
>permanent one. Keep in mind big deal of concurent transactions in
> Service
>commiter
>2. extreme memory load when keeping temporary cache in memory
>3. As long as user is not acquainted with ignite, working with cache
>must be transparent for him. Keep this in mind. User's node can evaluate
>logic with no transaction at all, so we should deal with both types of
>execution flow : transactional and non-transactional.Another one
> problem is
>transaction id support at the user node. We would have handled all this
>issues and many more.
>4. we cannot pessimistically lock entity.
>
> As a result, we decided to move on building distributed transaction. We put
> aside your "accumulation" approach until we realize how to solve
> difficulties above .
>
> чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin :
>
> > The problem "How to run millions of entities, and millions of operations
> on
> > a single Pentium3" is out of scope here. Do the math, plan capacity
> > reasonably.
> >
> > Sergi
> >
> > 2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > hmm, If we have millions of entities, and millions of operations, would
> > not
> > > this approache lead to memory overflow and perfomance degradation
> > >
> > > чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin  >:
> > >
> > > > 1. Actually you have to check versions on all the values you have
> read
> > > > during the tx.
> > > >
> > > > For example if we have [k1 => v1, k2 => v2] and do:
> > > >
> > > > put(k1, get(k2) + 5)
> > > >
> > > > We have to remember the version for k2. This logic can be relatively
> > > easily
> > > > encapsulated in a framework atop of Ignite. You need to implement one
> > to
> > > > make all this stuff usable.
> > > >
> > > > 2. I suggest to avoid any locking here, because you easily will end
> up
> > > with
> > > > deadlocks. If you do not have too frequent updates for your keys,
> > > > optimistic approach will work just fine.
> > > >
> > > > Theoretically in the Committer Service you can start a thread for the
> > > > lifetime of the whole distributed transaction, take a lock on the key
> > > using
> > > > IgniteCache.lock(K key) before executing any Services, wait for all
> the
> > > > services to complete, execute optimistic commit in the same thread
> > while
> > > > keeping this lock and then release it. Notice that all the Ignite
> > > > transactions inside of all Services must be optimistic here to be
> able
> > to
> > > > read this locked key.
> > > >
> > > > But again I do not recommend you using this approach until you have a
> > > > reliable deadlock avoidance scheme.
> > > >
> > > > Sergi
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > 2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > Yeah, now i got it.
> > > > > There are some doubts on this approach
> > > > > 1) During optimistic commit phase, when you assure no one altered
> the
> > > > > original values, you must check versions of other dependent keys.
> How
> > > > could
> > > > > we obtain those keys(in an automative manner, of course) ?
> > > > > 2) How could we lock a key before some Service A introduce changes?
> > So
> > > no
> > > > > other service is allowed to change this key-value?(sort of
> > pessimistic
> > > > > blocking)
> > > > > May be you know some implementations of such approach ?
> > > > >
> > > > > ср, 15 мар. 2017 г. в 17:54, ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > >
> > > > > >  Thank you very much for help.  I will answer later.
> > > > > >
> > > > > > ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin <
> > > sergi.vlady...@gmail.com
> > > > >:
> > > > > >
> > > > > > All the services do not update key in place, but only generate
> new
> > > keys
> > > > > > augmented by otx and store the updated value in the same cache +
> > > > remember
> > > > > > the keys and versions participating in the transaction in some
> > > separate
> > > > > > atomic cache.
> > > > > >
> > > > > > Follow this sequence of changes applied to cache contents by each
> > > > > Service:
> > > > > >
> > > > > > Initial cache contents:
> > > > > > [k1 => v1]
> > > > > > [k2 => v2]
> > > > > > [k3 => v3]
> > > > > >
> > > > > > Cache contents after Service A:
> > > > > > [k1 => v1]
> > > > > > [k2 => v2]
> > > > > > [k3 => v3]
> > > > > > [k1x => v1a]
> > > > > > [k2x => v2a]
> > > > > >
> > > > > >  + [x => (k1 -> ver1, k2 -> ver2)] in some separate
> atomic
> > > > cache
> > > > > >
> > > > > > Cache 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

we've discovered several problems regarding your "accumulation"
approach.These are

   1. perfomance issues when transfering data from temporary cache to
   permanent one. Keep in mind big deal of concurent transactions in Service
   commiter
   2. extreme memory load when keeping temporary cache in memory
   3. As long as user is not acquainted with ignite, working with cache
   must be transparent for him. Keep this in mind. User's node can evaluate
   logic with no transaction at all, so we should deal with both types of
   execution flow : transactional and non-transactional.Another one problem is
   transaction id support at the user node. We would have handled all this
   issues and many more.
   4. we cannot pessimistically lock entity.

As a result, we decided to move on building distributed transaction. We put
aside your "accumulation" approach until we realize how to solve
difficulties above .

чт, 16 мар. 2017 г. в 16:56, Sergi Vladykin :

> The problem "How to run millions of entities, and millions of operations on
> a single Pentium3" is out of scope here. Do the math, plan capacity
> reasonably.
>
> Sergi
>
> 2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > hmm, If we have millions of entities, and millions of operations, would
> not
> > this approache lead to memory overflow and perfomance degradation
> >
> > чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin :
> >
> > > 1. Actually you have to check versions on all the values you have read
> > > during the tx.
> > >
> > > For example if we have [k1 => v1, k2 => v2] and do:
> > >
> > > put(k1, get(k2) + 5)
> > >
> > > We have to remember the version for k2. This logic can be relatively
> > easily
> > > encapsulated in a framework atop of Ignite. You need to implement one
> to
> > > make all this stuff usable.
> > >
> > > 2. I suggest to avoid any locking here, because you easily will end up
> > with
> > > deadlocks. If you do not have too frequent updates for your keys,
> > > optimistic approach will work just fine.
> > >
> > > Theoretically in the Committer Service you can start a thread for the
> > > lifetime of the whole distributed transaction, take a lock on the key
> > using
> > > IgniteCache.lock(K key) before executing any Services, wait for all the
> > > services to complete, execute optimistic commit in the same thread
> while
> > > keeping this lock and then release it. Notice that all the Ignite
> > > transactions inside of all Services must be optimistic here to be able
> to
> > > read this locked key.
> > >
> > > But again I do not recommend you using this approach until you have a
> > > reliable deadlock avoidance scheme.
> > >
> > > Sergi
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > 2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > Yeah, now i got it.
> > > > There are some doubts on this approach
> > > > 1) During optimistic commit phase, when you assure no one altered the
> > > > original values, you must check versions of other dependent keys. How
> > > could
> > > > we obtain those keys(in an automative manner, of course) ?
> > > > 2) How could we lock a key before some Service A introduce changes?
> So
> > no
> > > > other service is allowed to change this key-value?(sort of
> pessimistic
> > > > blocking)
> > > > May be you know some implementations of such approach ?
> > > >
> > > > ср, 15 мар. 2017 г. в 17:54, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > >
> > > > >  Thank you very much for help.  I will answer later.
> > > > >
> > > > > ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > >:
> > > > >
> > > > > All the services do not update key in place, but only generate new
> > keys
> > > > > augmented by otx and store the updated value in the same cache +
> > > remember
> > > > > the keys and versions participating in the transaction in some
> > separate
> > > > > atomic cache.
> > > > >
> > > > > Follow this sequence of changes applied to cache contents by each
> > > > Service:
> > > > >
> > > > > Initial cache contents:
> > > > > [k1 => v1]
> > > > > [k2 => v2]
> > > > > [k3 => v3]
> > > > >
> > > > > Cache contents after Service A:
> > > > > [k1 => v1]
> > > > > [k2 => v2]
> > > > > [k3 => v3]
> > > > > [k1x => v1a]
> > > > > [k2x => v2a]
> > > > >
> > > > >  + [x => (k1 -> ver1, k2 -> ver2)] in some separate atomic
> > > cache
> > > > >
> > > > > Cache contents after Service B:
> > > > > [k1 => v1]
> > > > > [k2 => v2]
> > > > > [k3 => v3]
> > > > > [k1x => v1a]
> > > > > [k2x => v2ab]
> > > > > [k3x => v3b]
> > > > >
> > > > > + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some
> > separate
> > > > > atomic cache
> > > > >
> > > > > Finally the Committer Service 

Re: distributed transaction of non-single coordinator

[Sergi Vladykin]

The problem "How to run millions of entities, and millions of operations on
a single Pentium3" is out of scope here. Do the math, plan capacity
reasonably.

Sergi

2017-03-16 15:54 GMT+03:00 ALEKSEY KUZNETSOV :

> hmm, If we have millions of entities, and millions of operations, would not
> this approache lead to memory overflow and perfomance degradation
>
> чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin :
>
> > 1. Actually you have to check versions on all the values you have read
> > during the tx.
> >
> > For example if we have [k1 => v1, k2 => v2] and do:
> >
> > put(k1, get(k2) + 5)
> >
> > We have to remember the version for k2. This logic can be relatively
> easily
> > encapsulated in a framework atop of Ignite. You need to implement one to
> > make all this stuff usable.
> >
> > 2. I suggest to avoid any locking here, because you easily will end up
> with
> > deadlocks. If you do not have too frequent updates for your keys,
> > optimistic approach will work just fine.
> >
> > Theoretically in the Committer Service you can start a thread for the
> > lifetime of the whole distributed transaction, take a lock on the key
> using
> > IgniteCache.lock(K key) before executing any Services, wait for all the
> > services to complete, execute optimistic commit in the same thread while
> > keeping this lock and then release it. Notice that all the Ignite
> > transactions inside of all Services must be optimistic here to be able to
> > read this locked key.
> >
> > But again I do not recommend you using this approach until you have a
> > reliable deadlock avoidance scheme.
> >
> > Sergi
> >
> >
> >
> >
> >
> >
> >
> > 2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > Yeah, now i got it.
> > > There are some doubts on this approach
> > > 1) During optimistic commit phase, when you assure no one altered the
> > > original values, you must check versions of other dependent keys. How
> > could
> > > we obtain those keys(in an automative manner, of course) ?
> > > 2) How could we lock a key before some Service A introduce changes? So
> no
> > > other service is allowed to change this key-value?(sort of pessimistic
> > > blocking)
> > > May be you know some implementations of such approach ?
> > >
> > > ср, 15 мар. 2017 г. в 17:54, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > >
> > > >  Thank you very much for help.  I will answer later.
> > > >
> > > > ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > >
> > > > All the services do not update key in place, but only generate new
> keys
> > > > augmented by otx and store the updated value in the same cache +
> > remember
> > > > the keys and versions participating in the transaction in some
> separate
> > > > atomic cache.
> > > >
> > > > Follow this sequence of changes applied to cache contents by each
> > > Service:
> > > >
> > > > Initial cache contents:
> > > > [k1 => v1]
> > > > [k2 => v2]
> > > > [k3 => v3]
> > > >
> > > > Cache contents after Service A:
> > > > [k1 => v1]
> > > > [k2 => v2]
> > > > [k3 => v3]
> > > > [k1x => v1a]
> > > > [k2x => v2a]
> > > >
> > > >  + [x => (k1 -> ver1, k2 -> ver2)] in some separate atomic
> > cache
> > > >
> > > > Cache contents after Service B:
> > > > [k1 => v1]
> > > > [k2 => v2]
> > > > [k3 => v3]
> > > > [k1x => v1a]
> > > > [k2x => v2ab]
> > > > [k3x => v3b]
> > > >
> > > > + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some
> separate
> > > > atomic cache
> > > >
> > > > Finally the Committer Service takes this map of updated keys and
> their
> > > > versions from some separate atomic cache, starts Ignite transaction
> and
> > > > replaces all the values for k* keys to values taken from k*x keys.
> The
> > > > successful result must be the following:
> > > >
> > > > [k1 => v1a]
> > > > [k2 => v2ab]
> > > > [k3 => v3b]
> > > > [k1x => v1a]
> > > > [k2x => v2ab]
> > > > [k3x => v3b]
> > > >
> > > > + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some
> separate
> > > > atomic cache
> > > >
> > > > But Committer Service also has to check that no one updated the
> > original
> > > > values before us, because otherwise we can not give any
> serializability
> > > > guarantee for these distributed transactions. Here we may need to
> check
> > > not
> > > > only versions of the updated keys, but also versions of any other
> keys
> > > end
> > > > result depends on.
> > > >
> > > > After that Committer Service has to do a cleanup (may be outside of
> the
> > > > committing tx) to come to the following final state:
> > > >
> > > > [k1 => v1a]
> > > > [k2 => v2ab]
> > > > [k3 => v3b]
> > > >
> > > > 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

hmm, If we have millions of entities, and millions of operations, would not
this approache lead to memory overflow and perfomance degradation

чт, 16 мар. 2017 г. в 15:42, Sergi Vladykin :

> 1. Actually you have to check versions on all the values you have read
> during the tx.
>
> For example if we have [k1 => v1, k2 => v2] and do:
>
> put(k1, get(k2) + 5)
>
> We have to remember the version for k2. This logic can be relatively easily
> encapsulated in a framework atop of Ignite. You need to implement one to
> make all this stuff usable.
>
> 2. I suggest to avoid any locking here, because you easily will end up with
> deadlocks. If you do not have too frequent updates for your keys,
> optimistic approach will work just fine.
>
> Theoretically in the Committer Service you can start a thread for the
> lifetime of the whole distributed transaction, take a lock on the key using
> IgniteCache.lock(K key) before executing any Services, wait for all the
> services to complete, execute optimistic commit in the same thread while
> keeping this lock and then release it. Notice that all the Ignite
> transactions inside of all Services must be optimistic here to be able to
> read this locked key.
>
> But again I do not recommend you using this approach until you have a
> reliable deadlock avoidance scheme.
>
> Sergi
>
>
>
>
>
>
>
> 2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > Yeah, now i got it.
> > There are some doubts on this approach
> > 1) During optimistic commit phase, when you assure no one altered the
> > original values, you must check versions of other dependent keys. How
> could
> > we obtain those keys(in an automative manner, of course) ?
> > 2) How could we lock a key before some Service A introduce changes? So no
> > other service is allowed to change this key-value?(sort of pessimistic
> > blocking)
> > May be you know some implementations of such approach ?
> >
> > ср, 15 мар. 2017 г. в 17:54, ALEKSEY KUZNETSOV  >:
> >
> > >  Thank you very much for help.  I will answer later.
> > >
> > > ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin  >:
> > >
> > > All the services do not update key in place, but only generate new keys
> > > augmented by otx and store the updated value in the same cache +
> remember
> > > the keys and versions participating in the transaction in some separate
> > > atomic cache.
> > >
> > > Follow this sequence of changes applied to cache contents by each
> > Service:
> > >
> > > Initial cache contents:
> > > [k1 => v1]
> > > [k2 => v2]
> > > [k3 => v3]
> > >
> > > Cache contents after Service A:
> > > [k1 => v1]
> > > [k2 => v2]
> > > [k3 => v3]
> > > [k1x => v1a]
> > > [k2x => v2a]
> > >
> > >  + [x => (k1 -> ver1, k2 -> ver2)] in some separate atomic
> cache
> > >
> > > Cache contents after Service B:
> > > [k1 => v1]
> > > [k2 => v2]
> > > [k3 => v3]
> > > [k1x => v1a]
> > > [k2x => v2ab]
> > > [k3x => v3b]
> > >
> > > + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some separate
> > > atomic cache
> > >
> > > Finally the Committer Service takes this map of updated keys and their
> > > versions from some separate atomic cache, starts Ignite transaction and
> > > replaces all the values for k* keys to values taken from k*x keys. The
> > > successful result must be the following:
> > >
> > > [k1 => v1a]
> > > [k2 => v2ab]
> > > [k3 => v3b]
> > > [k1x => v1a]
> > > [k2x => v2ab]
> > > [k3x => v3b]
> > >
> > > + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some separate
> > > atomic cache
> > >
> > > But Committer Service also has to check that no one updated the
> original
> > > values before us, because otherwise we can not give any serializability
> > > guarantee for these distributed transactions. Here we may need to check
> > not
> > > only versions of the updated keys, but also versions of any other keys
> > end
> > > result depends on.
> > >
> > > After that Committer Service has to do a cleanup (may be outside of the
> > > committing tx) to come to the following final state:
> > >
> > > [k1 => v1a]
> > > [k2 => v2ab]
> > > [k3 => v3b]
> > >
> > > Makes sense?
> > >
> > > Sergi
> > >
> > >
> > > 2017-03-15 16:54 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > >- what do u mean by saying "
> > > > *in a single transaction checks value versions for all the old values
> > > > and replaces them with calculated new ones *"? Every time you
> > change
> > > >value(in some service), you store it to *some special atomic
> cache*
> > ,
> > > so
> > > >when all services ceased working, Service commiter got a values
> with
> > > the
> > > 

Re: distributed transaction of non-single coordinator

[Sergi Vladykin]

1. Actually you have to check versions on all the values you have read
during the tx.

For example if we have [k1 => v1, k2 => v2] and do:

put(k1, get(k2) + 5)

We have to remember the version for k2. This logic can be relatively easily
encapsulated in a framework atop of Ignite. You need to implement one to
make all this stuff usable.

2. I suggest to avoid any locking here, because you easily will end up with
deadlocks. If you do not have too frequent updates for your keys,
optimistic approach will work just fine.

Theoretically in the Committer Service you can start a thread for the
lifetime of the whole distributed transaction, take a lock on the key using
IgniteCache.lock(K key) before executing any Services, wait for all the
services to complete, execute optimistic commit in the same thread while
keeping this lock and then release it. Notice that all the Ignite
transactions inside of all Services must be optimistic here to be able to
read this locked key.

But again I do not recommend you using this approach until you have a
reliable deadlock avoidance scheme.

Sergi







2017-03-16 12:53 GMT+03:00 ALEKSEY KUZNETSOV :

> Yeah, now i got it.
> There are some doubts on this approach
> 1) During optimistic commit phase, when you assure no one altered the
> original values, you must check versions of other dependent keys. How could
> we obtain those keys(in an automative manner, of course) ?
> 2) How could we lock a key before some Service A introduce changes? So no
> other service is allowed to change this key-value?(sort of pessimistic
> blocking)
> May be you know some implementations of such approach ?
>
> ср, 15 мар. 2017 г. в 17:54, ALEKSEY KUZNETSOV :
>
> >  Thank you very much for help.  I will answer later.
> >
> > ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin :
> >
> > All the services do not update key in place, but only generate new keys
> > augmented by otx and store the updated value in the same cache + remember
> > the keys and versions participating in the transaction in some separate
> > atomic cache.
> >
> > Follow this sequence of changes applied to cache contents by each
> Service:
> >
> > Initial cache contents:
> > [k1 => v1]
> > [k2 => v2]
> > [k3 => v3]
> >
> > Cache contents after Service A:
> > [k1 => v1]
> > [k2 => v2]
> > [k3 => v3]
> > [k1x => v1a]
> > [k2x => v2a]
> >
> >  + [x => (k1 -> ver1, k2 -> ver2)] in some separate atomic cache
> >
> > Cache contents after Service B:
> > [k1 => v1]
> > [k2 => v2]
> > [k3 => v3]
> > [k1x => v1a]
> > [k2x => v2ab]
> > [k3x => v3b]
> >
> > + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some separate
> > atomic cache
> >
> > Finally the Committer Service takes this map of updated keys and their
> > versions from some separate atomic cache, starts Ignite transaction and
> > replaces all the values for k* keys to values taken from k*x keys. The
> > successful result must be the following:
> >
> > [k1 => v1a]
> > [k2 => v2ab]
> > [k3 => v3b]
> > [k1x => v1a]
> > [k2x => v2ab]
> > [k3x => v3b]
> >
> > + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some separate
> > atomic cache
> >
> > But Committer Service also has to check that no one updated the original
> > values before us, because otherwise we can not give any serializability
> > guarantee for these distributed transactions. Here we may need to check
> not
> > only versions of the updated keys, but also versions of any other keys
> end
> > result depends on.
> >
> > After that Committer Service has to do a cleanup (may be outside of the
> > committing tx) to come to the following final state:
> >
> > [k1 => v1a]
> > [k2 => v2ab]
> > [k3 => v3b]
> >
> > Makes sense?
> >
> > Sergi
> >
> >
> > 2017-03-15 16:54 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > >- what do u mean by saying "
> > > *in a single transaction checks value versions for all the old values
> > > and replaces them with calculated new ones *"? Every time you
> change
> > >value(in some service), you store it to *some special atomic cache*
> ,
> > so
> > >when all services ceased working, Service commiter got a values with
> > the
> > >last versions.
> > >- After "*does cleanup of temporary keys and values*" Service
> commiter
> > >persists them into permanent store, isn't it ?
> > >- I cant grasp your though, you say "*in case of version mismatch or
> > TX
> > >timeout just rollbacks*". But what versions would it match?
> > >
> > >
> > > ср, 15 мар. 2017 г. в 15:34, Sergi Vladykin  >:
> > >
> > > > Ok, here is what you actually need to implement at the application

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Yeah, now i got it.
There are some doubts on this approach
1) During optimistic commit phase, when you assure no one altered the
original values, you must check versions of other dependent keys. How could
we obtain those keys(in an automative manner, of course) ?
2) How could we lock a key before some Service A introduce changes? So no
other service is allowed to change this key-value?(sort of pessimistic
blocking)
May be you know some implementations of such approach ?

ср, 15 мар. 2017 г. в 17:54, ALEKSEY KUZNETSOV :

>  Thank you very much for help.  I will answer later.
>
> ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin :
>
> All the services do not update key in place, but only generate new keys
> augmented by otx and store the updated value in the same cache + remember
> the keys and versions participating in the transaction in some separate
> atomic cache.
>
> Follow this sequence of changes applied to cache contents by each Service:
>
> Initial cache contents:
> [k1 => v1]
> [k2 => v2]
> [k3 => v3]
>
> Cache contents after Service A:
> [k1 => v1]
> [k2 => v2]
> [k3 => v3]
> [k1x => v1a]
> [k2x => v2a]
>
>  + [x => (k1 -> ver1, k2 -> ver2)] in some separate atomic cache
>
> Cache contents after Service B:
> [k1 => v1]
> [k2 => v2]
> [k3 => v3]
> [k1x => v1a]
> [k2x => v2ab]
> [k3x => v3b]
>
> + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some separate
> atomic cache
>
> Finally the Committer Service takes this map of updated keys and their
> versions from some separate atomic cache, starts Ignite transaction and
> replaces all the values for k* keys to values taken from k*x keys. The
> successful result must be the following:
>
> [k1 => v1a]
> [k2 => v2ab]
> [k3 => v3b]
> [k1x => v1a]
> [k2x => v2ab]
> [k3x => v3b]
>
> + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some separate
> atomic cache
>
> But Committer Service also has to check that no one updated the original
> values before us, because otherwise we can not give any serializability
> guarantee for these distributed transactions. Here we may need to check not
> only versions of the updated keys, but also versions of any other keys end
> result depends on.
>
> After that Committer Service has to do a cleanup (may be outside of the
> committing tx) to come to the following final state:
>
> [k1 => v1a]
> [k2 => v2ab]
> [k3 => v3b]
>
> Makes sense?
>
> Sergi
>
>
> 2017-03-15 16:54 GMT+03:00 ALEKSEY KUZNETSOV :
>
> >- what do u mean by saying "
> > *in a single transaction checks value versions for all the old values
> > and replaces them with calculated new ones *"? Every time you change
> >value(in some service), you store it to *some special atomic cache* ,
> so
> >when all services ceased working, Service commiter got a values with
> the
> >last versions.
> >- After "*does cleanup of temporary keys and values*" Service commiter
> >persists them into permanent store, isn't it ?
> >- I cant grasp your though, you say "*in case of version mismatch or
> TX
> >timeout just rollbacks*". But what versions would it match?
> >
> >
> > ср, 15 мар. 2017 г. в 15:34, Sergi Vladykin :
> >
> > > Ok, here is what you actually need to implement at the application
> level.
> > >
> > > Lets say we have to call 2 services in the following order:
> > >  - Service A: wants to update keys [k1 => v1,   k2 => v2]  to  [k1 =>
> > v1a,
> > >   k2 => v2a]
> > >  - Service B: wants to update keys [k2 => v2a, k3 => v3]  to  [k2 =>
> > v2ab,
> > > k3 => v3b]
> > >
> > > The change
> > > from [ k1 => v1,   k2 => v2, k3 => v3   ]
> > > to [ k1 => v1a, k2 => v2ab, k3 => v3b ]
> > > must happen in a single transaction.
> > >
> > >
> > > Optimistic protocol to solve this:
> > >
> > > Each cache key must have a field `otx`, which is a unique orchestrator
> TX
> > > identifier - it must be a parameter passed to all the services. If
> `otx`
> > is
> > > set to some value it means that it is an intermediate key and is
> visible
> > > only inside of some transaction, for the finalized key `otx` must be
> > null -
> > > it means the key is committed and visible for everyone.
> > >
> > > Each cache value must have a field `ver` which is a version of that
> > value.
> > >
> > > For both fields (`otx` and `ver`) the safest way is to use UUID.
> > >
> > > Workflow is the following:
> > >
> > > Orchestrator starts the distributed transaction with `otx` = x and
> passes
> > > this parameter to all the services.
> > >
> > > Service A:
> > >  - does some computations
> > >  - stores [k1x => v1a, k2x => v2a]  with TTL = Za
> > >   where

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

 Thank you very much for help.  I will answer later.

ср, 15 мар. 2017 г. в 17:39, Sergi Vladykin :

> All the services do not update key in place, but only generate new keys
> augmented by otx and store the updated value in the same cache + remember
> the keys and versions participating in the transaction in some separate
> atomic cache.
>
> Follow this sequence of changes applied to cache contents by each Service:
>
> Initial cache contents:
> [k1 => v1]
> [k2 => v2]
> [k3 => v3]
>
> Cache contents after Service A:
> [k1 => v1]
> [k2 => v2]
> [k3 => v3]
> [k1x => v1a]
> [k2x => v2a]
>
>  + [x => (k1 -> ver1, k2 -> ver2)] in some separate atomic cache
>
> Cache contents after Service B:
> [k1 => v1]
> [k2 => v2]
> [k3 => v3]
> [k1x => v1a]
> [k2x => v2ab]
> [k3x => v3b]
>
> + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some separate
> atomic cache
>
> Finally the Committer Service takes this map of updated keys and their
> versions from some separate atomic cache, starts Ignite transaction and
> replaces all the values for k* keys to values taken from k*x keys. The
> successful result must be the following:
>
> [k1 => v1a]
> [k2 => v2ab]
> [k3 => v3b]
> [k1x => v1a]
> [k2x => v2ab]
> [k3x => v3b]
>
> + [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)] in some separate
> atomic cache
>
> But Committer Service also has to check that no one updated the original
> values before us, because otherwise we can not give any serializability
> guarantee for these distributed transactions. Here we may need to check not
> only versions of the updated keys, but also versions of any other keys end
> result depends on.
>
> After that Committer Service has to do a cleanup (may be outside of the
> committing tx) to come to the following final state:
>
> [k1 => v1a]
> [k2 => v2ab]
> [k3 => v3b]
>
> Makes sense?
>
> Sergi
>
>
> 2017-03-15 16:54 GMT+03:00 ALEKSEY KUZNETSOV :
>
> >- what do u mean by saying "
> > *in a single transaction checks value versions for all the old values
> > and replaces them with calculated new ones *"? Every time you change
> >value(in some service), you store it to *some special atomic cache* ,
> so
> >when all services ceased working, Service commiter got a values with
> the
> >last versions.
> >- After "*does cleanup of temporary keys and values*" Service commiter
> >persists them into permanent store, isn't it ?
> >- I cant grasp your though, you say "*in case of version mismatch or
> TX
> >timeout just rollbacks*". But what versions would it match?
> >
> >
> > ср, 15 мар. 2017 г. в 15:34, Sergi Vladykin :
> >
> > > Ok, here is what you actually need to implement at the application
> level.
> > >
> > > Lets say we have to call 2 services in the following order:
> > >  - Service A: wants to update keys [k1 => v1,   k2 => v2]  to  [k1 =>
> > v1a,
> > >   k2 => v2a]
> > >  - Service B: wants to update keys [k2 => v2a, k3 => v3]  to  [k2 =>
> > v2ab,
> > > k3 => v3b]
> > >
> > > The change
> > > from [ k1 => v1,   k2 => v2, k3 => v3   ]
> > > to [ k1 => v1a, k2 => v2ab, k3 => v3b ]
> > > must happen in a single transaction.
> > >
> > >
> > > Optimistic protocol to solve this:
> > >
> > > Each cache key must have a field `otx`, which is a unique orchestrator
> TX
> > > identifier - it must be a parameter passed to all the services. If
> `otx`
> > is
> > > set to some value it means that it is an intermediate key and is
> visible
> > > only inside of some transaction, for the finalized key `otx` must be
> > null -
> > > it means the key is committed and visible for everyone.
> > >
> > > Each cache value must have a field `ver` which is a version of that
> > value.
> > >
> > > For both fields (`otx` and `ver`) the safest way is to use UUID.
> > >
> > > Workflow is the following:
> > >
> > > Orchestrator starts the distributed transaction with `otx` = x and
> passes
> > > this parameter to all the services.
> > >
> > > Service A:
> > >  - does some computations
> > >  - stores [k1x => v1a, k2x => v2a]  with TTL = Za
> > >   where
> > >   Za - left time from max Orchestrator TX duration after
> Service
> > A
> > > end
> > >   k1x, k2x - new temporary keys with field `otx` = x
> > >   v2a has updated version `ver`
> > >  - returns a set of updated keys and all the old versions to the
> > > orchestrator
> > >or just stores it in some special atomic cache like
> > >[x => (k1 -> ver1, k2 -> ver2)] TTL = Za
> > >
> > > Service B:
> > >  - retrieves the updated value k2x => v2a because it knows `otx` = x
> > >  - does computations
> > >  - 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

   - what do u mean by saying "
*in a single transaction checks value versions for all the old values
and replaces them with calculated new ones *"? Every time you change
   value(in some service), you store it to *some special atomic cache* , so
   when all services ceased working, Service commiter got a values with the
   last versions.
   - After "*does cleanup of temporary keys and values*" Service commiter
   persists them into permanent store, isn't it ?
   - I cant grasp your though, you say "*in case of version mismatch or TX
   timeout just rollbacks*". But what versions would it match?


ср, 15 мар. 2017 г. в 15:34, Sergi Vladykin :

> Ok, here is what you actually need to implement at the application level.
>
> Lets say we have to call 2 services in the following order:
>  - Service A: wants to update keys [k1 => v1,   k2 => v2]  to  [k1 => v1a,
>   k2 => v2a]
>  - Service B: wants to update keys [k2 => v2a, k3 => v3]  to  [k2 => v2ab,
> k3 => v3b]
>
> The change
> from [ k1 => v1,   k2 => v2, k3 => v3   ]
> to [ k1 => v1a, k2 => v2ab, k3 => v3b ]
> must happen in a single transaction.
>
>
> Optimistic protocol to solve this:
>
> Each cache key must have a field `otx`, which is a unique orchestrator TX
> identifier - it must be a parameter passed to all the services. If `otx` is
> set to some value it means that it is an intermediate key and is visible
> only inside of some transaction, for the finalized key `otx` must be null -
> it means the key is committed and visible for everyone.
>
> Each cache value must have a field `ver` which is a version of that value.
>
> For both fields (`otx` and `ver`) the safest way is to use UUID.
>
> Workflow is the following:
>
> Orchestrator starts the distributed transaction with `otx` = x and passes
> this parameter to all the services.
>
> Service A:
>  - does some computations
>  - stores [k1x => v1a, k2x => v2a]  with TTL = Za
>   where
>   Za - left time from max Orchestrator TX duration after Service A
> end
>   k1x, k2x - new temporary keys with field `otx` = x
>   v2a has updated version `ver`
>  - returns a set of updated keys and all the old versions to the
> orchestrator
>or just stores it in some special atomic cache like
>[x => (k1 -> ver1, k2 -> ver2)] TTL = Za
>
> Service B:
>  - retrieves the updated value k2x => v2a because it knows `otx` = x
>  - does computations
>  - stores [k2x => v2ab, k3x => v3b] TTL = Zb
>  - updates the set of updated keys like [x => (k1 -> ver1, k2 -> ver2, k3
> -> ver3)] TTL = Zb
>
> Service Committer (may be embedded into Orchestrator):
>  - takes all the updated keys and versions for `otx` = x
>[x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)]
>  - in a single transaction checks value versions for all the old values
>and replaces them with calculated new ones
>  - does cleanup of temporary keys and values
>  - in case of version mismatch or TX timeout just rollbacks and signals
> to Orchestrator to restart the job with new `otx`
>
> PROFIT!!
>
> This approach even allows you to run independent parts of the graph in
> parallel (with TX transfer you will always run only one at a time). Also it
> does not require inventing any special fault tolerance technics because
> Ignite caches are already fault tolerant and all the intermediate results
> are virtually invisible and stored with TTL, thus in case of any crash you
> will not have inconsistent state or garbage.
>
> Sergi
>
>
> 2017-03-15 11:42 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > Okay, we are open for proposals on business task. I mean, we can make use
> > of some other thing, not distributed transaction. Not transaction yet.
> >
> > ср, 15 мар. 2017 г. в 11:24, Vladimir Ozerov :
> >
> > > IMO the use case makes sense. However, as Sergi already mentioned, the
> > > problem is far more complex, than simply passing TX state over a wire.
> > Most
> > > probably a kind of coordinator will be required still to manage all
> kinds
> > > of failures. This task should be started with clean design proposal
> > > explaining how we handle all these concurrent events. And only then,
> when
> > > we understand all implications, we should move to development stage.
> > >
> > > On Wed, Mar 15, 2017 at 10:38 AM, ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com> wrote:
> > >
> > > > Right
> > > >
> > > > ср, 15 мар. 2017 г. в 10:35, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > >
> > > > > Good! Basically your orchestrator just takes some predefined graph
> of
> > > > > distributed services to be invoked, calls them by some kind of RPC
> > and
> > > > > passes the needed parameters between them, right?
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > > >
> > > > > > orchestrator is a custom thing. He is responsible for managing
> 

Re: distributed transaction of non-single coordinator

[Sergi Vladykin]

Ok, here is what you actually need to implement at the application level.

Lets say we have to call 2 services in the following order:
 - Service A: wants to update keys [k1 => v1,   k2 => v2]  to  [k1 => v1a,
  k2 => v2a]
 - Service B: wants to update keys [k2 => v2a, k3 => v3]  to  [k2 => v2ab,
k3 => v3b]

The change
from [ k1 => v1,   k2 => v2, k3 => v3   ]
to [ k1 => v1a, k2 => v2ab, k3 => v3b ]
must happen in a single transaction.


Optimistic protocol to solve this:

Each cache key must have a field `otx`, which is a unique orchestrator TX
identifier - it must be a parameter passed to all the services. If `otx` is
set to some value it means that it is an intermediate key and is visible
only inside of some transaction, for the finalized key `otx` must be null -
it means the key is committed and visible for everyone.

Each cache value must have a field `ver` which is a version of that value.

For both fields (`otx` and `ver`) the safest way is to use UUID.

Workflow is the following:

Orchestrator starts the distributed transaction with `otx` = x and passes
this parameter to all the services.

Service A:
 - does some computations
 - stores [k1x => v1a, k2x => v2a]  with TTL = Za
  where
  Za - left time from max Orchestrator TX duration after Service A
end
  k1x, k2x - new temporary keys with field `otx` = x
  v2a has updated version `ver`
 - returns a set of updated keys and all the old versions to the
orchestrator
   or just stores it in some special atomic cache like
   [x => (k1 -> ver1, k2 -> ver2)] TTL = Za

Service B:
 - retrieves the updated value k2x => v2a because it knows `otx` = x
 - does computations
 - stores [k2x => v2ab, k3x => v3b] TTL = Zb
 - updates the set of updated keys like [x => (k1 -> ver1, k2 -> ver2, k3
-> ver3)] TTL = Zb

Service Committer (may be embedded into Orchestrator):
 - takes all the updated keys and versions for `otx` = x
   [x => (k1 -> ver1, k2 -> ver2, k3 -> ver3)]
 - in a single transaction checks value versions for all the old values
   and replaces them with calculated new ones
 - does cleanup of temporary keys and values
 - in case of version mismatch or TX timeout just rollbacks and signals
to Orchestrator to restart the job with new `otx`

PROFIT!!

This approach even allows you to run independent parts of the graph in
parallel (with TX transfer you will always run only one at a time). Also it
does not require inventing any special fault tolerance technics because
Ignite caches are already fault tolerant and all the intermediate results
are virtually invisible and stored with TTL, thus in case of any crash you
will not have inconsistent state or garbage.

Sergi


2017-03-15 11:42 GMT+03:00 ALEKSEY KUZNETSOV :

> Okay, we are open for proposals on business task. I mean, we can make use
> of some other thing, not distributed transaction. Not transaction yet.
>
> ср, 15 мар. 2017 г. в 11:24, Vladimir Ozerov :
>
> > IMO the use case makes sense. However, as Sergi already mentioned, the
> > problem is far more complex, than simply passing TX state over a wire.
> Most
> > probably a kind of coordinator will be required still to manage all kinds
> > of failures. This task should be started with clean design proposal
> > explaining how we handle all these concurrent events. And only then, when
> > we understand all implications, we should move to development stage.
> >
> > On Wed, Mar 15, 2017 at 10:38 AM, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com> wrote:
> >
> > > Right
> > >
> > > ср, 15 мар. 2017 г. в 10:35, Sergi Vladykin  >:
> > >
> > > > Good! Basically your orchestrator just takes some predefined graph of
> > > > distributed services to be invoked, calls them by some kind of RPC
> and
> > > > passes the needed parameters between them, right?
> > > >
> > > > Sergi
> > > >
> > > > 2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > orchestrator is a custom thing. He is responsible for managing
> > business
> > > > > scenarios flows. Many nodes are involved in scenarios. They
> exchange
> > > data
> > > > > and folow one another. If you acquinted with BPMN framework, so
> > > > > orchestrator is like bpmn engine.
> > > > >
> > > > > вт, 14 Мар 2017 г., 18:56 Sergi Vladykin  >:
> > > > >
> > > > > > What is Orchestrator for you? Is it a thing from Microsoft or
> your
> > > > custom
> > > > > > in-house software?
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > > >
> > > > > > > Fine. Let's say we've got multiple servers which fulfills
> custom
> > > > logic.
> > > > > > > This servers compound oriented graph (BPMN process) which
> > > controlled
> > > > by
> > > > > > > Orchestrator.
> > > > > > > For instance, *server1  *creates *variable 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Okay, we are open for proposals on business task. I mean, we can make use
of some other thing, not distributed transaction. Not transaction yet.

ср, 15 мар. 2017 г. в 11:24, Vladimir Ozerov :

> IMO the use case makes sense. However, as Sergi already mentioned, the
> problem is far more complex, than simply passing TX state over a wire. Most
> probably a kind of coordinator will be required still to manage all kinds
> of failures. This task should be started with clean design proposal
> explaining how we handle all these concurrent events. And only then, when
> we understand all implications, we should move to development stage.
>
> On Wed, Mar 15, 2017 at 10:38 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com> wrote:
>
> > Right
> >
> > ср, 15 мар. 2017 г. в 10:35, Sergi Vladykin :
> >
> > > Good! Basically your orchestrator just takes some predefined graph of
> > > distributed services to be invoked, calls them by some kind of RPC and
> > > passes the needed parameters between them, right?
> > >
> > > Sergi
> > >
> > > 2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > orchestrator is a custom thing. He is responsible for managing
> business
> > > > scenarios flows. Many nodes are involved in scenarios. They exchange
> > data
> > > > and folow one another. If you acquinted with BPMN framework, so
> > > > orchestrator is like bpmn engine.
> > > >
> > > > вт, 14 Мар 2017 г., 18:56 Sergi Vladykin :
> > > >
> > > > > What is Orchestrator for you? Is it a thing from Microsoft or your
> > > custom
> > > > > in-house software?
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > > >
> > > > > > Fine. Let's say we've got multiple servers which fulfills custom
> > > logic.
> > > > > > This servers compound oriented graph (BPMN process) which
> > controlled
> > > by
> > > > > > Orchestrator.
> > > > > > For instance, *server1  *creates *variable A *with value 1,
> > persists
> > > it
> > > > > to
> > > > > > IGNITE cache and creates *variable B *and sends it to* server2.
> > *The
> > > > > > latests receives *variable B*, do some logic with it and stores
> to
> > > > > IGNITE.
> > > > > > All the work made by both servers must be fulfilled in *one*
> > > > transaction.
> > > > > > Because we need all information done, or nothing(rollbacked). The
> > > > > scenario
> > > > > > is managed by orchestrator.
> > > > > >
> > > > > > вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin <
> > > sergi.vlady...@gmail.com
> > > > >:
> > > > > >
> > > > > > > Ok, it is not a business case, it is your wrong solution for
> it.
> > > > > > > Lets try again, what is the business case?
> > > > > > >
> > > > > > > Sergi
> > > > > > >
> > > > > > > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > The case is the following, One starts transaction in one
> node,
> > > and
> > > > > > commit
> > > > > > > > this transaction in another jvm node(or rollback it
> remotely).
> > > > > > > >
> > > > > > > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin <
> > > > > sergi.vlady...@gmail.com
> > > > > > >:
> > > > > > > >
> > > > > > > > > Because even if you make it work for some simplistic
> > scenario,
> > > > get
> > > > > > > ready
> > > > > > > > to
> > > > > > > > > write many fault tolerance tests and make sure that you TXs
> > > work
> > > > > > > > gracefully
> > > > > > > > > in all modes in case of crashes. Also make sure that we do
> > not
> > > > have
> > > > > > any
> > > > > > > > > performance drops after all your changes in existing
> > > benchmarks.
> > > > > All
> > > > > > in
> > > > > > > > all
> > > > > > > > > I don't believe these conditions will be met and your
> > > > contribution
> > > > > > will
> > > > > > > > be
> > > > > > > > > accepted.
> > > > > > > > >
> > > > > > > > > Better solution to what problem? Sending TX to another
> node?
> > > The
> > > > > > > problem
> > > > > > > > > statement itself is already wrong. What business case you
> are
> > > > > trying
> > > > > > to
> > > > > > > > > solve? I'm sure everything you need can be done in a much
> > more
> > > > > simple
> > > > > > > and
> > > > > > > > > efficient way at the application level.
> > > > > > > > >
> > > > > > > > > Sergi
> > > > > > > > >
> > > > > > > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > > alkuznetsov...@gmail.com
> > > > > > > >:
> > > > > > > > >
> > > > > > > > > > Why wrong ? You know the better solution?
> > > > > > > > > >
> > > > > > > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin <
> > > > > > > sergi.vlady...@gmail.com
> > > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > > Just serializing TX object and deserializing it on
> > another
> > > > node
> > > > > > is
> > > > > > > > > > > meaningless, because other nodes participating in the

Re: distributed transaction of non-single coordinator

[Vladimir Ozerov]

IMO the use case makes sense. However, as Sergi already mentioned, the
problem is far more complex, than simply passing TX state over a wire. Most
probably a kind of coordinator will be required still to manage all kinds
of failures. This task should be started with clean design proposal
explaining how we handle all these concurrent events. And only then, when
we understand all implications, we should move to development stage.

On Wed, Mar 15, 2017 at 10:38 AM, ALEKSEY KUZNETSOV <
alkuznetsov...@gmail.com> wrote:

> Right
>
> ср, 15 мар. 2017 г. в 10:35, Sergi Vladykin :
>
> > Good! Basically your orchestrator just takes some predefined graph of
> > distributed services to be invoked, calls them by some kind of RPC and
> > passes the needed parameters between them, right?
> >
> > Sergi
> >
> > 2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > orchestrator is a custom thing. He is responsible for managing business
> > > scenarios flows. Many nodes are involved in scenarios. They exchange
> data
> > > and folow one another. If you acquinted with BPMN framework, so
> > > orchestrator is like bpmn engine.
> > >
> > > вт, 14 Мар 2017 г., 18:56 Sergi Vladykin :
> > >
> > > > What is Orchestrator for you? Is it a thing from Microsoft or your
> > custom
> > > > in-house software?
> > > >
> > > > Sergi
> > > >
> > > > 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > Fine. Let's say we've got multiple servers which fulfills custom
> > logic.
> > > > > This servers compound oriented graph (BPMN process) which
> controlled
> > by
> > > > > Orchestrator.
> > > > > For instance, *server1  *creates *variable A *with value 1,
> persists
> > it
> > > > to
> > > > > IGNITE cache and creates *variable B *and sends it to* server2.
> *The
> > > > > latests receives *variable B*, do some logic with it and stores to
> > > > IGNITE.
> > > > > All the work made by both servers must be fulfilled in *one*
> > > transaction.
> > > > > Because we need all information done, or nothing(rollbacked). The
> > > > scenario
> > > > > is managed by orchestrator.
> > > > >
> > > > > вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > >:
> > > > >
> > > > > > Ok, it is not a business case, it is your wrong solution for it.
> > > > > > Lets try again, what is the business case?
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > > >
> > > > > > > The case is the following, One starts transaction in one node,
> > and
> > > > > commit
> > > > > > > this transaction in another jvm node(or rollback it remotely).
> > > > > > >
> > > > > > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin <
> > > > sergi.vlady...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > Because even if you make it work for some simplistic
> scenario,
> > > get
> > > > > > ready
> > > > > > > to
> > > > > > > > write many fault tolerance tests and make sure that you TXs
> > work
> > > > > > > gracefully
> > > > > > > > in all modes in case of crashes. Also make sure that we do
> not
> > > have
> > > > > any
> > > > > > > > performance drops after all your changes in existing
> > benchmarks.
> > > > All
> > > > > in
> > > > > > > all
> > > > > > > > I don't believe these conditions will be met and your
> > > contribution
> > > > > will
> > > > > > > be
> > > > > > > > accepted.
> > > > > > > >
> > > > > > > > Better solution to what problem? Sending TX to another node?
> > The
> > > > > > problem
> > > > > > > > statement itself is already wrong. What business case you are
> > > > trying
> > > > > to
> > > > > > > > solve? I'm sure everything you need can be done in a much
> more
> > > > simple
> > > > > > and
> > > > > > > > efficient way at the application level.
> > > > > > > >
> > > > > > > > Sergi
> > > > > > > >
> > > > > > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com
> > > > > > >:
> > > > > > > >
> > > > > > > > > Why wrong ? You know the better solution?
> > > > > > > > >
> > > > > > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin <
> > > > > > sergi.vlady...@gmail.com
> > > > > > > >:
> > > > > > > > >
> > > > > > > > > > Just serializing TX object and deserializing it on
> another
> > > node
> > > > > is
> > > > > > > > > > meaningless, because other nodes participating in the TX
> > have
> > > > to
> > > > > > know
> > > > > > > > > about
> > > > > > > > > > the new coordinator. This will require protocol changes,
> we
> > > > > > > definitely
> > > > > > > > > will
> > > > > > > > > > have fault tolerance and performance issues. IMO the
> whole
> > > idea
> > > > > is
> > > > > > > > wrong
> > > > > > > > > > and it makes no sense to waste time on it.
> > > > > > > > > >
> > > > > > > > > > Sergi
> > > > > > > > > >
> > > > > > > > > > 2017-03-14 10:57 GMT+03:00 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Right

ср, 15 мар. 2017 г. в 10:35, Sergi Vladykin :

> Good! Basically your orchestrator just takes some predefined graph of
> distributed services to be invoked, calls them by some kind of RPC and
> passes the needed parameters between them, right?
>
> Sergi
>
> 2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > orchestrator is a custom thing. He is responsible for managing business
> > scenarios flows. Many nodes are involved in scenarios. They exchange data
> > and folow one another. If you acquinted with BPMN framework, so
> > orchestrator is like bpmn engine.
> >
> > вт, 14 Мар 2017 г., 18:56 Sergi Vladykin :
> >
> > > What is Orchestrator for you? Is it a thing from Microsoft or your
> custom
> > > in-house software?
> > >
> > > Sergi
> > >
> > > 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > Fine. Let's say we've got multiple servers which fulfills custom
> logic.
> > > > This servers compound oriented graph (BPMN process) which controlled
> by
> > > > Orchestrator.
> > > > For instance, *server1  *creates *variable A *with value 1, persists
> it
> > > to
> > > > IGNITE cache and creates *variable B *and sends it to* server2. *The
> > > > latests receives *variable B*, do some logic with it and stores to
> > > IGNITE.
> > > > All the work made by both servers must be fulfilled in *one*
> > transaction.
> > > > Because we need all information done, or nothing(rollbacked). The
> > > scenario
> > > > is managed by orchestrator.
> > > >
> > > > вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > >
> > > > > Ok, it is not a business case, it is your wrong solution for it.
> > > > > Lets try again, what is the business case?
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > > >
> > > > > > The case is the following, One starts transaction in one node,
> and
> > > > commit
> > > > > > this transaction in another jvm node(or rollback it remotely).
> > > > > >
> > > > > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin <
> > > sergi.vlady...@gmail.com
> > > > >:
> > > > > >
> > > > > > > Because even if you make it work for some simplistic scenario,
> > get
> > > > > ready
> > > > > > to
> > > > > > > write many fault tolerance tests and make sure that you TXs
> work
> > > > > > gracefully
> > > > > > > in all modes in case of crashes. Also make sure that we do not
> > have
> > > > any
> > > > > > > performance drops after all your changes in existing
> benchmarks.
> > > All
> > > > in
> > > > > > all
> > > > > > > I don't believe these conditions will be met and your
> > contribution
> > > > will
> > > > > > be
> > > > > > > accepted.
> > > > > > >
> > > > > > > Better solution to what problem? Sending TX to another node?
> The
> > > > > problem
> > > > > > > statement itself is already wrong. What business case you are
> > > trying
> > > > to
> > > > > > > solve? I'm sure everything you need can be done in a much more
> > > simple
> > > > > and
> > > > > > > efficient way at the application level.
> > > > > > >
> > > > > > > Sergi
> > > > > > >
> > > > > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > Why wrong ? You know the better solution?
> > > > > > > >
> > > > > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin <
> > > > > sergi.vlady...@gmail.com
> > > > > > >:
> > > > > > > >
> > > > > > > > > Just serializing TX object and deserializing it on another
> > node
> > > > is
> > > > > > > > > meaningless, because other nodes participating in the TX
> have
> > > to
> > > > > know
> > > > > > > > about
> > > > > > > > > the new coordinator. This will require protocol changes, we
> > > > > > definitely
> > > > > > > > will
> > > > > > > > > have fault tolerance and performance issues. IMO the whole
> > idea
> > > > is
> > > > > > > wrong
> > > > > > > > > and it makes no sense to waste time on it.
> > > > > > > > >
> > > > > > > > > Sergi
> > > > > > > > >
> > > > > > > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > > alkuznetsov...@gmail.com
> > > > > > > >:
> > > > > > > > >
> > > > > > > > > > IgniteTransactionState implememntation contains
> > > IgniteTxEntry's
> > > > > > which
> > > > > > > > is
> > > > > > > > > > supposed to be transferable
> > > > > > > > > >
> > > > > > > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan <
> > > > > > > dsetrak...@apache.org
> > > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > > It sounds a little scary to me that we are passing
> > > > transaction
> > > > > > > > objects
> > > > > > > > > > > around. Such object may contain all sorts of Ignite
> > > context.
> > > > If
> > > > > > > some
> > > > > > > > > data
> > > > > > > > > > > needs to be passed across, we should create a special
> > > > transfer
> > > > > > > object
> 

Re: distributed transaction of non-single coordinator

[Sergi Vladykin]

Good! Basically your orchestrator just takes some predefined graph of
distributed services to be invoked, calls them by some kind of RPC and
passes the needed parameters between them, right?

Sergi

2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV :

> orchestrator is a custom thing. He is responsible for managing business
> scenarios flows. Many nodes are involved in scenarios. They exchange data
> and folow one another. If you acquinted with BPMN framework, so
> orchestrator is like bpmn engine.
>
> вт, 14 Мар 2017 г., 18:56 Sergi Vladykin :
>
> > What is Orchestrator for you? Is it a thing from Microsoft or your custom
> > in-house software?
> >
> > Sergi
> >
> > 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > Fine. Let's say we've got multiple servers which fulfills custom logic.
> > > This servers compound oriented graph (BPMN process) which controlled by
> > > Orchestrator.
> > > For instance, *server1  *creates *variable A *with value 1, persists it
> > to
> > > IGNITE cache and creates *variable B *and sends it to* server2. *The
> > > latests receives *variable B*, do some logic with it and stores to
> > IGNITE.
> > > All the work made by both servers must be fulfilled in *one*
> transaction.
> > > Because we need all information done, or nothing(rollbacked). The
> > scenario
> > > is managed by orchestrator.
> > >
> > > вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin  >:
> > >
> > > > Ok, it is not a business case, it is your wrong solution for it.
> > > > Lets try again, what is the business case?
> > > >
> > > > Sergi
> > > >
> > > > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > The case is the following, One starts transaction in one node, and
> > > commit
> > > > > this transaction in another jvm node(or rollback it remotely).
> > > > >
> > > > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > >:
> > > > >
> > > > > > Because even if you make it work for some simplistic scenario,
> get
> > > > ready
> > > > > to
> > > > > > write many fault tolerance tests and make sure that you TXs work
> > > > > gracefully
> > > > > > in all modes in case of crashes. Also make sure that we do not
> have
> > > any
> > > > > > performance drops after all your changes in existing benchmarks.
> > All
> > > in
> > > > > all
> > > > > > I don't believe these conditions will be met and your
> contribution
> > > will
> > > > > be
> > > > > > accepted.
> > > > > >
> > > > > > Better solution to what problem? Sending TX to another node? The
> > > > problem
> > > > > > statement itself is already wrong. What business case you are
> > trying
> > > to
> > > > > > solve? I'm sure everything you need can be done in a much more
> > simple
> > > > and
> > > > > > efficient way at the application level.
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > > >
> > > > > > > Why wrong ? You know the better solution?
> > > > > > >
> > > > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin <
> > > > sergi.vlady...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > Just serializing TX object and deserializing it on another
> node
> > > is
> > > > > > > > meaningless, because other nodes participating in the TX have
> > to
> > > > know
> > > > > > > about
> > > > > > > > the new coordinator. This will require protocol changes, we
> > > > > definitely
> > > > > > > will
> > > > > > > > have fault tolerance and performance issues. IMO the whole
> idea
> > > is
> > > > > > wrong
> > > > > > > > and it makes no sense to waste time on it.
> > > > > > > >
> > > > > > > > Sergi
> > > > > > > >
> > > > > > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com
> > > > > > >:
> > > > > > > >
> > > > > > > > > IgniteTransactionState implememntation contains
> > IgniteTxEntry's
> > > > > which
> > > > > > > is
> > > > > > > > > supposed to be transferable
> > > > > > > > >
> > > > > > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan <
> > > > > > dsetrak...@apache.org
> > > > > > > >:
> > > > > > > > >
> > > > > > > > > > It sounds a little scary to me that we are passing
> > > transaction
> > > > > > > objects
> > > > > > > > > > around. Such object may contain all sorts of Ignite
> > context.
> > > If
> > > > > > some
> > > > > > > > data
> > > > > > > > > > needs to be passed across, we should create a special
> > > transfer
> > > > > > object
> > > > > > > > in
> > > > > > > > > > this case.
> > > > > > > > > >
> > > > > > > > > > D.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > > > > > > > > > alkuznetsov...@gmail.com
> > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > well, there a couple of issues preventing transaction
> > > 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

orchestrator is a custom thing. He is responsible for managing business
scenarios flows. Many nodes are involved in scenarios. They exchange data
and folow one another. If you acquinted with BPMN framework, so
orchestrator is like bpmn engine.

вт, 14 Мар 2017 г., 18:56 Sergi Vladykin :

> What is Orchestrator for you? Is it a thing from Microsoft or your custom
> in-house software?
>
> Sergi
>
> 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > Fine. Let's say we've got multiple servers which fulfills custom logic.
> > This servers compound oriented graph (BPMN process) which controlled by
> > Orchestrator.
> > For instance, *server1  *creates *variable A *with value 1, persists it
> to
> > IGNITE cache and creates *variable B *and sends it to* server2. *The
> > latests receives *variable B*, do some logic with it and stores to
> IGNITE.
> > All the work made by both servers must be fulfilled in *one* transaction.
> > Because we need all information done, or nothing(rollbacked). The
> scenario
> > is managed by orchestrator.
> >
> > вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin :
> >
> > > Ok, it is not a business case, it is your wrong solution for it.
> > > Lets try again, what is the business case?
> > >
> > > Sergi
> > >
> > > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > The case is the following, One starts transaction in one node, and
> > commit
> > > > this transaction in another jvm node(or rollback it remotely).
> > > >
> > > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > >
> > > > > Because even if you make it work for some simplistic scenario, get
> > > ready
> > > > to
> > > > > write many fault tolerance tests and make sure that you TXs work
> > > > gracefully
> > > > > in all modes in case of crashes. Also make sure that we do not have
> > any
> > > > > performance drops after all your changes in existing benchmarks.
> All
> > in
> > > > all
> > > > > I don't believe these conditions will be met and your contribution
> > will
> > > > be
> > > > > accepted.
> > > > >
> > > > > Better solution to what problem? Sending TX to another node? The
> > > problem
> > > > > statement itself is already wrong. What business case you are
> trying
> > to
> > > > > solve? I'm sure everything you need can be done in a much more
> simple
> > > and
> > > > > efficient way at the application level.
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > > >
> > > > > > Why wrong ? You know the better solution?
> > > > > >
> > > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin <
> > > sergi.vlady...@gmail.com
> > > > >:
> > > > > >
> > > > > > > Just serializing TX object and deserializing it on another node
> > is
> > > > > > > meaningless, because other nodes participating in the TX have
> to
> > > know
> > > > > > about
> > > > > > > the new coordinator. This will require protocol changes, we
> > > > definitely
> > > > > > will
> > > > > > > have fault tolerance and performance issues. IMO the whole idea
> > is
> > > > > wrong
> > > > > > > and it makes no sense to waste time on it.
> > > > > > >
> > > > > > > Sergi
> > > > > > >
> > > > > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > IgniteTransactionState implememntation contains
> IgniteTxEntry's
> > > > which
> > > > > > is
> > > > > > > > supposed to be transferable
> > > > > > > >
> > > > > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan <
> > > > > dsetrak...@apache.org
> > > > > > >:
> > > > > > > >
> > > > > > > > > It sounds a little scary to me that we are passing
> > transaction
> > > > > > objects
> > > > > > > > > around. Such object may contain all sorts of Ignite
> context.
> > If
> > > > > some
> > > > > > > data
> > > > > > > > > needs to be passed across, we should create a special
> > transfer
> > > > > object
> > > > > > > in
> > > > > > > > > this case.
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > > > > > > > > alkuznetsov...@gmail.com
> > > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > well, there a couple of issues preventing transaction
> > > > proceeding.
> > > > > > > > > > At first, After transaction serialization and
> > deserialization
> > > > on
> > > > > > the
> > > > > > > > > remote
> > > > > > > > > > server, there is no txState. So im going to put it in
> > > > > > > > > > writeExternal()\readExternal()
> > > > > > > > > >
> > > > > > > > > > The last one is Deserialized transaction lacks of shared
> > > cache
> > > > > > > context
> > > > > > > > > > field at TransactionProxyImpl. Perhaps, it must be
> injected
> > > by
> > > > > > > > > > GridResourceProcessor ?
> > > > > > > > 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Fine. Let's say we've got multiple servers which fulfills custom logic.
This servers compound oriented graph (BPMN process) which controlled by
Orchestrator.
For instance, *server1  *creates *variable A *with value 1, persists it to
IGNITE cache and creates *variable B *and sends it to* server2. *The
latests receives *variable B*, do some logic with it and stores to IGNITE.
All the work made by both servers must be fulfilled in *one* transaction.
Because we need all information done, or nothing(rollbacked). The scenario
is managed by orchestrator.

вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin :

> Ok, it is not a business case, it is your wrong solution for it.
> Lets try again, what is the business case?
>
> Sergi
>
> 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > The case is the following, One starts transaction in one node, and commit
> > this transaction in another jvm node(or rollback it remotely).
> >
> > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin :
> >
> > > Because even if you make it work for some simplistic scenario, get
> ready
> > to
> > > write many fault tolerance tests and make sure that you TXs work
> > gracefully
> > > in all modes in case of crashes. Also make sure that we do not have any
> > > performance drops after all your changes in existing benchmarks. All in
> > all
> > > I don't believe these conditions will be met and your contribution will
> > be
> > > accepted.
> > >
> > > Better solution to what problem? Sending TX to another node? The
> problem
> > > statement itself is already wrong. What business case you are trying to
> > > solve? I'm sure everything you need can be done in a much more simple
> and
> > > efficient way at the application level.
> > >
> > > Sergi
> > >
> > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > Why wrong ? You know the better solution?
> > > >
> > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin <
> sergi.vlady...@gmail.com
> > >:
> > > >
> > > > > Just serializing TX object and deserializing it on another node is
> > > > > meaningless, because other nodes participating in the TX have to
> know
> > > > about
> > > > > the new coordinator. This will require protocol changes, we
> > definitely
> > > > will
> > > > > have fault tolerance and performance issues. IMO the whole idea is
> > > wrong
> > > > > and it makes no sense to waste time on it.
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > > >
> > > > > > IgniteTransactionState implememntation contains IgniteTxEntry's
> > which
> > > > is
> > > > > > supposed to be transferable
> > > > > >
> > > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan <
> > > dsetrak...@apache.org
> > > > >:
> > > > > >
> > > > > > > It sounds a little scary to me that we are passing transaction
> > > > objects
> > > > > > > around. Such object may contain all sorts of Ignite context. If
> > > some
> > > > > data
> > > > > > > needs to be passed across, we should create a special transfer
> > > object
> > > > > in
> > > > > > > this case.
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > > > > > > alkuznetsov...@gmail.com
> > > > > > > > wrote:
> > > > > > >
> > > > > > > > well, there a couple of issues preventing transaction
> > proceeding.
> > > > > > > > At first, After transaction serialization and deserialization
> > on
> > > > the
> > > > > > > remote
> > > > > > > > server, there is no txState. So im going to put it in
> > > > > > > > writeExternal()\readExternal()
> > > > > > > >
> > > > > > > > The last one is Deserialized transaction lacks of shared
> cache
> > > > > context
> > > > > > > > field at TransactionProxyImpl. Perhaps, it must be injected
> by
> > > > > > > > GridResourceProcessor ?
> > > > > > > >
> > > > > > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV <
> > > > > > alkuznetsov...@gmail.com
> > > > > > > >:
> > > > > > > >
> > > > > > > > > while starting and continuing transaction in different jvms
> > in
> > > > run
> > > > > > into
> > > > > > > > > serialization exception in writeExternalMeta :
> > > > > > > > >
> > > > > > > > > @Override public void writeExternal(ObjectOutput out)
> throws
> > > > > > > IOException
> > > > > > > > {
> > > > > > > > > writeExternalMeta(out);
> > > > > > > > >
> > > > > > > > > some meta is cannot be serialized.
> > > > > > > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> > > > > > > > alexey.goncha...@gmail.com
> > > > > > > > > >:
> > > > > > > > >
> > > > > > > > > Aleksey,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I think I am starting to get what you want, but I have a
> few
> > > > > > concerns:
> > > > > > > > >  - What is the API for the proposed change? In your test,
> you
> > > > pass
> > > > > an
> > > > > > > 

Re: distributed transaction of non-single coordinator

[Sergi Vladykin]

Ok, it is not a business case, it is your wrong solution for it.
Lets try again, what is the business case?

Sergi

2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV :

> The case is the following, One starts transaction in one node, and commit
> this transaction in another jvm node(or rollback it remotely).
>
> вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin :
>
> > Because even if you make it work for some simplistic scenario, get ready
> to
> > write many fault tolerance tests and make sure that you TXs work
> gracefully
> > in all modes in case of crashes. Also make sure that we do not have any
> > performance drops after all your changes in existing benchmarks. All in
> all
> > I don't believe these conditions will be met and your contribution will
> be
> > accepted.
> >
> > Better solution to what problem? Sending TX to another node? The problem
> > statement itself is already wrong. What business case you are trying to
> > solve? I'm sure everything you need can be done in a much more simple and
> > efficient way at the application level.
> >
> > Sergi
> >
> > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > Why wrong ? You know the better solution?
> > >
> > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin  >:
> > >
> > > > Just serializing TX object and deserializing it on another node is
> > > > meaningless, because other nodes participating in the TX have to know
> > > about
> > > > the new coordinator. This will require protocol changes, we
> definitely
> > > will
> > > > have fault tolerance and performance issues. IMO the whole idea is
> > wrong
> > > > and it makes no sense to waste time on it.
> > > >
> > > > Sergi
> > > >
> > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > IgniteTransactionState implememntation contains IgniteTxEntry's
> which
> > > is
> > > > > supposed to be transferable
> > > > >
> > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >:
> > > > >
> > > > > > It sounds a little scary to me that we are passing transaction
> > > objects
> > > > > > around. Such object may contain all sorts of Ignite context. If
> > some
> > > > data
> > > > > > needs to be passed across, we should create a special transfer
> > object
> > > > in
> > > > > > this case.
> > > > > >
> > > > > > D.
> > > > > >
> > > > > >
> > > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > > > > > alkuznetsov...@gmail.com
> > > > > > > wrote:
> > > > > >
> > > > > > > well, there a couple of issues preventing transaction
> proceeding.
> > > > > > > At first, After transaction serialization and deserialization
> on
> > > the
> > > > > > remote
> > > > > > > server, there is no txState. So im going to put it in
> > > > > > > writeExternal()\readExternal()
> > > > > > >
> > > > > > > The last one is Deserialized transaction lacks of shared cache
> > > > context
> > > > > > > field at TransactionProxyImpl. Perhaps, it must be injected by
> > > > > > > GridResourceProcessor ?
> > > > > > >
> > > > > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com
> > > > > > >:
> > > > > > >
> > > > > > > > while starting and continuing transaction in different jvms
> in
> > > run
> > > > > into
> > > > > > > > serialization exception in writeExternalMeta :
> > > > > > > >
> > > > > > > > @Override public void writeExternal(ObjectOutput out) throws
> > > > > > IOException
> > > > > > > {
> > > > > > > > writeExternalMeta(out);
> > > > > > > >
> > > > > > > > some meta is cannot be serialized.
> > > > > > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> > > > > > > alexey.goncha...@gmail.com
> > > > > > > > >:
> > > > > > > >
> > > > > > > > Aleksey,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > I think I am starting to get what you want, but I have a few
> > > > > concerns:
> > > > > > > >  - What is the API for the proposed change? In your test, you
> > > pass
> > > > an
> > > > > > > > instance of transaction created on ignite(0) to the ignite
> > > instance
> > > > > > > > ignite(1). This is obviously not possible in a truly
> > distributed
> > > > > > > > (multi-jvm) environment.
> > > > > > > > - How will you synchronize cache update actions and
> transaction
> > > > > commit?
> > > > > > > > Say, you have one node that decided to commit, but another
> node
> > > is
> > > > > > still
> > > > > > > > writing within this transaction. How do you make sure that
> two
> > > > nodes
> > > > > > will
> > > > > > > > not call commit() and rollback() simultaneously?
> > > > > > > >  - How do you make sure that either commit() or rollback() is
> > > > called
> > > > > if
> > > > > > > an
> > > > > > > > originator failed?
> > > > > > > >
> > > > > > > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов <
> > somefire...@gmail.com
> > > >:
> > > > > > > >
> > > > > > > > > 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

The case is the following, One starts transaction in one node, and commit
this transaction in another jvm node(or rollback it remotely).

вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin :

> Because even if you make it work for some simplistic scenario, get ready to
> write many fault tolerance tests and make sure that you TXs work gracefully
> in all modes in case of crashes. Also make sure that we do not have any
> performance drops after all your changes in existing benchmarks. All in all
> I don't believe these conditions will be met and your contribution will be
> accepted.
>
> Better solution to what problem? Sending TX to another node? The problem
> statement itself is already wrong. What business case you are trying to
> solve? I'm sure everything you need can be done in a much more simple and
> efficient way at the application level.
>
> Sergi
>
> 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > Why wrong ? You know the better solution?
> >
> > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin :
> >
> > > Just serializing TX object and deserializing it on another node is
> > > meaningless, because other nodes participating in the TX have to know
> > about
> > > the new coordinator. This will require protocol changes, we definitely
> > will
> > > have fault tolerance and performance issues. IMO the whole idea is
> wrong
> > > and it makes no sense to waste time on it.
> > >
> > > Sergi
> > >
> > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > IgniteTransactionState implememntation contains IgniteTxEntry's which
> > is
> > > > supposed to be transferable
> > > >
> > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >:
> > > >
> > > > > It sounds a little scary to me that we are passing transaction
> > objects
> > > > > around. Such object may contain all sorts of Ignite context. If
> some
> > > data
> > > > > needs to be passed across, we should create a special transfer
> object
> > > in
> > > > > this case.
> > > > >
> > > > > D.
> > > > >
> > > > >
> > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com
> > > > > > wrote:
> > > > >
> > > > > > well, there a couple of issues preventing transaction proceeding.
> > > > > > At first, After transaction serialization and deserialization on
> > the
> > > > > remote
> > > > > > server, there is no txState. So im going to put it in
> > > > > > writeExternal()\readExternal()
> > > > > >
> > > > > > The last one is Deserialized transaction lacks of shared cache
> > > context
> > > > > > field at TransactionProxyImpl. Perhaps, it must be injected by
> > > > > > GridResourceProcessor ?
> > > > > >
> > > > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > >
> > > > > > > while starting and continuing transaction in different jvms in
> > run
> > > > into
> > > > > > > serialization exception in writeExternalMeta :
> > > > > > >
> > > > > > > @Override public void writeExternal(ObjectOutput out) throws
> > > > > IOException
> > > > > > {
> > > > > > > writeExternalMeta(out);
> > > > > > >
> > > > > > > some meta is cannot be serialized.
> > > > > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> > > > > > alexey.goncha...@gmail.com
> > > > > > > >:
> > > > > > >
> > > > > > > Aleksey,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I think I am starting to get what you want, but I have a few
> > > > concerns:
> > > > > > >  - What is the API for the proposed change? In your test, you
> > pass
> > > an
> > > > > > > instance of transaction created on ignite(0) to the ignite
> > instance
> > > > > > > ignite(1). This is obviously not possible in a truly
> distributed
> > > > > > > (multi-jvm) environment.
> > > > > > > - How will you synchronize cache update actions and transaction
> > > > commit?
> > > > > > > Say, you have one node that decided to commit, but another node
> > is
> > > > > still
> > > > > > > writing within this transaction. How do you make sure that two
> > > nodes
> > > > > will
> > > > > > > not call commit() and rollback() simultaneously?
> > > > > > >  - How do you make sure that either commit() or rollback() is
> > > called
> > > > if
> > > > > > an
> > > > > > > originator failed?
> > > > > > >
> > > > > > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов <
> somefire...@gmail.com
> > >:
> > > > > > >
> > > > > > > > Alexey Goncharuk, heh, my initial understanding was that
> > > > transferring
> > > > > > of
> > > > > > > tx
> > > > > > > > ownership from one node to another will be happened
> > automatically
> > > > > when
> > > > > > > > originating node is gone down.
> > > > > > > >
> > > > > > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com
> > > > > > >:
> > > > > > > >
> > > > > > > > > Im aiming to span transaction on multiple threads, nodes,

Re: distributed transaction of non-single coordinator

[Sergi Vladykin]

Because even if you make it work for some simplistic scenario, get ready to
write many fault tolerance tests and make sure that you TXs work gracefully
in all modes in case of crashes. Also make sure that we do not have any
performance drops after all your changes in existing benchmarks. All in all
I don't believe these conditions will be met and your contribution will be
accepted.

Better solution to what problem? Sending TX to another node? The problem
statement itself is already wrong. What business case you are trying to
solve? I'm sure everything you need can be done in a much more simple and
efficient way at the application level.

Sergi

2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV :

> Why wrong ? You know the better solution?
>
> вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin :
>
> > Just serializing TX object and deserializing it on another node is
> > meaningless, because other nodes participating in the TX have to know
> about
> > the new coordinator. This will require protocol changes, we definitely
> will
> > have fault tolerance and performance issues. IMO the whole idea is wrong
> > and it makes no sense to waste time on it.
> >
> > Sergi
> >
> > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > IgniteTransactionState implememntation contains IgniteTxEntry's which
> is
> > > supposed to be transferable
> > >
> > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan  >:
> > >
> > > > It sounds a little scary to me that we are passing transaction
> objects
> > > > around. Such object may contain all sorts of Ignite context. If some
> > data
> > > > needs to be passed across, we should create a special transfer object
> > in
> > > > this case.
> > > >
> > > > D.
> > > >
> > > >
> > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > wrote:
> > > >
> > > > > well, there a couple of issues preventing transaction proceeding.
> > > > > At first, After transaction serialization and deserialization on
> the
> > > > remote
> > > > > server, there is no txState. So im going to put it in
> > > > > writeExternal()\readExternal()
> > > > >
> > > > > The last one is Deserialized transaction lacks of shared cache
> > context
> > > > > field at TransactionProxyImpl. Perhaps, it must be injected by
> > > > > GridResourceProcessor ?
> > > > >
> > > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > >
> > > > > > while starting and continuing transaction in different jvms in
> run
> > > into
> > > > > > serialization exception in writeExternalMeta :
> > > > > >
> > > > > > @Override public void writeExternal(ObjectOutput out) throws
> > > > IOException
> > > > > {
> > > > > > writeExternalMeta(out);
> > > > > >
> > > > > > some meta is cannot be serialized.
> > > > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> > > > > alexey.goncha...@gmail.com
> > > > > > >:
> > > > > >
> > > > > > Aleksey,
> > > > > >
> > > > > >
> > > > > >
> > > > > > I think I am starting to get what you want, but I have a few
> > > concerns:
> > > > > >  - What is the API for the proposed change? In your test, you
> pass
> > an
> > > > > > instance of transaction created on ignite(0) to the ignite
> instance
> > > > > > ignite(1). This is obviously not possible in a truly distributed
> > > > > > (multi-jvm) environment.
> > > > > > - How will you synchronize cache update actions and transaction
> > > commit?
> > > > > > Say, you have one node that decided to commit, but another node
> is
> > > > still
> > > > > > writing within this transaction. How do you make sure that two
> > nodes
> > > > will
> > > > > > not call commit() and rollback() simultaneously?
> > > > > >  - How do you make sure that either commit() or rollback() is
> > called
> > > if
> > > > > an
> > > > > > originator failed?
> > > > > >
> > > > > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов  >:
> > > > > >
> > > > > > > Alexey Goncharuk, heh, my initial understanding was that
> > > transferring
> > > > > of
> > > > > > tx
> > > > > > > ownership from one node to another will be happened
> automatically
> > > > when
> > > > > > > originating node is gone down.
> > > > > > >
> > > > > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > Im aiming to span transaction on multiple threads, nodes,
> > > > jvms(soon).
> > > > > > So
> > > > > > > > every node is able to rollback, or commit common
> transaction.It
> > > > > turned
> > > > > > > up i
> > > > > > > > need to transfer tx between nodes in order to commit
> > transaction
> > > in
> > > > > > > > different node(in the same jvm).
> > > > > > > >
> > > > > > > > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> > > > > > > alexey.goncha...@gmail.com
> > > > > > > > >:
> > > > > > > >
> > > > > > > > > Aleksey,
> > 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Why wrong ? You know the better solution?

вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin :

> Just serializing TX object and deserializing it on another node is
> meaningless, because other nodes participating in the TX have to know about
> the new coordinator. This will require protocol changes, we definitely will
> have fault tolerance and performance issues. IMO the whole idea is wrong
> and it makes no sense to waste time on it.
>
> Sergi
>
> 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > IgniteTransactionState implememntation contains IgniteTxEntry's which is
> > supposed to be transferable
> >
> > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan :
> >
> > > It sounds a little scary to me that we are passing transaction objects
> > > around. Such object may contain all sorts of Ignite context. If some
> data
> > > needs to be passed across, we should create a special transfer object
> in
> > > this case.
> > >
> > > D.
> > >
> > >
> > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > wrote:
> > >
> > > > well, there a couple of issues preventing transaction proceeding.
> > > > At first, After transaction serialization and deserialization on the
> > > remote
> > > > server, there is no txState. So im going to put it in
> > > > writeExternal()\readExternal()
> > > >
> > > > The last one is Deserialized transaction lacks of shared cache
> context
> > > > field at TransactionProxyImpl. Perhaps, it must be injected by
> > > > GridResourceProcessor ?
> > > >
> > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > >
> > > > > while starting and continuing transaction in different jvms in run
> > into
> > > > > serialization exception in writeExternalMeta :
> > > > >
> > > > > @Override public void writeExternal(ObjectOutput out) throws
> > > IOException
> > > > {
> > > > > writeExternalMeta(out);
> > > > >
> > > > > some meta is cannot be serialized.
> > > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com
> > > > > >:
> > > > >
> > > > > Aleksey,
> > > > >
> > > > >
> > > > >
> > > > > I think I am starting to get what you want, but I have a few
> > concerns:
> > > > >  - What is the API for the proposed change? In your test, you pass
> an
> > > > > instance of transaction created on ignite(0) to the ignite instance
> > > > > ignite(1). This is obviously not possible in a truly distributed
> > > > > (multi-jvm) environment.
> > > > > - How will you synchronize cache update actions and transaction
> > commit?
> > > > > Say, you have one node that decided to commit, but another node is
> > > still
> > > > > writing within this transaction. How do you make sure that two
> nodes
> > > will
> > > > > not call commit() and rollback() simultaneously?
> > > > >  - How do you make sure that either commit() or rollback() is
> called
> > if
> > > > an
> > > > > originator failed?
> > > > >
> > > > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов :
> > > > >
> > > > > > Alexey Goncharuk, heh, my initial understanding was that
> > transferring
> > > > of
> > > > > tx
> > > > > > ownership from one node to another will be happened automatically
> > > when
> > > > > > originating node is gone down.
> > > > > >
> > > > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > > >
> > > > > > > Im aiming to span transaction on multiple threads, nodes,
> > > jvms(soon).
> > > > > So
> > > > > > > every node is able to rollback, or commit common transaction.It
> > > > turned
> > > > > > up i
> > > > > > > need to transfer tx between nodes in order to commit
> transaction
> > in
> > > > > > > different node(in the same jvm).
> > > > > > >
> > > > > > > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> > > > > > alexey.goncha...@gmail.com
> > > > > > > >:
> > > > > > >
> > > > > > > > Aleksey,
> > > > > > > >
> > > > > > > > Do you mean that you want a concept of transferring of tx
> > > ownership
> > > > > > from
> > > > > > > > one node to another? My initial understanding was that you
> want
> > > to
> > > > be
> > > > > > > able
> > > > > > > > to update keys in a transaction from multiple threads in
> > > parallel.
> > > > > > > >
> > > > > > > > --AG
> > > > > > > >
> > > > > > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com
> > > > > > >:
> > > > > > > >
> > > > > > > > > Well. Consider transaction started in one node, and
> continued
> > > in
> > > > > > > another
> > > > > > > > > one.
> > > > > > > > > The following test describes my idea:
> > > > > > > > >
> > > > > > > > > Ignite ignite1 = ignite(0);
> > > > > > > > >
> > > > > > > > > IgniteTransactions transactions = ignite1.transactions();
> > > > > > > > >
> > > > > > > > > IgniteCache cache =
> > ignite1.getOrCreateCache("
> > > > > > > > > 

Re: distributed transaction of non-single coordinator

[Sergi Vladykin]

Just serializing TX object and deserializing it on another node is
meaningless, because other nodes participating in the TX have to know about
the new coordinator. This will require protocol changes, we definitely will
have fault tolerance and performance issues. IMO the whole idea is wrong
and it makes no sense to waste time on it.

Sergi

2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV :

> IgniteTransactionState implememntation contains IgniteTxEntry's which is
> supposed to be transferable
>
> пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan :
>
> > It sounds a little scary to me that we are passing transaction objects
> > around. Such object may contain all sorts of Ignite context. If some data
> > needs to be passed across, we should create a special transfer object in
> > this case.
> >
> > D.
> >
> >
> > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > wrote:
> >
> > > well, there a couple of issues preventing transaction proceeding.
> > > At first, After transaction serialization and deserialization on the
> > remote
> > > server, there is no txState. So im going to put it in
> > > writeExternal()\readExternal()
> > >
> > > The last one is Deserialized transaction lacks of shared cache context
> > > field at TransactionProxyImpl. Perhaps, it must be injected by
> > > GridResourceProcessor ?
> > >
> > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > >
> > > > while starting and continuing transaction in different jvms in run
> into
> > > > serialization exception in writeExternalMeta :
> > > >
> > > > @Override public void writeExternal(ObjectOutput out) throws
> > IOException
> > > {
> > > > writeExternalMeta(out);
> > > >
> > > > some meta is cannot be serialized.
> > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com
> > > > >:
> > > >
> > > > Aleksey,
> > > >
> > > >
> > > >
> > > > I think I am starting to get what you want, but I have a few
> concerns:
> > > >  - What is the API for the proposed change? In your test, you pass an
> > > > instance of transaction created on ignite(0) to the ignite instance
> > > > ignite(1). This is obviously not possible in a truly distributed
> > > > (multi-jvm) environment.
> > > > - How will you synchronize cache update actions and transaction
> commit?
> > > > Say, you have one node that decided to commit, but another node is
> > still
> > > > writing within this transaction. How do you make sure that two nodes
> > will
> > > > not call commit() and rollback() simultaneously?
> > > >  - How do you make sure that either commit() or rollback() is called
> if
> > > an
> > > > originator failed?
> > > >
> > > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов :
> > > >
> > > > > Alexey Goncharuk, heh, my initial understanding was that
> transferring
> > > of
> > > > tx
> > > > > ownership from one node to another will be happened automatically
> > when
> > > > > originating node is gone down.
> > > > >
> > > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > > >
> > > > > > Im aiming to span transaction on multiple threads, nodes,
> > jvms(soon).
> > > > So
> > > > > > every node is able to rollback, or commit common transaction.It
> > > turned
> > > > > up i
> > > > > > need to transfer tx between nodes in order to commit transaction
> in
> > > > > > different node(in the same jvm).
> > > > > >
> > > > > > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> > > > > alexey.goncha...@gmail.com
> > > > > > >:
> > > > > >
> > > > > > > Aleksey,
> > > > > > >
> > > > > > > Do you mean that you want a concept of transferring of tx
> > ownership
> > > > > from
> > > > > > > one node to another? My initial understanding was that you want
> > to
> > > be
> > > > > > able
> > > > > > > to update keys in a transaction from multiple threads in
> > parallel.
> > > > > > >
> > > > > > > --AG
> > > > > > >
> > > > > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > Well. Consider transaction started in one node, and continued
> > in
> > > > > > another
> > > > > > > > one.
> > > > > > > > The following test describes my idea:
> > > > > > > >
> > > > > > > > Ignite ignite1 = ignite(0);
> > > > > > > >
> > > > > > > > IgniteTransactions transactions = ignite1.transactions();
> > > > > > > >
> > > > > > > > IgniteCache cache =
> ignite1.getOrCreateCache("
> > > > > > > > testCache");
> > > > > > > >
> > > > > > > > Transaction tx = transactions.txStart(concurrency,
> isolation);
> > > > > > > >
> > > > > > > > cache.put("key1", 1);
> > > > > > > >
> > > > > > > > cache.put("key2", 2);
> > > > > > > >
> > > > > > > > tx.stop();
> > > > > > > >
> > > > > > > > IgniteInternalFuture fut = GridTestUtils.runAsync(()
> > ->
> > > {
> > > > > > > > IgniteTransactions ts = 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

IgniteTransactionState implememntation contains IgniteTxEntry's which is
supposed to be transferable

пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan :

> It sounds a little scary to me that we are passing transaction objects
> around. Such object may contain all sorts of Ignite context. If some data
> needs to be passed across, we should create a special transfer object in
> this case.
>
> D.
>
>
> On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > wrote:
>
> > well, there a couple of issues preventing transaction proceeding.
> > At first, After transaction serialization and deserialization on the
> remote
> > server, there is no txState. So im going to put it in
> > writeExternal()\readExternal()
> >
> > The last one is Deserialized transaction lacks of shared cache context
> > field at TransactionProxyImpl. Perhaps, it must be injected by
> > GridResourceProcessor ?
> >
> > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV  >:
> >
> > > while starting and continuing transaction in different jvms in run into
> > > serialization exception in writeExternalMeta :
> > >
> > > @Override public void writeExternal(ObjectOutput out) throws
> IOException
> > {
> > > writeExternalMeta(out);
> > >
> > > some meta is cannot be serialized.
> > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > > >:
> > >
> > > Aleksey,
> > >
> > >
> > >
> > > I think I am starting to get what you want, but I have a few concerns:
> > >  - What is the API for the proposed change? In your test, you pass an
> > > instance of transaction created on ignite(0) to the ignite instance
> > > ignite(1). This is obviously not possible in a truly distributed
> > > (multi-jvm) environment.
> > > - How will you synchronize cache update actions and transaction commit?
> > > Say, you have one node that decided to commit, but another node is
> still
> > > writing within this transaction. How do you make sure that two nodes
> will
> > > not call commit() and rollback() simultaneously?
> > >  - How do you make sure that either commit() or rollback() is called if
> > an
> > > originator failed?
> > >
> > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов :
> > >
> > > > Alexey Goncharuk, heh, my initial understanding was that transferring
> > of
> > > tx
> > > > ownership from one node to another will be happened automatically
> when
> > > > originating node is gone down.
> > > >
> > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > Im aiming to span transaction on multiple threads, nodes,
> jvms(soon).
> > > So
> > > > > every node is able to rollback, or commit common transaction.It
> > turned
> > > > up i
> > > > > need to transfer tx between nodes in order to commit transaction in
> > > > > different node(in the same jvm).
> > > > >
> > > > > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com
> > > > > >:
> > > > >
> > > > > > Aleksey,
> > > > > >
> > > > > > Do you mean that you want a concept of transferring of tx
> ownership
> > > > from
> > > > > > one node to another? My initial understanding was that you want
> to
> > be
> > > > > able
> > > > > > to update keys in a transaction from multiple threads in
> parallel.
> > > > > >
> > > > > > --AG
> > > > > >
> > > > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > > >
> > > > > > > Well. Consider transaction started in one node, and continued
> in
> > > > > another
> > > > > > > one.
> > > > > > > The following test describes my idea:
> > > > > > >
> > > > > > > Ignite ignite1 = ignite(0);
> > > > > > >
> > > > > > > IgniteTransactions transactions = ignite1.transactions();
> > > > > > >
> > > > > > > IgniteCache cache = ignite1.getOrCreateCache("
> > > > > > > testCache");
> > > > > > >
> > > > > > > Transaction tx = transactions.txStart(concurrency, isolation);
> > > > > > >
> > > > > > > cache.put("key1", 1);
> > > > > > >
> > > > > > > cache.put("key2", 2);
> > > > > > >
> > > > > > > tx.stop();
> > > > > > >
> > > > > > > IgniteInternalFuture fut = GridTestUtils.runAsync(()
> ->
> > {
> > > > > > > IgniteTransactions ts = ignite(1).transactions();
> > > > > > > Assert.assertNull(ts.tx());
> > > > > > > Assert.assertEquals(TransactionState.STOPPED, tx.state());
> > > > > > > ts.txStart(tx);
> > > > > > > Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> > > > > > > cache.put("key3", 3);
> > > > > > > Assert.assertTrue(cache.remove("key2"));
> > > > > > > tx.commit();
> > > > > > > return true;
> > > > > > > });
> > > > > > >
> > > > > > > fut.get();
> > > > > > >
> > > > > > > Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> > > > > > > Assert.assertEquals((long)1, (long)cache.get("key1"));
> > > > > > > Assert.assertEquals((long)3, 

Re: distributed transaction of non-single coordinator

[Dmitriy Setrakyan]

It sounds a little scary to me that we are passing transaction objects
around. Such object may contain all sorts of Ignite context. If some data
needs to be passed across, we should create a special transfer object in
this case.

D.


On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV  wrote:

> well, there a couple of issues preventing transaction proceeding.
> At first, After transaction serialization and deserialization on the remote
> server, there is no txState. So im going to put it in
> writeExternal()\readExternal()
>
> The last one is Deserialized transaction lacks of shared cache context
> field at TransactionProxyImpl. Perhaps, it must be injected by
> GridResourceProcessor ?
>
> пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV :
>
> > while starting and continuing transaction in different jvms in run into
> > serialization exception in writeExternalMeta :
> >
> > @Override public void writeExternal(ObjectOutput out) throws IOException
> {
> > writeExternalMeta(out);
> >
> > some meta is cannot be serialized.
> > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> >
> > Aleksey,
> >
> >
> >
> > I think I am starting to get what you want, but I have a few concerns:
> >  - What is the API for the proposed change? In your test, you pass an
> > instance of transaction created on ignite(0) to the ignite instance
> > ignite(1). This is obviously not possible in a truly distributed
> > (multi-jvm) environment.
> > - How will you synchronize cache update actions and transaction commit?
> > Say, you have one node that decided to commit, but another node is still
> > writing within this transaction. How do you make sure that two nodes will
> > not call commit() and rollback() simultaneously?
> >  - How do you make sure that either commit() or rollback() is called if
> an
> > originator failed?
> >
> > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов :
> >
> > > Alexey Goncharuk, heh, my initial understanding was that transferring
> of
> > tx
> > > ownership from one node to another will be happened automatically when
> > > originating node is gone down.
> > >
> > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > Im aiming to span transaction on multiple threads, nodes, jvms(soon).
> > So
> > > > every node is able to rollback, or commit common transaction.It
> turned
> > > up i
> > > > need to transfer tx between nodes in order to commit transaction in
> > > > different node(in the same jvm).
> > > >
> > > > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com
> > > > >:
> > > >
> > > > > Aleksey,
> > > > >
> > > > > Do you mean that you want a concept of transferring of tx ownership
> > > from
> > > > > one node to another? My initial understanding was that you want to
> be
> > > > able
> > > > > to update keys in a transaction from multiple threads in parallel.
> > > > >
> > > > > --AG
> > > > >
> > > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > >:
> > > > >
> > > > > > Well. Consider transaction started in one node, and continued in
> > > > another
> > > > > > one.
> > > > > > The following test describes my idea:
> > > > > >
> > > > > > Ignite ignite1 = ignite(0);
> > > > > >
> > > > > > IgniteTransactions transactions = ignite1.transactions();
> > > > > >
> > > > > > IgniteCache cache = ignite1.getOrCreateCache("
> > > > > > testCache");
> > > > > >
> > > > > > Transaction tx = transactions.txStart(concurrency, isolation);
> > > > > >
> > > > > > cache.put("key1", 1);
> > > > > >
> > > > > > cache.put("key2", 2);
> > > > > >
> > > > > > tx.stop();
> > > > > >
> > > > > > IgniteInternalFuture fut = GridTestUtils.runAsync(() ->
> {
> > > > > > IgniteTransactions ts = ignite(1).transactions();
> > > > > > Assert.assertNull(ts.tx());
> > > > > > Assert.assertEquals(TransactionState.STOPPED, tx.state());
> > > > > > ts.txStart(tx);
> > > > > > Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> > > > > > cache.put("key3", 3);
> > > > > > Assert.assertTrue(cache.remove("key2"));
> > > > > > tx.commit();
> > > > > > return true;
> > > > > > });
> > > > > >
> > > > > > fut.get();
> > > > > >
> > > > > > Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> > > > > > Assert.assertEquals((long)1, (long)cache.get("key1"));
> > > > > > Assert.assertEquals((long)3, (long)cache.get("key3"));
> > > > > > Assert.assertFalse(cache.containsKey("key2"));
> > > > > >
> > > > > > In method *ts.txStart(...)* we just rebind *tx* to current
> thread:
> > > > > >
> > > > > > public void txStart(Transaction tx) {
> > > > > > TransactionProxyImpl transactionProxy =
> > (TransactionProxyImpl)tx;
> > > > > > cctx.tm().reopenTx(transactionProxy.tx());
> > > > > > transactionProxy.bindToCurrentThread();
> > > > > > }
> > > > > >
> > > > > 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

well, there a couple of issues preventing transaction proceeding.
At first, After transaction serialization and deserialization on the remote
server, there is no txState. So im going to put it in
writeExternal()\readExternal()

The last one is Deserialized transaction lacks of shared cache context
field at TransactionProxyImpl. Perhaps, it must be injected by
GridResourceProcessor ?

пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV :

> while starting and continuing transaction in different jvms in run into
> serialization exception in writeExternalMeta :
>
> @Override public void writeExternal(ObjectOutput out) throws IOException {
> writeExternalMeta(out);
>
> some meta is cannot be serialized.
> пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk  >:
>
> Aleksey,
>
>
>
> I think I am starting to get what you want, but I have a few concerns:
>  - What is the API for the proposed change? In your test, you pass an
> instance of transaction created on ignite(0) to the ignite instance
> ignite(1). This is obviously not possible in a truly distributed
> (multi-jvm) environment.
> - How will you synchronize cache update actions and transaction commit?
> Say, you have one node that decided to commit, but another node is still
> writing within this transaction. How do you make sure that two nodes will
> not call commit() and rollback() simultaneously?
>  - How do you make sure that either commit() or rollback() is called if an
> originator failed?
>
> 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов :
>
> > Alexey Goncharuk, heh, my initial understanding was that transferring of
> tx
> > ownership from one node to another will be happened automatically when
> > originating node is gone down.
> >
> > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > Im aiming to span transaction on multiple threads, nodes, jvms(soon).
> So
> > > every node is able to rollback, or commit common transaction.It turned
> > up i
> > > need to transfer tx between nodes in order to commit transaction in
> > > different node(in the same jvm).
> > >
> > > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > > >:
> > >
> > > > Aleksey,
> > > >
> > > > Do you mean that you want a concept of transferring of tx ownership
> > from
> > > > one node to another? My initial understanding was that you want to be
> > > able
> > > > to update keys in a transaction from multiple threads in parallel.
> > > >
> > > > --AG
> > > >
> > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > Well. Consider transaction started in one node, and continued in
> > > another
> > > > > one.
> > > > > The following test describes my idea:
> > > > >
> > > > > Ignite ignite1 = ignite(0);
> > > > >
> > > > > IgniteTransactions transactions = ignite1.transactions();
> > > > >
> > > > > IgniteCache cache = ignite1.getOrCreateCache("
> > > > > testCache");
> > > > >
> > > > > Transaction tx = transactions.txStart(concurrency, isolation);
> > > > >
> > > > > cache.put("key1", 1);
> > > > >
> > > > > cache.put("key2", 2);
> > > > >
> > > > > tx.stop();
> > > > >
> > > > > IgniteInternalFuture fut = GridTestUtils.runAsync(() -> {
> > > > > IgniteTransactions ts = ignite(1).transactions();
> > > > > Assert.assertNull(ts.tx());
> > > > > Assert.assertEquals(TransactionState.STOPPED, tx.state());
> > > > > ts.txStart(tx);
> > > > > Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> > > > > cache.put("key3", 3);
> > > > > Assert.assertTrue(cache.remove("key2"));
> > > > > tx.commit();
> > > > > return true;
> > > > > });
> > > > >
> > > > > fut.get();
> > > > >
> > > > > Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> > > > > Assert.assertEquals((long)1, (long)cache.get("key1"));
> > > > > Assert.assertEquals((long)3, (long)cache.get("key3"));
> > > > > Assert.assertFalse(cache.containsKey("key2"));
> > > > >
> > > > > In method *ts.txStart(...)* we just rebind *tx* to current thread:
> > > > >
> > > > > public void txStart(Transaction tx) {
> > > > > TransactionProxyImpl transactionProxy =
> (TransactionProxyImpl)tx;
> > > > > cctx.tm().reopenTx(transactionProxy.tx());
> > > > > transactionProxy.bindToCurrentThread();
> > > > > }
> > > > >
> > > > > In method *reopenTx* we alter *threadMap* so that it binds
> > transaction
> > > > > to current thread.
> > > > >
> > > > > How do u think about it ?
> > > > >
> > > > >
> > > > > вт, 7 мар. 2017 г. в 22:38, Denis Magda :
> > > > >
> > > > > > Hi Alexey,
> > > > > >
> > > > > > Please share the rational behind this and the thoughts, design
> > ideas
> > > > you
> > > > > > have in mind.
> > > > > >
> > > > > > —
> > > > > > Denis
> > > > > >
> > > > > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com>
> > > > > > 

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

while starting and continuing transaction in different jvms in run into
serialization exception in writeExternalMeta :

@Override public void writeExternal(ObjectOutput out) throws IOException {
writeExternalMeta(out);

some meta is cannot be serialized.
пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk :

> Aleksey,
>
> I think I am starting to get what you want, but I have a few concerns:
>  - What is the API for the proposed change? In your test, you pass an
> instance of transaction created on ignite(0) to the ignite instance
> ignite(1). This is obviously not possible in a truly distributed
> (multi-jvm) environment.
> - How will you synchronize cache update actions and transaction commit?
> Say, you have one node that decided to commit, but another node is still
> writing within this transaction. How do you make sure that two nodes will
> not call commit() and rollback() simultaneously?
>  - How do you make sure that either commit() or rollback() is called if an
> originator failed?
>
> 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов :
>
> > Alexey Goncharuk, heh, my initial understanding was that transferring of
> tx
> > ownership from one node to another will be happened automatically when
> > originating node is gone down.
> >
> > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > Im aiming to span transaction on multiple threads, nodes, jvms(soon).
> So
> > > every node is able to rollback, or commit common transaction.It turned
> > up i
> > > need to transfer tx between nodes in order to commit transaction in
> > > different node(in the same jvm).
> > >
> > > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > > >:
> > >
> > > > Aleksey,
> > > >
> > > > Do you mean that you want a concept of transferring of tx ownership
> > from
> > > > one node to another? My initial understanding was that you want to be
> > > able
> > > > to update keys in a transaction from multiple threads in parallel.
> > > >
> > > > --AG
> > > >
> > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > Well. Consider transaction started in one node, and continued in
> > > another
> > > > > one.
> > > > > The following test describes my idea:
> > > > >
> > > > > Ignite ignite1 = ignite(0);
> > > > >
> > > > > IgniteTransactions transactions = ignite1.transactions();
> > > > >
> > > > > IgniteCache cache = ignite1.getOrCreateCache("
> > > > > testCache");
> > > > >
> > > > > Transaction tx = transactions.txStart(concurrency, isolation);
> > > > >
> > > > > cache.put("key1", 1);
> > > > >
> > > > > cache.put("key2", 2);
> > > > >
> > > > > tx.stop();
> > > > >
> > > > > IgniteInternalFuture fut = GridTestUtils.runAsync(() -> {
> > > > > IgniteTransactions ts = ignite(1).transactions();
> > > > > Assert.assertNull(ts.tx());
> > > > > Assert.assertEquals(TransactionState.STOPPED, tx.state());
> > > > > ts.txStart(tx);
> > > > > Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> > > > > cache.put("key3", 3);
> > > > > Assert.assertTrue(cache.remove("key2"));
> > > > > tx.commit();
> > > > > return true;
> > > > > });
> > > > >
> > > > > fut.get();
> > > > >
> > > > > Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> > > > > Assert.assertEquals((long)1, (long)cache.get("key1"));
> > > > > Assert.assertEquals((long)3, (long)cache.get("key3"));
> > > > > Assert.assertFalse(cache.containsKey("key2"));
> > > > >
> > > > > In method *ts.txStart(...)* we just rebind *tx* to current thread:
> > > > >
> > > > > public void txStart(Transaction tx) {
> > > > > TransactionProxyImpl transactionProxy =
> (TransactionProxyImpl)tx;
> > > > > cctx.tm().reopenTx(transactionProxy.tx());
> > > > > transactionProxy.bindToCurrentThread();
> > > > > }
> > > > >
> > > > > In method *reopenTx* we alter *threadMap* so that it binds
> > transaction
> > > > > to current thread.
> > > > >
> > > > > How do u think about it ?
> > > > >
> > > > >
> > > > > вт, 7 мар. 2017 г. в 22:38, Denis Magda :
> > > > >
> > > > > > Hi Alexey,
> > > > > >
> > > > > > Please share the rational behind this and the thoughts, design
> > ideas
> > > > you
> > > > > > have in mind.
> > > > > >
> > > > > > —
> > > > > > Denis
> > > > > >
> > > > > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi all! Im designing distributed transaction which can be
> started
> > > at
> > > > > one
> > > > > > > node, and continued at other one. Has anybody thoughts on it ?
> > > > > > > --
> > > > > > >
> > > > > > > *Best Regards,*
> > > > > > >
> > > > > > > *Kuznetsov Aleksey*
> > > > > >
> > > > > > --
> > > > >
> > > > > *Best Regards,*
> > > > >
> > > > > *Kuznetsov Aleksey*
> > > > >
> > > >
> > > --
> > >
> > > *Best Regards,*
> > >
> > > 

Re: distributed transaction of non-single coordinator

[Дмитрий Рябов]

What about to send special message to random/choosen node when we start
transaction? And when rollback procedure begins - this second node will
check state of originating node and if it is down then second node became
originating?

2017-03-10 17:25 GMT+03:00 Alexey Goncharuk :

> Aleksey,
>
> I think I am starting to get what you want, but I have a few concerns:
>  - What is the API for the proposed change? In your test, you pass an
> instance of transaction created on ignite(0) to the ignite instance
> ignite(1). This is obviously not possible in a truly distributed
> (multi-jvm) environment.
> - How will you synchronize cache update actions and transaction commit?
> Say, you have one node that decided to commit, but another node is still
> writing within this transaction. How do you make sure that two nodes will
> not call commit() and rollback() simultaneously?
>  - How do you make sure that either commit() or rollback() is called if an
> originator failed?
>
> 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов :
>
> > Alexey Goncharuk, heh, my initial understanding was that transferring of
> tx
> > ownership from one node to another will be happened automatically when
> > originating node is gone down.
> >
> > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > Im aiming to span transaction on multiple threads, nodes, jvms(soon).
> So
> > > every node is able to rollback, or commit common transaction.It turned
> > up i
> > > need to transfer tx between nodes in order to commit transaction in
> > > different node(in the same jvm).
> > >
> > > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > > >:
> > >
> > > > Aleksey,
> > > >
> > > > Do you mean that you want a concept of transferring of tx ownership
> > from
> > > > one node to another? My initial understanding was that you want to be
> > > able
> > > > to update keys in a transaction from multiple threads in parallel.
> > > >
> > > > --AG
> > > >
> > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > Well. Consider transaction started in one node, and continued in
> > > another
> > > > > one.
> > > > > The following test describes my idea:
> > > > >
> > > > > Ignite ignite1 = ignite(0);
> > > > >
> > > > > IgniteTransactions transactions = ignite1.transactions();
> > > > >
> > > > > IgniteCache cache = ignite1.getOrCreateCache("
> > > > > testCache");
> > > > >
> > > > > Transaction tx = transactions.txStart(concurrency, isolation);
> > > > >
> > > > > cache.put("key1", 1);
> > > > >
> > > > > cache.put("key2", 2);
> > > > >
> > > > > tx.stop();
> > > > >
> > > > > IgniteInternalFuture fut = GridTestUtils.runAsync(() -> {
> > > > > IgniteTransactions ts = ignite(1).transactions();
> > > > > Assert.assertNull(ts.tx());
> > > > > Assert.assertEquals(TransactionState.STOPPED, tx.state());
> > > > > ts.txStart(tx);
> > > > > Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> > > > > cache.put("key3", 3);
> > > > > Assert.assertTrue(cache.remove("key2"));
> > > > > tx.commit();
> > > > > return true;
> > > > > });
> > > > >
> > > > > fut.get();
> > > > >
> > > > > Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> > > > > Assert.assertEquals((long)1, (long)cache.get("key1"));
> > > > > Assert.assertEquals((long)3, (long)cache.get("key3"));
> > > > > Assert.assertFalse(cache.containsKey("key2"));
> > > > >
> > > > > In method *ts.txStart(...)* we just rebind *tx* to current thread:
> > > > >
> > > > > public void txStart(Transaction tx) {
> > > > > TransactionProxyImpl transactionProxy =
> (TransactionProxyImpl)tx;
> > > > > cctx.tm().reopenTx(transactionProxy.tx());
> > > > > transactionProxy.bindToCurrentThread();
> > > > > }
> > > > >
> > > > > In method *reopenTx* we alter *threadMap* so that it binds
> > transaction
> > > > > to current thread.
> > > > >
> > > > > How do u think about it ?
> > > > >
> > > > >
> > > > > вт, 7 мар. 2017 г. в 22:38, Denis Magda :
> > > > >
> > > > > > Hi Alexey,
> > > > > >
> > > > > > Please share the rational behind this and the thoughts, design
> > ideas
> > > > you
> > > > > > have in mind.
> > > > > >
> > > > > > —
> > > > > > Denis
> > > > > >
> > > > > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi all! Im designing distributed transaction which can be
> started
> > > at
> > > > > one
> > > > > > > node, and continued at other one. Has anybody thoughts on it ?
> > > > > > > --
> > > > > > >
> > > > > > > *Best Regards,*
> > > > > > >
> > > > > > > *Kuznetsov Aleksey*
> > > > > >
> > > > > > --
> > > > >
> > > > > *Best Regards,*
> > > > >
> > > > > *Kuznetsov Aleksey*
> > > > >
> > > >
> > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> > >
> >
>

Re: distributed transaction of non-single coordinator

[Alexey Goncharuk]

Aleksey,

I think I am starting to get what you want, but I have a few concerns:
 - What is the API for the proposed change? In your test, you pass an
instance of transaction created on ignite(0) to the ignite instance
ignite(1). This is obviously not possible in a truly distributed
(multi-jvm) environment.
- How will you synchronize cache update actions and transaction commit?
Say, you have one node that decided to commit, but another node is still
writing within this transaction. How do you make sure that two nodes will
not call commit() and rollback() simultaneously?
 - How do you make sure that either commit() or rollback() is called if an
originator failed?

2017-03-10 15:38 GMT+03:00 Дмитрий Рябов :

> Alexey Goncharuk, heh, my initial understanding was that transferring of tx
> ownership from one node to another will be happened automatically when
> originating node is gone down.
>
> 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > Im aiming to span transaction on multiple threads, nodes, jvms(soon). So
> > every node is able to rollback, or commit common transaction.It turned
> up i
> > need to transfer tx between nodes in order to commit transaction in
> > different node(in the same jvm).
> >
> > пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> >
> > > Aleksey,
> > >
> > > Do you mean that you want a concept of transferring of tx ownership
> from
> > > one node to another? My initial understanding was that you want to be
> > able
> > > to update keys in a transaction from multiple threads in parallel.
> > >
> > > --AG
> > >
> > > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV  >:
> > >
> > > > Well. Consider transaction started in one node, and continued in
> > another
> > > > one.
> > > > The following test describes my idea:
> > > >
> > > > Ignite ignite1 = ignite(0);
> > > >
> > > > IgniteTransactions transactions = ignite1.transactions();
> > > >
> > > > IgniteCache cache = ignite1.getOrCreateCache("
> > > > testCache");
> > > >
> > > > Transaction tx = transactions.txStart(concurrency, isolation);
> > > >
> > > > cache.put("key1", 1);
> > > >
> > > > cache.put("key2", 2);
> > > >
> > > > tx.stop();
> > > >
> > > > IgniteInternalFuture fut = GridTestUtils.runAsync(() -> {
> > > > IgniteTransactions ts = ignite(1).transactions();
> > > > Assert.assertNull(ts.tx());
> > > > Assert.assertEquals(TransactionState.STOPPED, tx.state());
> > > > ts.txStart(tx);
> > > > Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> > > > cache.put("key3", 3);
> > > > Assert.assertTrue(cache.remove("key2"));
> > > > tx.commit();
> > > > return true;
> > > > });
> > > >
> > > > fut.get();
> > > >
> > > > Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> > > > Assert.assertEquals((long)1, (long)cache.get("key1"));
> > > > Assert.assertEquals((long)3, (long)cache.get("key3"));
> > > > Assert.assertFalse(cache.containsKey("key2"));
> > > >
> > > > In method *ts.txStart(...)* we just rebind *tx* to current thread:
> > > >
> > > > public void txStart(Transaction tx) {
> > > > TransactionProxyImpl transactionProxy = (TransactionProxyImpl)tx;
> > > > cctx.tm().reopenTx(transactionProxy.tx());
> > > > transactionProxy.bindToCurrentThread();
> > > > }
> > > >
> > > > In method *reopenTx* we alter *threadMap* so that it binds
> transaction
> > > > to current thread.
> > > >
> > > > How do u think about it ?
> > > >
> > > >
> > > > вт, 7 мар. 2017 г. в 22:38, Denis Magda :
> > > >
> > > > > Hi Alexey,
> > > > >
> > > > > Please share the rational behind this and the thoughts, design
> ideas
> > > you
> > > > > have in mind.
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY KUZNETSOV <
> > > > alkuznetsov...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > Hi all! Im designing distributed transaction which can be started
> > at
> > > > one
> > > > > > node, and continued at other one. Has anybody thoughts on it ?
> > > > > > --
> > > > > >
> > > > > > *Best Regards,*
> > > > > >
> > > > > > *Kuznetsov Aleksey*
> > > > >
> > > > > --
> > > >
> > > > *Best Regards,*
> > > >
> > > > *Kuznetsov Aleksey*
> > > >
> > >
> > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
> >
>

Re: distributed transaction of non-single coordinator

[Дмитрий Рябов]

Alexey Goncharuk, heh, my initial understanding was that transferring of tx
ownership from one node to another will be happened automatically when
originating node is gone down.

2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV :

> Im aiming to span transaction on multiple threads, nodes, jvms(soon). So
> every node is able to rollback, or commit common transaction.It turned up i
> need to transfer tx between nodes in order to commit transaction in
> different node(in the same jvm).
>
> пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk  >:
>
> > Aleksey,
> >
> > Do you mean that you want a concept of transferring of tx ownership from
> > one node to another? My initial understanding was that you want to be
> able
> > to update keys in a transaction from multiple threads in parallel.
> >
> > --AG
> >
> > 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV :
> >
> > > Well. Consider transaction started in one node, and continued in
> another
> > > one.
> > > The following test describes my idea:
> > >
> > > Ignite ignite1 = ignite(0);
> > >
> > > IgniteTransactions transactions = ignite1.transactions();
> > >
> > > IgniteCache cache = ignite1.getOrCreateCache("
> > > testCache");
> > >
> > > Transaction tx = transactions.txStart(concurrency, isolation);
> > >
> > > cache.put("key1", 1);
> > >
> > > cache.put("key2", 2);
> > >
> > > tx.stop();
> > >
> > > IgniteInternalFuture fut = GridTestUtils.runAsync(() -> {
> > > IgniteTransactions ts = ignite(1).transactions();
> > > Assert.assertNull(ts.tx());
> > > Assert.assertEquals(TransactionState.STOPPED, tx.state());
> > > ts.txStart(tx);
> > > Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> > > cache.put("key3", 3);
> > > Assert.assertTrue(cache.remove("key2"));
> > > tx.commit();
> > > return true;
> > > });
> > >
> > > fut.get();
> > >
> > > Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> > > Assert.assertEquals((long)1, (long)cache.get("key1"));
> > > Assert.assertEquals((long)3, (long)cache.get("key3"));
> > > Assert.assertFalse(cache.containsKey("key2"));
> > >
> > > In method *ts.txStart(...)* we just rebind *tx* to current thread:
> > >
> > > public void txStart(Transaction tx) {
> > > TransactionProxyImpl transactionProxy = (TransactionProxyImpl)tx;
> > > cctx.tm().reopenTx(transactionProxy.tx());
> > > transactionProxy.bindToCurrentThread();
> > > }
> > >
> > > In method *reopenTx* we alter *threadMap* so that it binds transaction
> > > to current thread.
> > >
> > > How do u think about it ?
> > >
> > >
> > > вт, 7 мар. 2017 г. в 22:38, Denis Magda :
> > >
> > > > Hi Alexey,
> > > >
> > > > Please share the rational behind this and the thoughts, design ideas
> > you
> > > > have in mind.
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Hi all! Im designing distributed transaction which can be started
> at
> > > one
> > > > > node, and continued at other one. Has anybody thoughts on it ?
> > > > > --
> > > > >
> > > > > *Best Regards,*
> > > > >
> > > > > *Kuznetsov Aleksey*
> > > >
> > > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> > >
> >
> --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Im aiming to span transaction on multiple threads, nodes, jvms(soon). So
every node is able to rollback, or commit common transaction.It turned up i
need to transfer tx between nodes in order to commit transaction in
different node(in the same jvm).

пт, 10 мар. 2017 г. в 15:20, Alexey Goncharuk :

> Aleksey,
>
> Do you mean that you want a concept of transferring of tx ownership from
> one node to another? My initial understanding was that you want to be able
> to update keys in a transaction from multiple threads in parallel.
>
> --AG
>
> 2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV :
>
> > Well. Consider transaction started in one node, and continued in another
> > one.
> > The following test describes my idea:
> >
> > Ignite ignite1 = ignite(0);
> >
> > IgniteTransactions transactions = ignite1.transactions();
> >
> > IgniteCache cache = ignite1.getOrCreateCache("
> > testCache");
> >
> > Transaction tx = transactions.txStart(concurrency, isolation);
> >
> > cache.put("key1", 1);
> >
> > cache.put("key2", 2);
> >
> > tx.stop();
> >
> > IgniteInternalFuture fut = GridTestUtils.runAsync(() -> {
> > IgniteTransactions ts = ignite(1).transactions();
> > Assert.assertNull(ts.tx());
> > Assert.assertEquals(TransactionState.STOPPED, tx.state());
> > ts.txStart(tx);
> > Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> > cache.put("key3", 3);
> > Assert.assertTrue(cache.remove("key2"));
> > tx.commit();
> > return true;
> > });
> >
> > fut.get();
> >
> > Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> > Assert.assertEquals((long)1, (long)cache.get("key1"));
> > Assert.assertEquals((long)3, (long)cache.get("key3"));
> > Assert.assertFalse(cache.containsKey("key2"));
> >
> > In method *ts.txStart(...)* we just rebind *tx* to current thread:
> >
> > public void txStart(Transaction tx) {
> > TransactionProxyImpl transactionProxy = (TransactionProxyImpl)tx;
> > cctx.tm().reopenTx(transactionProxy.tx());
> > transactionProxy.bindToCurrentThread();
> > }
> >
> > In method *reopenTx* we alter *threadMap* so that it binds transaction
> > to current thread.
> >
> > How do u think about it ?
> >
> >
> > вт, 7 мар. 2017 г. в 22:38, Denis Magda :
> >
> > > Hi Alexey,
> > >
> > > Please share the rational behind this and the thoughts, design ideas
> you
> > > have in mind.
> > >
> > > —
> > > Denis
> > >
> > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com>
> > > wrote:
> > > >
> > > > Hi all! Im designing distributed transaction which can be started at
> > one
> > > > node, and continued at other one. Has anybody thoughts on it ?
> > > > --
> > > >
> > > > *Best Regards,*
> > > >
> > > > *Kuznetsov Aleksey*
> > >
> > > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
> >
>
-- 

*Best Regards,*

*Kuznetsov Aleksey*

Re: distributed transaction of non-single coordinator

[Alexey Goncharuk]

Aleksey,

Do you mean that you want a concept of transferring of tx ownership from
one node to another? My initial understanding was that you want to be able
to update keys in a transaction from multiple threads in parallel.

--AG

2017-03-10 15:01 GMT+03:00 ALEKSEY KUZNETSOV :

> Well. Consider transaction started in one node, and continued in another
> one.
> The following test describes my idea:
>
> Ignite ignite1 = ignite(0);
>
> IgniteTransactions transactions = ignite1.transactions();
>
> IgniteCache cache = ignite1.getOrCreateCache("
> testCache");
>
> Transaction tx = transactions.txStart(concurrency, isolation);
>
> cache.put("key1", 1);
>
> cache.put("key2", 2);
>
> tx.stop();
>
> IgniteInternalFuture fut = GridTestUtils.runAsync(() -> {
> IgniteTransactions ts = ignite(1).transactions();
> Assert.assertNull(ts.tx());
> Assert.assertEquals(TransactionState.STOPPED, tx.state());
> ts.txStart(tx);
> Assert.assertEquals(TransactionState.ACTIVE, tx.state());
> cache.put("key3", 3);
> Assert.assertTrue(cache.remove("key2"));
> tx.commit();
> return true;
> });
>
> fut.get();
>
> Assert.assertEquals(TransactionState.COMMITTED, tx.state());
> Assert.assertEquals((long)1, (long)cache.get("key1"));
> Assert.assertEquals((long)3, (long)cache.get("key3"));
> Assert.assertFalse(cache.containsKey("key2"));
>
> In method *ts.txStart(...)* we just rebind *tx* to current thread:
>
> public void txStart(Transaction tx) {
> TransactionProxyImpl transactionProxy = (TransactionProxyImpl)tx;
> cctx.tm().reopenTx(transactionProxy.tx());
> transactionProxy.bindToCurrentThread();
> }
>
> In method *reopenTx* we alter *threadMap* so that it binds transaction
> to current thread.
>
> How do u think about it ?
>
>
> вт, 7 мар. 2017 г. в 22:38, Denis Magda :
>
> > Hi Alexey,
> >
> > Please share the rational behind this and the thoughts, design ideas you
> > have in mind.
> >
> > —
> > Denis
> >
> > > On Mar 7, 2017, at 3:19 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com>
> > wrote:
> > >
> > > Hi all! Im designing distributed transaction which can be started at
> one
> > > node, and continued at other one. Has anybody thoughts on it ?
> > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> >
> > --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>

Re: distributed transaction of non-single coordinator

[ALEKSEY KUZNETSOV]

Well. Consider transaction started in one node, and continued in another
one.
The following test describes my idea:

Ignite ignite1 = ignite(0);

IgniteTransactions transactions = ignite1.transactions();

IgniteCache cache = ignite1.getOrCreateCache("testCache");

Transaction tx = transactions.txStart(concurrency, isolation);

cache.put("key1", 1);

cache.put("key2", 2);

tx.stop();

IgniteInternalFuture fut = GridTestUtils.runAsync(() -> {
IgniteTransactions ts = ignite(1).transactions();
Assert.assertNull(ts.tx());
Assert.assertEquals(TransactionState.STOPPED, tx.state());
ts.txStart(tx);
Assert.assertEquals(TransactionState.ACTIVE, tx.state());
cache.put("key3", 3);
Assert.assertTrue(cache.remove("key2"));
tx.commit();
return true;
});

fut.get();

Assert.assertEquals(TransactionState.COMMITTED, tx.state());
Assert.assertEquals((long)1, (long)cache.get("key1"));
Assert.assertEquals((long)3, (long)cache.get("key3"));
Assert.assertFalse(cache.containsKey("key2"));

In method *ts.txStart(...)* we just rebind *tx* to current thread:

public void txStart(Transaction tx) {
TransactionProxyImpl transactionProxy = (TransactionProxyImpl)tx;
cctx.tm().reopenTx(transactionProxy.tx());
transactionProxy.bindToCurrentThread();
}

In method *reopenTx* we alter *threadMap* so that it binds transaction
to current thread.

How do u think about it ?


вт, 7 мар. 2017 г. в 22:38, Denis Magda :

> Hi Alexey,
>
> Please share the rational behind this and the thoughts, design ideas you
> have in mind.
>
> —
> Denis
>
> > On Mar 7, 2017, at 3:19 AM, ALEKSEY KUZNETSOV 
> wrote:
> >
> > Hi all! Im designing distributed transaction which can be started at one
> > node, and continued at other one. Has anybody thoughts on it ?
> > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
>
> --

*Best Regards,*

*Kuznetsov Aleksey*