Okay, we are open for proposals on business task. I mean, we can make use of some other thing, not distributed transaction. Not transaction yet.
ср, 15 мар. 2017 г. в 11:24, Vladimir Ozerov <voze...@gridgain.com>: > IMO the use case makes sense. However, as Sergi already mentioned, the > problem is far more complex, than simply passing TX state over a wire. Most > probably a kind of coordinator will be required still to manage all kinds > of failures. This task should be started with clean design proposal > explaining how we handle all these concurrent events. And only then, when > we understand all implications, we should move to development stage. > > On Wed, Mar 15, 2017 at 10:38 AM, ALEKSEY KUZNETSOV < > alkuznetsov...@gmail.com> wrote: > > > Right > > > > ср, 15 мар. 2017 г. в 10:35, Sergi Vladykin <sergi.vlady...@gmail.com>: > > > > > Good! Basically your orchestrator just takes some predefined graph of > > > distributed services to be invoked, calls them by some kind of RPC and > > > passes the needed parameters between them, right? > > > > > > Sergi > > > > > > 2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV <alkuznetsov...@gmail.com > >: > > > > > > > orchestrator is a custom thing. He is responsible for managing > business > > > > scenarios flows. Many nodes are involved in scenarios. They exchange > > data > > > > and folow one another. If you acquinted with BPMN framework, so > > > > orchestrator is like bpmn engine. > > > > > > > > вт, 14 Мар 2017 г., 18:56 Sergi Vladykin <sergi.vlady...@gmail.com>: > > > > > > > > > What is Orchestrator for you? Is it a thing from Microsoft or your > > > custom > > > > > in-house software? > > > > > > > > > > Sergi > > > > > > > > > > 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV < > > alkuznetsov...@gmail.com > > > >: > > > > > > > > > > > Fine. Let's say we've got multiple servers which fulfills custom > > > logic. > > > > > > This servers compound oriented graph (BPMN process) which > > controlled > > > by > > > > > > Orchestrator. > > > > > > For instance, *server1 *creates *variable A *with value 1, > > persists > > > it > > > > > to > > > > > > IGNITE cache and creates *variable B *and sends it to* server2. > > *The > > > > > > latests receives *variable B*, do some logic with it and stores > to > > > > > IGNITE. > > > > > > All the work made by both servers must be fulfilled in *one* > > > > transaction. > > > > > > Because we need all information done, or nothing(rollbacked). The > > > > > scenario > > > > > > is managed by orchestrator. > > > > > > > > > > > > вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin < > > > sergi.vlady...@gmail.com > > > > >: > > > > > > > > > > > > > Ok, it is not a business case, it is your wrong solution for > it. > > > > > > > Lets try again, what is the business case? > > > > > > > > > > > > > > Sergi > > > > > > > > > > > > > > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV < > > > > alkuznetsov...@gmail.com > > > > > >: > > > > > > > > > > > > > > > The case is the following, One starts transaction in one > node, > > > and > > > > > > commit > > > > > > > > this transaction in another jvm node(or rollback it > remotely). > > > > > > > > > > > > > > > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin < > > > > > sergi.vlady...@gmail.com > > > > > > >: > > > > > > > > > > > > > > > > > Because even if you make it work for some simplistic > > scenario, > > > > get > > > > > > > ready > > > > > > > > to > > > > > > > > > write many fault tolerance tests and make sure that you TXs > > > work > > > > > > > > gracefully > > > > > > > > > in all modes in case of crashes. Also make sure that we do > > not > > > > have > > > > > > any > > > > > > > > > performance drops after all your changes in existing > > > benchmarks. > > > > > All > > > > > > in > > > > > > > > all > > > > > > > > > I don't believe these conditions will be met and your > > > > contribution > > > > > > will > > > > > > > > be > > > > > > > > > accepted. > > > > > > > > > > > > > > > > > > Better solution to what problem? Sending TX to another > node? > > > The > > > > > > > problem > > > > > > > > > statement itself is already wrong. What business case you > are > > > > > trying > > > > > > to > > > > > > > > > solve? I'm sure everything you need can be done in a much > > more > > > > > simple > > > > > > > and > > > > > > > > > efficient way at the application level. > > > > > > > > > > > > > > > > > > Sergi > > > > > > > > > > > > > > > > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV < > > > > > > alkuznetsov...@gmail.com > > > > > > > >: > > > > > > > > > > > > > > > > > > > Why wrong ? You know the better solution? > > > > > > > > > > > > > > > > > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin < > > > > > > > sergi.vlady...@gmail.com > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > Just serializing TX object and deserializing it on > > another > > > > node > > > > > > is > > > > > > > > > > > meaningless, because other nodes participating in the > TX > > > have > > > > > to > > > > > > > know > > > > > > > > > > about > > > > > > > > > > > the new coordinator. This will require protocol > changes, > > we > > > > > > > > definitely > > > > > > > > > > will > > > > > > > > > > > have fault tolerance and performance issues. IMO the > > whole > > > > idea > > > > > > is > > > > > > > > > wrong > > > > > > > > > > > and it makes no sense to waste time on it. > > > > > > > > > > > > > > > > > > > > > > Sergi > > > > > > > > > > > > > > > > > > > > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV < > > > > > > > > alkuznetsov...@gmail.com > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > IgniteTransactionState implememntation contains > > > > > IgniteTxEntry's > > > > > > > > which > > > > > > > > > > is > > > > > > > > > > > > supposed to be transferable > > > > > > > > > > > > > > > > > > > > > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan < > > > > > > > > > dsetrak...@apache.org > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > It sounds a little scary to me that we are passing > > > > > > transaction > > > > > > > > > > objects > > > > > > > > > > > > > around. Such object may contain all sorts of Ignite > > > > > context. > > > > > > If > > > > > > > > > some > > > > > > > > > > > data > > > > > > > > > > > > > needs to be passed across, we should create a > special > > > > > > transfer > > > > > > > > > object > > > > > > > > > > > in > > > > > > > > > > > > > this case. > > > > > > > > > > > > > > > > > > > > > > > > > > D. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV > < > > > > > > > > > > > > > alkuznetsov...@gmail.com > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > well, there a couple of issues preventing > > transaction > > > > > > > > proceeding. > > > > > > > > > > > > > > At first, After transaction serialization and > > > > > > deserialization > > > > > > > > on > > > > > > > > > > the > > > > > > > > > > > > > remote > > > > > > > > > > > > > > server, there is no txState. So im going to put > it > > in > > > > > > > > > > > > > > writeExternal()\readExternal() > > > > > > > > > > > > > > > > > > > > > > > > > > > > The last one is Deserialized transaction lacks of > > > > shared > > > > > > > cache > > > > > > > > > > > context > > > > > > > > > > > > > > field at TransactionProxyImpl. Perhaps, it must > be > > > > > injected > > > > > > > by > > > > > > > > > > > > > > GridResourceProcessor ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV < > > > > > > > > > > > > alkuznetsov...@gmail.com > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > while starting and continuing transaction in > > > > different > > > > > > jvms > > > > > > > > in > > > > > > > > > > run > > > > > > > > > > > > into > > > > > > > > > > > > > > > serialization exception in writeExternalMeta : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > @Override public void > writeExternal(ObjectOutput > > > out) > > > > > > > throws > > > > > > > > > > > > > IOException > > > > > > > > > > > > > > { > > > > > > > > > > > > > > > writeExternalMeta(out); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > some meta is cannot be serialized. > > > > > > > > > > > > > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk < > > > > > > > > > > > > > > alexey.goncha...@gmail.com > > > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Aleksey, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think I am starting to get what you want, > but I > > > > have > > > > > a > > > > > > > few > > > > > > > > > > > > concerns: > > > > > > > > > > > > > > > - What is the API for the proposed change? In > > your > > > > > test, > > > > > > > you > > > > > > > > > > pass > > > > > > > > > > > an > > > > > > > > > > > > > > > instance of transaction created on ignite(0) to > > the > > > > > > ignite > > > > > > > > > > instance > > > > > > > > > > > > > > > ignite(1). This is obviously not possible in a > > > truly > > > > > > > > > distributed > > > > > > > > > > > > > > > (multi-jvm) environment. > > > > > > > > > > > > > > > - How will you synchronize cache update actions > > and > > > > > > > > transaction > > > > > > > > > > > > commit? > > > > > > > > > > > > > > > Say, you have one node that decided to commit, > > but > > > > > > another > > > > > > > > node > > > > > > > > > > is > > > > > > > > > > > > > still > > > > > > > > > > > > > > > writing within this transaction. How do you > make > > > sure > > > > > > that > > > > > > > > two > > > > > > > > > > > nodes > > > > > > > > > > > > > will > > > > > > > > > > > > > > > not call commit() and rollback() > simultaneously? > > > > > > > > > > > > > > > - How do you make sure that either commit() or > > > > > > rollback() > > > > > > > is > > > > > > > > > > > called > > > > > > > > > > > > if > > > > > > > > > > > > > > an > > > > > > > > > > > > > > > originator failed? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов < > > > > > > > > > somefire...@gmail.com > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Alexey Goncharuk, heh, my initial > understanding > > > was > > > > > > that > > > > > > > > > > > > transferring > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > tx > > > > > > > > > > > > > > > > ownership from one node to another will be > > > happened > > > > > > > > > > automatically > > > > > > > > > > > > > when > > > > > > > > > > > > > > > > originating node is gone down. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV > < > > > > > > > > > > > > > alkuznetsov...@gmail.com > > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Im aiming to span transaction on multiple > > > > threads, > > > > > > > nodes, > > > > > > > > > > > > > jvms(soon). > > > > > > > > > > > > > > > So > > > > > > > > > > > > > > > > > every node is able to rollback, or commit > > > common > > > > > > > > > > transaction.It > > > > > > > > > > > > > > turned > > > > > > > > > > > > > > > > up i > > > > > > > > > > > > > > > > > need to transfer tx between nodes in order > to > > > > > commit > > > > > > > > > > > transaction > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > different node(in the same jvm). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > пт, 10 мар. 2017 г. в 15:20, Alexey > > Goncharuk < > > > > > > > > > > > > > > > > alexey.goncha...@gmail.com > > > > > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Aleksey, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do you mean that you want a concept of > > > > > transferring > > > > > > > of > > > > > > > > tx > > > > > > > > > > > > > ownership > > > > > > > > > > > > > > > > from > > > > > > > > > > > > > > > > > > one node to another? My initial > > understanding > > > > was > > > > > > > that > > > > > > > > > you > > > > > > > > > > > want > > > > > > > > > > > > > to > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > able > > > > > > > > > > > > > > > > > > to update keys in a transaction from > > multiple > > > > > > threads > > > > > > > > in > > > > > > > > > > > > > parallel. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --AG > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY > > KUZNETSOV > > > < > > > > > > > > > > > > > > > alkuznetsov...@gmail.com > > > > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Well. Consider transaction started in > one > > > > node, > > > > > > and > > > > > > > > > > > continued > > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > another > > > > > > > > > > > > > > > > > > > one. > > > > > > > > > > > > > > > > > > > The following test describes my idea: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ignite ignite1 = ignite(0); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IgniteTransactions transactions = > > > > > > > > > ignite1.transactions(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IgniteCache<String, Integer> cache = > > > > > > > > > > > > ignite1.getOrCreateCache(" > > > > > > > > > > > > > > > > > > > testCache"); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Transaction tx = transactions.txStart( > > > > > > concurrency, > > > > > > > > > > > > isolation); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > cache.put("key1", 1); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > cache.put("key2", 2); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > tx.stop(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IgniteInternalFuture<Boolean> fut = > > > > > > > > > > > GridTestUtils.runAsync(() > > > > > > > > > > > > > -> > > > > > > > > > > > > > > { > > > > > > > > > > > > > > > > > > > IgniteTransactions ts = > > > > > > > ignite(1).transactions(); > > > > > > > > > > > > > > > > > > > Assert.assertNull(ts.tx()); > > > > > > > > > > > > > > > > > > > Assert.assertEquals( > > > > > > TransactionState.STOPPED, > > > > > > > > > > > > tx.state()); > > > > > > > > > > > > > > > > > > > ts.txStart(tx); > > > > > > > > > > > > > > > > > > > > > > > > Assert.assertEquals(TransactionState.ACTIVE, > > > > > > > > > > > > tx.state()); > > > > > > > > > > > > > > > > > > > cache.put("key3", 3); > > > > > > > > > > > > > > > > > > > > > > Assert.assertTrue(cache.remove("key2")); > > > > > > > > > > > > > > > > > > > tx.commit(); > > > > > > > > > > > > > > > > > > > return true; > > > > > > > > > > > > > > > > > > > }); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > fut.get(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Assert.assertEquals( > > > > TransactionState.COMMITTED, > > > > > > > > > > > tx.state()); > > > > > > > > > > > > > > > > > > > Assert.assertEquals((long)1, > > > > > > > > (long)cache.get("key1")); > > > > > > > > > > > > > > > > > > > Assert.assertEquals((long)3, > > > > > > > > (long)cache.get("key3")); > > > > > > > > > > > > > > > > > > > Assert.assertFalse(cache. > > > > containsKey("key2")); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In method *ts.txStart(...)* we just > > rebind > > > > *tx* > > > > > > to > > > > > > > > > > current > > > > > > > > > > > > > > thread: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > public void txStart(Transaction tx) { > > > > > > > > > > > > > > > > > > > TransactionProxyImpl > > transactionProxy = > > > > > > > > > > > > > > > (TransactionProxyImpl)tx; > > > > > > > > > > > > > > > > > > > cctx.tm().reopenTx( > > > > transactionProxy.tx()); > > > > > > > > > > > > > > > > > > > transactionProxy. > > bindToCurrentThread(); > > > > > > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In method *reopenTx* we alter > *threadMap* > > > so > > > > > that > > > > > > > it > > > > > > > > > > binds > > > > > > > > > > > > > > > > transaction > > > > > > > > > > > > > > > > > > > to current thread. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > How do u think about it ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > вт, 7 мар. 2017 г. в 22:38, Denis > Magda < > > > > > > > > > > dma...@apache.org > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Alexey, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please share the rational behind this > > and > > > > the > > > > > > > > > thoughts, > > > > > > > > > > > > > design > > > > > > > > > > > > > > > > ideas > > > > > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > > > > > > have in mind. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > — > > > > > > > > > > > > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY > > > > > > KUZNETSOV < > > > > > > > > > > > > > > > > > > > alkuznetsov...@gmail.com> > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi all! Im designing distributed > > > > > transaction > > > > > > > > which > > > > > > > > > > can > > > > > > > > > > > be > > > > > > > > > > > > > > > started > > > > > > > > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > one > > > > > > > > > > > > > > > > > > > > > node, and continued at other one. > Has > > > > > anybody > > > > > > > > > > thoughts > > > > > > > > > > > on > > > > > > > > > > > > > it > > > > > > > > > > > > > > ? > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > *Best Regards,* > > > > > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > > > > > > > -- > > > > > > > > *Best Regards,* > > > > > > > > *Kuznetsov Aleksey* > > > > > > > > > -- > > > > *Best Regards,* > > > > *Kuznetsov Aleksey* > > > -- *Best Regards,* *Kuznetsov Aleksey*