IMO the use case makes sense. However, as Sergi already mentioned, the
problem is far more complex, than simply passing TX state over a wire. Most
probably a kind of coordinator will be required still to manage all kinds
of failures. This task should be started with clean design proposal
explaining how we handle all these concurrent events. And only then, when
we understand all implications, we should move to development stage.
On Wed, Mar 15, 2017 at 10:38 AM, ALEKSEY KUZNETSOV <
alkuznetsov...@gmail.com> wrote:
> Right
>
> ср, 15 мар. 2017 г. в 10:35, Sergi Vladykin <sergi.vlady...@gmail.com>:
>
> > Good! Basically your orchestrator just takes some predefined graph of
> > distributed services to be invoked, calls them by some kind of RPC and
> > passes the needed parameters between them, right?
> >
> > Sergi
> >
> > 2017-03-14 22:46 GMT+03:00 ALEKSEY KUZNETSOV <alkuznetsov...@gmail.com>:
> >
> > > orchestrator is a custom thing. He is responsible for managing business
> > > scenarios flows. Many nodes are involved in scenarios. They exchange
> data
> > > and folow one another. If you acquinted with BPMN framework, so
> > > orchestrator is like bpmn engine.
> > >
> > > вт, 14 Мар 2017 г., 18:56 Sergi Vladykin <sergi.vlady...@gmail.com>:
> > >
> > > > What is Orchestrator for you? Is it a thing from Microsoft or your
> > custom
> > > > in-house software?
> > > >
> > > > Sergi
> > > >
> > > > 2017-03-14 18:00 GMT+03:00 ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > >:
> > > >
> > > > > Fine. Let's say we've got multiple servers which fulfills custom
> > logic.
> > > > > This servers compound oriented graph (BPMN process) which
> controlled
> > by
> > > > > Orchestrator.
> > > > > For instance, *server1 *creates *variable A *with value 1,
> persists
> > it
> > > > to
> > > > > IGNITE cache and creates *variable B *and sends it to* server2.
> *The
> > > > > latests receives *variable B*, do some logic with it and stores to
> > > > IGNITE.
> > > > > All the work made by both servers must be fulfilled in *one*
> > > transaction.
> > > > > Because we need all information done, or nothing(rollbacked). The
> > > > scenario
> > > > > is managed by orchestrator.
> > > > >
> > > > > вт, 14 мар. 2017 г. в 17:31, Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > >:
> > > > >
> > > > > > Ok, it is not a business case, it is your wrong solution for it.
> > > > > > Lets try again, what is the business case?
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-03-14 16:42 GMT+03:00 ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > >:
> > > > > >
> > > > > > > The case is the following, One starts transaction in one node,
> > and
> > > > > commit
> > > > > > > this transaction in another jvm node(or rollback it remotely).
> > > > > > >
> > > > > > > вт, 14 мар. 2017 г. в 16:30, Sergi Vladykin <
> > > > sergi.vlady...@gmail.com
> > > > > >:
> > > > > > >
> > > > > > > > Because even if you make it work for some simplistic
> scenario,
> > > get
> > > > > > ready
> > > > > > > to
> > > > > > > > write many fault tolerance tests and make sure that you TXs
> > work
> > > > > > > gracefully
> > > > > > > > in all modes in case of crashes. Also make sure that we do
> not
> > > have
> > > > > any
> > > > > > > > performance drops after all your changes in existing
> > benchmarks.
> > > > All
> > > > > in
> > > > > > > all
> > > > > > > > I don't believe these conditions will be met and your
> > > contribution
> > > > > will
> > > > > > > be
> > > > > > > > accepted.
> > > > > > > >
> > > > > > > > Better solution to what problem? Sending TX to another node?
> > The
> > > > > > problem
> > > > > > > > statement itself is already wrong. What business case you are
> > > > trying
> > > > > to
> > > > > > > > solve? I'm sure everything you need can be done in a much
> more
> > > > simple
> > > > > > and
> > > > > > > > efficient way at the application level.
> > > > > > > >
> > > > > > > > Sergi
> > > > > > > >
> > > > > > > > 2017-03-14 16:03 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com
> > > > > > >:
> > > > > > > >
> > > > > > > > > Why wrong ? You know the better solution?
> > > > > > > > >
> > > > > > > > > вт, 14 мар. 2017 г. в 15:46, Sergi Vladykin <
> > > > > > sergi.vlady...@gmail.com
> > > > > > > >:
> > > > > > > > >
> > > > > > > > > > Just serializing TX object and deserializing it on
> another
> > > node
> > > > > is
> > > > > > > > > > meaningless, because other nodes participating in the TX
> > have
> > > > to
> > > > > > know
> > > > > > > > > about
> > > > > > > > > > the new coordinator. This will require protocol changes,
> we
> > > > > > > definitely
> > > > > > > > > will
> > > > > > > > > > have fault tolerance and performance issues. IMO the
> whole
> > > idea
> > > > > is
> > > > > > > > wrong
> > > > > > > > > > and it makes no sense to waste time on it.
> > > > > > > > > >
> > > > > > > > > > Sergi
> > > > > > > > > >
> > > > > > > > > > 2017-03-14 10:57 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > > > alkuznetsov...@gmail.com
> > > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > > IgniteTransactionState implememntation contains
> > > > IgniteTxEntry's
> > > > > > > which
> > > > > > > > > is
> > > > > > > > > > > supposed to be transferable
> > > > > > > > > > >
> > > > > > > > > > > пн, 13 мар. 2017 г. в 19:32, Dmitriy Setrakyan <
> > > > > > > > dsetrak...@apache.org
> > > > > > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > It sounds a little scary to me that we are passing
> > > > > transaction
> > > > > > > > > objects
> > > > > > > > > > > > around. Such object may contain all sorts of Ignite
> > > > context.
> > > > > If
> > > > > > > > some
> > > > > > > > > > data
> > > > > > > > > > > > needs to be passed across, we should create a special
> > > > > transfer
> > > > > > > > object
> > > > > > > > > > in
> > > > > > > > > > > > this case.
> > > > > > > > > > > >
> > > > > > > > > > > > D.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Mar 13, 2017 at 9:10 AM, ALEKSEY KUZNETSOV <
> > > > > > > > > > > > alkuznetsov...@gmail.com
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > well, there a couple of issues preventing
> transaction
> > > > > > > proceeding.
> > > > > > > > > > > > > At first, After transaction serialization and
> > > > > deserialization
> > > > > > > on
> > > > > > > > > the
> > > > > > > > > > > > remote
> > > > > > > > > > > > > server, there is no txState. So im going to put it
> in
> > > > > > > > > > > > > writeExternal()\readExternal()
> > > > > > > > > > > > >
> > > > > > > > > > > > > The last one is Deserialized transaction lacks of
> > > shared
> > > > > > cache
> > > > > > > > > > context
> > > > > > > > > > > > > field at TransactionProxyImpl. Perhaps, it must be
> > > > injected
> > > > > > by
> > > > > > > > > > > > > GridResourceProcessor ?
> > > > > > > > > > > > >
> > > > > > > > > > > > > пн, 13 мар. 2017 г. в 17:27, ALEKSEY KUZNETSOV <
> > > > > > > > > > > alkuznetsov...@gmail.com
> > > > > > > > > > > > >:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > while starting and continuing transaction in
> > > different
> > > > > jvms
> > > > > > > in
> > > > > > > > > run
> > > > > > > > > > > into
> > > > > > > > > > > > > > serialization exception in writeExternalMeta :
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > @Override public void writeExternal(ObjectOutput
> > out)
> > > > > > throws
> > > > > > > > > > > > IOException
> > > > > > > > > > > > > {
> > > > > > > > > > > > > > writeExternalMeta(out);
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > some meta is cannot be serialized.
> > > > > > > > > > > > > > пт, 10 мар. 2017 г. в 17:25, Alexey Goncharuk <
> > > > > > > > > > > > > alexey.goncha...@gmail.com
> > > > > > > > > > > > > > >:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Aleksey,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I think I am starting to get what you want, but I
> > > have
> > > > a
> > > > > > few
> > > > > > > > > > > concerns:
> > > > > > > > > > > > > > - What is the API for the proposed change? In
> your
> > > > test,
> > > > > > you
> > > > > > > > > pass
> > > > > > > > > > an
> > > > > > > > > > > > > > instance of transaction created on ignite(0) to
> the
> > > > > ignite
> > > > > > > > > instance
> > > > > > > > > > > > > > ignite(1). This is obviously not possible in a
> > truly
> > > > > > > > distributed
> > > > > > > > > > > > > > (multi-jvm) environment.
> > > > > > > > > > > > > > - How will you synchronize cache update actions
> and
> > > > > > > transaction
> > > > > > > > > > > commit?
> > > > > > > > > > > > > > Say, you have one node that decided to commit,
> but
> > > > > another
> > > > > > > node
> > > > > > > > > is
> > > > > > > > > > > > still
> > > > > > > > > > > > > > writing within this transaction. How do you make
> > sure
> > > > > that
> > > > > > > two
> > > > > > > > > > nodes
> > > > > > > > > > > > will
> > > > > > > > > > > > > > not call commit() and rollback() simultaneously?
> > > > > > > > > > > > > > - How do you make sure that either commit() or
> > > > > rollback()
> > > > > > is
> > > > > > > > > > called
> > > > > > > > > > > if
> > > > > > > > > > > > > an
> > > > > > > > > > > > > > originator failed?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2017-03-10 15:38 GMT+03:00 Дмитрий Рябов <
> > > > > > > > somefire...@gmail.com
> > > > > > > > > >:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Alexey Goncharuk, heh, my initial understanding
> > was
> > > > > that
> > > > > > > > > > > transferring
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > tx
> > > > > > > > > > > > > > > ownership from one node to another will be
> > happened
> > > > > > > > > automatically
> > > > > > > > > > > > when
> > > > > > > > > > > > > > > originating node is gone down.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 2017-03-10 15:36 GMT+03:00 ALEKSEY KUZNETSOV <
> > > > > > > > > > > > alkuznetsov...@gmail.com
> > > > > > > > > > > > > >:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Im aiming to span transaction on multiple
> > > threads,
> > > > > > nodes,
> > > > > > > > > > > > jvms(soon).
> > > > > > > > > > > > > > So
> > > > > > > > > > > > > > > > every node is able to rollback, or commit
> > common
> > > > > > > > > transaction.It
> > > > > > > > > > > > > turned
> > > > > > > > > > > > > > > up i
> > > > > > > > > > > > > > > > need to transfer tx between nodes in order to
> > > > commit
> > > > > > > > > > transaction
> > > > > > > > > > > in
> > > > > > > > > > > > > > > > different node(in the same jvm).
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > пт, 10 мар. 2017 г. в 15:20, Alexey
> Goncharuk <
> > > > > > > > > > > > > > > alexey.goncha...@gmail.com
> > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Aleksey,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Do you mean that you want a concept of
> > > > transferring
> > > > > > of
> > > > > > > tx
> > > > > > > > > > > > ownership
> > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > one node to another? My initial
> understanding
> > > was
> > > > > > that
> > > > > > > > you
> > > > > > > > > > want
> > > > > > > > > > > > to
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > able
> > > > > > > > > > > > > > > > > to update keys in a transaction from
> multiple
> > > > > threads
> > > > > > > in
> > > > > > > > > > > > parallel.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > --AG
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > 2017-03-10 15:01 GMT+03:00 ALEKSEY
> KUZNETSOV
> > <
> > > > > > > > > > > > > > alkuznetsov...@gmail.com
> > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Well. Consider transaction started in one
> > > node,
> > > > > and
> > > > > > > > > > continued
> > > > > > > > > > > > in
> > > > > > > > > > > > > > > > another
> > > > > > > > > > > > > > > > > > one.
> > > > > > > > > > > > > > > > > > The following test describes my idea:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Ignite ignite1 = ignite(0);
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > IgniteTransactions transactions =
> > > > > > > > ignite1.transactions();
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > IgniteCache<String, Integer> cache =
> > > > > > > > > > > ignite1.getOrCreateCache("
> > > > > > > > > > > > > > > > > > testCache");
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Transaction tx = transactions.txStart(
> > > > > concurrency,
> > > > > > > > > > > isolation);
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > cache.put("key1", 1);
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > cache.put("key2", 2);
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > tx.stop();
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > IgniteInternalFuture<Boolean> fut =
> > > > > > > > > > GridTestUtils.runAsync(()
> > > > > > > > > > > > ->
> > > > > > > > > > > > > {
> > > > > > > > > > > > > > > > > > IgniteTransactions ts =
> > > > > > ignite(1).transactions();
> > > > > > > > > > > > > > > > > > Assert.assertNull(ts.tx());
> > > > > > > > > > > > > > > > > > Assert.assertEquals(
> > > > > TransactionState.STOPPED,
> > > > > > > > > > > tx.state());
> > > > > > > > > > > > > > > > > > ts.txStart(tx);
> > > > > > > > > > > > > > > > > >
> > > > Assert.assertEquals(TransactionState.ACTIVE,
> > > > > > > > > > > tx.state());
> > > > > > > > > > > > > > > > > > cache.put("key3", 3);
> > > > > > > > > > > > > > > > > >
> > Assert.assertTrue(cache.remove("key2"));
> > > > > > > > > > > > > > > > > > tx.commit();
> > > > > > > > > > > > > > > > > > return true;
> > > > > > > > > > > > > > > > > > });
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > fut.get();
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Assert.assertEquals(
> > > TransactionState.COMMITTED,
> > > > > > > > > > tx.state());
> > > > > > > > > > > > > > > > > > Assert.assertEquals((long)1,
> > > > > > > (long)cache.get("key1"));
> > > > > > > > > > > > > > > > > > Assert.assertEquals((long)3,
> > > > > > > (long)cache.get("key3"));
> > > > > > > > > > > > > > > > > > Assert.assertFalse(cache.
> > > containsKey("key2"));
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > In method *ts.txStart(...)* we just
> rebind
> > > *tx*
> > > > > to
> > > > > > > > > current
> > > > > > > > > > > > > thread:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > public void txStart(Transaction tx) {
> > > > > > > > > > > > > > > > > > TransactionProxyImpl
> transactionProxy =
> > > > > > > > > > > > > > (TransactionProxyImpl)tx;
> > > > > > > > > > > > > > > > > > cctx.tm().reopenTx(
> > > transactionProxy.tx());
> > > > > > > > > > > > > > > > > > transactionProxy.
> bindToCurrentThread();
> > > > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > In method *reopenTx* we alter *threadMap*
> > so
> > > > that
> > > > > > it
> > > > > > > > > binds
> > > > > > > > > > > > > > > transaction
> > > > > > > > > > > > > > > > > > to current thread.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > How do u think about it ?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > вт, 7 мар. 2017 г. в 22:38, Denis Magda <
> > > > > > > > > dma...@apache.org
> > > > > > > > > > >:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Hi Alexey,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Please share the rational behind this
> and
> > > the
> > > > > > > > thoughts,
> > > > > > > > > > > > design
> > > > > > > > > > > > > > > ideas
> > > > > > > > > > > > > > > > > you
> > > > > > > > > > > > > > > > > > > have in mind.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > —
> > > > > > > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > On Mar 7, 2017, at 3:19 AM, ALEKSEY
> > > > > KUZNETSOV <
> > > > > > > > > > > > > > > > > > alkuznetsov...@gmail.com>
> > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Hi all! Im designing distributed
> > > > transaction
> > > > > > > which
> > > > > > > > > can
> > > > > > > > > > be
> > > > > > > > > > > > > > started
> > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > one
> > > > > > > > > > > > > > > > > > > > node, and continued at other one. Has
> > > > anybody
> > > > > > > > > thoughts
> > > > > > > > > > on
> > > > > > > > > > > > it
> > > > > > > > > > > > > ?
> > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > *Best Regards,*
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > *Best Regards,*
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > *Best Regards,*
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *Best Regards,*
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > >
> > > > > > > > > > > > > *Best Regards,*
> > > > > > > > > > > > >
> > > > > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > >
> > > > > > > > > > > *Best Regards,*
> > > > > > > > > > >
> > > > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > *Best Regards,*
> > > > > > > > >
> > > > > > > > > *Kuznetsov Aleksey*
> > > > > > > > >
> > > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > *Best Regards,*
> > > > > > >
> > > > > > > *Kuznetsov Aleksey*
> > > > > > >
> > > > > >
> > > > > --
> > > > >
> > > > > *Best Regards,*
> > > > >
> > > > > *Kuznetsov Aleksey*
> > > > >
> > > >
> > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> > >
> >
> --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>
