Thanks for starting this, great ideas. We should start exploring such options but I am thinking of some more limitations that we'll have to overcome. To keep the discussion from getting hopelessly complex, let's try to clarify the purpose:
The goal is to solve two problems which are strongly related: # performance / efficiency # better ACIDity of "transactions" # Efficiency Radim makes a great point, it would be nice to be able to address the optimal node for any "task" we might want to perform. JIRA: - https://issues.jboss.org/browse/ISPN-8770 However while we wait for the Infinispan team to provide such refinements, I think we should go ahead with this plan already, not least for the other reasons, but also to be able to eventually provide quick feedback on such improvements. Clearly this means we'll incur (possibly?) a performance hit when running a simple task - such as writing a single key - but it shouldn't matter much for more complex operations: especially with a small sized cluster chances are high to hit a node owning at least one of these. For the moment our client could detect if it's preferrable to run a task or a "simple op"; essentially we could implement some of ISPN-8770 in our Dialect but rather than making this too complex on our side I'd contribute to the Infinispan improvement first. # "one transaction, one task" ? Just to clarify, we can't promise that the ORM won't need to flush multiple times before the transaction is finally committed. Make sure you read about auto-flush strategies, either in our docs or there are many good articles about the topic. In a nutshell, if there are dirty entities of some type A, and then a query is run on that same type A, the ORM engine needs to flush the pending changes on all As first to make sure the queries are accurate. We could debate if this is appropriate and maybe we shouldn't do it on all NoSQL stores, but it's certainly debatable as it's a change in semantics compared to what people are used to. It's not entirely bad news though; while a single transaction might actually be performed by multiple, independent although ordered "Task RPC"s this will give us some opportunity to at least make sure that operations which really need to be atomic will happen atomically; for example when we delete an entity we can make sure that all references are cleared up correctly. However it's not a real transaction still and we need to be clear about that, no big deal as "real transactions" will come soon in Infinispan Remote too. # Usability Just a note on this. I hope we can try to work on this while expecting that the Infinispan Server won't need to have specific code deployed which depends on the specific application, such as the object model. If it helps we can think of an "OGM extensions" to be included with the Infinispan Server, but the code we include would then need to be able to work with any schema. Ideally, even with multiple versions of OGM, so I'd hope that we can design the models of each task we'll need as Protobuf Entities. Fabio, do you want to start working on such a POC? Thanks, Sanne On 5 February 2018 at 15:05, Fabio Massimo Ercoli <ferc...@redhat.com> wrote: > Hi Radim, > > thank you for the alternative, it is very interesting. > I've to study deeply the feature :P > > Fabio > > > On Mon, Feb 5, 2018 at 9:53 AM, Radim Vansa <rva...@redhat.com> wrote: > >> Hi Fabio, >> >> thinking about the cons, Hot Rod can intelligently pick the server where >> should a given key reside, to reduce the number of hops (therefore, >> latency). RemoteCache does not expose any way to route according to some >> key; feel free to file a feature request in Infinispan JIRA. >> >> Note that in order to reduce the number of round-trips, Infinispan 9.2 >> will support true-async operations: Previously the putAsync et all just >> moved the execution to different thread, since 9.2.0.CR2 it sends the >> request to the wire right await and the response is received through a >> CompletableFuture. At this moment multiple requests will need a distinct >> TCP connection for each concurrent operation, but this limitation is >> likely to be removed in 9.3. The goal is to let you write the batch down >> one-by-one using async API and just wait for all the operations to >> complete. >> >> I know this won't help for all the types of operation (a lack of >> client-side API sometimes forces OGM to use get() + CAS in a loop). >> >> I am not trying to talk you out of the remote execution API, just >> spreading news about the emerging alternatives. >> >> Radim >> >> On 02/02/2018 09:24 AM, Fabio Massimo Ercoli wrote: >> > I'm Fabio, nice to meet you. >> > >> > Speaking of the current implementation of the Infinispan remote dialect >> of >> > Hibernate OGM, working on issue OGM-1206 and talking with Davide I >> noticed >> > that the unit of work commands are executed (flushed to datastore) at the >> > end of the transaction itself. >> > In particular I noticed that the commands are stored in a transaction >> > scoped object of type org.hibernate.ogm.dialect. >> batch.spi.OperationsQueue. >> > >> > Instead of perfom one remote invocation for each command in the method >> > org.hibernate.ogm.dialect.impl.BatchOperationsDelegator::executeBatch >> > maybe we could invoke a proper Infispan Remote Server Task to execute the >> > command queue on server side as a bulk operation. >> > >> > Moving the execution of the server-side command list (Infinispan) we >> would >> > have the advantage of reducing remote interactions. Moreover and above >> all >> > the execution of the command queue would be a transactional work unit, on >> > which could be apply a Repeteable Read isolation level, for instance. >> > >> > The solution would not solve the need for an XA client instead, but it >> > could be adopted by customers interested in local transactions. >> > >> > What do you think about it? >> > Can I open a Jira issue? >> > >> > Fabio >> > >> >> -- >> Radim Vansa <rva...@redhat.com> >> JBoss Performance Team >> >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > > > > -- > > FABIO MASSIMO ERCOLI > > Senior Software Engineer - Hibernate & Data Platform > > Red Hat <https://www.redhat.com/> > > fabio.erc...@redhat.com M: (+39)-329.8681441 > <http://redhatemailsignature-marketing.itos.redhat.com/> > <https://red.ht/sig> > _______________________________________________ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev