Also, I've updated IEP regarding features masks (new backward compatibility mechanism for thin clients) which was discussed earlier in this thread.
вт, 31 мар. 2020 г. в 02:42, Alex Plehanov <plehanov.a...@gmail.com>: > Guys, > > I've finished IEP implementation for server-side and java thin client side > (ticket: [1], PR: [2]). Can you please review the patch? > > [1]: https://issues.apache.org/jira/browse/IGNITE-12835 > [2]: https://github.com/apache/ignite/pull/7572 > > пт, 27 мар. 2020 г. в 19:11, Pavel Tupitsyn <ptupit...@apache.org>: > >> Agree with Igor, let's set deployment aside for now, it is out of scope of >> this IEP. >> >> >> >> On Fri, Mar 27, 2020 at 6:52 PM Igor Sapego <isap...@apache.org> wrote: >> >> > Hi guys, >> > >> > I like the proposal in general. >> > >> > Now for the task deployment - I think we should have separate API >> > for it (and separate IEP I believe). Also, I'm not sure that this API >> > should be a part of the API of any thin client as it seems weird to me >> > to use Python client to deploy Java tasks. Control.sh or visor proposal >> > sounds much better. >> > >> > Best Regards, >> > Igor >> > >> > >> > On Thu, Mar 26, 2020 at 10:56 PM Denis Magda <dma...@apache.org> wrote: >> > >> > > > >> > > > Deployment API definitely needed as one of the next steps. >> Currently, >> > we >> > > > are talking only about the first step (execution of already deployed >> > > > tasks). >> > > > Also, I'm not sure about automatic redeploy and peer-class-loading >> for >> > > thin >> > > > clients, I think it's better to have more control here and provide >> API >> > to >> > > > explicitly deploy classes or jar files. WDYT? >> > > >> > > >> > > Alex, agree that automatic redeployment is better suited for the >> > management >> > > APIs. How about adding this capability to our command-line tool >> > > (control.sh, or visor cmd, or one new holistic tool). >> > > >> > > - >> > > Denis >> > > >> > > >> > > On Wed, Mar 25, 2020 at 1:04 PM Alex Plehanov < >> plehanov.a...@gmail.com> >> > > wrote: >> > > >> > > > Pavel, >> > > > >> > > > 1. Actually it can be solved on the client-side (and already solved >> in >> > > PoC >> > > > implementation). But I agreed it brings extra complexity for >> > client-side >> > > > implementation, will try to provide such guarantees on the >> server-side. >> > > > 2. ComputeTask has also "reduce" step which is executed on the >> > initiator >> > > > node. Binary-rest client implementation, for example, has such >> affinity >> > > > methods (to execute the task by name). I'm ok with removing it. At >> > least >> > > if >> > > > someone will need it we can implement it again at any time in the >> > future >> > > > without protocol change. >> > > > I've fixed IEP. >> > > > >> > > > Denis, >> > > > >> > > > Deployment API definitely needed as one of the next steps. >> Currently, >> > we >> > > > are talking only about the first step (execution of already deployed >> > > > tasks). >> > > > Also, I'm not sure about automatic redeploy and peer-class-loading >> for >> > > thin >> > > > clients, I think it's better to have more control here and provide >> API >> > to >> > > > explicitly deploy classes or jar files. WDYT? >> > > > >> > > > ср, 25 мар. 2020 г. в 21:17, Denis Magda <dma...@apache.org>: >> > > > >> > > > > Alex, thanks for preparing the outline. >> > > > > >> > > > > I'd like us to discuss an approach for compute tasks update with >> no >> > > > > downtimes on the servers' end. For instance, let's assume that a >> > > > > Python/C++/Node.JS developer requested to update a compute task he >> > > called >> > > > > from the app. Should we introduce some system level API to the >> binary >> > > > > protocol that can take a jar file (or class) and redeploy it >> > > > automatically >> > > > > with the usage of peer-class-loading? >> > > > > >> > > > > - >> > > > > Denis >> > > > > >> > > > > >> > > > > On Wed, Mar 25, 2020 at 5:47 AM Alex Plehanov < >> > plehanov.a...@gmail.com >> > > > >> > > > > wrote: >> > > > > >> > > > > > Hello guys. >> > > > > > >> > > > > > I've implemented PoC and created IEP [1] for thin client compute >> > grid >> > > > > > functionality. Please have a look. >> > > > > > >> > > > > > [1]: >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-42+Thin+client%3A+compute+support >> > > > > > >> > > > > > пт, 24 янв. 2020 г. в 16:56, Alex Plehanov < >> > plehanov.a...@gmail.com >> > > >: >> > > > > > >> > > > > > > We've discussed thin client compute protocol with Pavel >> Tupitsyn >> > > and >> > > > > Igor >> > > > > > > Sapego and come to the conclusion that approach with two-way >> > > requests >> > > > > > > should be used: client generates taskId and send a request to >> the >> > > > > server >> > > > > > to >> > > > > > > execute a task. The server responds that the request has been >> > > > accepted. >> > > > > > > After task has finished the server notifies the client (send a >> > > > request >> > > > > > > without waiting for a response). The client can cancel the >> task >> > by >> > > > > > sending >> > > > > > > a corresponding request to the server. >> > > > > > > >> > > > > > > Also, a node list should be passed (optionally) with a >> request to >> > > > limit >> > > > > > > nodes to execute the task. >> > > > > > > >> > > > > > > I will create IEP and file detailed protocol changes shortly. >> > > > > > > >> > > > > > > вт, 21 янв. 2020 г. в 18:46, Alex Plehanov < >> > > plehanov.a...@gmail.com >> > > > >: >> > > > > > > >> > > > > > >> Igor, thanks for the reply. >> > > > > > >> >> > > > > > >> > Approach with taskId will require a lot of changes in >> protocol >> > > and >> > > > > > thus >> > > > > > >> more "heavy" for implementation >> > > > > > >> Do you mean approach with server notifications mechanism? >> Yes, >> > it >> > > > will >> > > > > > >> require a lot of changes. But in most recent messages we've >> > > > discussed >> > > > > > with >> > > > > > >> Pavel approach without server notifications mechanism. This >> > > approach >> > > > > > have >> > > > > > >> the same complexity and performance as an approach with >> > requestId. >> > > > > > >> >> > > > > > >> > But such clients as Python, Node.js, PHP, Go most probably >> > won't >> > > > > have >> > > > > > >> support for this API, at least for now. >> > > > > > >> Without a server notifications mechanism, there will be no >> > > breaking >> > > > > > >> changes in the protocol, so client implementation can just >> skip >> > > this >> > > > > > >> feature and protocol version and implement the next one. >> > > > > > >> >> > > > > > >> > Or never. >> > > > > > >> I think it still useful to execute java compute tasks from >> > > non-java >> > > > > thin >> > > > > > >> clients. Also, we can provide some out-of-the-box java tasks, >> > for >> > > > > > example >> > > > > > >> ExecutePythonScriptTask with python compute implementation, >> > which >> > > > can >> > > > > > run >> > > > > > >> python script on server node. >> > > > > > >> >> > > > > > >> > So, maybe it's a good time for us to change our backward >> > > > > compatibility >> > > > > > >> mechanism from protocol versioning to feature masks? >> > > > > > >> I like the idea with feature masks, but it will force us to >> > > support >> > > > > both >> > > > > > >> backward compatibility mechanisms, protocol versioning and >> > feature >> > > > > > masks. >> > > > > > >> >> > > > > > >> пн, 20 янв. 2020 г. в 20:34, Pavel Tupitsyn < >> > ptupit...@apache.org >> > > >: >> > > > > > >> >> > > > > > >>> Huge +1 from me for Feature Masks. >> > > > > > >>> I think this should be our top priority for thin client >> > protocol, >> > > > > since >> > > > > > >>> it >> > > > > > >>> simplifies change management a lot. >> > > > > > >>> >> > > > > > >>> On Mon, Jan 20, 2020 at 8:21 PM Igor Sapego < >> > isap...@apache.org> >> > > > > > wrote: >> > > > > > >>> >> > > > > > >>> > Sorry for the late reply. >> > > > > > >>> > >> > > > > > >>> > Approach with taskId will require a lot of changes in >> > protocol >> > > > and >> > > > > > thus >> > > > > > >>> > more "heavy" for implementation, but it definitely looks >> to >> > me >> > > > less >> > > > > > >>> hacky >> > > > > > >>> > than reqId-approach. Moreover, as was mentioned, server >> > > > > notifications >> > > > > > >>> > mechanism will be required in a future anyway with high >> > > > > probability. >> > > > > > So >> > > > > > >>> > from this point of view I like taskId-approach. >> > > > > > >>> > >> > > > > > >>> > On the other hand, what we should also consider here is >> > > > > performance. >> > > > > > >>> > Speaking of latency, it looks like reqId will have better >> > > results >> > > > > in >> > > > > > >>> case >> > > > > > >>> > of >> > > > > > >>> > small and fast tasks. The only question here, if we want >> to >> > > > > optimize >> > > > > > >>> thin >> > > > > > >>> > clients for this case. >> > > > > > >>> > >> > > > > > >>> > Also, what are you talking about mostly involves clients >> on >> > > > > platforms >> > > > > > >>> > that already have Compute API for thick clients. Let me >> > mention >> > > > one >> > > > > > >>> > more point of view here and another concern here. >> > > > > > >>> > >> > > > > > >>> > The changes you propose are going to change protocol >> version >> > > for >> > > > > > sure. >> > > > > > >>> > In case with taskId approach and server notifications - >> even >> > > more >> > > > > so. >> > > > > > >>> > >> > > > > > >>> > But such clients as Python, Node.js, PHP, Go most probably >> > > won't >> > > > > have >> > > > > > >>> > support for this API, at least for now. Or never. But >> current >> > > > > > >>> > backward-compatibility mechanism implies protocol versions >> > > where >> > > > we >> > > > > > >>> > imply that client that supports version 1.5 also supports >> all >> > > the >> > > > > > >>> features >> > > > > > >>> > introduced in all the previous versions of the protocol. >> > > > > > >>> > >> > > > > > >>> > Thus implementing Compute API in any of the proposed ways >> > *may* >> > > > > > >>> > force mentioned clients to support changes in protocol >> which >> > > they >> > > > > not >> > > > > > >>> > necessarily need in order to introduce new features in the >> > > > future. >> > > > > > >>> > >> > > > > > >>> > So, maybe it's a good time for us to change our backward >> > > > > > compatibility >> > > > > > >>> > mechanism from protocol versioning to feature masks? >> > > > > > >>> > >> > > > > > >>> > WDYT? >> > > > > > >>> > >> > > > > > >>> > Best Regards, >> > > > > > >>> > Igor >> > > > > > >>> > >> > > > > > >>> > >> > > > > > >>> > On Fri, Jan 17, 2020 at 9:37 AM Alex Plehanov < >> > > > > > plehanov.a...@gmail.com >> > > > > > >>> > >> > > > > > >>> > wrote: >> > > > > > >>> > >> > > > > > >>> > > Looks like we didn't rich consensus here. >> > > > > > >>> > > >> > > > > > >>> > > Igor, as thin client maintainer, can you please share >> your >> > > > > opinion? >> > > > > > >>> > > >> > > > > > >>> > > Everyone else also welcome, please share your thoughts >> > about >> > > > > > options >> > > > > > >>> to >> > > > > > >>> > > implement operations for compute. >> > > > > > >>> > > >> > > > > > >>> > > >> > > > > > >>> > > чт, 28 нояб. 2019 г. в 10:02, Alex Plehanov < >> > > > > > plehanov.a...@gmail.com >> > > > > > >>> >: >> > > > > > >>> > > >> > > > > > >>> > > > > Since all thin client operations are inherently >> async, >> > we >> > > > > > should >> > > > > > >>> be >> > > > > > >>> > > able >> > > > > > >>> > > > to cancel any of them >> > > > > > >>> > > > It's illogical to have such ability. What should do >> > cancel >> > > > > > >>> operation of >> > > > > > >>> > > > cancel operation? Moreover, sometimes it's dangerous, >> for >> > > > > > example, >> > > > > > >>> > create >> > > > > > >>> > > > cache operation should never be canceled. There >> should be >> > > an >> > > > > > >>> explicit >> > > > > > >>> > set >> > > > > > >>> > > > of processes that we can cancel: queries, >> transactions, >> > > > tasks, >> > > > > > >>> > services. >> > > > > > >>> > > > The lifecycle of services is more complex than the >> > > lifecycle >> > > > of >> > > > > > >>> tasks. >> > > > > > >>> > > With >> > > > > > >>> > > > services, I suppose, we can't use request >> cancelation, so >> > > > tasks >> > > > > > >>> will be >> > > > > > >>> > > the >> > > > > > >>> > > > only process with an exceptional pattern. >> > > > > > >>> > > > >> > > > > > >>> > > > > The request would be "execute task with specified >> node >> > > > > filter" >> > > > > > - >> > > > > > >>> > simple >> > > > > > >>> > > > and efficient. >> > > > > > >>> > > > It's not simple: every compute or service request >> should >> > > > > contain >> > > > > > >>> > complex >> > > > > > >>> > > > node filtering logic, which duplicates the same logic >> for >> > > > > cluster >> > > > > > >>> API. >> > > > > > >>> > > > It's not efficient: for example, we can't implement >> > > > > > forPredicate() >> > > > > > >>> > > > filtering in this case. >> > > > > > >>> > > > >> > > > > > >>> > > > >> > > > > > >>> > > > ср, 27 нояб. 2019 г. в 19:25, Pavel Tupitsyn < >> > > > > > ptupit...@apache.org >> > > > > > >>> >: >> > > > > > >>> > > > >> > > > > > >>> > > >> > The request is already processed (task is >> started), >> > we >> > > > > can't >> > > > > > >>> cancel >> > > > > > >>> > > the >> > > > > > >>> > > >> request >> > > > > > >>> > > >> The request is not "start a task". It is "execute >> task" >> > > (and >> > > > > get >> > > > > > >>> > > result). >> > > > > > >>> > > >> Same as "cache get" - you get a result in the end, we >> > > don't >> > > > > > "start >> > > > > > >>> > cache >> > > > > > >>> > > >> get" then "end cache get". >> > > > > > >>> > > >> >> > > > > > >>> > > >> Since all thin client operations are inherently >> async, >> > we >> > > > > should >> > > > > > >>> be >> > > > > > >>> > able >> > > > > > >>> > > >> to >> > > > > > >>> > > >> cancel any of them >> > > > > > >>> > > >> by sending another request with an id of prior >> request >> > to >> > > be >> > > > > > >>> > cancelled. >> > > > > > >>> > > >> That's why I'm advocating for this approach - it will >> > work >> > > > for >> > > > > > >>> > anything, >> > > > > > >>> > > >> no >> > > > > > >>> > > >> special cases. >> > > > > > >>> > > >> And it keeps "happy path" as simple as it is right >> now. >> > > > > > >>> > > >> >> > > > > > >>> > > >> Queries are different because we retrieve results in >> > > pages, >> > > > we >> > > > > > >>> can't >> > > > > > >>> > do >> > > > > > >>> > > >> them as one request. >> > > > > > >>> > > >> Transactions are also different because client >> controls >> > > when >> > > > > > they >> > > > > > >>> > should >> > > > > > >>> > > >> end. >> > > > > > >>> > > >> There is no reason for task execution to be a special >> > case >> > > > > like >> > > > > > >>> > queries >> > > > > > >>> > > or >> > > > > > >>> > > >> transactions. >> > > > > > >>> > > >> >> > > > > > >>> > > >> > we always need to send 2 requests to server to >> > execute >> > > > the >> > > > > > task >> > > > > > >>> > > >> Nope. We don't need to get nodes on client at all. >> > > > > > >>> > > >> The request would be "execute task with specified >> node >> > > > > filter" - >> > > > > > >>> > simple >> > > > > > >>> > > >> and >> > > > > > >>> > > >> efficient. >> > > > > > >>> > > >> >> > > > > > >>> > > >> >> > > > > > >>> > > >> On Wed, Nov 27, 2019 at 4:31 PM Alex Plehanov < >> > > > > > >>> > plehanov.a...@gmail.com> >> > > > > > >>> > > >> wrote: >> > > > > > >>> > > >> >> > > > > > >>> > > >> > > We do cancel a request to perform a task. We may >> > and >> > > > > should >> > > > > > >>> use >> > > > > > >>> > > this >> > > > > > >>> > > >> to >> > > > > > >>> > > >> > cancel any other request in future. >> > > > > > >>> > > >> > The request is already processed (task is >> started), we >> > > > can't >> > > > > > >>> cancel >> > > > > > >>> > > the >> > > > > > >>> > > >> > request. As you mentioned before, we already do >> almost >> > > the >> > > > > > same >> > > > > > >>> for >> > > > > > >>> > > >> queries >> > > > > > >>> > > >> > (close the cursor, but not cancel the request to >> run a >> > > > > query), >> > > > > > >>> it's >> > > > > > >>> > > >> better >> > > > > > >>> > > >> > to do such things in a common way. We have a >> pattern: >> > > > start >> > > > > > some >> > > > > > >>> > > process >> > > > > > >>> > > >> > (query, transaction), get id of this process, end >> > > process >> > > > by >> > > > > > >>> this >> > > > > > >>> > id. >> > > > > > >>> > > >> The >> > > > > > >>> > > >> > "Execute task" process should match the same >> pattern. >> > In >> > > > my >> > > > > > >>> opinion, >> > > > > > >>> > > >> > implementation with two-way requests is the best >> > option >> > > to >> > > > > > match >> > > > > > >>> > this >> > > > > > >>> > > >> > pattern (we can even reuse OP_RESOURCE_CLOSE >> operation >> > > > type >> > > > > in >> > > > > > >>> this >> > > > > > >>> > > >> case). >> > > > > > >>> > > >> > Sometime in the future, we will need two-way >> requests >> > > for >> > > > > some >> > > > > > >>> other >> > > > > > >>> > > >> > functionality (continuous queries, event listening, >> > > etc). >> > > > > But >> > > > > > >>> even >> > > > > > >>> > > >> without >> > > > > > >>> > > >> > two-way requests introducing some process id (task >> id >> > in >> > > > our >> > > > > > >>> case) >> > > > > > >>> > > will >> > > > > > >>> > > >> be >> > > > > > >>> > > >> > closer to existing pattern than canceling tasks by >> > > request >> > > > > id. >> > > > > > >>> > > >> > >> > > > > > >>> > > >> > > So every new request will apply those filters on >> > > server >> > > > > > side, >> > > > > > >>> > using >> > > > > > >>> > > >> the >> > > > > > >>> > > >> > most recent set of nodes. >> > > > > > >>> > > >> > In this case, we always need to send 2 requests to >> > > server >> > > > to >> > > > > > >>> execute >> > > > > > >>> > > the >> > > > > > >>> > > >> > task. First - to get nodes by the filter, second - >> to >> > > > > actually >> > > > > > >>> > execute >> > > > > > >>> > > >> the >> > > > > > >>> > > >> > task. It seems like overhead. The same will be for >> > > > services. >> > > > > > >>> Cluster >> > > > > > >>> > > >> group >> > > > > > >>> > > >> > remains the same if the topology hasn't changed. We >> > can >> > > > use >> > > > > > this >> > > > > > >>> > fact >> > > > > > >>> > > >> and >> > > > > > >>> > > >> > bind "execute task" request to topology. If >> topology >> > has >> > > > > > >>> changed - >> > > > > > >>> > get >> > > > > > >>> > > >> > nodes for new topology and retry request. >> > > > > > >>> > > >> > >> > > > > > >>> > > >> > вт, 26 нояб. 2019 г. в 17:44, Pavel Tupitsyn < >> > > > > > >>> ptupit...@apache.org >> > > > > > >>> > >: >> > > > > > >>> > > >> > >> > > > > > >>> > > >> > > > After all, we don't cancel request >> > > > > > >>> > > >> > > We do cancel a request to perform a task. We may >> and >> > > > > should >> > > > > > >>> use >> > > > > > >>> > this >> > > > > > >>> > > >> to >> > > > > > >>> > > >> > > cancel any other request in future. >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > > > Client uses some cluster group filtration (for >> > > example >> > > > > > >>> > > forServers() >> > > > > > >>> > > >> > > cluster group) >> > > > > > >>> > > >> > > Please see above - Aleksandr Shapkin described >> how >> > we >> > > > > store >> > > > > > >>> > > >> > > filtered cluster groups on client. >> > > > > > >>> > > >> > > We don't store node IDs, we store actual >> filters. So >> > > > every >> > > > > > new >> > > > > > >>> > > request >> > > > > > >>> > > >> > will >> > > > > > >>> > > >> > > apply those filters on server side, >> > > > > > >>> > > >> > > using the most recent set of nodes. >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > > var myGrp = >> > cluster.forServers().forAttribute("foo"); >> > > // >> > > > > > This >> > > > > > >>> does >> > > > > > >>> > > not >> > > > > > >>> > > >> > > issue any server requests, just builds an object >> > with >> > > > > > filters >> > > > > > >>> on >> > > > > > >>> > > >> client >> > > > > > >>> > > >> > > while (true) myGrp.compute().executeTask("bar"); >> // >> > > > Every >> > > > > > >>> request >> > > > > > >>> > > >> > includes >> > > > > > >>> > > >> > > filters, and filters are applied on the server >> side >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > > On Tue, Nov 26, 2019 at 1:42 PM Alex Plehanov < >> > > > > > >>> > > >> plehanov.a...@gmail.com> >> > > > > > >>> > > >> > > wrote: >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > > > > Anyway, my point stands. >> > > > > > >>> > > >> > > > I can't agree. Why you don't want to use task >> id >> > for >> > > > > this? >> > > > > > >>> After >> > > > > > >>> > > >> all, >> > > > > > >>> > > >> > we >> > > > > > >>> > > >> > > > don't cancel request (request is already >> > processed), >> > > > we >> > > > > > >>> cancel >> > > > > > >>> > the >> > > > > > >>> > > >> > task. >> > > > > > >>> > > >> > > So >> > > > > > >>> > > >> > > > it's more convenient to use task id here. >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > > > Can you please provide equivalent use case >> with >> > > > > existing >> > > > > > >>> > "thick" >> > > > > > >>> > > >> > > client? >> > > > > > >>> > > >> > > > For example: >> > > > > > >>> > > >> > > > Cluster consists of one server node. >> > > > > > >>> > > >> > > > Client uses some cluster group filtration (for >> > > example >> > > > > > >>> > > forServers() >> > > > > > >>> > > >> > > cluster >> > > > > > >>> > > >> > > > group). >> > > > > > >>> > > >> > > > Client starts to send periodically (for >> example 1 >> > > per >> > > > > > >>> minute) >> > > > > > >>> > > >> long-term >> > > > > > >>> > > >> > > > (for example 1 hour long) tasks to the cluster. >> > > > > > >>> > > >> > > > Meanwhile, several server nodes joined the >> > cluster. >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > > In case of thick client: All server nodes will >> be >> > > > used, >> > > > > > >>> tasks >> > > > > > >>> > will >> > > > > > >>> > > >> be >> > > > > > >>> > > >> > > load >> > > > > > >>> > > >> > > > balanced. >> > > > > > >>> > > >> > > > In case of thin client: Only one server node >> will >> > be >> > > > > used, >> > > > > > >>> > client >> > > > > > >>> > > >> will >> > > > > > >>> > > >> > > > detect topology change after an hour. >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > > вт, 26 нояб. 2019 г. в 11:50, Pavel Tupitsyn < >> > > > > > >>> > > ptupit...@apache.org >> > > > > > >>> > > >> >: >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > > > > I can't see any usage of request id in >> query >> > > > > cursors >> > > > > > >>> > > >> > > > > You are right, cursor id is a separate thing. >> > > > > > >>> > > >> > > > > Anyway, my point stands. >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > > > client sends long term tasks to nodes and >> > wants >> > > to >> > > > > do >> > > > > > it >> > > > > > >>> > with >> > > > > > >>> > > >> load >> > > > > > >>> > > >> > > > > balancing >> > > > > > >>> > > >> > > > > I still don't get it. Can you please provide >> > > > > equivalent >> > > > > > >>> use >> > > > > > >>> > case >> > > > > > >>> > > >> with >> > > > > > >>> > > >> > > > > existing "thick" client? >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > > On Mon, Nov 25, 2019 at 11:59 PM Alex >> Plehanov < >> > > > > > >>> > > >> > > plehanov.a...@gmail.com> >> > > > > > >>> > > >> > > > > wrote: >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > > > > And it is fine to use request ID to >> identify >> > > > > compute >> > > > > > >>> tasks >> > > > > > >>> > > >> (as we >> > > > > > >>> > > >> > > do >> > > > > > >>> > > >> > > > > with >> > > > > > >>> > > >> > > > > > query cursors). >> > > > > > >>> > > >> > > > > > I can't see any usage of request id in >> query >> > > > > cursors. >> > > > > > We >> > > > > > >>> > send >> > > > > > >>> > > >> query >> > > > > > >>> > > >> > > > > request >> > > > > > >>> > > >> > > > > > and get cursor id in response. After that, >> we >> > > only >> > > > > use >> > > > > > >>> > cursor >> > > > > > >>> > > id >> > > > > > >>> > > >> > (to >> > > > > > >>> > > >> > > > get >> > > > > > >>> > > >> > > > > > next pages and to close the resource). Did >> I >> > > miss >> > > > > > >>> something? >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > > > Looks like I'm missing something - how is >> > > > topology >> > > > > > >>> change >> > > > > > >>> > > >> > relevant >> > > > > > >>> > > >> > > to >> > > > > > >>> > > >> > > > > > executing compute tasks from client? >> > > > > > >>> > > >> > > > > > It's not relevant directly. But there are >> some >> > > > cases >> > > > > > >>> where >> > > > > > >>> > it >> > > > > > >>> > > >> will >> > > > > > >>> > > >> > be >> > > > > > >>> > > >> > > > > > helpful. For example, if client sends long >> > term >> > > > > tasks >> > > > > > to >> > > > > > >>> > nodes >> > > > > > >>> > > >> and >> > > > > > >>> > > >> > > > wants >> > > > > > >>> > > >> > > > > to >> > > > > > >>> > > >> > > > > > do it with load balancing it will detect >> > > topology >> > > > > > change >> > > > > > >>> > only >> > > > > > >>> > > >> after >> > > > > > >>> > > >> > > > some >> > > > > > >>> > > >> > > > > > time in the future with the first >> response, so >> > > > load >> > > > > > >>> > balancing >> > > > > > >>> > > >> will >> > > > > > >>> > > >> > no >> > > > > > >>> > > >> > > > > work. >> > > > > > >>> > > >> > > > > > Perhaps we can add optional "topology >> version" >> > > > field >> > > > > > to >> > > > > > >>> the >> > > > > > >>> > > >> > > > > > OP_COMPUTE_EXECUTE_TASK request to solve >> this >> > > > > problem. >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > > пн, 25 нояб. 2019 г. в 22:42, Pavel >> Tupitsyn < >> > > > > > >>> > > >> ptupit...@apache.org >> > > > > > >>> > > >> > >: >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > > > Alex, >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > > > we will mix entities from different >> layers >> > > > > > >>> (transport >> > > > > > >>> > > layer >> > > > > > >>> > > >> and >> > > > > > >>> > > >> > > > > request >> > > > > > >>> > > >> > > > > > > body) >> > > > > > >>> > > >> > > > > > > I would not call our message header >> (which >> > > > > includes >> > > > > > >>> the >> > > > > > >>> > id) >> > > > > > >>> > > >> > > > "transport >> > > > > > >>> > > >> > > > > > > layer". >> > > > > > >>> > > >> > > > > > > TCP is our transport layer. And it is >> fine >> > to >> > > > use >> > > > > > >>> request >> > > > > > >>> > ID >> > > > > > >>> > > >> to >> > > > > > >>> > > >> > > > > identify >> > > > > > >>> > > >> > > > > > > compute tasks (as we do with query >> cursors). >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > > > we still can't be sure that the task is >> > > > > > successfully >> > > > > > >>> > > started >> > > > > > >>> > > >> > on a >> > > > > > >>> > > >> > > > > > server >> > > > > > >>> > > >> > > > > > > The request to start the task will fail >> and >> > > > we'll >> > > > > > get >> > > > > > >>> a >> > > > > > >>> > > >> response >> > > > > > >>> > > >> > > > > > indicating >> > > > > > >>> > > >> > > > > > > that right away >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > > > we won't ever know about topology >> change >> > > > > > >>> > > >> > > > > > > Looks like I'm missing something - how is >> > > > topology >> > > > > > >>> change >> > > > > > >>> > > >> > relevant >> > > > > > >>> > > >> > > to >> > > > > > >>> > > >> > > > > > > executing compute tasks from client? >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > > On Mon, Nov 25, 2019 at 10:17 PM Alex >> > > Plehanov < >> > > > > > >>> > > >> > > > > plehanov.a...@gmail.com> >> > > > > > >>> > > >> > > > > > > wrote: >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > > > Pavel, in this case, we will mix >> entities >> > > from >> > > > > > >>> different >> > > > > > >>> > > >> layers >> > > > > > >>> > > >> > > > > > > (transport >> > > > > > >>> > > >> > > > > > > > layer and request body), it's not very >> > good. >> > > > The >> > > > > > >>> same >> > > > > > >>> > > >> behavior >> > > > > > >>> > > >> > we >> > > > > > >>> > > >> > > > can >> > > > > > >>> > > >> > > > > > > > achieve with generated on client-side >> task >> > > id, >> > > > > but >> > > > > > >>> there >> > > > > > >>> > > >> will >> > > > > > >>> > > >> > be >> > > > > > >>> > > >> > > no >> > > > > > >>> > > >> > > > > > > > inter-layer data intersection and I >> think >> > it >> > > > > will >> > > > > > be >> > > > > > >>> > > easier >> > > > > > >>> > > >> to >> > > > > > >>> > > >> > > > > > implement >> > > > > > >>> > > >> > > > > > > on >> > > > > > >>> > > >> > > > > > > > both client and server-side. But we >> still >> > > > can't >> > > > > be >> > > > > > >>> sure >> > > > > > >>> > > that >> > > > > > >>> > > >> > the >> > > > > > >>> > > >> > > > task >> > > > > > >>> > > >> > > > > > is >> > > > > > >>> > > >> > > > > > > > successfully started on a server. We >> won't >> > > > ever >> > > > > > know >> > > > > > >>> > about >> > > > > > >>> > > >> > > topology >> > > > > > >>> > > >> > > > > > > change, >> > > > > > >>> > > >> > > > > > > > because topology changed flag will be >> sent >> > > > from >> > > > > > >>> server >> > > > > > >>> > to >> > > > > > >>> > > >> > client >> > > > > > >>> > > >> > > > only >> > > > > > >>> > > >> > > > > > > with >> > > > > > >>> > > >> > > > > > > > a response when the task will be >> > completed. >> > > > Are >> > > > > we >> > > > > > >>> > accept >> > > > > > >>> > > >> that? >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > > пн, 25 нояб. 2019 г. в 19:07, Pavel >> > > Tupitsyn < >> > > > > > >>> > > >> > > ptupit...@apache.org >> > > > > > >>> > > >> > > > >: >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > > > Alex, >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > I have a simpler idea. We already do >> > > request >> > > > > id >> > > > > > >>> > handling >> > > > > > >>> > > >> in >> > > > > > >>> > > >> > the >> > > > > > >>> > > >> > > > > > > protocol, >> > > > > > >>> > > >> > > > > > > > > so: >> > > > > > >>> > > >> > > > > > > > > - Client sends a normal request to >> > execute >> > > > > > compute >> > > > > > >>> > task. >> > > > > > >>> > > >> > > Request >> > > > > > >>> > > >> > > > ID >> > > > > > >>> > > >> > > > > > is >> > > > > > >>> > > >> > > > > > > > > generated as usual. >> > > > > > >>> > > >> > > > > > > > > - As soon as task is completed, a >> > response >> > > > is >> > > > > > >>> > received. >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > As for cancellation - client can >> send a >> > > new >> > > > > > >>> request >> > > > > > >>> > > (with >> > > > > > >>> > > >> new >> > > > > > >>> > > >> > > > > request >> > > > > > >>> > > >> > > > > > > ID) >> > > > > > >>> > > >> > > > > > > > > and (in the body) pass the request ID >> > from >> > > > > above >> > > > > > >>> > > >> > > > > > > > > as a task identifier. As a result, >> there >> > > are >> > > > > two >> > > > > > >>> > > >> responses: >> > > > > > >>> > > >> > > > > > > > > - Cancellation response >> > > > > > >>> > > >> > > > > > > > > - Task response (with proper >> cancelled >> > > > status) >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > That's it, no need to modify the >> core of >> > > the >> > > > > > >>> protocol. >> > > > > > >>> > > One >> > > > > > >>> > > >> > > > request >> > > > > > >>> > > >> > > > > - >> > > > > > >>> > > >> > > > > > > one >> > > > > > >>> > > >> > > > > > > > > response. >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > On Mon, Nov 25, 2019 at 6:20 PM Alex >> > > > Plehanov >> > > > > < >> > > > > > >>> > > >> > > > > > plehanov.a...@gmail.com >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > > > wrote: >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Pavel, we need to inform the client >> > when >> > > > the >> > > > > > >>> task is >> > > > > > >>> > > >> > > completed, >> > > > > > >>> > > >> > > > > we >> > > > > > >>> > > >> > > > > > > need >> > > > > > >>> > > >> > > > > > > > > the >> > > > > > >>> > > >> > > > > > > > > > ability to cancel the task. I see >> > > several >> > > > > ways >> > > > > > >>> to >> > > > > > >>> > > >> implement >> > > > > > >>> > > >> > > > this: >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > 1. Сlient sends a request to the >> > server >> > > to >> > > > > > >>> start a >> > > > > > >>> > > task, >> > > > > > >>> > > >> > > server >> > > > > > >>> > > >> > > > > > > return >> > > > > > >>> > > >> > > > > > > > > task >> > > > > > >>> > > >> > > > > > > > > > id in response. Server notifies >> client >> > > > when >> > > > > > >>> task is >> > > > > > >>> > > >> > completed >> > > > > > >>> > > >> > > > > with >> > > > > > >>> > > >> > > > > > a >> > > > > > >>> > > >> > > > > > > > new >> > > > > > >>> > > >> > > > > > > > > > request (from server to client). >> > Client >> > > > can >> > > > > > >>> cancel >> > > > > > >>> > the >> > > > > > >>> > > >> task >> > > > > > >>> > > >> > > by >> > > > > > >>> > > >> > > > > > > sending >> > > > > > >>> > > >> > > > > > > > a >> > > > > > >>> > > >> > > > > > > > > > new request with operation type >> > "cancel" >> > > > and >> > > > > > >>> task >> > > > > > >>> > id. >> > > > > > >>> > > In >> > > > > > >>> > > >> > this >> > > > > > >>> > > >> > > > > case, >> > > > > > >>> > > >> > > > > > > we >> > > > > > >>> > > >> > > > > > > > > > should implement 2-ways requests. >> > > > > > >>> > > >> > > > > > > > > > 2. Client generates unique task id >> and >> > > > > sends a >> > > > > > >>> > request >> > > > > > >>> > > >> to >> > > > > > >>> > > >> > the >> > > > > > >>> > > >> > > > > > server >> > > > > > >>> > > >> > > > > > > to >> > > > > > >>> > > >> > > > > > > > > > start a task, server don't reply >> > > > immediately >> > > > > > but >> > > > > > >>> > wait >> > > > > > >>> > > >> until >> > > > > > >>> > > >> > > > task >> > > > > > >>> > > >> > > > > is >> > > > > > >>> > > >> > > > > > > > > > completed. Client can cancel task >> by >> > > > sending >> > > > > > new >> > > > > > >>> > > request >> > > > > > >>> > > >> > with >> > > > > > >>> > > >> > > > > > > operation >> > > > > > >>> > > >> > > > > > > > > > type "cancel" and task id. In this >> > case, >> > > > we >> > > > > > >>> should >> > > > > > >>> > > >> decouple >> > > > > > >>> > > >> > > > > request >> > > > > > >>> > > >> > > > > > > and >> > > > > > >>> > > >> > > > > > > > > > response on the server-side >> (currently >> > > > > > response >> > > > > > >>> is >> > > > > > >>> > > sent >> > > > > > >>> > > >> > right >> > > > > > >>> > > >> > > > > after >> > > > > > >>> > > >> > > > > > > > > request >> > > > > > >>> > > >> > > > > > > > > > was processed). Also, we can't be >> sure >> > > > that >> > > > > > >>> task is >> > > > > > >>> > > >> > > > successfully >> > > > > > >>> > > >> > > > > > > > started >> > > > > > >>> > > >> > > > > > > > > on >> > > > > > >>> > > >> > > > > > > > > > a server. >> > > > > > >>> > > >> > > > > > > > > > 3. Client sends a request to the >> > server >> > > to >> > > > > > >>> start a >> > > > > > >>> > > task, >> > > > > > >>> > > >> > > server >> > > > > > >>> > > >> > > > > > > return >> > > > > > >>> > > >> > > > > > > > id >> > > > > > >>> > > >> > > > > > > > > > in response. Client periodically >> asks >> > > the >> > > > > > server >> > > > > > >>> > about >> > > > > > >>> > > >> task >> > > > > > >>> > > >> > > > > status. >> > > > > > >>> > > >> > > > > > > > > Client >> > > > > > >>> > > >> > > > > > > > > > can cancel the task by sending new >> > > request >> > > > > > with >> > > > > > >>> > > >> operation >> > > > > > >>> > > >> > > type >> > > > > > >>> > > >> > > > > > > "cancel" >> > > > > > >>> > > >> > > > > > > > > and >> > > > > > >>> > > >> > > > > > > > > > task id. This case brings some >> > overhead >> > > to >> > > > > the >> > > > > > >>> > > >> > communication >> > > > > > >>> > > >> > > > > > channel. >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Personally, I think that the case >> with >> > > > > 2-ways >> > > > > > >>> > requests >> > > > > > >>> > > >> is >> > > > > > >>> > > >> > > > better, >> > > > > > >>> > > >> > > > > > but >> > > > > > >>> > > >> > > > > > > > I'm >> > > > > > >>> > > >> > > > > > > > > > open to any other ideas. >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Aleksandr, >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Filtering logic for >> > > > > > >>> OP_CLUSTER_GROUP_GET_NODE_IDS >> > > > > > >>> > > looks >> > > > > > >>> > > >> > > > > > > > overcomplicated. >> > > > > > >>> > > >> > > > > > > > > Do >> > > > > > >>> > > >> > > > > > > > > > we need server-side filtering at >> all? >> > > > > Wouldn't >> > > > > > >>> it be >> > > > > > >>> > > >> better >> > > > > > >>> > > >> > > to >> > > > > > >>> > > >> > > > > send >> > > > > > >>> > > >> > > > > > > > basic >> > > > > > >>> > > >> > > > > > > > > > info (ids, order, flags) for all >> nodes >> > > > > (there >> > > > > > is >> > > > > > >>> > > >> relatively >> > > > > > >>> > > >> > > > small >> > > > > > >>> > > >> > > > > > > > amount >> > > > > > >>> > > >> > > > > > > > > of >> > > > > > >>> > > >> > > > > > > > > > data) and extended info >> (attributes) >> > for >> > > > > > >>> selected >> > > > > > >>> > list >> > > > > > >>> > > >> of >> > > > > > >>> > > >> > > > nodes? >> > > > > > >>> > > >> > > > > In >> > > > > > >>> > > >> > > > > > > > this >> > > > > > >>> > > >> > > > > > > > > > case, we can do basic node >> filtration >> > on >> > > > > > >>> client-side >> > > > > > >>> > > >> > > > > (forClients(), >> > > > > > >>> > > >> > > > > > > > > > forServers(), forNodeIds(), >> > forOthers(), >> > > > > etc). >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > Do you use standard ClusterNode >> > > > > serialization? >> > > > > > >>> There >> > > > > > >>> > > are >> > > > > > >>> > > >> > also >> > > > > > >>> > > >> > > > > > metrics >> > > > > > >>> > > >> > > > > > > > > > serialized with ClusterNode, do we >> > need >> > > it >> > > > > on >> > > > > > >>> thin >> > > > > > >>> > > >> client? >> > > > > > >>> > > >> > > > There >> > > > > > >>> > > >> > > > > > are >> > > > > > >>> > > >> > > > > > > > > other >> > > > > > >>> > > >> > > > > > > > > > interfaces exist to show metrics, I >> > > think >> > > > > it's >> > > > > > >>> > > >> redundant to >> > > > > > >>> > > >> > > > > export >> > > > > > >>> > > >> > > > > > > > > metrics >> > > > > > >>> > > >> > > > > > > > > > to thin clients too. >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > What do you think? >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > пт, 22 нояб. 2019 г. в 20:15, >> > Aleksandr >> > > > > > Shapkin >> > > > > > >>> < >> > > > > > >>> > > >> > > > > lexw...@gmail.com >> > > > > > >>> > > >> > > > > > >: >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Alex, >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > I think you can create a new IEP >> > page >> > > > and >> > > > > I >> > > > > > >>> will >> > > > > > >>> > > fill >> > > > > > >>> > > >> it >> > > > > > >>> > > >> > > with >> > > > > > >>> > > >> > > > > the >> > > > > > >>> > > >> > > > > > > > > Cluster >> > > > > > >>> > > >> > > > > > > > > > > API details. >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > In short, I’ve introduced several >> > new >> > > > > codes: >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Cluster API is pretty >> > straightforward: >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_IS_ACTIVE = 5000 >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_CHANGE_STATE = 5001 >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_CHANGE_WAL_STATE = >> 5002 >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_GET_WAL_STATE = 5003 >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Cluster group codes: >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_GROUP_GET_NODE_IDS = >> 5100 >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > OP_CLUSTER_GROUP_GET_NODE_INFO = >> > 5101 >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > The underlying implementation is >> > based >> > > > on >> > > > > > the >> > > > > > >>> > thick >> > > > > > >>> > > >> > client >> > > > > > >>> > > >> > > > > logic. >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > For every request, we provide a >> > known >> > > > > > topology >> > > > > > >>> > > version >> > > > > > >>> > > >> > and >> > > > > > >>> > > >> > > if >> > > > > > >>> > > >> > > > > it >> > > > > > >>> > > >> > > > > > > has >> > > > > > >>> > > >> > > > > > > > > > > changed, >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > a client updates it firstly and >> then >> > > > > > re-sends >> > > > > > >>> the >> > > > > > >>> > > >> > filtering >> > > > > > >>> > > >> > > > > > > request. >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Alongside the topVer a client >> sends >> > a >> > > > > > >>> serialized >> > > > > > >>> > > nodes >> > > > > > >>> > > >> > > > > projection >> > > > > > >>> > > >> > > > > > > > > object >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > that could be considered as a >> code >> > to >> > > > > value >> > > > > > >>> > mapping. >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Consider: [{Code = 1, Value= >> > > [“DotNet”, >> > > > > > >>> > > >> “MyAttribute”}, >> > > > > > >>> > > >> > > > > {Code=2, >> > > > > > >>> > > >> > > > > > > > > > Value=1}] >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > Where “1” stands for Attribute >> > > filtering >> > > > > and >> > > > > > >>> “2” – >> > > > > > >>> > > >> > > > > > serverNodesOnly >> > > > > > >>> > > >> > > > > > > > > flag. >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > As a result of request >> processing, a >> > > > > server >> > > > > > >>> sends >> > > > > > >>> > > >> nodeId >> > > > > > >>> > > >> > > > UUIDs >> > > > > > >>> > > >> > > > > > and >> > > > > > >>> > > >> > > > > > > a >> > > > > > >>> > > >> > > > > > > > > > > current topVer. >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > When a client obtains nodeIds, it >> > can >> > > > > > perform >> > > > > > >>> a >> > > > > > >>> > > >> NODE_INFO >> > > > > > >>> > > >> > > > call >> > > > > > >>> > > >> > > > > to >> > > > > > >>> > > >> > > > > > > > get a >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > serialized ClusterNode object. In >> > > > addition >> > > > > > >>> there >> > > > > > >>> > > >> should >> > > > > > >>> > > >> > be >> > > > > > >>> > > >> > > a >> > > > > > >>> > > >> > > > > > > > different >> > > > > > >>> > > >> > > > > > > > > > API >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > method for accessing/updating >> node >> > > > > metrics. >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > чт, 21 нояб. 2019 г. в 12:32, >> Sergey >> > > > > Kozlov >> > > > > > < >> > > > > > >>> > > >> > > > > > skoz...@gridgain.com >> > > > > > >>> > > >> > > > > > > >: >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > Hi Pavel >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > On Thu, Nov 21, 2019 at 11:30 >> AM >> > > Pavel >> > > > > > >>> Tupitsyn >> > > > > > >>> > < >> > > > > > >>> > > >> > > > > > > > > ptupit...@apache.org> >> > > > > > >>> > > >> > > > > > > > > > > > wrote: >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > 1. I believe that Cluster >> > > operations >> > > > > for >> > > > > > >>> Thin >> > > > > > >>> > > >> Client >> > > > > > >>> > > >> > > > > protocol >> > > > > > >>> > > >> > > > > > > are >> > > > > > >>> > > >> > > > > > > > > > > already >> > > > > > >>> > > >> > > > > > > > > > > > > in the works >> > > > > > >>> > > >> > > > > > > > > > > > > by Alexandr Shapkin. Can't >> find >> > > the >> > > > > > ticket >> > > > > > >>> > > though. >> > > > > > >>> > > >> > > > > > > > > > > > > Alexandr, can you please >> confirm >> > > and >> > > > > > >>> attach >> > > > > > >>> > the >> > > > > > >>> > > >> > ticket >> > > > > > >>> > > >> > > > > > number? >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > 2. Proposed changes will work >> > only >> > > > for >> > > > > > >>> Java >> > > > > > >>> > > tasks >> > > > > > >>> > > >> > that >> > > > > > >>> > > >> > > > are >> > > > > > >>> > > >> > > > > > > > already >> > > > > > >>> > > >> > > > > > > > > > > > deployed >> > > > > > >>> > > >> > > > > > > > > > > > > on server nodes. >> > > > > > >>> > > >> > > > > > > > > > > > > This is mostly useless for >> other >> > > > thin >> > > > > > >>> clients >> > > > > > >>> > we >> > > > > > >>> > > >> have >> > > > > > >>> > > >> > > > > > (Python, >> > > > > > >>> > > >> > > > > > > > PHP, >> > > > > > >>> > > >> > > > > > > > > > > .NET, >> > > > > > >>> > > >> > > > > > > > > > > > > C++). >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > I don't guess so. The task >> > > (execution) >> > > > > is >> > > > > > a >> > > > > > >>> way >> > > > > > >>> > to >> > > > > > >>> > > >> > > > implement >> > > > > > >>> > > >> > > > > > own >> > > > > > >>> > > >> > > > > > > > > layer >> > > > > > >>> > > >> > > > > > > > > > > for >> > > > > > >>> > > >> > > > > > > > > > > > the thin client application. >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > We should think of a way to >> make >> > > > this >> > > > > > >>> useful >> > > > > > >>> > for >> > > > > > >>> > > >> all >> > > > > > >>> > > >> > > > > clients. >> > > > > > >>> > > >> > > > > > > > > > > > > For example, we may allow >> > sending >> > > > > tasks >> > > > > > in >> > > > > > >>> > some >> > > > > > >>> > > >> > > scripting >> > > > > > >>> > > >> > > > > > > > language >> > > > > > >>> > > >> > > > > > > > > > like >> > > > > > >>> > > >> > > > > > > > > > > > > Javascript. >> > > > > > >>> > > >> > > > > > > > > > > > > Thoughts? >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > The arbitrary code execution >> from >> > a >> > > > > remote >> > > > > > >>> > client >> > > > > > >>> > > >> must >> > > > > > >>> > > >> > be >> > > > > > >>> > > >> > > > > > > protected >> > > > > > >>> > > >> > > > > > > > > > > > from malicious code. >> > > > > > >>> > > >> > > > > > > > > > > > I don't know how it could be >> > > designed >> > > > > but >> > > > > > >>> > without >> > > > > > >>> > > >> that >> > > > > > >>> > > >> > we >> > > > > > >>> > > >> > > > > open >> > > > > > >>> > > >> > > > > > > the >> > > > > > >>> > > >> > > > > > > > > hole >> > > > > > >>> > > >> > > > > > > > > > > to >> > > > > > >>> > > >> > > > > > > > > > > > kill cluster. >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > On Thu, Nov 21, 2019 at >> 11:21 AM >> > > > > Sergey >> > > > > > >>> > Kozlov < >> > > > > > >>> > > >> > > > > > > > > skoz...@gridgain.com >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > wrote: >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > Hi Alex >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > The idea is great. But I >> have >> > > some >> > > > > > >>> concerns >> > > > > > >>> > > that >> > > > > > >>> > > >> > > > probably >> > > > > > >>> > > >> > > > > > > > should >> > > > > > >>> > > >> > > > > > > > > be >> > > > > > >>> > > >> > > > > > > > > > > > taken >> > > > > > >>> > > >> > > > > > > > > > > > > > into account for design: >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > 1. We need to have the >> > > ability >> > > > to >> > > > > > >>> stop a >> > > > > > >>> > > task >> > > > > > >>> > > >> > > > > execution, >> > > > > > >>> > > >> > > > > > > > smth >> > > > > > >>> > > >> > > > > > > > > > like >> > > > > > >>> > > >> > > > > > > > > > > > > > OP_COMPUTE_CANCEL_TASK >> > > > operation >> > > > > > >>> (client >> > > > > > >>> > > to >> > > > > > >>> > > >> > > server) >> > > > > > >>> > > >> > > > > > > > > > > > > > 2. What's about task >> > > execution >> > > > > > >>> timeout? >> > > > > > >>> > It >> > > > > > >>> > > >> may >> > > > > > >>> > > >> > > help >> > > > > > >>> > > >> > > > to >> > > > > > >>> > > >> > > > > > the >> > > > > > >>> > > >> > > > > > > > > > cluster >> > > > > > >>> > > >> > > > > > > > > > > > > > survival for buggy tasks >> > > > > > >>> > > >> > > > > > > > > > > > > > 3. Ignite doesn't have >> > > > > > >>> > roles/authorization >> > > > > > >>> > > >> > > > > functionality >> > > > > > >>> > > >> > > > > > > for >> > > > > > >>> > > >> > > > > > > > > > now. >> > > > > > >>> > > >> > > > > > > > > > > > But >> > > > > > >>> > > >> > > > > > > > > > > > > a >> > > > > > >>> > > >> > > > > > > > > > > > > > task is the risky >> operation >> > > for >> > > > > > >>> cluster >> > > > > > >>> > > (for >> > > > > > >>> > > >> > > > security >> > > > > > >>> > > >> > > > > > > > > reasons). >> > > > > > >>> > > >> > > > > > > > > > > > Could >> > > > > > >>> > > >> > > > > > > > > > > > > we >> > > > > > >>> > > >> > > > > > > > > > > > > > add for Ignite >> > configuration >> > > > new >> > > > > > >>> options: >> > > > > > >>> > > >> > > > > > > > > > > > > > - Explicit turning on >> > for >> > > > > > compute >> > > > > > >>> task >> > > > > > >>> > > >> > support >> > > > > > >>> > > >> > > > for >> > > > > > >>> > > >> > > > > > thin >> > > > > > >>> > > >> > > > > > > > > > > protocol >> > > > > > >>> > > >> > > > > > > > > > > > > > (disabled by default) >> > for >> > > > > whole >> > > > > > >>> > cluster >> > > > > > >>> > > >> > > > > > > > > > > > > > - Explicit turning on >> > for >> > > > > > compute >> > > > > > >>> task >> > > > > > >>> > > >> > support >> > > > > > >>> > > >> > > > for >> > > > > > >>> > > >> > > > > a >> > > > > > >>> > > >> > > > > > > node >> > > > > > >>> > > >> > > > > > > > > > > > > > - The list of task >> names >> > > > > > (classes) >> > > > > > >>> > > >> allowed to >> > > > > > >>> > > >> > > > > execute >> > > > > > >>> > > >> > > > > > > by >> > > > > > >>> > > >> > > > > > > > > thin >> > > > > > >>> > > >> > > > > > > > > > > > > client. >> > > > > > >>> > > >> > > > > > > > > > > > > > 4. Support the labeling >> for >> > > > task >> > > > > > >>> that may >> > > > > > >>> > > >> help >> > > > > > >>> > > >> > to >> > > > > > >>> > > >> > > > > > > > investigate >> > > > > > >>> > > >> > > > > > > > > > > issues >> > > > > > >>> > > >> > > > > > > > > > > > > on >> > > > > > >>> > > >> > > > > > > > > > > > > > cluster (the idea from >> > IEP-34 >> > > > > [1]) >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > 1. >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > >> > > > > > >>> > > >> >> > > > > > >>> > > >> > > > > > >>> > >> > > > > > >>> >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > On Thu, Nov 21, 2019 at >> 10:58 >> > AM >> > > > > Alex >> > > > > > >>> > > Plehanov < >> > > > > > >>> > > >> > > > > > > > > > > > plehanov.a...@gmail.com> >> > > > > > >>> > > >> > > > > > > > > > > > > > wrote: >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Hello, Igniters! >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > I have plans to start >> > > > > implementation >> > > > > > >>> of >> > > > > > >>> > > >> Compute >> > > > > > >>> > > >> > > > > interface >> > > > > > >>> > > >> > > > > > > for >> > > > > > >>> > > >> > > > > > > > > > > Ignite >> > > > > > >>> > > >> > > > > > > > > > > > > thin >> > > > > > >>> > > >> > > > > > > > > > > > > > > client and want to >> discuss >> > > > > features >> > > > > > >>> that >> > > > > > >>> > > >> should >> > > > > > >>> > > >> > be >> > > > > > >>> > > >> > > > > > > > implemented. >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > We already have Compute >> > > > > > >>> implementation for >> > > > > > >>> > > >> > > > binary-rest >> > > > > > >>> > > >> > > > > > > > clients >> > > > > > >>> > > >> > > > > > > > > > > > > > > (GridClientCompute), >> which >> > > have >> > > > > the >> > > > > > >>> > > following >> > > > > > >>> > > >> > > > > > > functionality: >> > > > > > >>> > > >> > > > > > > > > > > > > > > - Filtering cluster nodes >> > > > > > >>> (projection) for >> > > > > > >>> > > >> > compute >> > > > > > >>> > > >> > > > > > > > > > > > > > > - Executing task by the >> name >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > I think we can implement >> > this >> > > > > > >>> > functionality >> > > > > > >>> > > >> in a >> > > > > > >>> > > >> > > thin >> > > > > > >>> > > >> > > > > > > client >> > > > > > >>> > > >> > > > > > > > as >> > > > > > >>> > > >> > > > > > > > > > > well. >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > First of all, we need >> some >> > > > > operation >> > > > > > >>> types >> > > > > > >>> > > to >> > > > > > >>> > > >> > > > request a >> > > > > > >>> > > >> > > > > > > list >> > > > > > >>> > > >> > > > > > > > of >> > > > > > >>> > > >> > > > > > > > > > all >> > > > > > >>> > > >> > > > > > > > > > > > > > > available nodes and >> probably >> > > > node >> > > > > > >>> > attributes >> > > > > > >>> > > >> (by >> > > > > > >>> > > >> > a >> > > > > > >>> > > >> > > > list >> > > > > > >>> > > >> > > > > > of >> > > > > > >>> > > >> > > > > > > > > > nodes). >> > > > > > >>> > > >> > > > > > > > > > > > Node >> > > > > > >>> > > >> > > > > > > > > > > > > > > attributes will be >> helpful >> > if >> > > we >> > > > > > will >> > > > > > >>> > decide >> > > > > > >>> > > >> to >> > > > > > >>> > > >> > > > > implement >> > > > > > >>> > > >> > > > > > > > > analog >> > > > > > >>> > > >> > > > > > > > > > of >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> ClusterGroup#forAttribute or >> > > > > > >>> > > >> > > > ClusterGroup#forePredicate >> > > > > > >>> > > >> > > > > > > > methods >> > > > > > >>> > > >> > > > > > > > > > in >> > > > > > >>> > > >> > > > > > > > > > > > the >> > > > > > >>> > > >> > > > > > > > > > > > > > thin >> > > > > > >>> > > >> > > > > > > > > > > > > > > client. Perhaps they can >> be >> > > > > > requested >> > > > > > >>> > > lazily. >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > From the protocol point >> of >> > > view >> > > > > > there >> > > > > > >>> will >> > > > > > >>> > > be >> > > > > > >>> > > >> two >> > > > > > >>> > > >> > > new >> > > > > > >>> > > >> > > > > > > > > operations: >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > OP_CLUSTER_GET_NODES >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: empty >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: long >> > > topologyVersion, >> > > > > int >> > > > > > >>> > > >> > > > > minorTopologyVersion, >> > > > > > >>> > > >> > > > > > > int >> > > > > > >>> > > >> > > > > > > > > > > > > nodesCount, >> > > > > > >>> > > >> > > > > > > > > > > > > > > for each node set of node >> > > fields >> > > > > > (UUID >> > > > > > >>> > > nodeId, >> > > > > > >>> > > >> > > Object >> > > > > > >>> > > >> > > > > or >> > > > > > >>> > > >> > > > > > > > String >> > > > > > >>> > > >> > > > > > > > > > > > > > > consistentId, long order, >> > etc) >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > OP_CLUSTER_GET_NODE_ATTRIBUTES >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: int nodesCount, >> for >> > > > each >> > > > > > >>> node: >> > > > > > >>> > UUID >> > > > > > >>> > > >> > nodeId >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: int nodesCount, >> > for >> > > > each >> > > > > > >>> node: >> > > > > > >>> > int >> > > > > > >>> > > >> > > > > > > attributesCount, >> > > > > > >>> > > >> > > > > > > > > for >> > > > > > >>> > > >> > > > > > > > > > > > each >> > > > > > >>> > > >> > > > > > > > > > > > > > node >> > > > > > >>> > > >> > > > > > > > > > > > > > > attribute: String name, >> > Object >> > > > > value >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > To execute tasks we need >> > > > something >> > > > > > >>> like >> > > > > > >>> > > these >> > > > > > >>> > > >> > > methods >> > > > > > >>> > > >> > > > > in >> > > > > > >>> > > >> > > > > > > the >> > > > > > >>> > > >> > > > > > > > > > client >> > > > > > >>> > > >> > > > > > > > > > > > > API: >> > > > > > >>> > > >> > > > > > > > > > > > > > > Object execute(String >> task, >> > > > Object >> > > > > > >>> arg) >> > > > > > >>> > > >> > > > > > > > > > > > > > > Future<Object> >> > > > executeAsync(String >> > > > > > >>> task, >> > > > > > >>> > > >> Object >> > > > > > >>> > > >> > > arg) >> > > > > > >>> > > >> > > > > > > > > > > > > > > Object >> > affinityExecute(String >> > > > > task, >> > > > > > >>> String >> > > > > > >>> > > >> cache, >> > > > > > >>> > > >> > > > > Object >> > > > > > >>> > > >> > > > > > > key, >> > > > > > >>> > > >> > > > > > > > > > > Object >> > > > > > >>> > > >> > > > > > > > > > > > > arg) >> > > > > > >>> > > >> > > > > > > > > > > > > > > Future<Object> >> > > > > > >>> affinityExecuteAsync(String >> > > > > > >>> > > >> task, >> > > > > > >>> > > >> > > > String >> > > > > > >>> > > >> > > > > > > > cache, >> > > > > > >>> > > >> > > > > > > > > > > Object >> > > > > > >>> > > >> > > > > > > > > > > > > > key, >> > > > > > >>> > > >> > > > > > > > > > > > > > > Object arg) >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Which can be mapped to >> > > protocol >> > > > > > >>> > operations: >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > OP_COMPUTE_EXECUTE_TASK >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: UUID nodeId, >> String >> > > > > > taskName, >> > > > > > >>> > > Object >> > > > > > >>> > > >> arg >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: Object result >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > OP_COMPUTE_EXECUTE_TASK_AFFINITY >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: String >> cacheName, >> > > > Object >> > > > > > key, >> > > > > > >>> > > String >> > > > > > >>> > > >> > > > taskName, >> > > > > > >>> > > >> > > > > > > > Object >> > > > > > >>> > > >> > > > > > > > > > arg >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: Object result >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > The second operation is >> > needed >> > > > > > >>> because we >> > > > > > >>> > > >> > sometimes >> > > > > > >>> > > >> > > > > can't >> > > > > > >>> > > >> > > > > > > > > > calculate >> > > > > > >>> > > >> > > > > > > > > > > > and >> > > > > > >>> > > >> > > > > > > > > > > > > > > connect to affinity node >> on >> > > the >> > > > > > >>> > client-side >> > > > > > >>> > > >> > > (affinity >> > > > > > >>> > > >> > > > > > > > awareness >> > > > > > >>> > > >> > > > > > > > > > can >> > > > > > >>> > > >> > > > > > > > > > > > be >> > > > > > >>> > > >> > > > > > > > > > > > > > > disabled, custom affinity >> > > > function >> > > > > > >>> can be >> > > > > > >>> > > >> used or >> > > > > > >>> > > >> > > > there >> > > > > > >>> > > >> > > > > > can >> > > > > > >>> > > >> > > > > > > > be >> > > > > > >>> > > >> > > > > > > > > no >> > > > > > >>> > > >> > > > > > > > > > > > > > > connection between client >> > and >> > > > > > affinity >> > > > > > >>> > > node), >> > > > > > >>> > > >> but >> > > > > > >>> > > >> > > we >> > > > > > >>> > > >> > > > > can >> > > > > > >>> > > >> > > > > > > make >> > > > > > >>> > > >> > > > > > > > > > best >> > > > > > >>> > > >> > > > > > > > > > > > > effort >> > > > > > >>> > > >> > > > > > > > > > > > > > > to send request to target >> > node >> > > > if >> > > > > > >>> affinity >> > > > > > >>> > > >> > > awareness >> > > > > > >>> > > >> > > > is >> > > > > > >>> > > >> > > > > > > > > enabled. >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Currently, on the >> > server-side >> > > > > > requests >> > > > > > >>> > > always >> > > > > > >>> > > >> > > > processed >> > > > > > >>> > > >> > > > > > > > > > > synchronously >> > > > > > >>> > > >> > > > > > > > > > > > > and >> > > > > > >>> > > >> > > > > > > > > > > > > > > responses are sent right >> > after >> > > > > > >>> request was >> > > > > > >>> > > >> > > processed. >> > > > > > >>> > > >> > > > > To >> > > > > > >>> > > >> > > > > > > > > execute >> > > > > > >>> > > >> > > > > > > > > > > long >> > > > > > >>> > > >> > > > > > > > > > > > > > tasks >> > > > > > >>> > > >> > > > > > > > > > > > > > > async we should whether >> > change >> > > > > this >> > > > > > >>> logic >> > > > > > >>> > or >> > > > > > >>> > > >> > > > introduce >> > > > > > >>> > > >> > > > > > some >> > > > > > >>> > > >> > > > > > > > > kind >> > > > > > >>> > > >> > > > > > > > > > > > > two-way >> > > > > > >>> > > >> > > > > > > > > > > > > > > communication between >> client >> > > and >> > > > > > >>> server >> > > > > > >>> > (now >> > > > > > >>> > > >> only >> > > > > > >>> > > >> > > > > one-way >> > > > > > >>> > > >> > > > > > > > > > requests >> > > > > > >>> > > >> > > > > > > > > > > > from >> > > > > > >>> > > >> > > > > > > > > > > > > > > client to server are >> > allowed). >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Two-way communication can >> > also >> > > > be >> > > > > > >>> useful >> > > > > > >>> > in >> > > > > > >>> > > >> the >> > > > > > >>> > > >> > > > future >> > > > > > >>> > > >> > > > > if >> > > > > > >>> > > >> > > > > > > we >> > > > > > >>> > > >> > > > > > > > > will >> > > > > > >>> > > >> > > > > > > > > > > > send >> > > > > > >>> > > >> > > > > > > > > > > > > > some >> > > > > > >>> > > >> > > > > > > > > > > > > > > server-side generated >> events >> > > to >> > > > > > >>> clients. >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > In case of two-way >> > > communication >> > > > > > >>> there can >> > > > > > >>> > > be >> > > > > > >>> > > >> new >> > > > > > >>> > > >> > > > > > > operations >> > > > > > >>> > > >> > > > > > > > > > > > > introduced: >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > OP_COMPUTE_EXECUTE_TASK >> > (from >> > > > > client >> > > > > > >>> to >> > > > > > >>> > > >> server) >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: UUID nodeId, >> String >> > > > > > taskName, >> > > > > > >>> > > Object >> > > > > > >>> > > >> arg >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: long taskId >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > OP_COMPUTE_TASK_FINISHED >> > (from >> > > > > > server >> > > > > > >>> to >> > > > > > >>> > > >> client) >> > > > > > >>> > > >> > > > > > > > > > > > > > > Request: taskId, Object >> > result >> > > > > > >>> > > >> > > > > > > > > > > > > > > Response: empty >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > The same for affinity >> > > requests. >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > Also, we can implement >> not >> > > only >> > > > > > >>> execute >> > > > > > >>> > task >> > > > > > >>> > > >> > > > operation, >> > > > > > >>> > > >> > > > > > but >> > > > > > >>> > > >> > > > > > > > > some >> > > > > > >>> > > >> > > > > > > > > > > > other >> > > > > > >>> > > >> > > > > > > > > > > > > > > operations from >> > IgniteCompute >> > > > > > >>> (broadcast, >> > > > > > >>> > > run, >> > > > > > >>> > > >> > > call), >> > > > > > >>> > > >> > > > > but >> > > > > > >>> > > >> > > > > > > it >> > > > > > >>> > > >> > > > > > > > > will >> > > > > > >>> > > >> > > > > > > > > > > be >> > > > > > >>> > > >> > > > > > > > > > > > > > useful >> > > > > > >>> > > >> > > > > > > > > > > > > > > only for java thin >> client. >> > And >> > > > > even >> > > > > > >>> with >> > > > > > >>> > > java >> > > > > > >>> > > >> > thin >> > > > > > >>> > > >> > > > > client >> > > > > > >>> > > >> > > > > > > we >> > > > > > >>> > > >> > > > > > > > > > should >> > > > > > >>> > > >> > > > > > > > > > > > > > whether >> > > > > > >>> > > >> > > > > > > > > > > > > > > implement >> peer-class-loading >> > > for >> > > > > > thin >> > > > > > >>> > > clients >> > > > > > >>> > > >> > (this >> > > > > > >>> > > >> > > > > also >> > > > > > >>> > > >> > > > > > > > > requires >> > > > > > >>> > > >> > > > > > > > > > > > > two-way >> > > > > > >>> > > >> > > > > > > > > > > > > > > client-server >> communication) >> > > or >> > > > > put >> > > > > > >>> > classes >> > > > > > >>> > > >> with >> > > > > > >>> > > >> > > > > executed >> > > > > > >>> > > >> > > > > > > > > > closures >> > > > > > >>> > > >> > > > > > > > > > > to >> > > > > > >>> > > >> > > > > > > > > > > > > the >> > > > > > >>> > > >> > > > > > > > > > > > > > > server locally. >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > > What do you think about >> > > proposed >> > > > > > >>> protocol >> > > > > > >>> > > >> > changes? >> > > > > > >>> > > >> > > > > > > > > > > > > > > Do we need two-way >> requests >> > > > > between >> > > > > > >>> client >> > > > > > >>> > > and >> > > > > > >>> > > >> > > > server? >> > > > > > >>> > > >> > > > > > > > > > > > > > > Do we need support of >> > compute >> > > > > > methods >> > > > > > >>> > other >> > > > > > >>> > > >> than >> > > > > > >>> > > >> > > > > "execute >> > > > > > >>> > > >> > > > > > > > > task"? >> > > > > > >>> > > >> > > > > > > > > > > > > > > What do you think about >> > > > > > >>> peer-class-loading >> > > > > > >>> > > for >> > > > > > >>> > > >> > thin >> > > > > > >>> > > >> > > > > > > clients? >> > > > > > >>> > > >> > > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > > -- >> > > > > > >>> > > >> > > > > > > > > > > > > > Sergey Kozlov >> > > > > > >>> > > >> > > > > > > > > > > > > > GridGain Systems >> > > > > > >>> > > >> > > > > > > > > > > > > > www.gridgain.com >> > > > > > >>> > > >> > > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > > -- >> > > > > > >>> > > >> > > > > > > > > > > > Sergey Kozlov >> > > > > > >>> > > >> > > > > > > > > > > > GridGain Systems >> > > > > > >>> > > >> > > > > > > > > > > > www.gridgain.com >> > > > > > >>> > > >> > > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > > -- >> > > > > > >>> > > >> > > > > > > > > > > Alex. >> > > > > > >>> > > >> > > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > > >> > > > > > >>> > > >> > > > > > > > > >> > > > > > >>> > > >> > > > > > > > >> > > > > > >>> > > >> > > > > > > >> > > > > > >>> > > >> > > > > > >> > > > > > >>> > > >> > > > > >> > > > > > >>> > > >> > > > >> > > > > > >>> > > >> > > >> > > > > > >>> > > >> > >> > > > > > >>> > > >> >> > > > > > >>> > > > >> > > > > > >>> > > >> > > > > > >>> > >> > > > > > >>> >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >> >
